Interactive Human-Machine Collaborative Path Planning and Manipulator Follow-up Control System and Method

Abstract

The present invention discloses an interactive human-machine collaborative path planning and robotic arm follow-up control system and method, which relates to the technical field of path planning control of medical robots. The control system includes an optical positioning camera, a robotic arm, a patient tracer, an interactor, and a controller; the robotic arm includes a robotic arm main controller, a robotic arm end tracer, and a robotic arm end guide; the interactor includes an interactor tracer; the optical positioning camera is used to obtain the poses of the interactor tracer, the robotic arm end tracer, and the patient tracer in the camera coordinate system; the robotic arm main controller is used to obtain the pose of the end of the robotic arm in the robotic arm base coordinate system; the controller is used to realize the follow-up display of the two-dimensional section of the three-dimensional image with the nail placement position of the interactor according to the real-time position mapping, and control the robotic arm to aim at the nail placement path. The present invention realizes a more intuitive, efficient, and safe non-contact human-machine interaction.

Description

Technical Field

[0001] The present invention relates to the technical field of path planning control of medical robots, and particularly relates to an interactive human-machine collaborative path planning and robotic arm follow-up control system and method. Background Art

[0002] In minimally invasive orthopedic nail insertion surgery, the commonly used interactive control method for robot-assisted navigation and positioning is based on medical image planning. That is, after completing the preoperative registration work, the positional relationships among the robotic arm operation space, the three-dimensional anatomical space of the affected area, and the medical image space are obtained. The doctor manually plans the nail insertion position in the two-dimensional medical image, and through the spatial registration matrix, maps the nail insertion position to the three-dimensional anatomical space, and controls the robotic arm to move to the nail insertion position in the robotic arm operation space.

[0003] However, the planning information of the above method is not intuitive enough. Due to the small incision in minimally invasive surgery, doctors are required to have rich anatomical knowledge and surgical experience to confirm the exact nail insertion position in a single two-dimensional medical image. Secondly, this method requires a sufficient learning curve. Doctors need to gradually adjust parameters such as the nail insertion angle and entry / exit points on the software planning interface, and doctors are required to be proficient in software operation. In addition, doctors need to move back and forth between the surgical area environment and the equipment, which not only reduces the surgical efficiency but also is not conducive to protecting the aseptic environment of the surgery.

[0004] Therefore, in the surgery of robot-assisted nail insertion, it is necessary to study a new type of interactive control strategy to achieve more intuitive, efficient, and safe intelligent interaction between doctors and the robot system. Summary of the Invention

[0005] In view of the above-mentioned defects or deficiencies in the prior art, the present invention provides an interactive human-machine collaborative path planning and robotic arm follow-up control system and method, which can wholly or partially solve the above-mentioned technical problems.

[0006] One aspect of the present invention provides an interactive human-machine collaborative path planning and robotic arm follow-up control system, comprising: an optical positioning camera, a robotic arm, a patient tracer, an interactor, and a controller; the robotic arm includes a robotic arm main controller, a robotic arm end tracer, and a robotic arm end guide; the interactor includes an interactor tracer; the optical positioning camera is configured to obtain the poses of the interactor tracer, the robotic arm end tracer, and the patient tracer in the camera coordinate system; the robotic arm main controller is configured to obtain the pose of the end of the robotic arm in the robotic arm base coordinate system; the controller is configured to calculate, based on the poses of the interactor tracer, the robotic arm end tracer, and the patient tracer in the camera coordinate system obtained by the optical positioning camera, and based on the pose of the current end of the robotic arm in the robotic arm base coordinate system obtained by the robotic arm main controller, a real-time position mapping of the interactor nail-insertion line segment in the three-dimensional image space, and to realize the follow-up display of the two-dimensional section of the three-dimensional image following the nail-insertion position of the interactor according to this real-time position mapping, and to control the movement of the robotic arm end guide until the nail-insertion line segment of the robotic arm end guide and the nail-insertion line segment of the interactor coincide in the robotic arm base coordinate system.

[0007] Another aspect of the present invention further provides a control method based on the above-mentioned interactive human-machine collaborative path planning and robotic arm follow-up control device, comprising: instructing the optical positioning camera to obtain the poses of the interactor tracer, the robotic arm end tracer, and the patient tracer in the camera coordinate system; instructing the robotic arm main controller to obtain the pose of the end of the robotic arm in the robotic arm base coordinate system; instructing the controller to calculate, based on the poses of the interactor tracer, the robotic arm end tracer, and the patient tracer in the camera coordinate system obtained by the optical positioning camera, and based on the pose of the current end of the robotic arm in the robotic arm base coordinate system obtained by the robotic arm main controller, a real-time position mapping of the interactor nail-insertion line segment in the three-dimensional image space, and to realize the follow-up display of the two-dimensional section of the three-dimensional image following the nail-insertion position of the interactor according to this real-time position mapping, and to control the movement of the robotic arm end guide until the nail-insertion line segment of the robotic arm end guide and the nail-insertion line segment of the interactor coincide in the robotic arm base coordinate system.

[0008] The interactive human-machine collaborative path planning and robotic arm follow-up control system and method provided by the present invention can, on the one hand, realize the real-time follow-up display of the two-dimensional sectional view of the medical image of the affected anatomical tissue along the interactor coordinate system, thereby enhancing the intuitiveness of the information of the affected area during the operation; on the other hand, it can realize the nail-insertion path planning along the axial direction of the interactor, and cooperate with the remote device to control the robotic arm to execute to the position where the interactor is located, realizing the non-contact interaction between the doctor and the robotic arm and improving the safety of the sterile environment during the operation; the third aspect is that the reading of the information of the affected area, the planning of the nail insertion, and the control execution of the robotic arm do not require manual operation of the software, shortening the learning curve of the doctor and improving the operation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Other features, objects, and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments read in conjunction with the accompanying drawings:

[0010] Figure 1 is a real-scene schematic diagram of an interactive human-machine collaborative path planning and robotic arm follow-up control system provided by an embodiment of the present application;

[0011] Figure 2 is a schematic diagram of the logical principle of an interactive human-machine collaborative path planning and robotic arm follow-up control system provided by an embodiment of the present application;

[0012] Figure 3 is a schematic diagram of the coordinate system and coordinate transformation of an interactive human-machine collaborative path planning and robotic arm follow-up control system provided by an embodiment of the present application. Detailed implementation manners

[0013] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0014] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "the", and "said" used in the embodiments of the present invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0015] It should be understood that although the terms first, second, third, etc. may be used in the embodiments of the present invention to describe the acquisition modules, these acquisition modules should not be limited to these terms. These terms are only used to distinguish the acquisition modules from each other.

[0016] Depending on the context, the word "if" as used herein can be interpreted as "when" or "while" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "if determined" or "if detecting (stated condition or event)" can be interpreted as "when determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)".

[0017] It should be noted that the orientation terms such as "upper", "lower", "left", and "right" described in the embodiments of the present invention are described from the angles shown in the drawings, and should not be construed as limitations on the embodiments of the present invention. In addition, in the context, it should also be understood that when it is mentioned that one element is formed "on" or "under" another element, it can not only be directly formed "on" or "under" another element, but also be indirectly formed "on" or "under" another element through an intermediate element.

[0018] Currently, there is a problem of poor human-computer interaction in minimally invasive orthopedic surgery assisted by robotic nail insertion. Doctors need to operate software to determine the nail insertion position in two-dimensional medical images. This method has non-intuitive planning information, a long learning curve, low surgical efficiency, and poor aseptic environmental protection.

[0019] To solve the above-mentioned technical problems existing in the prior art, refer to Figure 1 An embodiment of the present application provides an interactive human-machine collaborative path planning and robotic arm follow-up control system 100, including:

[0020] An optical positioning camera 101, a robotic arm 102, a patient tracer 103, an interactor 104, and a controller (not shown in the figure). Among them, the robotic arm 102 includes a robotic arm main controller 1021, a robotic arm end tracer 1022, and a robotic arm end guide 1023. The interactor 104 includes an interactor tracer.

[0021] Refer to Figure 2 , and the basic operation process of the control system 100 is as follows:

[0022] In the first step, preoperative medical image acquisition and registration are performed, and then the interactor 104 is calibrated to obtain the position conversion relationship between the interactor space and the medical three-dimensional image space.

[0023] In the second step: The doctor holds the interactor 104 and freely moves it at the affected area (not limited to adjusting the position, angle, and depth). The two-dimensional image information of the affected area under the current section where the interactor 104 is located can be real-time displayed on the software interface.

[0024] In the third step: The current position and pose of the interactor 104 are saved as the position and pose parameters for nail insertion, and the robotic arm is controlled to execute to the nail insertion position, thereby realizing rapid planning and navigation control.

[0025] The following details the connection relationships, structures, functions, and roles of the various components in the interactive human-machine collaborative path planning and robotic arm follow-up control system 100.

[0026] The optical positioning camera 101 is used to obtain the poses of the interactor tracer, the robotic arm end tracer 1022, and the patient tracer 103 in the camera coordinate system.

[0027] The manipulator master controller 1021 is used to obtain the pose of the end of the manipulator in the manipulator base coordinate system.

[0028] The controller is used to calculate the real-time position mapping of the nail-inserting line segment of the interactor in the three-dimensional image space according to the poses of the interactor tracer, the manipulator end tracer 1022, and the patient tracer 103 in the camera coordinate system obtained by the optical positioning camera 101, and the pose of the current manipulator end in the manipulator base coordinate system obtained by the manipulator master controller 1021. According to this real-time position mapping, the two-dimensional section of the three-dimensional image is displayed following the nail-inserting position of the interactor, and the movement of the manipulator end guide 1023 is controlled until the nail-inserting line segment of the manipulator end guide coincides with the nail-inserting line segment of the interactor in the manipulator base coordinate system.

[0029] See Figure 3 , specifically, the controller is used to: calculate the homogeneous transformation matrix from the camera coordinate system to the interactor tracer coordinate system according to the poses of the interactor tracer, the manipulator end tracer 1022, and the patient tracer 103 in the camera coordinate system . In addition, calculate the homogeneous transformation matrix from the camera coordinate system to the manipulator end coordinate system and the homogeneous transformation matrix from the camera coordinate system to the patient tracer coordinate system . Moreover, calculate the homogeneous transformation matrix from the manipulator base coordinate system to the manipulator end coordinate system according to the pose of the current manipulator end in the manipulator base coordinate system obtained by the manipulator master controller 1021 . . to the manipulator end coordinate system .

[0030] See Figure 3 , the controller is further used to realize the follow-up display of the two-dimensional section of the three-dimensional image with the nail-inserting position of the interactor:

[0031] Take the end point on the axial direction of the interactor as the nail-inserting end point , determine the nail-inserting starting point according to the nail-inserting length , and obtain the nail-inserting line segment of the interactor in the interactor tracer coordinate system ;

[0032] Calculate the homogeneous transformation matrix from the patient tracer coordinate system to the interactor tracer coordinate system​​ Homogeneous transformation matrix :

[0033] = ;

[0034] Wherein, is the camera coordinate system to the patient tracer coordinate system Homogeneous transformation matrix Inverse matrix of.

[0035] Calculate the starting point of the interactor nail placement line segment under the patient tracer coordinate system and the end point and the position of: According to the following formula:

[0036] = ;

[0037] = ;

[0038] Calculate the starting point of the interactor nail placement line segment in the three-dimensional image space and the end point and the position of:

[0039] = ;

[0040] = ;

[0041] Wherein, is the pre-acquired spatial transformation matrix from the patient tracer coordinate system to the three-dimensional image space coordinate system Inverse matrix of. It should be noted that the spatial transformation matrix can be calculated by the point cloud registration algorithm of ICP (Iterative Closest Point), which will not be elaborated here.

[0042] Finally, according to the real-time positions of the starting point and the end point , the two-dimensional section of the three-dimensional image can be realized to follow the movement display with the nail placement position of the interactor through the image processing algorithm.

[0043] See Figure 3 , the controller is further used to implement nail placement planning and robotic arm follow-up control:

[0044] Calculate the homogeneous transformation matrix from the manipulator base coordinate system to the interactuator tracer coordinate system as follows :

[0045] = ;

[0046] where is the inverse matrix of the homogeneous transformation matrix from the camera coordinate system to the manipulator end coordinate system ;

[0047] Convert the interactuator nailing line segment in the interactuator tracer coordinate system to the interactuator nailing line segment in the manipulator base coordinate system :

[0048] = ;

[0049] = ;

[0050] On the axial direction of the manipulator end guide Set the starting point of nailing at a preset distance offset from the lower edge of the manipulator end guide 1023 , preferably at an offset of 80 mm, and set the ending point of nailing at a distance offset from the starting point of nailing by the nail length to obtain the interactuator nailing line segment of the manipulator end coordinate system ; ;

[0051] Convert the interactuator nailing line segment of the manipulator end coordinate system to the interactuator nailing line segment of the manipulator base coordinate system as follows

[0052] = ;

[0053] = ;

[0054] The interactuator nailing line segment in the manipulator base coordinate system As the target position, the nail - inserting line segment of the end effector of the robotic arm in the robotic arm base coordinate system is used As the current position, through the inverse kinematics solution algorithm of the robotic arm, the movement of the end effector 1023 of the robotic arm is controlled until the nail - inserting line segment of the end effector of the robotic arm and the nail - inserting line segment of the interactor coincide in the robotic arm base coordinate system , thus realizing the nail - inserting planning at the specified position of the interactor 104 and the follow - up control of the robotic arm 102.

[0055] An interactive human - machine collaborative path planning and robotic arm follow - up control system 100 provided in this embodiment uses the interactor 104 to realize the intuitive display of three - dimensional medical image sections, the real - time planning of the nail - inserting position, and the follow - up control of the robotic arm, achieving a more intuitive, efficient, and safe non - contact human - machine interaction.

[0056] Another embodiment of this application further provides an interactive human - machine collaborative path planning and robotic arm follow - up control method. This control method is based on the control system in the above - mentioned device embodiment and realizes the planning and control of the nail - inserting path of the robotic arm.

[0057] Specifically, the interactive human - machine collaborative path planning and robotic arm follow - up control method of this embodiment includes the following steps:

[0058] Step S101: Let the optical positioning camera obtain the poses of the interactor tracer, the end - effector tracer of the robotic arm, and the patient tracer in the camera coordinate system;

[0059] Step S102: Let the main controller of the robotic arm obtain the pose of the end of the robotic arm in the robotic arm base coordinate system;

[0060] Step S103: Let the controller calculate the real - time position mapping of the nail - inserting line segment of the interactor in the three - dimensional image space according to the poses of the interactor tracer, the end - effector tracer of the robotic arm, and the patient tracer in the camera coordinate system obtained by the optical positioning camera, and according to the pose of the current end of the robotic arm in the robotic arm base coordinate system obtained by the main controller of the robotic arm. According to this real - time position mapping, realize the follow - up display of the two - dimensional section of the three - dimensional image with the nail - inserting position of the interactor, and control the movement of the end effector of the robotic arm until the nail - inserting line segment of the end effector of the robotic arm and the nail - inserting line segment of the interactor coincide in the robotic arm base coordinate system.

[0061] Further, step S103 further includes the following step S1031:

[0062] Let the controller calculate the homogeneous transformation matrix from the camera coordinate system to the interactor tracer coordinate system based on the poses of the interactor tracer, the end-effector tracer of the robotic arm, and the patient tracer in the camera coordinate system obtained by the optical positioning camera. , the homogeneous transformation matrix from the camera coordinate system to the end-effector coordinate system of the robotic arm and the homogeneous transformation matrix from the camera coordinate system to the patient tracer coordinate system .

[0063] Furthermore, step S103 further includes the following step S1032:

[0064] Let the controller calculate the homogeneous transformation matrix from the robotic arm base coordinate system to the end-effector coordinate system of the robotic arm based on the pose of the current end-effector of the robotic arm obtained by the robotic arm master controller. .

[0065] Furthermore, step S103 further includes the following step S1033:

[0066] Let the controller use the end point on the axial direction of the interactor as the nail insertion end point , determine the nail insertion start point according to the nail insertion length , and obtain the interactor nail insertion line segment in the interactor tracer coordinate system ;

[0067] Calculate the homogeneous transformation matrix from the patient tracer coordinate system to the interactor tracer coordinate system according to the following formula :

[0068] = ;

[0069] where is the inverse matrix of the homogeneous transformation matrix from the camera coordinate system to the patient tracer coordinate system.

[0070] Calculate the positions of the start point and the end point of the interactor nail insertion line segment in the patient tracer coordinate system according to the following formula:

[0071] = ;

[0072] = ;

[0073] Calculate the start point and the end point of the interactor nail insertion line segment in the three-dimensional image space according to the following formulaPosition:

[0074] = ;

[0075] = ;

[0076] wherein, is the inverse matrix of the spatial transformation matrix from the patient tracer coordinate system to the three-dimensional image space coordinate system obtained in advance; Inverse matrix;

[0077] According to the real-time positions of the starting point and the ending point The two-dimensional section of the three-dimensional image is realized to be displayed following the position of the nail placed by the interactor.

[0078] Furthermore, step S103 further includes the following step S1034:

[0079] Let the controller calculate the homogeneous transformation matrix from the robotic arm base coordinate system to the interactor tracer coordinate system according to the following formula :

[0080] = ;

[0081] wherein, is the inverse matrix of the homogeneous transformation matrix from the camera coordinate system to the robotic arm end coordinate system Inverse matrix;

[0082] Convert the interactor nail placement line segment in the interactor tracer coordinate system to the interactor nail placement line segment in the robotic arm base coordinate system:

[0083] = ;

[0084] = ;

[0085] On the axial direction of the robotic arm end guide, set the lower edge of the robotic arm end guide offset by a preset distance as the nail placement starting point , and set the nail placement starting point offset by the nail length as the nail placement ending point , to obtain the robotic arm end guide nail placement line segment in the robotic arm end coordinate system;

[0086] Convert the robotic arm end guide nail placement line segment in the robotic arm end coordinate system through the following formulaThe nail - driving line segment of the end - effector of the robotic arm in the base coordinate system of the robotic arm :

[0087] = ;

[0088] = ;

[0089] Take the nail - driving line segment of the interactor in the base coordinate system of the robotic arm as the target position, and take the nail - driving line segment of the end - effector of the robotic arm in the base coordinate system of the robotic arm as the current position, and control the movement of the end - effector of the robotic arm until the nail - driving line segment of the end - effector of the robotic arm and the nail - driving line segment of the interactor coincide in the base coordinate system of the robotic arm.

[0090] It should be noted that the interactive human - machine collaborative path - planning and robotic - arm follow - up control method provided in this embodiment can operate the control system 100 in the device embodiment by means of a computer program or a field - programmable logic hardware. Its implementation principle and technical effects are similar to the functions of each component in the device embodiment, and will not be elaborated here.

[0091] The above description is only a preferred embodiment of the present invention. Those skilled in the art should understand that the disclosed scope in the present invention is not limited to the technical solution formed by the specific combination of the above - mentioned technical features, and should also cover other technical solutions formed by any combination of the above - mentioned technical features or their equivalent features without departing from the above - mentioned disclosed concept. For example, the technical solutions formed by mutually replacing the above - mentioned features with the (but not limited to) technical features with similar functions disclosed in the present invention.

Claims

1. A path planning and follow-up system for minimally invasive orthopedic nail insertion, characterized in that, Comprising: An optical positioning camera, a robotic arm, a patient tracer, an interactor, and a controller; The robotic arm includes a robotic arm master controller, a robotic arm end tracer, and a robotic arm end guide, and the interactor includes an interactor tracer; The optical positioning camera is used to obtain the poses of the interactor tracer, the robotic arm end tracer, and the patient tracer in the camera coordinate system; The robotic arm master controller is used to obtain the pose of the end of the robotic arm in the robotic arm base coordinate system; A controller, configured to calculate a homogeneous transformation matrix from the camera coordinate system to the interactor tracer coordinate system based on the poses of the interactor tracer, the end effector tracer, and the patient tracer obtained by an optical positioning camera in the camera coordinate system , a homogeneous transformation matrix from the camera coordinate system to the end effector coordinate system , and a homogeneous transformation matrix from the camera coordinate system to the patient tracer coordinate system ; According to the pose of the end of the current robotic arm obtained by the robotic arm master controller in the robotic arm base coordinate system, the homogeneous transformation matrix from the robotic arm base coordinate system to the robotic arm end coordinate system is calculated ; Take the end point in the axial direction of the interactor as the nail - placing end point , determine the nail - placing starting point according to the nail - placing length , and obtain the nail - placing line segment of the interactor in the coordinate system of the interactor tracer ; Calculate the homogeneous transformation matrix from the patient tracer coordinate system to the interactor tracer coordinate system according to the following formula : = ； wherein, is the inverse matrix of the homogeneous transformation matrix from the camera coordinate system to the patient tracker coordinate system , Calculate the starting point and the ending point of the interactor nail-insertion line segment in the patient tracer coordinate system according to the following formula: = ； = ； The starting point and the ending point of the nail - inserting line segment of the interactor in the three - dimensional image space are calculated according to the following formula: Position: = ； = ； Among them, is the inverse matrix of the spatial transformation matrix from the pre-acquired patient tracer coordinate system to the three-dimensional image space coordinate system ; According to the starting point and the end point Based on the real-time positions of the starting point and the end point, a two-dimensional section of the three-dimensional image is realized to be displayed in real-time following the nail placement position of the interactive device; Calculate the homogeneous transformation matrix from the robotic arm base coordinate system to the interactor tracer coordinate system according to the following formula : = ； Among them, is the homogeneous transformation matrix from the camera coordinate system to the end-effector coordinate system of the robotic arm and its inverse matrix; Convert the interactor nail-insertion line segment in the interactor tracer coordinate system to the interactor nail-insertion line segment in the manipulator base coordinate system : = ； = ； In the axial direction of the end effector of the robotic arm, a nail insertion starting point is set at a preset distance offset from the lower edge of the end effector of the robotic arm. , the nail insertion starting point is set as the nail insertion end point at a distance offset equal to the nail length. , and a nail insertion line segment of the end effector of the robotic arm in the end effector coordinate system of the robotic arm is obtained. ; The nail - inserting line segment of the end - effector of the robotic arm in the end - effector coordinate system is converted into the nail - inserting line segment of the end - effector of the robotic arm in the base coordinate system of the robotic arm by the following formula : = ； = ； Take the nail - placing line segment of the interactuator in the base coordinate system of the robotic arm as the target position, and take the nail - placing line segment of the end effector of the robotic arm in the base coordinate system of the robotic arm as the current position. Control the movement of the end effector of the robotic arm until the nail - placing line segment of the end effector of the robotic arm and the nail - placing line segment of the interactuator coincide in the base coordinate system of the robotic arm.

2. A control method for a path planning and servo system for minimally invasive orthopedic nail insertion as described in claim 1, characterized in that, Comprising: Enabling the optical positioning camera to obtain the poses of the interactor tracer, the robotic arm end tracer, and the patient tracer in the camera coordinate system; Enabling the robotic arm master controller to obtain the pose of the end of the robotic arm in the robotic arm base coordinate system; Let the controller calculate the homogeneous transformation matrix from the camera coordinate system to the interactor tracer coordinate system, the homogeneous transformation matrix from the camera coordinate system to the end-effector coordinate system of the robotic arm, and the homogeneous transformation matrix from the camera coordinate system to the patient tracer coordinate system according to the poses of the interactor tracer, the end-effector tracer of the robotic arm, and the patient tracer in the camera coordinate system obtained by the optical positioning camera. , the homogeneous transformation matrix from the camera coordinate system to the end-effector coordinate system of the robotic arm and the homogeneous transformation matrix from the camera coordinate system to the patient tracer coordinate system ; Let the controller calculate the homogeneous transformation matrix from the robot base coordinate system to the robot end coordinate system according to the pose of the current robot end in the robot base coordinate system obtained by the robot master controller. ; Let the controller take the end point in the axial direction of the interactor as the nail - setting end point , determine the nail - setting starting point according to the nail - setting length , and obtain the nail - setting line segment of the interactor in the coordinate system of the interactor tracer ; Calculate the homogeneous transformation matrix from the patient tracer coordinate system to the interactor tracer coordinate system according to the following formula : = ； Among them, is the homogeneous transformation matrix from the camera coordinate system to the patient tracker coordinate system is the inverse matrix of Calculate the starting point and the ending point of the interactor's nail-insertion line segment in the patient tracer coordinate system according to the following formula: The positions are: = ； = ； The starting point and the ending point of the nail - inserting line segment of the interactuator in the three - dimensional image space are calculated according to the following formula: Position: = ； = ； Among them, is the inverse matrix of the spatial transformation matrix from the pre-acquired patient tracer coordinate system to the three-dimensional image space coordinate system ; According to the starting point and the ending point of the real-time position, the two-dimensional section of the three-dimensional image is realized to be displayed in real-time following the nail-setting position of the interactive device; Let the controller calculate the homogeneous transformation matrix from the base coordinate system of the robotic arm to the tracer coordinate system of the interactor according to the following formula : = ； Among them, is the inverse matrix of the homogeneous transformation matrix from the camera coordinate system to the end coordinate system of the robotic arm; of the inverse matrix; Convert the interactor nail - placing line segment in the interactor tracer coordinate system into the interactor nail - placing line segment in the manipulator base coordinate system : = ； = ； In the axial direction of the end effector of the robotic arm, set the starting point of screw insertion at a preset distance offset from the lower edge of the end effector of the robotic arm. , set the starting point of screw insertion and set the ending point of screw insertion at a distance offset equal to the screw length to obtain the screw insertion line segment of the end effector of the robotic arm in the end effector coordinate system of the robotic arm. ; Convert the nail-insertion line segment of the manipulator end effector in the end effector coordinate system to the nail-insertion line segment of the manipulator end effector in the manipulator base coordinate system by the following formula :​ = ； = ； Take the nail - placing line segment of the interactuator in the base coordinate system of the robotic arm as the target position, and take the nail - placing line segment of the end effector of the robotic arm in the base coordinate system of the robotic arm as the current position. Control the movement of the end effector of the robotic arm until the nail - placing line segment of the end effector of the robotic arm and the nail - placing line segment of the interactuator coincide in the base coordinate system of the robotic arm.

Images