Control Method for Master-Slave Movement, Robot System, Device and Storage Medium

Abstract

The present disclosure relates to the field of robots, and discloses a control method for master-slave motion, a robot system, a device, and a storage medium. The control method for master-slave motion includes: determining the current pose of the slave tool; determining the target pose of the handle of the master manipulator based on the current pose of the slave tool; and generating a control signal for the master manipulator based on the target pose of the handle of the master manipulator. It is possible to achieve pose matching between the slave tool and the handle of the master manipulator, and improve the accuracy of master-slave motion control.

Description

Technical Field

[0001] The present disclosure relates to the field of robots, and particularly to a control method for master-slave motion, a robot system, a device, and a storage medium. Background Art

[0002] With the development of technology, the use of medical robots to assist medical staff in performing surgeries has developed rapidly. Medical robots can not only help medical staff with a series of medical diagnoses and assisted treatments, but also effectively alleviate the shortage of medical resources.

[0003] Generally, a medical robot includes a slave tool for performing operations and a master manipulator for controlling the movement of the slave tool. In an actual scenario, the slave tool is configured to be able to enter the operation area, and the medical staff controls the movement of the slave tool in the operation area by remotely operating the master manipulator to perform medical operations.

[0004] However, the number of slave tools that are generally remotely operated is greater than the number of master manipulators. Therefore, in surgeries, there may be a situation where the slave tool controlled by the master manipulator is changed. Moreover, at the beginning or during the operation, the master manipulator needs to establish a mapping with the slave tool first and then perform master-slave control. Since the master manipulator and the corresponding slave tool are not pre-matched in terms of posture, there may be a mismatch in the posture (such as orientation or angle) between the master manipulator and the slave tool. If the two are directly matched for master-slave mapping, the control accuracy of the slave tool will be reduced, deteriorating the human-machine interaction experience of medical staff (such as surgeons). Therefore, before remote operation after the master manipulator and the slave tool are matched and connected, the posture of the master manipulator needs to be correspondingly matched with the posture of the slave tool to improve the accuracy of the master manipulator's posture control of the slave tool. Summary of the Invention

[0005] In some embodiments, the present disclosure provides a control method for master-slave motion, including: determining the current posture of the slave tool; determining the target posture of the handle of the master manipulator based on the current posture of the slave tool; and generating a control signal for the master manipulator based on the target posture of the handle of the master manipulator.

[0006] In some embodiments, the present disclosure provides a robotic system, including: a master manipulator, including a robotic arm, a handle disposed on the robotic arm, and at least one motor and at least one master manipulator sensor disposed at at least one joint of the robotic arm, where the at least one master manipulator sensor is configured to obtain joint information of the at least one joint; a slave tool, including a flexible arm body and an end instrument disposed at the end of the flexible arm body; a driving device configured to drive the flexible arm body of the slave tool, the driving device including at least one driving device sensor configured to obtain driving information; and a control device communicatively connected to the master manipulator and the driving device, the control device being configured to execute the master-slave motion control method provided in any one of the above embodiments.

[0007] In some embodiments, the present disclosure provides a computer device, including: a memory configured to store at least one instruction; and a processor coupled to the memory and configured to execute the at least one instruction to execute the master-slave motion control method provided in any one of the above embodiments.

[0008] In some embodiments, the present disclosure provides a computer-readable storage medium configured to store at least one instruction, where when the at least one instruction is executed by a computer, the robotic system is caused to implement the master-slave motion control method provided in any one of the above embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] To more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for the description of the embodiments of the present disclosure. The drawings in the following description only show some embodiments of the present disclosure. For those of ordinary skill in the art, other embodiments can be obtained based on the content of the embodiments of the present disclosure and these drawings without creative efforts.

[0010] Figure 1 A flowchart showing a master-slave motion control method according to some embodiments of the present disclosure;

[0011] Figure 2 A schematic structural diagram of a robotic system according to some embodiments of the present disclosure;

[0012] Figure 3 A schematic diagram showing a master manipulator according to some embodiments of the present disclosure;

[0013] Figure 4 A schematic diagram of a robotic system according to some embodiments of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0014] To make the technical problems solved by the present disclosure, the technical solutions adopted, and the achieved technical effects clearer, the technical solutions of the embodiments of the present disclosure will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only exemplary embodiments of the present disclosure, rather than all embodiments.

[0015] In the description of the present disclosure, it should be noted that the orientation or positional relationship indicated by terms such as "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the accompanying drawings. It is only for the convenience of describing the present disclosure and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present disclosure. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In the description of the present disclosure, it should be noted that unless otherwise clearly specified and limited, the terms "mounted", "connected", "coupled" should be understood in a broad sense. For example, it can be a fixed connection or a detachable connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium; it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present disclosure can be understood according to specific situations. In the present disclosure, the end close to the operator (such as a doctor) is defined as the proximal end, the proximal part, the rear end, or the rear part, and the end close to the surgical patient is defined as the distal end, the distal part, the front end, or the front part. Those skilled in the art can understand that the embodiments of the present disclosure can be used for medical devices or surgical robots, and can also be used for other non-medical devices.

[0016] In the present disclosure, the term "position" refers to the positioning of an object or a part of an object in three-dimensional space (for example, three translational degrees of freedom can be described by changes in Cartesian X, Y, and Z coordinates, such as three translational degrees of freedom along the Cartesian X-axis, Y-axis, and Z-axis respectively). In the present disclosure, the term "orientation" refers to the rotational setting of an object or a part of an object (for example, three rotational degrees of freedom, which can be described by roll, pitch, and yaw). In the present disclosure, the term "pose" refers to the combination of the position and orientation of an object or a part of an object, and can be described by six parameters among the six degrees of freedom mentioned above. In the present disclosure, the pose of the handle of the master manipulator can be represented by a set of joint information of the master manipulator joints (for example, a one-dimensional matrix composed of these joint information). The pose of the slave tool can be determined by the drive information of the slave tool (for example, the drive information of the flexible arm body of the slave tool). In the present disclosure, the joint information of a joint can include the angle rotated by the corresponding joint relative to the corresponding joint axis or the distance moved relative to the initial position.

[0017] Figure 1 FIG. 100 is a flowchart of a control method for master-slave motion according to some embodiments of the present disclosure. Figure 2 FIG. 200 is a schematic structural diagram of a robot system according to some embodiments of the present disclosure. The method 100 can be implemented or executed by hardware, software, or firmware. In some embodiments, the method 100 can be executed by a robot system (e.g., Figure 2 the robot system 200 shown). In some embodiments, the method 100 can be implemented as computer-readable instructions. These instructions can be read and executed by a general-purpose processor or a dedicated processor (e.g., Figure 2 the control device 220 shown). For example, the control device for the robot system 200 can include a processor configured to execute the method 100. In some embodiments, these instructions can be stored on a computer-readable medium.

[0018] In some embodiments, as Figure 2 shown, the robot system 200 can include a master control cart 210, a surgical cart 230, and a control device 220. The control device 220 can be communicatively connected to the master control cart 210 and the surgical cart 230, for example, through a cable connection or a wireless connection, to enable communication between the master control cart 210 and the surgical cart 230. The master control cart 210 includes a master operator for the operator to remotely operate, and the surgical cart 2,30 includes a slave tool for performing surgery. Through the control device 220, a master-slave mapping between the master operator in the master control cart and the slave tool in the surgical cart is achieved, and the motion control of the slave tool by the master operator is realized. In some embodiments, the surgical cart includes at least one slave tool (such as a surgical tool or a vision tool) provided on the surgical cart. And the slave tool is arranged to be able to enter the operation area through a sheath, where the sheath can be fixed at the surgical opening (such as an incision or a natural opening) of the patient, and the operation area can be the area where the surgery is performed. The slave tool can include an arm body and a distal instrument. The arm body of the slave tool can be a flexible arm body, and the distal instrument can be provided at the distal end of the flexible arm body. The distal instrument of the surgical tool can include, but is not limited to, surgical forceps, an electrosurgical knife, an electrocautery, etc. The distal instrument of the vision tool can include, but is not limited to, an imaging device or a lighting device, etc. In some embodiments, the master control cart includes a master operator, a display, and a foot pedal. Those skilled in the art can understand that the master control cart 210 and the surgical cart 230 can adopt other structures or forms, such as a base, a bracket, or a building, etc.

[0019] In step 101, the current posture of the slave tool can be determined. In some embodiments, the current driving information (such as an angle) of the slave tool is obtained through a driving device sensor, and the current posture of the slave tool is determined based on the current driving information. For example, the current posture of the slave tool can be calculated through a forward kinematics algorithm.

[0020] The driving device sensor can be arranged on the driving device. The driving device is used to drive the flexible arm body of the driven tool. The driving device sensor is used to obtain driving information and determine the current pose of the driven tool according to the driving information. For example, the driving device can include at least one driving motor, and the driving device sensor is coupled with the driving motor to record and output motor data. For example, the motor data can include binary or hexadecimal numbers, and the current pose of the driven tool can be obtained through conversion. The driving device sensor can include a potentiometer or an encoder. Information such as an angle is obtained through the potentiometer or the encoder, and then the current pose of the driven tool is determined.

[0021] In some embodiments, an attitude sensor can be used to obtain the attitude of the driven tool. For example, the attitude sensor can also be an optical fiber sensor, which is arranged through the arm body of the driven tool and is used to sense the position and attitude of the driven tool.

[0022] In some embodiments, the current pose of the driven tool is the current pose of the driven tool relative to the base coordinate system of the driven tool. The driven tool includes a flexible arm body and an end instrument arranged at the end of the flexible arm body. The current pose of the driven tool includes the pose of the end instrument of the driven tool relative to the base coordinate system of the driven tool. The base coordinate system of the driven tool can be the coordinate system of the base on which the driven tool is installed (for example, the end of the moving arm of a surgical robot), the coordinate system of the sheath tube through which the driven tool passes (for example, the coordinate system of the sheath tube outlet), the coordinate system of the remote center of motion (RCM) of the driven tool, etc. For example, the base coordinate system of the driven tool can be set at the sheath tube outlet position, and during teleoperation, the base coordinate system of the driven tool remains fixed. The coordinate transformation of the current pose of the end instrument can be performed to obtain the pose relative to other coordinate systems.

[0023] In some other embodiments, the current pose of the driven tool is the current pose of the image of the driven tool in the display relative to the world coordinate system. The world coordinate system can be the coordinate system of the space where the operator or the master manipulator is located. Therefore, the pose of the image of the driven tool in the display relative to the world coordinate system is the pose perceived by the operator. The driven tool includes a surgical tool and a vision tool. During the surgical process, the surgical tool performs surgery on the patient's body, and the vision tool uses a camera to collect images of the patient's body and transmits the collected images to the surgical cart. The images are processed by a video processing module in the surgical cart and then displayed on the display of the master cart. The operator obtains the current position and pose of the driven tool through the images in the display.

[0024] In some embodiments, the current pose of the image of the slave tool in the display relative to the world coordinate system can be obtained by a sensor. For example, through a drive device sensor or a pose sensor of the slave tool, the pose of the end instrument of the slave tool relative to the base coordinate system of the slave tool can be obtained. In other embodiments, the current pose of the image of the slave tool in the display relative to the world coordinate system can be obtained by coordinate transformation. For example, based on the base coordinate system of the slave tool, the coordinate system of the camera of the vision tool, the base coordinate system of the vision tool, the coordinate system of the display, and the world coordinate system, the current pose of the image of the slave tool in the display relative to the world coordinate system can be obtained.

[0025] In step 103, based on the current pose of the slave tool, the target pose of the handle of the master manipulator can be determined. In some embodiments, the current pose of the slave tool is the current pose relative to the base coordinate system of the slave tool, or the current pose of the slave tool is the current pose of the image of the slave tool in the display relative to the world coordinate system. The target pose of the handle of the master manipulator is the pose relative to the base coordinate system of the master manipulator. The base coordinate system of the master manipulator can be the coordinate system of the base (such as the main control cart 210) to which the master manipulator is connected. In some embodiments, there is a fixed transformation relationship between the base coordinate system of the master manipulator and the base coordinate system of the slave tool.

[0026] In some embodiments, the current pose of the slave tool is the same as the target pose of the handle, for example, the same or having a fixed difference. For example, before teleoperation, keep the current pose of the slave tool unchanged and use the current pose of the slave tool as the target pose of the handle, and adjust the current pose of the handle to the target pose to achieve the pose matching between the handle and the slave tool.

[0027] In step 105, based on the target pose, a control signal for the handle of the master manipulator can be generated. In some embodiments, determine the current pose of the handle of the master manipulator; and based on the target pose and the current pose of the handle of the master manipulator, generate a control signal for the master manipulator. The current pose of the handle of the master manipulator is the pose of the handle of the master manipulator relative to the base coordinate system of the master manipulator. In some embodiments, determine the control signal corresponding to the handle reaching the target pose from the current pose based on the current pose and the target pose of the handle.

[0028] In some embodiments, the master manipulator includes at least one posture joint for controlling the posture of the handle. Determining the current posture of the handle of the master manipulator includes: obtaining joint information of at least one posture joint; and determining the current posture of the master manipulator based on the joint information of at least one posture joint. The master manipulator includes a robotic arm, the robotic arm includes position joints and posture joints. The posture joints serve as the orientation module of the master manipulator, and the master manipulator is controlled to reach the target posture through one or more posture joints. The position joints serve as the positioning module of the master manipulator, and the master manipulator is controlled to reach the target position through one or more position joints. The master manipulator sensor is disposed at the posture joints of the robotic arm for acquiring joint information (such as angles) corresponding to the posture joints, and determining the current posture of the handle of the master manipulator relative to the base coordinate system of the master manipulator according to the acquired joint information. For example, the master manipulator includes 7 joints, where joints 1, 2, 5, 6, and 7 are posture joints for controlling the posture of the handle of the master manipulator. The joint information is acquired by the master manipulator sensors of the posture joints, and the current posture of the master manipulator is calculated based on the forward kinematics algorithm. In some embodiments, the master manipulator includes at least one posture joint for controlling the posture of the handle of the master manipulator, and the control signal includes a control signal for controlling one or more of the at least one posture joint. The posture of the handle of the master manipulator is adjusted by adjusting one or more posture joints, so as to achieve the posture matching between the handle of the master manipulator and the slave tool.

[0029] In some embodiments, the control signal includes a control signal for controlling one or more of the at least one posture joint, and one or more of the at least one posture joint include uncoupled posture joints. A coupled joint may refer to a joint for adjusting the position and posture of the master manipulator. An uncoupled joint may refer to a joint that can only be used to adjust the position (referred to as an uncoupled position joint in the present disclosure) or the posture (referred to as an uncoupled posture joint in the present disclosure) of the master manipulator. In some embodiments, the master manipulator may include at least one coupled joint. For example, the master manipulator may include 7 joints, as Figure 3 shown, where joints 1, 2, and 3 are position joints, joints 1, 2, 5, 6, and 7 are posture joints, joints 1 and 2 are coupled joints that can both adjust the position and the posture of the master manipulator, and joints 5, 6, and 7 are uncoupled posture joints that can only adjust the posture of the master manipulator. In some embodiments, the posture of the handle of the master manipulator can be adjusted by calculating the control signals of the uncoupled posture joints (such as joints 5, 6, and 7), so as to achieve the posture matching between the handle of the master manipulator and the slave tool, providing conditions for subsequent teleoperation.

[0030] such asFigure 3 As shown Figure 3 FIG. 300 shows a schematic diagram of a master manipulator according to some embodiments of the present disclosure. In Figure 3 it, the master manipulator includes 7 joints (numbered i = 1...7), the base coordinate system of the master manipulator is b, and the coordinate system of the handle is d. In Figure 3 it, the base coordinate system b is a coordinate system established with the base virtual as a point, and its direction can be determined based on its physical structure. Similarly, the coordinate system d of the handle is a coordinate system established with the handle virtual as a point, and its direction can be determined based on its physical structure. The origin of the coordinate system d of the handle can coincide with the origin of the coordinate systems of joints 5, 6, and 7, and the position and orientation of the coordinate system d of the handle relative to the base coordinate system of the master manipulator can be determined by the joint information of joints 1-7.

[0031] In some embodiments, the master manipulator sensor acquires the joint information q i (i is the number of the joint). In some embodiments, each joint information q i may include the angular value θ i of the joint. For example, the joint information q1 of joint 1, the joint information q2 of joint 2, the joint information q3 of joint 3, the joint information q4 of joint 4, the joint information q5 of joint 5, the joint information q6 of joint 6, and the joint information q7 of joint 7 are acquired. Joint 4 is a slave joint of joint 3, and the joint angle of joint 4 has the same absolute value as that of joint 3 but in the opposite direction. Therefore, the angles of the six joints of the master manipulator are represented as a 6*1 matrix q, and the joint angle of joint 4 may not be reflected in the matrix q. Each joint information q i can be represented as θ i , and the structure of the master manipulator has six degrees of freedom, as shown in formula (1):

[0032] q = (q1 q2 q3 q5 q6 q7) T (1)

[0033] Joints 1, 2, and 3 are position joints, and q1, q2, and q3 determine the position of the handle of the master manipulator. Joints 1, 2, 5, 6, and 7 are attitude joints, and q1, q2, q5, q6, and q7 determine the attitude of the handle. In some embodiments, to determine the attitude of the handle of the master manipulator, one may not care about the position controlled by joints 1, 2, and 3, but care about the attitude (such as direction) determined by joints 1, 2, 5, 6, and 7. In some embodiments, when driven by a motor, joints 1, 2, and 3 are kept stationary, q5, q6, and q7 corresponding to joints 5, 6, and 7 are determined, and control signals are calculated based on q5, q6, and q7 to achieve attitude adjustment of the handle.

[0034] Those skilled in the art can understand that there can be many solutions for the multi-joint master manipulator to reach the target posture. In some embodiments, one or more of the posture joints in at least one posture joint can be adjusted to adjust the posture of the master manipulator handle. For example, in one embodiment, the coupled posture joint 1, the coupled posture joint 2, and the uncoupled position joint 3 can be kept unchanged, and the posture of the master manipulator handle can be adjusted by adjusting the uncoupled posture joints 5, 6, and 7.

[0035] In some embodiments, the joint information of other posture joints except one or more posture joints in at least one posture joint is obtained. Based on the joint information of other posture joints, the transformation matrix of other posture joints can be determined. For example, based on the master manipulator sensor, the joint information of other posture joints is obtained, and the transformation matrix of other posture joints is determined based on the joint information of other posture joints. As Figure 3 shown, the joint information of the coupled posture joints 1 and 2 can be obtained, and the transformation matrix is calculated.

[0036] In some embodiments, the uncoupled posture joints (e.g., joint 5, joint 6, and joint 7) in one or more posture joints can be adjusted without adjusting other posture joints, such as coupled joints (e.g., joint 1 and joint 2). Based on q1 and q2 corresponding to other posture joints (e.g., joint 1 and joint 2), the transformation matrix of other posture joints (e.g., the transformation matrix of other posture joints relative to the joint starting point 0 0 R4) is determined. Based on the target posture of the handle of the master manipulator and the transformation matrix 0 R4 of other posture joints, the control signal of the master manipulator is generated, as shown in formulas (2) to (4).

[0037] Those skilled in the art can understand that joints 3 and 4 are uncoupled position joints, and the transformation matrix 0 R4 of other posture joints determined based on q1, q2, and q3 0 is consistent with the transformation matrix

[0038] 4 R7 = 0 R4 T · b R0 T · b R d · 7 R d T (2)

[0039] In formula (2), the transformation matrix 0R4 is determined by the inputs q1, q2 or q1, q2, q3. b is the base coordinate system of the master manipulator, and d is the coordinate system of the handle of the master manipulator. b R d is the attitude of the handle of the master manipulator relative to the base coordinate system of the master manipulator. b R0 is the existing angular relationship between the base and the starting point of the joint, which is a structural constant. 7 R d is the existing angular relationship between joint 7 and the handle, which is a structural constant.

[0040] 4 R7 = 4 R5 · 5 R6 · 6 R7(3)

[0041] R(q5, q6, q7) = 0 R4 T · b R0 T ·R t · 7 R d T (4)

[0042] In formula (4), R t is the current attitude of the slave tool and is the same as b R d Same. 4 R5, 5 R6 and 6 R7 respectively correspond to the quantities to be solved q5, q6, q7. Based on the obtained q5, q6, q, determine the control signal, and based on the control signal, adjust the attitude of the master manipulator to achieve the matching of the master-slave attitudes. Those skilled in the art can understand that R t can be the current attitude of the end instrument of the slave tool relative to the base coordinate system of the slave tool, or the current attitude of the image of the end instrument of the slave tool in the display relative to the world coordinate system. R t can be consistent with b R d For example, the same or having a specific ratio or difference. In some embodiments, according to the control signal, determine the joint target values of one or more attitude joints in the handle, and convert the joint target values into driving amounts and send them to the driving device. The driving device drives the motors of one or more attitude joints of the master manipulator to make one or more attitude joints of the master manipulator move, so as to achieve the matching of the attitude of the handle of the master manipulator and the attitude of the end instrument of the slave tool.

[0043] In some embodiments, the mathematical structure model of the master manipulator can be constructed based on the DH parameter method or the exponential product representation method. For example, determine the DH matrix corresponding to the joints of the master manipulator, and determine the mathematical structure model of the master manipulator based on the DH matrix of the joints. The DH matrix of each joint of the master manipulator is expressed as formula (5).

[0044]

[0045] The corresponding relationship between the DH matrix and q is shown in Table 1.

[0046] Table 1 Corresponding relationship between the DH matrix and q

[0047] <![CDATA 0 T1]]> <![CDATA 1 T2]]> <![CDATA 2 T3]]> <![CDATA 3 T4]]> <![CDATA 4 T5]]> <![CDATA 5 T6]]> <![CDATA 6 T7]]> <![CDATA[q1]]> <![CDATA[q2]]> <![CDATA[q3]]> <![CDATA[q4 (q4 = -q3)]]> <![CDATA[q5]]> <![CDATA[q6]]> <![CDATA[q7]]> Joint 1 Joint 2 Joint 3 Joint 4 (driven joint of Joint 3) Joint 5 Joint 6 Joint 7

[0048] In formula (5), j is the joint number, Rot(x,α j ) is a rotation of α j degrees about the x-axis, Rot(z,θ j ) is a rotation of θ degrees about the z-axis, Trans(x,α j ) moves α j in the x-direction, Trans(z,d j ) moves d j in the z-direction, Rot(x,α j ), Trans(x,α j ) etc. are all 4*4 matrices. As Figure 3 shown in the structure of the multi-joint master manipulator, the z-axis is the rotation axis of the joint, and the x-axis points to the next joint. The y-axis direction can be determined according to the left / right hand rule of the Cartesian coordinate system. Rot(x,α j ), Trans(x,α j ) The fourth-order matrix represents a rotation of a certain angle in a certain direction or a translation of a certain distance in a certain direction.

[0049] The mathematical structure model of the master manipulator is described by multiplying the DH matrices of all joints, as shown in formula (6):

[0050] 0 T7 = 0 T1· 1 T2· 2 T3· 3 T4· 4 T5· 5 T6· 6 T7 (6)

[0051] In formula (6), T can be understood as a matrix with q as the main variable, which represents different parts of the mathematical model according to the identification of the subscripts. The upper left 3*3 part of the matrix T is the rotation matrix R.

[0052] The master manipulator includes an arm body and a handle, and the arm body includes joints and linkages. The operator controls the position and posture of the slave tool by remotely operating the handle of the master manipulator. It can be understood that when starting the remote operation, if the posture of the handle (such as the orientation or angle) is inconsistent with the posture of the corresponding controlled slave tool (such as the orientation or angle), it will result in a poor human-machine interaction experience for the operator (such as a surgeon) during the operation process, affecting the operation accuracy of the slave tool. Therefore, after the master manipulator is matched and connected to the slave tool and before the master manipulator remotely operates the slave tool (such as when the operator holds the handle of the master manipulator to obtain the control right of the corresponding slave tool but has not started the master-slave remote operation), the posture of the handle is matched and adjusted with the posture of the slave tool. When the postures of the two are consistent, the remote operation of the slave tool by the master manipulator can be executed, which can improve the accuracy and experience of the subsequent remote operation.

[0053] In some embodiments, in response to a predetermined condition being satisfied, the posture matching degree between the master manipulator and the slave tool can be determined. The predetermined condition includes the triggering of the remote operation control right. In some embodiments, the triggering of the remote operation control right can be achieved through a triggering device. The triggering device can be a switch arranged on the master manipulator or the display for the operator to easily touch, press, or swipe. The triggering methods include but are not limited to, keeping close, touching, swiping, clicking, or long-pressing, etc. The triggering method of the triggering device can be toggling the switch on the master manipulator, touching the sensing position on the master manipulator, long-pressing or clicking the button on the master manipulator, stepping on the foot pedal of the main console, operating the display screen of the main console, etc.

[0054] Matching means that the posture of the handle and the posture of the slave tool satisfy a preset relationship (for example, being consistent), and the posture matching degree refers to the degree of matching between the current posture of the handle and the current posture of the slave tool. In some embodiments, in response to a predetermined condition being satisfied, the joint information of the master manipulator and the slave tool is obtained through sensors, the current postures of the handle and the slave tool are determined through the forward kinematics algorithm, and based on the current posture of the handle of the master manipulator and the current posture of the slave tool, the posture matching degree between the master manipulator and the slave tool is determined. When the posture matching degree is lower than the preset threshold, in response to the posture matching degree being lower than the preset threshold, a control signal for adjusting the current posture of the handle of the master manipulator is generated to make the posture matching degree higher than or equal to the preset threshold. In this way, when the postures of the two do not match, the posture can be automatically adjusted to achieve the consistency of the two postures. When the current postures of the two are consistent or basically consistent (the posture matching degree is higher than or equal to the preset threshold), in response to the posture matching degree being higher than or equal to the preset threshold, a master-slave mapping between the master manipulator and the slave tool is established, and thus the next remote operation process can be executed.

[0055] In some embodiments, the method for adjusting the posture of the handle of the master manipulator to be consistent with the posture of the slave tool includes: keeping the current posture of the slave tool unchanged, and adjusting the posture of the handle of the master manipulator so that the posture of the handle of the master manipulator is consistent with the posture of the slave tool.

[0056] The target posture of the handle of the master-slave master manipulator is consistent with the current posture of the slave tool. By establishing a master-slave mapping between the master manipulator and the slave tool, teleoperation of the master manipulator on the slave tool can be performed, improving the operation accuracy of the teleoperation and the experience of the teleoperation. Those skilled in the art can understand that postural consistency means basically consistent postures. There may be a certain error between the target posture of the handle of the master manipulator and the current posture of the slave tool, but the range of the error is within an acceptable range.

[0057] In the above embodiments, the postures of the handle and the slave tool are matched before teleoperation. When the operator starts to operate (such as pressing the clamp button of the handle of the master manipulator), the teleoperation can be quickly established. In addition, only the current posture of the slave tool is maintained, and the operator can still move the position of the handle of the master manipulator in the non-operating state and move it to a suitable position before performing the teleoperation matching, greatly increasing the movement space of the handle of the master manipulator. Moreover, the provided master-slave motion control method can be applied to various slave ends with different principles and forms, and the calculation process is highly targeted and has a small amount of calculation, which also reduces the driving amount when adjusting the handle of the master manipulator to the target posture.

[0058] In the above embodiments, by establishing a connection between the master manipulator and the slave tool and realizing the transfer of control rights, the posture matching degree between the handle of the master manipulator and the slave tool is determined in the state of connection and transfer of control rights. If the posture matching degree meets the preset threshold condition, a master-slave mapping between the master manipulator and the slave tool is established, and the teleoperation step is executed. If the posture matching degree does not meet the preset threshold condition, at this time, the posture of the handle of the master manipulator needs to be adjusted to be consistent with the current posture of the slave tool, and then a master-slave mapping between the master manipulator and the slave tool is established, and the teleoperation of the handle of the master manipulator is executed. Before establishing a teleoperation relationship between the master manipulator and the slave tool, the posture of the handle of the master manipulator is adjusted to be consistent with the posture of the slave tool in time, realizing the accuracy of the master-slave mapping between the handle of the master manipulator and the slave tool, improving the operation experience of the operator during teleoperation, realizing high-precision matching of operation actions and actual actions, and at the same time avoiding operation restrictions caused by inconsistent motion control boundaries between the master manipulator and the slave tool.

[0059] Figure 4 FIG. 400 shows a schematic diagram of a robot system according to some embodiments of the present disclosure. As Figure 4As shown, the robot system 400 includes: a master manipulator 410, a control device, a driving device, and a slave tool. The master manipulator 410 includes a robotic arm, a handle disposed on the robotic arm, and at least one master manipulator sensor disposed at at least one joint of the robotic arm. The at least one master manipulator sensor is configured to obtain joint information of the at least one joint. The slave tool 420 includes a flexible arm body and an end instrument. The driving device 430 is configured to drive the flexible arm body of the slave tool and includes at least one driving device sensor for obtaining driving information. The control device 440 is communicatively connected to the master manipulator 410 and the driving device 430. The control device 440 is configured to execute a master-slave motion control method according to some embodiments of the present disclosure.

[0060] In some embodiments, the master manipulator 410 includes a six-degree-of-freedom robotic arm, and a master manipulator sensor is disposed at each joint of the six-degree-of-freedom robotic arm to generate joint information (such as joint angle data) through the master manipulator sensor of each joint. In some embodiments, the master manipulator sensor employs a potentiometer and / or an encoder.

[0061] In some embodiments, the slave tool 420 includes a multi-link six-degree-of-freedom flexible arm body.

[0062] In some embodiments, the driving device 430 is configured to drive the flexible arm body of the slave tool 420 and obtain the corresponding driving information of the slave tool through the driving device sensor.

[0063] In some embodiments, the control device 440 is communicatively connected to the master manipulator 410 and the driving device 430. For example, the master manipulator 410, the driving device 430, and the control device 440 may be connected through a data transmission bus, including but not limited to wireless data transmission, wired data connection, or a combination of multiple data communication methods. The data transmission bus may employ a Controller Area Network (CAN) bus.

[0064] The control device 440 is configured to execute a master-slave motion control method in some embodiments of the present disclosure. For example, the control device is configured to receive network data packets (such as joint information) sent by the master manipulator sensor and the driving device sensor. The control device calculates the joint target value for the handle of the master manipulator to reach a target pose consistent with the current pose of the slave tool based on the joint information of the slave tool and the joint information of the master manipulator, and converts it into a driving signal and sends it to the driving device 430. The driving device 430 receives the driving signal through the network data packet, for example, sends it to each Epos control tool through the CAN bus, and drives the respective motors of the master manipulator to move the master manipulator in place, thereby achieving the pose matching between the handle of the master manipulator and the slave tool.

[0065] In some embodiments, a controller may be provided in the master manipulator. The controller may calculate the pose data of the master manipulator based on the joint information obtained by each master manipulator sensor, and send the calculated pose data to the control device. In some other embodiments, the control device may also calculate the pose data of the master manipulator based on the joint data sent by the master manipulator sensor.

[0066] In the above embodiments, when changing the controlled object of the master manipulator (for example, the slave tool), it is very likely that the front-end orientation of the slave tool entering the abdomen is different from the current orientation of the handle of the master manipulator. The method provided by the present disclosure can adjust the pose of the handle of the master manipulator to be consistent with the current pose of the slave tool before establishing the master-slave mapping relationship between the master manipulator and the slave tool and before the operator actually operates, so as to achieve a good operation experience for the operator and a high-precision match between the action expectation and the reality, and at the same time avoid the operation limitation caused by the inconsistent motion control boundaries between the master manipulator and the slave tool.

[0067] The present disclosure also discloses the following:

[0068] 1. A control method for master-slave motion, including:

[0069] Determine the current pose of the slave tool;

[0070] Based on the current pose of the slave tool, determine the target pose of the handle of the master manipulator; and

[0071] Generate a control signal for the master manipulator based on the target pose of the handle of the master manipulator.

[0072] 2. The control method according to item 1, further including:

[0073] Determine the current pose of the handle of the master manipulator; and

[0074] Generate a control signal for the master manipulator based on the target pose and the current pose of the handle of the master manipulator.

[0075] 3. The control method according to item 2, the master manipulator includes at least one pose joint for controlling the pose of the handle, and determining the current pose of the handle of the master manipulator includes:

[0076] Obtain the joint information of the at least one pose joint; and

[0077] Based on the joint information of the at least one pose joint, determine the current pose of the master manipulator.

[0078] 4. The control method according to any one of Items 1-3, wherein the slave tool includes a flexible arm body and an end instrument provided at the end of the flexible arm body. Determining the current pose of the slave tool includes:

[0079] Determining the current pose of the end instrument of the slave tool relative to the base coordinate system of the slave tool; or

[0080] Determining the current pose of the image of the end instrument of the slave tool in the display relative to the world coordinate system.

[0081] 5. The control method according to any one of Items 1-4, wherein the target pose of the handle of the master manipulator is the pose relative to the base coordinate system of the master manipulator.

[0082] 6. The control method according to any one of Items 1-5, wherein the master manipulator includes at least one pose joint for controlling the pose of the handle of the master manipulator, and the control signal includes a control signal for controlling one or more of the at least one pose joint.

[0083] 7. The control method according to Item 6, wherein one or more of the at least one pose joint include non-coupled pose joints.

[0084] 8. The control method according to any one of Items 6-7, further comprising:

[0085] Obtaining joint information of other pose joints of the at least one pose joint except the one or more pose joints; and

[0086] Determining a transformation matrix of the other pose joints based on the joint information of the other pose joints.

[0087] 9. The control method according to Item 8, further comprising:

[0088] Generating the control signal of the master manipulator based on the target pose of the handle of the master manipulator and the transformation matrix of the other pose joints.

[0089] 10. The control method according to any one of Items 1-9, further comprising:

[0090] In response to a predetermined condition being satisfied, determining the pose matching degree between the handle of the master manipulator and the slave tool, the predetermined condition including the triggering of the teleoperation control right.

[0091] 11. The control method according to Item 10, further comprising:

[0092] Determine the pose matching degree between the handle of the master manipulator and the slave tool based on the current pose of the handle of the master manipulator and the current pose of the slave tool.

[0093] 12. The control method according to any one of claims 10-11 further includes:

[0094] In response to the pose matching degree being lower than a preset threshold, generate the control signal of the handle of the master manipulator so that the pose matching degree is higher than or equal to the preset threshold.

[0095] 13. The control method according to any one of claims 10-12 further includes:

[0096] In response to the pose matching degree being higher than or equal to the preset threshold, establish a master-slave mapping between the master manipulator and the slave tool.

[0097] 14. In the control method according to any one of claims 1-13, the target pose of the handle of the master manipulator is consistent with the current pose of the slave tool.

[0098] 15. A robot system includes:

[0099] A master manipulator, including a robotic arm, a handle disposed on the robotic arm, and at least one motor and at least one master manipulator sensor disposed at at least one joint of the robotic arm, the at least one master manipulator sensor being configured to obtain joint information of the at least one joint;

[0100] A slave tool, including a flexible arm body and an end instrument disposed at the end of the flexible arm body;

[0101] A driving device for driving the flexible arm body of the slave tool, the driving device including at least one driving device sensor for obtaining driving information; and

[0102] A control device communicatively connected to the master manipulator and the driving device, the control device being configured to execute the master-slave motion control method according to any one of claims 1-14.

[0103] 16. A computer device, the computer device includes:

[0104] A memory for storing at least one instruction; and

[0105] A processor coupled to the memory and configured to execute the at least one instruction to execute the master-slave motion control method according to any one of claims 1-14.

[0106] 17. A computer-readable storage medium for storing at least one instruction, which, when executed by a computer, causes a robot system to implement the master-slave motion control method according to any one of items 1-14.

[0107] Note that the above are only exemplary embodiments of the present disclosure and the applied technical principles. Those skilled in the art will understand that the present disclosure is not limited to the specific embodiments here, and various obvious changes, re-adjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present disclosure. Therefore, although the present disclosure has been described in more detail through the above embodiments, the present disclosure is not limited to the above embodiments only. Without departing from the concept of the present disclosure, more other equivalent embodiments can be included, and the scope of the present disclosure is determined by the scope of the appended claims.

Claims

1. A method for controlling a master-slave motion, comprising: Controlling the robot system to enter a non-teleoperation state, so as to allow the operator to adjust the motion space of the handle of the main manipulator of the robot system by moving the position of the handle of the main manipulator in the non-teleoperation state; In response to the triggering of the teleoperation control right, determining a current posture of a driven tool of the robotic system; determining a target posture of the handle of the master manipulator based on the current posture of the slave tool, the target posture of the handle of the master manipulator being consistent with the current posture of the slave tool; and generating a control signal for the master manipulator based on the target posture of the handle of the master manipulator, so that the handle of the main manipulator moves to the target posture; as well as In response to a start operation, a teleoperation relationship is established between the master operator and the slave tool to control the movement of the slave tool by teleoperating the master operator.

2. The control method according to claim 1, characterized in that: Also includes: Determining a current posture of a handle of the master manipulator; as well as A control signal of the master manipulator is generated based on a target posture and a current posture of a handle of the master manipulator.

3. The control method according to claim 2, characterized in that: The main manipulator includes at least one posture joint for controlling the posture of the handle, and determining the current posture of the handle of the main manipulator includes: obtaining joint information of the at least one posture joint; and The current posture of the master manipulator is determined based on the joint information of the at least one posture joint.

4. The control method according to claim 1, wherein: The driven tool includes a flexible arm and an end tool disposed at an end of the flexible arm, and determining the current posture of the driven tool includes: Determining a current posture of the end tool of the driven tool relative to a base coordinate system of the driven tool; or A current posture of the image of the end tool of the driven tool on the display relative to the world coordinate system is determined.

5. The control method according to claim 1, characterized in that: The target posture of the handle of the master manipulator is a posture relative to the base coordinate system of the master manipulator.

6. The control method according to claim 1, characterized in that: The master manipulator includes at least one posture joint for controlling the posture of a handle of the master manipulator, and the control signal includes a control signal for controlling one or more posture joints of the at least one posture joint.

7. The control method according to claim 6, characterized in that: The one or more posture joints of the at least one posture joint include uncoupled posture joints.

8. The control method according to claim 6, characterized in that: Also includes: Obtaining joint information of other posture joints in the at least one posture joint except the one or more posture joints; as well as Based on the joint information of the other posture joints, a transformation matrix of the other posture joints is determined.

9. The control method according to claim 8, characterized in that: Also includes: The control signal of the master manipulator is generated based on the target posture of the handle of the master manipulator and the transformation matrix of the other posture joints.

10. The control method according to claim 1, characterized in that: Also includes: In response to the triggering of the teleoperation control right, a posture matching degree between the handle of the master manipulator and the slave tool is determined.

11. The control method according to claim 10, characterized in that: Also includes: Based on the current posture of the handle of the master manipulator and the current posture of the slave tool, a posture matching degree between the handle of the master manipulator and the slave tool is determined.

12. The control method according to claim 10, characterized in that: Also includes: In response to the posture matching degree being lower than a preset threshold, the control signal of the handle of the master operator is generated so that the posture matching degree is higher than or equal to the preset threshold.

13. The control method according to claim 10, characterized in that: Also includes: In response to the posture matching degree being higher than or equal to a preset threshold, a master-slave mapping is established between the master manipulator and the slave tool.

14. A robotic system comprising: A main manipulator, comprising a robotic arm, a handle provided on the robotic arm, at least one motor provided at at least one joint of the robotic arm, and at least one main manipulator sensor, wherein the at least one main manipulator sensor is used to obtain joint information of the at least one joint; The driven tool comprises a flexible arm and an end instrument arranged at the end of the flexible arm; a driving device for driving the flexible arm of the driven tool, the driving device comprising at least one driving device sensor for obtaining driving information; as well as A control device is communicatively connected to the master operator and the drive device, and is configured to execute the master-slave motion control method according to any one of claims 1 to 13.

15. A computer device, comprising: A memory for storing at least one instruction; as well as A processor is coupled to the memory and configured to execute the at least one instruction to perform the master-slave motion control method according to any one of claims 1 to 13. 16 . A computer-readable storage medium for storing at least one instruction, wherein when the at least one instruction is executed by a computer, the robot system implements the master-slave motion control method according to claim 1 .

Images