Master-slave motion control method, robot system, device, and storage medium

Abstract

The present disclosure relates to the field of robots, and discloses a master-slave motion control method, a robot system, a device and a storage medium. The master-slave motion control method comprises: determining a current pose of a master manipulator, the current pose comprising a current position and a current attitude; determining a target pose of a slave tool based on the current pose of the master manipulator and a pose relationship between the master manipulator and the slave tool; and generating a control signal of the slave tool based on the target pose of the slave tool. The master manipulator can realize remote operation on the slave tool, and the control accuracy of the master manipulator on the remote operation of the slave tool is improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of robots, and in particular to a master-slave motion control method, a robot system, a device and a storage medium. BACKGROUND

[0002] With the development of technology, the operation of medical staff assisted by medical robots has been rapidly developed. Medical robots can not only help medical staff to perform a series of medical diagnosis and auxiliary treatment, but also effectively alleviate the problem of tight medical resources.

[0003] Generally, a medical robot includes a slave tool for performing an operation and a master manipulator for controlling the motion of the slave tool. In actual scenarios, the slave tool is arranged to be able to enter an operation area, and a medical staff controls the motion of the slave tool in the operation area by remotely operating the master manipulator, thereby realizing medical operation. Generally, a medical robot realizes the motion control of the master manipulator on the slave tool through the motion conversion between the master manipulator and the slave tool.

[0004] The operation precision and human-computer interaction experience of a surgical robot are very high. For example, the surgical robot can acquire an image of the slave tool through a camera and display the image through a display. An operator (for example, a surgical doctor) obtains the position and attitude of the slave tool through the image in the display and controls the position and attitude of the slave tool through the master manipulator by remote operation. In this way, the position and attitude of the camera and the display will affect the operation direction and distance of the operator in the next step. Therefore, it is necessary to improve the precision of the master-slave motion control of the surgical robot to realize the consistency between the target result of the master-slave motion control between the master manipulator and the slave tool and the sensory intention of the operator. SUMMARY

[0005] In some embodiments, the present disclosure provides a master-slave motion control method, comprising: determining a current pose of a master manipulator, the current pose comprising a current position and a current attitude; determining a target pose of a slave tool based on the current pose of the master manipulator and a pose relationship between the master manipulator and the slave tool; and generating a control signal of the slave tool based on the target pose of the slave tool.

[0006] In some embodiments, the present disclosure provides a robot system, comprising: a master operator comprising a mechanical arm, a handle disposed on the mechanical arm, and at least one master operator sensor disposed at at least one joint of the mechanical arm, the at least one master operator sensor being configured to obtain joint information of the at least one joint; a slave tool comprising a flexible arm body and an end instrument; at least one driving device configured to drive the flexible arm body of the slave tool; at least one driving device sensor coupled with the at least one driving device and configured to obtain driving information; and a control device communicatively connected with the master operator and the at least one driving device, the control device being configured to perform the master-slave motion control method provided in any one of the above embodiments.

[0007] In some embodiments, the present disclosure provides a computer device, comprising: a memory configured to store at least one instruction; and a processor coupled with the memory and configured to execute the at least one instruction to perform the master-slave motion control method provided in any one of the above embodiments.

[0008] In some embodiments, the present disclosure provides a computer-readable storage medium configured to store at least one instruction, the at least one instruction being executed by a computer to cause the robot system to implement the master-slave motion control method provided in any one of the above embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings needed to be used in the description of the embodiments of the present disclosure will be briefly introduced as follows. The drawings in the following description only show some embodiments of the present disclosure, and other embodiments can be obtained by those skilled in the art without creative labor on the basis of the contents of the embodiments of the present disclosure and these drawings.

[0010] Figure 1 A flow chart of a master-slave motion control method according to some embodiments of the present disclosure is shown;

[0011] Figure 2 A structural schematic diagram of a robot system according to some embodiments of the present disclosure is shown;

[0012] Figure 3 A coordinate system schematic diagram in master-slave motion mapping according to some embodiments of the present disclosure is shown;

[0013] Figure 4 A schematic diagram of a robot system according to some embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0014] To make the technical problems solved by this disclosure, the technical solutions adopted, and the technical effects achieved clearer, the technical solutions of the embodiments of this disclosure will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely exemplary embodiments of this disclosure, and not all embodiments.

[0015] In the description of this disclosure, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing this disclosure and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this disclosure. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. In the description of this disclosure, it should be noted that unless otherwise expressly specified and limited, the terms "installed," "connected," "coupled," and "coupled" should be interpreted broadly. For example, they can refer to fixed connections or detachable connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal connections between two components. Those skilled in the art can understand the specific meaning of the above terms in this disclosure according to the specific circumstances. In this disclosure, the end closer to the operator (e.g., a doctor) is defined as the proximal end, proximal or rear end, or posterior end, and the end closer to the surgical patient is defined as the distal end, distal or anterior end, or anterior end. Those skilled in the art will understand that embodiments of this disclosure can be used in medical devices or surgical robots, as well as in other non-medical devices.

[0016] In this disclosure, the term "position" refers to the location of an object or part of an object in three-dimensional space (e.g., three translational degrees of freedom can be described using variations in Cartesian X, Y, and Z coordinates, such as three translational degrees of freedom along the Cartesian X, Y, and Z axes, respectively). In this disclosure, the term "attitude" refers to the rotational setting of an object or part of an object (e.g., three rotational degrees of freedom, which can be described using roll, pitch, and yaw). In this disclosure, the term "pose" refers to a combination of the position and attitude of an object or part of an object, which can be described, for example, using six parameters from the six degrees of freedom mentioned above. In this disclosure, the pose of the master manipulator's handle can be represented by a set of joint information of the master manipulator joints (e.g., a one-dimensional matrix composed of this joint information). The pose of the slave tool can be determined by the drive information of the slave tool (e.g., drive information of the flexible arm of the slave tool). In this disclosure, the joint information of a joint can include the angle of rotation of the respective joint relative to the respective joint axis or the distance moved relative to the initial position.

[0017] Figure 1 A flowchart 100 showing a master-slave motion control method according to some embodiments of the present disclosure is provided. Figure 2 A schematic diagram 200 of a robot system according to some embodiments of the present disclosure is shown. Method 100 may be implemented or executed by hardware, software, or firmware. In some embodiments, method 100 may be implemented by a robot system (e.g., Figure 2 The robot system 200 shown is executed. In some embodiments, method 100 can be implemented as computer-readable instructions. These instructions can be executed by a general-purpose processor or a special-purpose processor (e.g., [missing information]). Figure 2 The control device 220 shown reads and executes the instructions. For example, the control device for the robot system 200 may include a processor configured to execute method 100. In some embodiments, these instructions may be stored on a computer-readable medium.

[0018] In some embodiments, such as Figure 3 As shown, the robotic system 200 may include a master control carriage 210, a surgical carriage 230, and a control device 220. The control device 220 can communicate with the master control carriage 210 and the surgical carriage 230, for example, via cable or wireless connection, to enable communication between the master control carriage 210 and the surgical carriage 230. The master control carriage 210 includes a master manipulator for remote operation by the operator, and the surgical carriage 230 includes a slave tool for performing surgery. The control device 220 enables master-slave mapping between the master manipulator in the master control carriage and the slave tool in the surgical carriage, allowing the master manipulator to control the motion of the slave tool. In some embodiments, the surgical carriage includes at least one slave tool (such as a surgical instrument or visual tool) mounted on the surgical carriage. The slave tool is configured to enter the operating area through a sheath, which may be fixed at the patient's surgical opening (e.g., an incision or natural opening), and the operating area may be the area where surgery is performed. The slave tool may include an arm and an end effector. The arm of the driven tool can be a flexible arm, and the end effector can be located at the distal end of the flexible arm. The end effector of the surgical tool can include, but is not limited to, surgical forceps, electrosurgical units, and electro-hooks. The end effector of the visual tool can include, but is not limited to, imaging devices or lighting devices. In some embodiments, the main control carriage includes a main operator, a display, and a foot pedal. Those skilled in the art will understand that the main control carriage 210 and the surgical carriage 230 can adopt other structures or forms, such as a base, support, or building.

[0019] Figure 3 A schematic diagram of the coordinate system in master-slave motion mapping according to some embodiments of the present disclosure is shown. Figure 3 The definitions of each coordinate system are as follows:

[0020] {Eb}: ​​Camera base coordinate system, with the origin located at the base of the vision tool or the exit of the in-situ sheath. Aligned with the extension line of the base or the axial direction of the abdominal sheath. Direction such as Figure 3 As shown.

[0021] {Tb}: The base coordinate system of the driven tool, with the origin located at the base of the driven tool or the exit of the insertion sheath. It is aligned with the extension line of the base or the axial direction of the abdominal sheath. Direction such as Figure 3 As shown.

[0022] {Cam}: Camera coordinate system, with its origin at the camera center and the camera axis pointing in the direction of... Direction, after the field of vision is straightened, the upper part is direction.

[0023] {Tt}: The coordinate system of the end effector of the driven tool, with the origin located at the end effector of the driven tool. Aligned with the axial direction of the end effector. Direction such as Figure 3 As shown.

[0024] {ITt}: The image coordinate system of the end effector of the driven tool, associated with the image of the end effector displayed on the monitor.

[0025] {W}: Reference coordinate system, which can be the coordinate system of the space where the main controller is located or the world coordinate system. It can be based on the operator's body sensation. When the operator is sitting in front of the main control console, the upward body sensation is... Direction, the perceived forward direction is... direction.

[0026] {Screen}: The monitor's coordinate system, with its origin at the center of the monitor and its coordinates extending inwards from the screen image. Positive direction, the top of the screen is direction.

[0027] {CombX}: Main operator base coordinate system, coordinate axis directions are as follows: Figure 3 As shown.

[0028] {H}: The coordinate system of the main controller's handles, with coordinate axes oriented as follows: Figure 3 As shown.

[0029] In the following text, Figure 1 The master-slave motion control method 100 is described using the coordinate system shown as an example. However, those skilled in the art will understand that other coordinate systems can be used to implement the master-slave motion control method 100.

[0030] likeFigure 4 As shown, in step 101, the current pose of the main manipulator can be determined, which includes the current position and the current orientation. In some embodiments, the current pose of the main manipulator is the pose relative to the main manipulator base coordinate system. For example, the pose of the main manipulator is the pose of the coordinate system defined by the main manipulator handle or a portion thereof relative to the main manipulator base coordinate system (e.g., the coordinate system defined by the support or base on which the main manipulator is located, or the world coordinate system). In some embodiments, determining the current position of the main manipulator includes determining the current position of the main manipulator handle relative to the main manipulator base coordinate system, and determining the current orientation of the main manipulator includes determining the current orientation of the main manipulator handle relative to the main manipulator base coordinate system.

[0031] In some embodiments, the current pose of the master operator can be determined based on coordinate transformation. For example, the current pose of the handle can be determined based on the transformation relationship between the coordinate system of the master operator's handle and the master operator's base coordinate system. Typically, the master operator's base coordinate system can be set on the bracket or base on which the master operator is located, and the master operator's base coordinate system remains unchanged during teleoperation.

[0032] In some embodiments, the current pose of the master manipulator can be determined based on a master manipulator sensor. In some embodiments, current joint information of at least one joint of the master manipulator is received, and the current pose of the master manipulator is determined based on the current joint information of at least one joint. For example, the current pose of the master manipulator is determined based on the current joint information of at least one joint obtained by the master manipulator sensor. The master manipulator sensor is disposed at at least one joint position of the master manipulator. For example, the master manipulator includes at least one joint, and at least one master manipulator sensor is disposed at at at least one joint. The current pose of the master manipulator is calculated based on the joint information (position or angle) of the corresponding joint obtained by the master manipulator sensor. For example, the current position and current orientation of the master manipulator are calculated based on a forward kinematics algorithm.

[0033] In some embodiments, the master manipulator includes at least one attitude joint for controlling the orientation of the handle. Determining the current orientation of the handle of the master manipulator includes: obtaining joint information of at least one attitude joint, and determining the current orientation of the master manipulator based on the joint information of at least one attitude joint. The master manipulator includes a robotic arm, which includes position joints and attitude joints. The attitude joints adjust the orientation of the master manipulator, controlling the master manipulator to reach a target orientation through one or more attitude joints. The position joints adjust the position of the master manipulator, controlling the master manipulator to reach a target position through one or more position joints. Master manipulator sensors are disposed at the attitude joints and position joints of the robotic arm to acquire joint information (position or angle) corresponding to the attitude joints and position joints. Based on the acquired joint information, the current orientation of the handle of the master manipulator relative to the base coordinate system of the master manipulator can be determined. For example, the master manipulator includes 7 joints, of which joints 5, 6, and 7 are attitude joints used to adjust the orientation of the handle of the master manipulator. Based on the joint information (such as angle) acquired by the master manipulator sensors of the attitude joints and a forward kinematics algorithm, the current orientation of the master manipulator is calculated. Joints 1, 2, and 3 are position joints used to adjust the position of the main manipulator's handle. Based on joint information (such as position) acquired by the main manipulator's sensors and a forward kinematics algorithm, the current position of the main manipulator is calculated.

[0034] In step 103, the target pose of the slave tool can be determined based on the current pose of the master manipulator and the pose relationship between the master manipulator and the slave tool. For example, a master-slave mapping relationship can be established between the master manipulator and the slave tool, and the pose of the slave tool can be controlled by teleoperating the master manipulator. The pose relationship includes the relationship between the pose of the slave tool or its image on the display relative to a reference coordinate system and the pose of the master manipulator relative to the reference coordinate system. The reference coordinate system includes the coordinate system of the space where the master manipulator is located or the world coordinate system. In some embodiments, the reference coordinate system can be based on the operator's haptic perception, thus taking into account the pose of the slave tool's image on the display relative to the reference coordinate system.

[0035] In some embodiments, the pose relationship between the master manipulator and the slave tool may include the relationship between the pose change of the master manipulator and the pose change of the slave tool, such as being equal or proportional. Determining the target pose of the slave tool includes: determining the previous pose of the master manipulator, determining the current pose of the slave tool, and determining the target pose of the slave tool based on the previous pose, the current pose of the master manipulator, and the current pose of the slave tool. The previous pose and the current pose of the master manipulator may be the pose of the handle of the master manipulator relative to the master manipulator base coordinate system. The current pose and the target pose of the slave tool may be the pose of the end effector of the slave tool relative to the slave tool base coordinate system.

[0036] The driven tool includes a flexible arm and an end effector located at the end of the flexible arm. The pose of the driven tool includes the orientation of the end effector relative to the tool's base coordinate system. The base coordinate system of the driven tool can be the coordinate system of the base on which it is mounted (e.g., the end effector of a surgical robot's arm), the coordinate system of the sheath through which the driven tool passes (e.g., the coordinate system of the sheath exit), or the coordinate system of the remote center of motion (RCM) of the driven tool, etc. For example, the base coordinate system of the driven tool can be set at the sheath exit position and remains unchanged during teleoperation. The current orientation of the end effector can be transformed to obtain its orientation relative to other coordinate systems.

[0037] In some embodiments, previous joint information of at least one joint of the master manipulator can be received, and the previous pose of the master manipulator can be determined based on the previous joint information of at least one joint. For example, the previous pose and current pose of the master manipulator's handle can be determined based on joint information of the master manipulator read from the master manipulator's sensors at a previous time and at the current time. The position change of the master manipulator's handle can be determined based on the previous position and current position of the handle relative to the master manipulator's base coordinate system. The attitude change of the master manipulator's handle can be determined based on the previous attitude and current attitude of the handle relative to the master manipulator's base coordinate system.

[0038] In some embodiments, current drive information of at least one drive device of the driven tool can be received, wherein the at least one drive device is used to drive the flexible arm of the driven tool. The current pose of the driven tool is determined based on the current drive information of the at least one drive device. In some embodiments, the current drive information (e.g., angle) of the driven tool is obtained through a drive device sensor, and the current attitude of the driven tool is determined based on the current drive information. For example, the current attitude of the driven tool can be calculated using a forward kinematics algorithm.

[0039] A drive unit sensor can be mounted on the drive unit, which drives the flexible arm of the driven tool. The drive unit sensor acquires drive information. Based on this drive information, the current posture of the driven tool can be determined. For example, the drive unit may include at least one drive motor, and the drive unit sensor is coupled to the drive motor to record and output motor data. For example, the motor data may include binary or hexadecimal numbers, which can be converted to obtain the current posture of the driven tool. The drive unit sensor may include a potentiometer or an encoder. Information such as angles is acquired through the potentiometer or encoder, thereby determining the current posture of the driven tool. In some embodiments, a pose sensor can be used to obtain the posture of the driven tool. For example, the pose sensor may also be a fiber optic sensor, which is disposed throughout the arm of the driven tool to sense the position and posture of the driven tool.

[0040] In some embodiments, the pose change of the master manipulator can be determined based on its previous and current poses. The pose change of the slave tool can be determined based on the pose change of the master manipulator and the pose relationship between the master manipulator and the slave tool. The target pose of the slave tool can be determined based on its current pose and the pose change of the slave tool.

[0041] Positional relationships can include both positional relationships and attitude relationships. The positional relationship between the master manipulator and the slave tool can include the relationship between the positional changes of the master manipulator and the positional changes of the slave tool, while the attitude relationship between the master manipulator and the slave tool can include the relationship between the attitude changes of the master manipulator and the attitude changes of the slave tool.

[0042] In some embodiments, method 100 further includes: determining the current position of the handle of the master operator relative to the master operator base coordinate system; determining the previous position of the handle relative to the master operator base coordinate system; determining the current position of the end effector of the driven tool relative to the driven tool base coordinate system; and determining the target position of the end effector relative to the driven tool base coordinate system based on the previous and current positions of the handle relative to the master operator base coordinate system and the current position of the end effector relative to the driven tool base coordinate system. For example, the previous position of the master operator is determined based on joint information of the master operator read by the master operator sensor at a previous time; the current position of the master operator is determined based on joint information of the master operator read by the master operator sensor at the current time. The position change of the master operator is determined based on the previous and current positions of the handle relative to the master operator base coordinate system. The current position of the driven tool is determined based on joint information of the driven tool read by the driven tool sensor at the current time. The position change of the driven tool is determined based on the position change of the master operator and the pose relationship between the master operator and the driven tool. The target position of the driven tool is determined based on the current position of the driven tool and the position change of the driven tool.

[0043] In some embodiments, the pose relationship includes the relationship between the pose of the slave tool's image on the display relative to a reference coordinate system and the pose of the master manipulator relative to the reference coordinate system. The slave tool includes surgical tools and visual tools. During surgery, the surgical tools perform the procedure inside the patient's body, and the visual tools acquire images inside the patient's body and transmit the acquired images to the operating table. After processing by the video processing module in the operating table, the images are displayed on the monitor of the master control table. The operator obtains the pose of the slave tool relative to the reference coordinate system in real time through the images on the monitor. The pose of the master manipulator relative to the reference coordinate system is the posture actually perceived by the operator. The pose change perceived by the operator through remote operation of the master manipulator and the pose change of the slave tool perceived by the operator on the display conform to a preset pose relationship. Thus, by remotely operating the master manipulator, the pose transformation of the master manipulator is converted into a pose change of the slave tool based on the preset pose relationship, thereby achieving pose control of the slave tool.

[0044] In some embodiments, the pose relationship includes: the change in position of the image of the driven tool on the display relative to the reference coordinate system is proportional to the change in position of the master operator relative to the reference coordinate system. Thus, when the operator holds the handle of the master operator, based on the principle of intuitive operation, the change in position of the image of the end effector of the driven tool perceived by the operator is proportional to the change in position of the handle of the master operator perceived by the operator, thereby improving the accuracy of teleoperation.

[0045] The change in position of the slave tool's image on the monitor relative to the reference coordinate system is proportional to the change in position of the master operator relative to the reference coordinate system, which can be expressed as:

[0046] W Δp ITt =k· W Δp H (1)

[0047] In formula (1), the left side W Δp ITt This represents the change in position of the end effector of the driven tool relative to the reference coordinate system in the image on the monitor. (Right side) W Δp H This indicates the change in position of the main controller's handle relative to the reference coordinate system. W Δp ITt and W Δp H They are proportional, with a proportionality constant of k.

[0048] In some embodiments, the position of the driven tool's image on the display relative to a reference coordinate system can be used as the basis. W p ITt(t0) and target location W p ITt Determine the change in position of the driven tool. W Δp ITt It can be based on the previous position of the master operator relative to the reference coordinate system. W p H(t0) and current location W p H Determine the position change of the master operator. W Δp H For example, when a teleoperation command is triggered or at time t0 in the previous control cycle, the previous position of the master operator's handle relative to the reference coordinate system can be determined based on the joint information of the master operator obtained from the master operator's sensors. W p H(t0) In the current control loop, denoted as time t1, the current position of the master manipulator's handle relative to the reference coordinate system can be determined based on the joint information of the master manipulator obtained from the master manipulator's sensors. W p H Based on the master operator's previous position at time t0. W p H(t0) And the current position of the master operator at time t1 W p H The position change of the master operator is obtained. W Δp H Furthermore, at time t0, the current position of the driven tool in the image on the display can be determined based on the driven tool's driving information. W p ITt(t0) It can be based on the change in the position of the handle. W Δp H And the current position of the driven tool's image on the monitor. W p ITt(t0) Determine the target position of the driven tool in the image on the monitor. W p ITt .

[0049] In some embodiments, the left side of formula (1) represents the change in position of the end effector of the driven tool relative to the reference coordinate system in the image on the display. W Δp ITt It can be based on the change in position of the driven tool's image on the monitor relative to the monitor coordinate system. Screen Δp ITt and the transformation relationship between the display coordinate system and the world coordinate system. W R Screen Sure.

[0050] Specifically, as shown in formula (2),

[0051] W Δp ITt = W R Screen Screen Δp ITt (2)

[0052] In some embodiments, the display coordinate system (Screen) and the camera coordinate system (Cam) are defined identically for the field of view direction. Therefore, the change in position of the driven tool's image on the display relative to the display coordinate system... Screen Δp ITt The change in position of the driven tool relative to the camera coordinate system Cam Δp Tt Consistent. See formula (3) for details.

[0053] Screen Δp ITt = Cam Δp Tt (3)

[0054] Based on formulas (2) and (3), formula (4) is obtained.

[0055] W Δp ITt = W R Screen Cam Δp Tt (4)

[0056] In formula (4), the change in position of the driven tool's image on the monitor relative to the reference coordinate system is... W Δp ITt It can be based on the change in position of the end effector of the driven tool relative to the camera coordinate system. Cam Δp Tt and the transformation relationship between the display coordinate system and the world coordinate system. W R Screen Confirmed. The display coordinate system and the reference coordinate system have a predetermined transformation relationship. W R Screen .

[0057] In formula (4), the change in position of the end effector of the driven tool relative to the camera coordinate system is... Cam Δp Tt It can be based on the position of the end effector relative to the camera coordinate system at time t0. Cam p Tt(t0) and the position of the end effector relative to the camera coordinate system at time t. Cam ΔpTt The difference is determined as shown in formula (5).

[0058] Cam Δp Tt = Cam p Tt - Cam p Tt(t0) (5)

[0059] In formulas (4) and (5), the change in position of the end effector of the driven tool in the camera coordinate system is... Cam Δp Tt It can be based on the transformation relationship between the camera base coordinate system and the camera coordinate system. Cam R Eb The relationship between the camera base coordinate system and the driven tool base coordinate system Eb R Tb and the change in position of the end effector of the driven tool relative to the base coordinate system of the driven tool. Tb Δp Tt Confirmed. See formula (6) for details.

[0060] Cam Δp Tt = Cam R Eb Eb R Tb Tb Δp Tt (6)

[0061] in, Tb Δp Tt It can be determined based on the driving information of the driven tool.

[0062] In some embodiments, the position change of the main operator's handle relative to the reference coordinate system is on the right side of formula (1). W Δp H It can be based on the transformation relationship between the master operator's base coordinate system and the reference coordinate system. W R CombX and the change in position of the handle relative to the main controller's base coordinate system CombX Δp H As confirmed in formula (7),

[0063] W Δp H = W R CombX CombX Δp H (7)

[0064] In formula (7), W R CombXThe orientation of the main operator is determined by its position. In some embodiments, the reference coordinate system and the main operator's base coordinate system have a predetermined transformation relationship.

[0065] In formula (7), the change in position of the handle relative to the base coordinate system of the main controller is... CombX Δp H It can be based on the current attitude of the master controller's handle relative to the master controller's base coordinate system. CombX p H and the previous attitude of the handle relative to the main controller base coordinate system. CombX p H(t0) Please refer to formula (8) for details.

[0066] CombX Δp H = CombX p H - CombX p H(t0) (8)

[0067] Combining formulas (1) to (8), the target position of the driven tool during teleoperation can be obtained. Tb p Tt The expression is as shown in formula (9).

[0068] Tb p Tt = Tb p Tt(t0) +k· Tb R Cam Screen R CombX ( CombX p H - CombX p H(t0) (9)

[0069] Based on formula (9), in some embodiments, the previous position of the handle relative to the master operator base coordinate system can be used as a basis. CombX p H(t0) and current location CombX p H The current position of the end effector relative to the driven tool base coordinate system Tb p Tt(t0) Transformation relationship between the main operator's base coordinate system and the display coordinate system Screen R CombX And the transformation relationship between the camera coordinate system and the driven tool base coordinate system. Tb R Cam Determine the target position of the end effector relative to the driven tool base coordinate system. Tb p Tt .

[0070] In some embodiments, the transformation relationship between the camera coordinate system and the driven tool base coordinate system Tb R Cam It can be based on the transformation relationship between the camera coordinate system and the camera base coordinate system. Eb R Cam And the transformation relationship between the camera base coordinate system and the driven tool base coordinate system. Tb R Eb Sure.

[0071] In some embodiments, the transformation relationship between the main operator base coordinate system and the display coordinate system Screen R CombX It can be based on the transformation relationship between the master operator's base coordinate system and the reference coordinate system. W R CombX Transformation relationship between reference coordinate system and display coordinate system Screen R W Sure.

[0072] In some embodiments, the attitude change of the image of the slave tool on the display relative to the reference coordinate system is consistent with the attitude change of the master controller relative to the reference coordinate system. Thus, when the operator holds the master controller handle, based on the principle of intuitive operation, the attitude change of the image of the end effector of the surgical tool perceived by the operator is consistent with the attitude change of the master controller handle perceived by the operator, thereby improving the accuracy of teleoperation.

[0073] The change in pose of the followed tool's image on the monitor relative to the reference coordinate system. W R ITt(t0)ITt attitude change of the master manipulator relative to the reference coordinate system W R H(t0)H Consistency can be expressed as follows:

[0074] W R ITt(t0)ITt = W R H(t0)H (10)

[0075] In some embodiments, the orientation of the master operator relative to the reference coordinate system at time t0 can be used as a basis. W R H(t0) and the current attitude at time t1 W R H Determine the attitude change of the master manipulator W R H(t0)H Moreover, it can be based on the amount of attitude change. W R H(t0)H And the current pose of the slave tool's image on the display relative to the reference coordinate system at time t0. W R ITt(t0)Determine the target pose of the driven tool in the image on the monitor. W R ITt .

[0076] For example, when a teleoperation command is triggered or at time t0 in the previous control loop, the previous orientation of the master operator's handle relative to the reference coordinate system can be determined based on the joint information of the master operator obtained from the master operator's sensors. W R H(t0) In the current control loop, at time t1, based on the joint information of the master manipulator obtained from the master manipulator sensors, the current attitude of the master manipulator's handle relative to the reference coordinate system is determined. W R H Based on the master operator's previous attitude at time t0. W R H(t0) and the current attitude of the master operator at time t1 W R H The attitude change of the master operator is obtained. W R H(t0)H Furthermore, at time t0, the current pose of the driven tool in the image on the display can be determined based on the driven tool's driving information. W R ITt(t0) It can be based on the change in the handle's attitude. W R H(t0)H and the current pose of the slave tool's image on the monitor. W R ITt(t0) Determine the target pose of the driven tool in the image on the monitor. W R ITt .

[0077] In some embodiments, regarding the left side of equation (10), the pose change of the image of the driven tool on the display relative to the reference coordinate system is... W R ITt(t0)ITt It can be based on the transformation relationship between the reference coordinate system and the display coordinate system. W R Screen Transformation relationship between display coordinate system and camera coordinate system Screen R Cam The previous orientation of the end effector of the driven tool relative to the camera coordinate system. Cam R Tt(t0) and the current pose of the end effector relative to the camera coordinate system Cam R Tt Confirmed. See formula (11) for details.

[0078] W R ITt(t0)ITt =( W R Screen Screen RCam Cam R Tt ()( W R Screen Screen R Cam Cam R Tt(t0) ) T (11)

[0079] In some embodiments, regarding the right side of equation (10), the attitude change of the master manipulator relative to the reference coordinate system W R H(t0)H It can be based on the transformation relationship between the reference coordinate system and the main operator's base coordinate system. W R CombX The previous attitude of the master controller's handle relative to the master controller's base coordinate system. CombX R H(t0) and the current attitude of the master controller's handle relative to the master controller's base coordinate system. CombX R H Confirmed. See formula (12) for details.

[0080] W R H(t0)H = W R CombX CombX R H ( W R CombX CombX R H(t0) ) T (12)

[0081] Combining formulas (10) to (12), the target attitude expression of the driven tool during teleoperation can be obtained as shown in formula (13).

[0082] Tb R Tt = Tb R CombX ( CombX R H ( CombX R H(t0) ) T ) CombX R Tb Tb R Tt(t0) (13)

[0083] Based on formula (13), in some embodiments, the previous orientation of the handle relative to the master operator base coordinate system can be used as a basis. CombX R H(t0) and current posture CombX R HThe current attitude of the end effector of the driven tool relative to the base coordinate system of the driven tool. Tb R Tt(t0) and the transformation relationship between the driven tool base coordinate system and the master operator base coordinate system. CombX R Tb Determine the target attitude of the end effector relative to the base coordinate system of the driven tool. Tb R Tt .

[0084] In some embodiments, the transformation relationship between the driven tool base coordinate system and the master operator base coordinate system CombX R Tb It can be based on the transformation relationship between the driven tool base coordinate system and the camera coordinate system. Cam R Tb Transformation relationship between camera coordinate system and display coordinate system Screen R Cam Transformation relationship between the display coordinate system and the main operator base coordinate system CombX R Screen Sure.

[0085] In some embodiments, the transformation relationship between the driven tool base coordinate system and the camera coordinate system Cam R Tb It can be based on the transformation relationship between the driven tool base coordinate system and the camera base coordinate system. Eb R Tb Transformation relationship between camera coordinate system and camera base coordinate system Cam R Eb The transformation relationship between the display coordinate system and the main operator's base coordinate system is confirmed. CombX R Screen It can be based on the transformation relationship between the display coordinate system and the reference coordinate system. W R Screen Transformation relationship between the reference coordinate system and the main operator base coordinate system CombX R W It's confirmed.

[0086] In some embodiments, the camera may be positioned at the end of a drivable flexible arm, and the transformation relationship between the camera coordinate system and the camera base coordinate system is determined based on the driving information of the flexible arm.

[0087] In some embodiments, the slave tool's base coordinate system and the camera's base coordinate system have a predetermined transformation relationship. In some embodiments, before entering the master-slave motion control phase, the vision tool has already finished its motion, and the current pose of the camera is represented in the camera's base coordinate system. It will no longer change.

[0088] In some embodiments, the display coordinate system and the reference coordinate system have a predetermined transformation relationship. In some embodiments, the reference coordinate system and the main operator base coordinate system have a predetermined transformation relationship. In some embodiments, the display coordinate system and the main operator base coordinate system have a predetermined transformation relationship.

[0089] In some embodiments, the attitude change of the driven tool relative to the reference coordinate system is consistent with the attitude change of the master operator relative to the reference coordinate system. In some embodiments, the position change of the driven tool relative to the reference coordinate system is proportional to the position change of the master operator relative to the reference coordinate system. Thus, when the operator moves the handle of the master operator to operate the driven tool, based on the principle of intuitive operation, the position change of the end effector of the driven tool perceived by the operator maintains a certain proportional relationship with the position change of the master operator perceived by the operator, thereby improving the accuracy of teleoperation.

[0090] The change in position of the driven tool relative to the reference coordinate system is proportional to the change in position of the master operator relative to the reference coordinate system, which can be expressed as:

[0091] W Δp Tt =k· W Δp H (14)

[0092] In formula (14), the left side W Δp Tt This indicates the change in position of the driven tool relative to the reference coordinate system; the right side... W Δp H This represents the change in position of the master operator relative to the reference coordinate system. Furthermore, W Δp Tt and W Δp H They are in a proportional relationship, with a proportionality coefficient of k.

[0093] In some embodiments, the previous position of the driven tool relative to the reference coordinate system can be used as a basis. W p Tt(t0) and current location W p Tt Determine the change in position of the driven tool. W Δp Tt It can be based on the previous position of the master operator relative to the reference coordinate system. W p H(t0) and current location W p H Determine the position change of the master operator W Δp HFor example, when a teleoperation command is triggered or at time t0 in the previous control cycle, the previous position of the master operator's handle relative to the reference coordinate system can be determined based on the joint information of the master operator obtained from the master operator's sensors. W p H(t0) In the current control loop, at time t1, based on the joint information of the master manipulator obtained by the master manipulator sensor, the current position of the master manipulator's handle relative to the reference coordinate system is determined. W p H Based on the master operator's previous position at time t0. W p H(t0) And the current position of the master operator at time t1 W p H The position change of the master operator is obtained. W Δp H Furthermore, at time t0, the current position of the driven tool relative to the reference coordinate system can be determined based on the driven tool's driving information. W p Tt(t0) It can be based on the change in the position of the handle. W Δp H and the current position of the driven tool relative to the reference coordinate system. W p Tt(t0) Determine the target position of the driven tool relative to the reference coordinate system. W p Tt .

[0094] In formula (14), the change in position of the end effector of the driven tool relative to the reference coordinate system is... W Δp Tt The position of the end effector relative to the reference coordinate system at time t0 can be determined by... W p Tt and the position of the end effector relative to the reference coordinate system at time t. W p Tt(t0) The difference is represented as shown in formula (15).

[0095] W Δp Tt = W p Tt - W p Tt(t0) (15)

[0096] In formula (14), the change in position of the master operator relative to the world coordinate system is... W Δp H The position of the master operator relative to the reference coordinate system at time t0 can be determined by the position of the master operator. W p H(t0) and the position of the master operator relative to the reference coordinate system at time t.W p H The difference is represented as shown in formula (16).

[0097] W Δp H = W p H - W p H(t0) (16)

[0098] In some embodiments, the same matrix is ​​multiplied on both sides of formula (14). Tb R W Based on formulas (14) to (16), formula (17) is obtained.

[0099] Tb R W ( W p Tt - W p Tt(t0) )=k· Tb R W ( W p H - W p H(t0) (17)

[0100] Formula (18) is derived from the left side of formula (17).

[0101] Tb R W ( W p Tt - W p Tt(t0) )= Tb p Tt(t0) (18)

[0102] Formula (19) is derived from the right side of formula (17).

[0103] k· Tb R W ( W p H - W p H(t0) )=k· Tb R CombX ( CombX p H - CombX p H(t0) (19)

[0104] Formula (20) is derived from formulas (18) and (19).

[0105] Tb pTt =k· Tb R CombX ( CombX p H - CombX p H(t0) )+ Tb p Tt(t0) (20)

[0106] Based on formula (20), in some embodiments, the position of the handle relative to the master operator base coordinate system can be used as a basis. CombX p H(t0) and current location CombX p H The current position of the end effector relative to the driven tool base coordinate system Tb p Tt(t0) Transformation relationship between the master operator base coordinate system and the slave tool base coordinate system Tb R CombX Determine the target position of the end effector relative to the driven tool base coordinate system. Tb p Tt .

[0107] In some embodiments, the attitude change of the driven tool relative to the reference coordinate system is consistent with the attitude change of the master manipulator relative to the reference coordinate system. Thus, when the operator moves the handle of the master manipulator to operate the driven tool, based on the principle of intuitive operation, the attitude change of the end effector of the surgical tool perceived by the operator is consistent with the attitude change of the master manipulator perceived by the operator, thereby improving the accuracy of teleoperation.

[0108] The attitude change of the driven tool relative to the reference coordinate system is the same as the attitude change of the master operator relative to the reference coordinate system, which can be expressed as:

[0109]

[0110] In formula (21), the left side This represents the change in the attitude of the driven tool relative to the reference coordinate system, on the right side. W R H(t0)H This represents the change in attitude of the master operator relative to the reference coordinate system.

[0111] In some embodiments, the current pose of the driven tool relative to the reference coordinate system can be used as a basis. and target attitude Determine the attitude change of the driven tool Based on the previous attitude of the master operator relative to the reference coordinate system W R H(t0) and current posture W R HDetermine the attitude change of the master manipulator W R H(t0)H For example, when a teleoperation command is triggered or at time t0 in the previous control loop, the previous orientation of the master operator's handle relative to the reference coordinate system can be determined based on the joint information of the master operator obtained from the master operator's sensors. W R H(t0)H In the current control loop, at time t1, based on the joint information of the master manipulator obtained from the master manipulator sensors, the current attitude of the master manipulator's handle relative to the reference coordinate system is determined. W R H It can be based on the master operator's previous posture at time t0. W R H(t0)H and the current attitude of the master operator at time t1 W R H The attitude change of the master operator is obtained. W Δp H Furthermore, at time t0, the current attitude of the driven tool relative to the reference coordinate system can be determined based on the driving information of the driven tool. It can be based on the change in the handle's attitude. W R H(t0)H and the current attitude of the driven tool relative to the reference coordinate system. Determine the target attitude of the driven tool relative to the reference coordinate system.

[0112] In formula (21), the attitude change of the driven tool relative to the reference coordinate system is the amount of attitude change. It can be based on the current attitude of the end effector relative to the reference coordinate system at time t0. and the target attitude of the end effector relative to the reference coordinate system at time t. Confirmed. Attitude change of the master operator relative to the reference coordinate system. W R H(t0)H It can be based on the handle's previous orientation relative to the reference coordinate system at time t0. And the current attitude of the handle relative to the reference coordinate system at time t. W R H Confirmed. See formula (22) for details.

[0113]

[0114] In some embodiments, the same matrix is ​​multiplied on both sides of formula (22). Tb R W ( Tb R W ) T Based on formula (22), formula (23) is obtained.

[0115]

[0116] Formula (24) is derived from the left side of formula (23).

[0117]

[0118] Formula (25) is derived from the right side of formula (23).

[0119]

[0120] Combining formulas (21) to (25), the target attitude of the driven tool during teleoperation can be obtained. The expression is as shown in formula (26).

[0121]

[0122] Based on formula (26), in some embodiments, the previous orientation of the handle relative to the master operator base coordinate system can be used as a basis. CombX R H(t0) and current posture CombX R H The current attitude of the end effector of the driven tool relative to the base coordinate system of the driven tool. Tb R Tt(t0) and the transformation relationship between the driven tool base coordinate system and the master operator base coordinate system. CombX R Tb Determine the target attitude of the end effector relative to the base coordinate system of the driven tool. Tb R Tt .

[0123] In step 105, a control signal for the driven tool can be generated based on the target pose of the driven tool. In some embodiments, a drive signal for driving at least one drive device of the driven tool is generated based on the target pose of the driven tool. In some embodiments, a drive signal for driving at least one drive device of the driven tool is generated based on the current pose and the target pose of the driven tool. For example, the drive signal for driving at least one drive device of the driven tool is calculated based on an inverse kinematics algorithm.

[0124] In some embodiments, the master-slave motion control method further includes: performing master-slave motion control at a predetermined period. For example, during master-slave motion control, the current pose of the master manipulator is read at a predetermined period, the target pose of the slave tool is determined based on the current pose of the master manipulator and the pose relationship between the master and slave, and the slave tool is controlled to move to the target pose, thereby realizing the master manipulator's motion control over the slave tool.

[0125] In some embodiments, for each current cycle, the previous pose of the master operator includes the pose when the master operator and the slave tool initially established a master-slave mapping relationship or in the previous cycle. The current pose of the master operator includes the pose of the master operator in the current cycle (e.g., the pose after teleoperation). The current pose of the slave tool includes the pose of the current cycle (e.g., the pose before master-slave control was performed in the current cycle), and the target pose includes the target pose for master-slave control in the current cycle. The current pose of the master operator matches the target pose of the slave tool.

[0126] Surgical robots demand high precision in operation and a superior human-computer interaction experience. They typically perform surgical procedures using a vision system. The pose of the camera in this vision system is usually adjustable to modify the angle and range of the field of view. However, adjusting the camera pose can alter the operator's (e.g., surgeon's) observation experience. Some embodiments of this disclosure consider the position and orientation of the camera and display, optimizing the operator's experience. This allows for accurate and rapid control of the driven tool to the desired position and can exceed the movement limits of the master manipulator, achieving a wider operating range. Therefore, some embodiments of this disclosure can mitigate or even avoid the visual-to-realistic discrepancies caused by the direct mapping of the master manipulator's pose to the driven tool's pose, as well as the limitations on the driven tool's movement range imposed by the master manipulator's control over the driven tool's pose.

[0127] Figure 4 A schematic diagram 400 of a robot system according to some embodiments of the present disclosure is shown. For example... ​ As shown, a robot system 400 includes a master manipulator 410, a slave tool 420, a drive unit 430, and a control unit 440. The master manipulator 410 includes a robotic arm, a handle mounted on the robotic arm, and at least one master manipulator sensor mounted at at least one joint on the robotic arm. The at least one master manipulator sensor is used to obtain joint information of at least one joint. The slave tool 420 includes a flexible arm body and an end effector. The drive unit 430 is used to drive the flexible arm body of the slave tool, and at least one drive unit sensor is coupled to the at least one drive unit and used to obtain drive information. The control unit 440 is communicatively connected to the master manipulator 410 and the at least one drive unit 430. The control unit 440 is configured to execute a master-slave motion control method according to some embodiments of this disclosure.

[0128] In some embodiments, the master manipulator 410 includes a six-degree-of-freedom robotic arm, with a master manipulator sensor located at each joint of the six-degree-of-freedom robotic arm. Joint information (such as joint angle data) is generated through the master manipulator sensor at each joint. In some embodiments, the master manipulator sensor employs a potentiometer and / or an encoder.

[0129] In some embodiments, the driven tool 420 includes a multi-jointed six-DOF flexible arm.

[0130] In some embodiments, the drive device 430 is used to drive the flexible arm of the driven tool 420 and obtain the drive information corresponding to the driven tool through the drive device sensor.

[0131] In some embodiments, the control device 440 is communicatively connected to the master operator 410 and the drive device 430. For example, the master operator 410, the drive device 430, and the control device 440 may be connected via a data transmission bus, including but not limited to wireless data transmission, wired data connection, or a combination of multiple data communication methods. The data transmission bus may be a communication protocol bus, such as a Controller Area Network (CAN) bus.

[0132] Control device 440 is configured to execute a master-slave motion control method according to some embodiments of this disclosure. For example, the control device is used to receive network data packets (such as joint information) sent by the master manipulator sensor and the drive device sensor. The control device calculates the joint target value for the master manipulator handle to reach the target posture consistent with the current posture of the slave tool based on the joint information of the slave tool and the joint information of the master manipulator, and converts it into a drive signal and sends it to drive device 430. Drive device 430 receives the drive signal through network data packets, sends it to each Epos control tool through the CAN bus, drives the motors of the master manipulator to move so that the master manipulator moves into position, and realizes the posture matching between the master manipulator handle and the slave tool.

[0133] In some embodiments, the master manipulator may include a controller, which calculates the master manipulator's posture data based on joint information obtained from the master manipulator sensors and sends the calculated posture data to a control device. In other embodiments, the control device may also calculate the master manipulator's posture data based on the joint data sent by the master manipulator sensors.

[0134] 1. A method for controlling master-slave motion, comprising:

[0135] Determine the current pose of the main operator, which includes the current position and the current orientation;

[0136] Based on the current pose of the master manipulator and the pose relationship between the master manipulator and the slave tool, the target pose of the slave tool is determined; and

[0137] Based on the target pose of the driven tool, a control signal for the driven tool is generated.

[0138] 2. According to the control method described in item 1, the pose relationship includes the relationship between the pose of the slave tool or the image of the slave tool on the display relative to the reference coordinate system and the pose of the master manipulator relative to the reference coordinate system.

[0139] 3. According to the control method described in item 2, the pose relationship includes at least one of the following:

[0140] The change in position of the driven tool or its image on the display relative to the reference coordinate system is proportional to the change in position of the master operator relative to the reference coordinate system; or

[0141] The attitude change of the slave tool or the image of the slave tool on the display relative to the reference coordinate system is consistent with the attitude change of the master operator relative to the reference coordinate system.

[0142] 4. In the control method according to any one of 2-3, the reference coordinate system includes the coordinate system of the space where the master operator is located or the world coordinate system.

[0143] 5. The control method according to any one of claims 1-4 further includes:

[0144] Determine the previous pose of the master operator;

[0145] Determine the current pose of the driven tool;

[0146] The target pose of the slave tool is determined based on the previous and current poses of the master manipulator and the current pose of the slave tool.

[0147] 6. The control method according to any one of claims 1-5 further includes:

[0148] Determine the current position of the handle of the main operator relative to the main operator base coordinate system;

[0149] Determine the previous position of the handle relative to the master operator base coordinate system;

[0150] Determine the current position of the end effector of the driven tool relative to the base coordinate system of the driven tool; and

[0151] Based on the previous and current positions of the handle relative to the master operator base coordinate system and the current position of the end effector relative to the driven tool base coordinate system, the target position of the end effector relative to the driven tool base coordinate system is determined.

[0152] 7. Determining the target position of the end effector relative to the driven tool base coordinate system according to the control method described in item 6 includes:

[0153] Based on the previous and current positions of the handle relative to the master operator base coordinate system, the transformation relationship between the master operator base coordinate system and the display coordinate system, the transformation relationship between the camera coordinate system and the driven tool base coordinate system, and the current position of the end effector relative to the driven tool base coordinate system, the target position of the end effector relative to the driven tool base coordinate system is determined.

[0154] 8. According to the control method described in item 7, the display coordinate system and the camera coordinate system have the same definition for the field of view direction.

[0155] 9. Determining the target position of the end effector relative to the driven tool base coordinate system according to any one of claims 6-8 includes:

[0156] Based on the previous and current positions of the handle relative to the master operator base coordinate system, the transformation relationship between the master operator base coordinate system and the driven tool base coordinate system, and the current position of the end effector relative to the driven tool base coordinate system, the target position of the end effector relative to the driven tool base coordinate system is determined.

[0157] 10. The control method according to any one of claims 1-9 further includes:

[0158] Determine the current orientation of the handle of the master operator relative to the master operator base coordinate system;

[0159] Determine the previous orientation of the handle relative to the master operator base coordinate system;

[0160] Determine the current orientation of the end effector of the driven tool relative to the driven tool base coordinate system; and

[0161] Based on the previous and current orientations of the handle relative to the master operator's base coordinate system, the transformation relationship between the slave tool's base coordinate system and the master operator's base coordinate system, and the current orientation of the end effector of the slave tool relative to the slave tool's base coordinate system, the target orientation of the end effector relative to the slave tool's base coordinate system is determined.

[0162] 11. According to the control method described in 10, the transformation relationship between the driven tool base coordinate system and the master operator base coordinate system is determined based on the transformation relationship between the driven tool base coordinate system and the camera coordinate system, the transformation relationship between the camera coordinate system and the display coordinate system, and the transformation relationship between the display coordinate system and the master operator base coordinate system.

[0163] 12. According to the control method described in item 7 or 11, the transformation relationship between the driven tool base coordinate system and the camera coordinate system is determined based on the transformation relationship between the driven tool base coordinate system and the camera base coordinate system and the transformation relationship between the camera coordinate system and the camera base coordinate system.

[0164] 13. According to the control method described in item 7 or 11, the transformation relationship between the display coordinate system and the main operator base coordinate system is determined based on the transformation relationship between the display coordinate system and the reference coordinate system and the transformation relationship between the reference coordinate system and the main operator base coordinate system.

[0165] 14. In the control method according to any one of 11-13, the driven tool base coordinate system and the camera base coordinate system have a predetermined transformation relationship.

[0166] 15. According to the control method described in item 13 or 14, the display coordinate system and the reference coordinate system have a predetermined transformation relationship.

[0167] 16. In the control method according to any one of 13-15, the reference coordinate system and the main operator base coordinate system have a predetermined transformation relationship.

[0168] 17. The control method according to any one of 12-16, wherein the camera is disposed at the end of a drivable flexible arm, and the transformation relationship between the camera coordinate system and the camera base coordinate system is determined based on the driving information of the flexible arm.

[0169] 18. The control method according to any one of claims 1-17 further comprises:

[0170] Receive current joint information of at least one joint of the master manipulator; and

[0171] Based on the joint information of the at least one joint, the current pose of the main manipulator is determined.

[0172] 19. The control method according to any one of claims 1-18, generating the control signal for the driven tool includes:

[0173] Based on the target pose of the driven tool, a drive signal is generated for at least one drive device to drive the driven tool.

[0174] 20. The control method according to any one of claims 5-19 further includes:

[0175] Receive previous joint information of at least one joint of the master manipulator;

[0176] Based on the previous joint information of the at least one joint, the previous pose of the master manipulator is determined, the previous pose including the previous position and the previous orientation;

[0177] Receive current drive information of at least one drive device of the driven tool, wherein the at least one drive device is used to drive the flexible arm of the driven tool; and

[0178] Based on the current driving information of the at least one driving device, the current pose of the driven tool is determined, and the current pose includes the current position and the current orientation.

[0179] 21. The control method according to any one of claims 1-20 further includes: controlling the master-slave motion at a predetermined period.

[0180] 22. A robot system, comprising:

[0181] The master manipulator includes a robotic arm, a handle disposed on the robotic arm, and at least one master manipulator sensor disposed at at least one joint on the robotic arm, wherein the at least one master manipulator sensor is used to obtain joint information of the at least one joint;

[0182] Driven tools, including flexible arms and end effectors;

[0183] At least one driving device for driving the flexible arm of the driven tool;

[0184] At least one drive sensor, coupled to the at least one drive device and used to obtain drive information; and

[0185] A control device, communicatively connected to the master operator and the at least one drive device, is configured to perform a master-slave motion control method as described in any one of claims 1-21.

[0186] 23. A computer device, the computer device comprising:

[0187] Memory for storing at least one instruction; and

[0188] A processor, coupled to the memory and configured to execute the at least one instruction to perform the master-slave motion control method as described in any one of claims 1-21.

[0189] 24. A computer-readable storage medium for storing at least one instruction, which, when executed by a computer, causes a robot system to perform a master-slave motion control method as described in any one of claims 1-21.

[0190] Note that the above are merely exemplary embodiments and technical principles of this disclosure. Those skilled in the art will understand that this disclosure is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this disclosure. Therefore, although this disclosure has been described in detail through the above embodiments, this disclosure is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of this disclosure, the scope of which is determined by the scope of the appended claims.

Claims

1. A computer-readable storage medium storing at least one instruction which, when executed by a computer, causes a robotic system to implement a master-slave motion control method, the master-slave motion control method comprising: controlling a flexible arm body motion of a vision tool of a plurality of slave tools to adjust a pose of a camera at a distal end of the flexible arm body of the vision tool to adjust a field of view of the camera; controlling a master operator to move to a target pose consistent with a current pose of a surgical tool of the plurality of slave tools; and performing master-slave motion control of the master operator on the surgical tool through a plurality of control cycles; wherein performing master-slave motion control of the master operator on the surgical tool through a plurality of control cycles comprises, in each control cycle, determining a current pose of the master operator; determining a previous pose of the master operator; determining a current pose of the surgical tool; determining a target pose of the surgical tool based on the current and previous poses of the master operator, the current pose of the surgical tool, and a pose relationship between the master operator and the surgical tool; and generating a control signal for the surgical tool to control the surgical tool to move to the target pose based on the target pose of the surgical tool; the current pose of the master operator comprises a current position of a handle of the master operator relative to a master operator base coordinate system and a current pose of the handle of the master operator relative to the master operator base coordinate system; the previous pose of the master operator comprises a previous position of the handle of the master operator relative to the master operator base coordinate system and a previous pose of the handle of the master operator relative to the master operator base coordinate system; the surgical tool comprises a flexible arm body and an end effector disposed at a distal end of the flexible arm body; the current pose of the surgical tool comprises a current position of the end effector of the surgical tool relative to a surgical tool base coordinate system and a current pose of the end effector of the surgical tool relative to the surgical tool base coordinate system; determining the current pose of the surgical tool comprises: receiving current drive information of at least one drive device of the surgical tool, wherein the at least one drive device is used to drive the flexible arm body of the surgical tool; and determining the current pose of the surgical tool based on the current drive information of the at least one drive device; determining the target pose of the surgical tool comprises determining a target position of the end effector relative to the surgical tool base coordinate system based on the previous and current positions of the handle of the master operator relative to the master operator base coordinate system, a transformation relationship between the master operator base coordinate system and a display coordinate system, a transformation relationship between a camera coordinate system and the surgical tool base coordinate system, and the current position of the end effector relative to the surgical tool base coordinate system; and / or ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ The determining the target pose of the surgical tool comprises: determining a target pose of the end effector of the surgical tool relative to the surgical tool base coordinate system based on a previous pose and a current pose of the handle of the master manipulator relative to a master manipulator base coordinate system, a transformation relationship between the surgical tool base coordinate system and the master manipulator base coordinate system, and a current pose of the end effector of the surgical tool relative to the surgical tool base coordinate system; the transformation relationship between the surgical tool base coordinate system and the master manipulator base coordinate system is determined based on a transformation relationship between the surgical tool base coordinate system and a camera coordinate system, a transformation relationship between the camera coordinate system and a display coordinate system, and a transformation relationship between the display coordinate system and the master manipulator base coordinate system; The transformation relationship between the camera coordinate system and the surgical tool base coordinate system and / or the transformation relationship between the surgical tool base coordinate system and the camera coordinate system is determined based on a transformation relationship between the surgical tool base coordinate system and a camera base coordinate system and a transformation relationship between the camera coordinate system and the camera base coordinate system; The transformation relationship between the camera coordinate system and the camera base coordinate system is determined based on driving information of a flexible arm body of the visual tool.

2. The computer-readable storage medium of claim 1, wherein, The pose relationship comprises a relationship between a pose of the surgical tool or an image of the surgical tool in the display relative to a reference coordinate system and a pose of the master manipulator relative to the reference coordinate system.

3. The computer-readable storage medium of claim 2, wherein, The pose relationship comprises at least one of: A change in position of the surgical tool or the image of the surgical tool in the display relative to the reference coordinate system is proportional to a change in position of the master manipulator relative to the reference coordinate system; or A change in pose of the surgical tool or the image of the surgical tool in the display relative to the reference coordinate system is consistent with a change in pose of the master manipulator relative to the reference coordinate system.

4. The computer-readable storage medium of claim 2, wherein, The reference coordinate system comprises a coordinate system of a space in which the master manipulator is located or a world coordinate system.

5. The computer-readable storage medium of claim 1, wherein, The display coordinate system and the camera coordinate system are consistent in definition of a field of view direction.

6. The computer-readable storage medium of claim 1, wherein, The transformation relationship between the master manipulator base coordinate system and the display coordinate system and / or the transformation relationship between the display coordinate system and the master manipulator base coordinate system is determined based on a transformation relationship between the display coordinate system and a reference coordinate system and a transformation relationship between the reference coordinate system and the master manipulator base coordinate system.

7. The computer-readable storage medium of claim 1, wherein, The surgical tool base coordinate system and the camera base coordinate system have a predetermined transformation relationship.

8. The computer-readable storage medium of claim 1, wherein, Further comprising: receiving current joint information of at least one joint of the master manipulator; and determining a current pose of the master manipulator based on the joint information of the at least one joint. Generating the control signal of the surgical tool comprises:

9. The computer-readable storage medium of claim 1, wherein, generating a driving signal for driving at least one driving device of the surgical tool based on the target pose of the surgical tool. Further comprising:

10. The computer-readable storage medium of claim 1, wherein, receiving previous joint information of at least one joint of the master manipulator; determining a previous pose of the master manipulator based on the previous joint information of the at least one joint, the previous pose comprising a previous position and a previous pose.

11. A robot system comprising: ​ a master manipulator including a robot arm, a handle disposed on the robot arm, and at least one master manipulator sensor disposed at at least one joint of the robot arm, the at least one master manipulator sensor being configured to obtain joint information of the at least one joint; a plurality of slave tools including a surgical tool and a vision tool, the surgical tool including a flexible arm body and an end instrument, the end instrument of the vision tool including a camera; at least one driving device configured to drive the flexible arm body of the slave tool; at least one driving device sensor coupled to the at least one driving device and configured to obtain driving information; and a control device communicatively coupled to the master manipulator and the at least one driving device, the control device being configured to execute the at least one instruction stored in the computer readable storage medium as claimed in any one of claims 1-10 to implement the control method of master-slave motion.

12. A computer device, the computer device comprising: a memory configured to store at least one instruction; and a processor coupled to the memory and configured to execute the at least one instruction stored in the computer readable storage medium as claimed in any one of claims 1-10 to implement the control method of master-slave motion. ​

Images