Surgical robot and method of controlling the same, computer readable storage medium

Abstract

The present disclosure relates to a surgical robot and a control method thereof, and a computer readable storage medium. The surgical robot comprises a manipulator with a joint assembly, a medical instrument being installed and driven by the manipulator; an operating part; a controller configured to: acquire a target pose of the end of the operating part when the pose of the operating part follows the pose change of the end of the surgical instrument, and acquire a current pose of the end of the medical instrument; in the case that the current pose reaches the motion boundary of the end of the medical instrument and the target pose exceeds the motion boundary, determine a first resistance acting on the end of the operating part based on the target pose and the current pose, determine a second resistance acting on the end of the manipulator based on the first resistance; determine a joint target driving force expected to be output by the joint assembly based on the second resistance, and control the joint assembly to output the joint target driving force. Through the above implementation, the manipulator can realize force feedback with a prompt effect when the operating part exceeds the motion boundary.

Description

Technical Field

[0001] This disclosure relates to the field of medical devices, and in particular to a surgical robot and its control method, and a computer-readable storage medium. Background Technology

[0002] Minimally invasive surgery refers to a surgical procedure performed inside the human body using modern medical instruments and equipment such as laparoscopes and thoracoscopes. Compared to traditional surgical methods, minimally invasive surgery has advantages such as less trauma, less pain, and faster recovery.

[0003] With advancements in technology, minimally invasive surgical robot technology has matured and is widely used. A surgical robot includes a master control panel and slave control devices. The slave control devices include multiple medical instruments with end effectors, such as imaging instruments with image end effectors and surgical instruments with manipulation end effectors. The master control panel includes a monitor and a control unit. The surgeon operates the control unit to manipulate the imaging instruments or surgical instruments within the field of view provided by the imaging instruments displayed on the monitor.

[0004] However, since the range of motion of the operating part and the range of motion of the medical device are usually inconsistent, when one of the operating part and the medical device reaches the boundary of the range of motion, continuous manipulation or being manipulated in the direction beyond the boundary can easily lead to problems that endanger safety and reliability. Summary of the Invention

[0005] Therefore, it is necessary to provide a surgical robot and its control method, as well as a computer-readable storage medium, that can ensure safety and reliability.

[0006] On the one hand, this disclosure provides a surgical robot, including:

[0007] A first manipulator, which is equipped with and drives a first medical device, has a first joint assembly;

[0008] The second manipulator is equipped with and drives the second medical device, wherein one of the first medical device and the second medical device is a surgical instrument and the other is an imaging device;

[0009] A main control panel includes an operating unit and a display. The operating unit is configured to, in response to a first manipulator moving relative to a second manipulator to a target pose, cause the endpiece of the surgical instrument to move to a second target pose in the endoscopic coordinate system of the imaging instrument, and to move to a first target pose in the display coordinate system of the display; and

[0010] The controller, coupled to the first actuator, the second actuator, and the operating unit, is configured to:

[0011] Obtain the first target pose of the end of the operating unit, and obtain the current pose of the end of the operating unit in the display coordinate system;

[0012] When the current posture reaches the motion boundary of the end of the operating part and the first target posture exceeds the motion boundary, a first resistance that is hypothetically acting on the end of the operating part is determined based on the first target posture and the current posture, and a second resistance that is actually acting on the end of the first manipulator is determined based on the first resistance. The first resistance is used to hypothetically resist the movement of the end of the operating part toward the first target posture, and the second resistance is used to actually resist the movement of the end of the first manipulator toward the target posture.

[0013] Based on the second resistance, the desired joint target driving force output by the first joint assembly is determined, and the first joint assembly is controlled to output the joint target driving force to achieve force feedback in the first manipulator;

[0014] The first resistance, the second resistance, and the joint target driving force are generalized forces.

[0015] On the other hand, this disclosure also provides a method for controlling a surgical robot, the surgical robot comprising:

[0016] A first manipulator, which is equipped with and drives a first medical device, has a first joint assembly;

[0017] The second manipulator is equipped with and drives the second medical device, wherein one of the first medical device and the second medical device is a surgical instrument and the other is an imaging device;

[0018] The main control panel includes an operating unit and a display. The operating unit is configured to respond to the movement of the first manipulator relative to the second manipulator to a target pose, resulting in the end of the surgical instrument moving to a second target pose in the endoscopic coordinate system of the imaging instrument and to a first target pose in the display coordinate system of the display.

[0019] The control method includes:

[0020] Obtain the first target pose of the end of the operating unit, and obtain the current pose of the end of the operating unit in the display coordinate system;

[0021] When the current posture reaches the motion boundary of the end of the operating part and the first target posture exceeds the motion boundary, a first resistance that is hypothetically acting on the end of the operating part is determined based on the first target posture and the current posture, and a second resistance that is actually acting on the end of the first manipulator is determined based on the first resistance. The first resistance is used to hypothetically resist the movement of the end of the operating part toward the first target posture, and the second resistance is used to actually resist the movement of the end of the first manipulator toward the target posture.

[0022] Based on the second resistance, the desired joint target driving force output by the first joint assembly is determined, and the first joint assembly is controlled to output the joint target driving force to achieve force feedback in the first manipulator;

[0023] The first resistance, the second resistance, and the joint target driving force are generalized forces.

[0024] On the other hand, this disclosure also provides a computer-readable storage medium storing a computer program configured to be loaded by a processor and execute steps implementing the control method as described in any of the above embodiments.

[0025] The surgical robot, its control method, and the computer-readable storage medium disclosed herein have the following beneficial effects:

[0026] When the posture of the end of the manipulator follows the posture change of the end of the surgical instrument, when the end of the manipulator exceeds the motion boundary, a virtual force is applied to the end and converted into a real force of the first manipulator. This virtual force is then used to control the joint assembly in the first manipulator to output a target joint driving force associated with the real force. Even when there is no and / or it is not possible to install a force sensor at the end of the manipulator to sense when the end of the manipulator exceeds the motion boundary, the operator can still have a clear force sensation at the first manipulator when the end of the manipulator exceeds the motion boundary. That is, the operator can feel the resistance caused by the end of the manipulator exceeding its boundary, thus avoiding over-manipulation of the end of the manipulator and helping to ensure safety and reliability. At the same time, since there is no need to install a force sensor at the end of the manipulator, the cost can be reduced and the structure of the end can be simplified. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the operating device of an embodiment of the surgical robot disclosed herein;

[0028] Figure 2 for Figure 1 The diagram shown is a structural schematic of a manipulator assembly from one embodiment of the operating device;

[0029] Figure 3This is a schematic diagram of the operating device of another embodiment of the surgical robot disclosed herein;

[0030] Figure 4 This is a schematic diagram of the main operating table of an embodiment of the surgical robot disclosed herein;

[0031] Figure 5 This is a schematic diagram of the operation section of an embodiment of the main control panel of this disclosure;

[0032] Figure 6 for Figure 5 A partial structural schematic diagram of one embodiment of the operating unit shown;

[0033] Figure 7 This is a flowchart of an embodiment of the control method for the surgical robot disclosed herein;

[0034] Figure 8 This is a flowchart of an embodiment of the control method for the surgical robot disclosed herein;

[0035] Figure 9 This is a flowchart of an embodiment of the control method for the surgical robot disclosed herein;

[0036] Figure 10 This is a flowchart of an embodiment of the control method for the surgical robot disclosed herein;

[0037] Figure 11 This is a flowchart of an embodiment of the control method for the surgical robot disclosed herein;

[0038] Figure 12 This is a flowchart of an embodiment of the control method for the surgical robot disclosed herein;

[0039] Figure 13 This is a flowchart of an embodiment of the control method for the surgical robot disclosed herein;

[0040] Figure 14 This is a partial schematic diagram of a surgical robot according to an embodiment of the present disclosure during the surgical procedure.

[0041] Figure 15 This is a partial schematic diagram of the surgical robot of another embodiment of the present disclosure in a surgical state;

[0042] Figure 16 This is a flowchart of an embodiment of the control method for the surgical robot disclosed herein;

[0043] Figure 17 This is a schematic diagram of an embodiment of the display interface related to the transboundary section in the surgical robot of this disclosure;

[0044] Figure 18 This is a schematic diagram of another embodiment of the display interface related to the transboundary section in the surgical robot of this disclosure;

[0045] Figure 19 This is a schematic diagram of another embodiment of the display interface related to the transboundary section in the surgical robot of this disclosure;

[0046] Figure 20 This is a schematic diagram of another embodiment of the display interface related to the transboundary section in the surgical robot of this disclosure;

[0047] Figure 21 This is a schematic diagram of another embodiment of the display interface related to the transboundary section in the surgical robot of this disclosure;

[0048] Figure 22 This is a schematic diagram of another embodiment of the display interface related to the transboundary section in the surgical robot of this disclosure;

[0049] Figure 23 This is a schematic diagram of another embodiment of the display interface related to the transboundary section in the surgical robot of this disclosure;

[0050] Figure 24 This is a schematic diagram of another embodiment of the display interface related to the transboundary section in the surgical robot of this disclosure;

[0051] Figure 25 This is a flowchart of an embodiment of the control method for the surgical robot disclosed herein;

[0052] Figure 26 This is a schematic diagram of an embodiment of the display interface related to the transboundary section in the surgical robot of this disclosure;

[0053] Figure 27 This is a flowchart of an embodiment of the control method for the surgical robot disclosed herein;

[0054] Figure 28 This is a schematic diagram of an embodiment of the display interface related to the transboundary section in the surgical robot of this disclosure;

[0055] Figure 29 This is a flowchart of an embodiment of the control method for the surgical robot disclosed herein;

[0056] Figure 30 A flowchart illustrating another embodiment of the control method for the surgical robot disclosed herein;

[0057] Figure 31 This is a schematic diagram of the structure of a controller for a surgical robot according to an embodiment of the present disclosure. Implementation

[0058] To facilitate understanding of this disclosure, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of this disclosure are shown in the drawings. However, this disclosure can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to enable a more thorough and complete understanding of the disclosure.

[0059] It should be noted that when an element is referred to as being "set on" another element, it can be directly on the other element or there may be an intervening element. When an element is considered to be "connected" to another element, it can be directly connected to the other element or there may be an intervening element. When an element is considered to be "coupled" to another element, it can be directly coupled to the other element or there may be an intervening element. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this disclosure are for illustrative purposes only and do not represent the only possible embodiments. The terms "distal" (i.e., end) and "proximal" used in this disclosure are directional terms commonly used in the field of interventional medical devices, where "distal" refers to the end away from the operator during surgery, and "proximal" refers to the end closer to the operator during surgery. The terms "first / second," etc., used in this disclosure can refer to a component or two or more components of a class having common characteristics.

[0060] Unless otherwise defined, all technical and scientific terms used in this disclosure have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of this disclosure. The term "and / or" as used in this disclosure includes any and all combinations of one or more of the associated listed items. The terms "each" and "a plurality" as used in this disclosure include one or more.

[0061] Surgical robots encompass various types, including laparoscopic surgical robots for thoracic and abdominal surgeries, and endoscopic surgical robots for natural cavity surgeries such as bronchial surgeries. These types of surgical robots typically include two or more relatively independent arms with articulated components, where the movement of one arm can serve as input to control the associated movement of the other arm. The control methods disclosed herein are applicable to surgical robots with two or more arms, where the movement of one arm can cause the movement of the other arm.

[0062] In some embodiments, a laparoscopic surgical robot includes slave operating devices and a master operating console that can manipulate the slave operating devices. Different slave operating devices can be operated by a master operating console of the same structure or by a master operating console of different structures; therefore, the specific type of laparoscopic surgical robot is not usually distinguished by the master operating console. However, the specific type of laparoscopic surgical robot can be distinguished based on the different structures of the slave operating devices. For example, a laparoscopic surgical robot with multiple medical instruments inserted into the patient through different surgical channels provided by a single trocar connected to the patient can be called a single-port laparoscopic surgical robot. As another example, a laparoscopic surgical robot with multiple medical instruments inserted into the patient through multiple surgical channels provided by different trocars connected to the patient can be called a multi-port laparoscopic surgical robot. These medical instruments typically include imaging instruments and surgical instruments; the imaging instruments provide a field of view, and the surgical instruments are used for surgical operations such as cutting and suturing.

[0063] Figure 1 This illustrates a single-port laparoscopic surgical robot. (Example) Figure 1 As shown, in a single-port laparoscopic surgical robot, the operating device 100 includes a drive arm, which comprises a main arm 110 and a manipulator assembly 120 connected in sequence. Both have joint components, and the main arm 110 can adjust the position and / or posture of the manipulator assembly 120. (Combined with...) Figure 2 Referring to the description, the manipulator assembly 120 includes a housing 130 connected to the end of the main arm 110, a manipulator 140 housed within the housing 130, and a medical device 150 detachably mounted on the manipulator 140. Multiple manipulators 140 are included, each housed within the housing 130 and connected in parallel to the end of the main arm 110 via the housing 130. Movement of the main arm 110 can cause coordinated movement of the multiple manipulators 140 in position and / or orientation. Typically, the manipulator 140 has at least one articulated assembly to provide, for example, a feed degree of freedom in the operating space to adjust the depth of insertion of the medical device 150 into the patient. The medical device 150 may include multiple components, typically the same number as or fewer than the number of manipulators 140. The medical device 150 also includes multiple joint assemblies. Driven by the manipulators 140, the joint assemblies of the medical device 150 can provide multiple degrees of freedom in the operating space to achieve, for example, all degrees of freedom except for the feed degree of freedom, including horizontal translation, vertical translation, rotation, yaw, and pitch. The feed, horizontal translation, and vertical translation degrees of freedom are the three positional degrees of freedom in the operating space, while the rotation, yaw, and pitch degrees of freedom are the three attitude degrees of freedom in the operating space. Figure 1 In this process, multiple medical devices 150 are inserted into the patient's body through the same puncture device 400.

[0064] Figure 3 This illustrates a multi-port laparoscopic surgical robot. For example... Figure 3 As shown, in the multi-port laparoscopic surgical robot, the operating device 200 includes a drive arm, which comprises a main arm 210, an adjusting arm 220, and a manipulator assembly 230 connected in sequence, all three having joint components. Multiple adjusting arms 220 are included, each with its proximal end connected to the end of the main arm 210. The end of the main arm 210 includes a directional platform 215, to which the proximal end of each adjusting arm 220 is connected. The main arm 210 can adjust the position and / or orientation of the adjusting arms 220 and the manipulator assembly 230. Multiple manipulator assemblies 230 are included, the number being the same as the number of adjusting arms 220. The proximal end of each manipulator assembly 230 is connected to the end of the adjusting arm 220, and the adjusting arm 220 can adjust the position and / or orientation of the manipulator assembly 230. The manipulator assembly 230 includes a manipulator 240 and a medical device 250 detachably mounted on the manipulator 240, both of which include multiple joint components. Manipulator 240 includes a parallelogram mechanism. Utilizing the parallelogram principle, manipulator 240 can be limited to rotational movement around a remote center of motion (RC). The joint assembly of manipulator 240 can include multiple degrees of freedom in the operating space, such as feed, yaw, and pitch, with the yaw and pitch degrees of freedom both rotating around the RC. The joint assembly of medical device 250 can provide multiple degrees of freedom in the operating space, such as rotation, yaw, and pitch, driven by manipulator 240. Specifically, the feed degree of freedom of manipulator 240 adjusts the depth of insertion of medical device 250 into the patient. The yaw and pitch degrees of freedom of manipulator 240 can affect both the position and orientation of medical device 250. The rotation, yaw, and pitch degrees of freedom of medical device 250 primarily affect the orientation, while their impact on the position can be ignored. Figure 2 In this process, multiple medical devices 250 are inserted into the patient's body through different puncture instruments 500.

[0065] In some embodiments, such as Figure 4 The main control panel 300 shown can be connected with Figure 1 The operating device 100 shown is or Figure 2 The setup shown is relatively independent of the operating device 200. They can be set up close together, for example, in the same room, within a few meters of each other; or they can be set up far apart, for example, in different cities, thousands of kilometers apart.

[0066] Continue reading Figure 4The main control panel 300 may include an operating unit 310 and a display. A main control panel 300 may include more than one operating unit 310, for example, two. The operating unit 310 may include multiple joint assemblies, and the operating unit 310 with multiple joint assemblies can be regarded as an arm in a surgical robot.

[0067] Please see Figure 5 and Figure 6 The operating unit 310 may include a connector 101, a base 12, an operating component 15, and at least one drive arm, which are rotatably connected in sequence. The end of the connector 101 away from the base 12 is connected to a relevant component of the main operating console 300. The operating component 15 is configured to receive operations from the operator and may include a handle 151. The user performs relevant operations on the operating unit 310 by gripping the handle 151 to send control commands to the slave operating device 100 or slave operating device 200. The end of the operating unit 310 mentioned in this disclosure may, for example, refer to the handle 151.

[0068] The number of drive arms can be determined according to the actual execution requirements. For example, when the motion to be performed is relatively simple, one drive arm can be selected. One end of the drive arm is connected to the base 12, and the other end of the drive arm is connected to the operating component 15. When the motion to be performed is more complex, two or more drive arms can be selected. The more drive arms there are, the higher the degree of freedom, and the more complex operations can be performed.

[0069] This embodiment uses two drive arms as an example. The two drive arms are the first arm 13 and the second arm 14. The second arm 14 is rotatably connected to the first arm 13. The end of the first arm 13 away from the second arm 14 is rotatably connected to the base 12. The end of the second arm 14 away from the first arm 13 is rotatably connected to the operating component 15. The rotation of the first arm 13 and the second arm 14 provides the operating component 15 with at least two degrees of freedom of movement.

[0070] In this embodiment, the base 12 rotates relative to the connector 101 along a first axis A1, the first arm 13 rotates relative to the base 12 along a second axis A2, and the second arm 14 rotates relative to the first arm 13 along a third axis A3. The first axis A1 is perpendicular to the second axis A2 and the third axis A3, and the second axis A2 and the third axis A3 are parallel, thus allowing the rotation of the base 12, the first arm 13, and the second arm 14 to enable the movement of the operating component 15 in three-dimensional space.

[0071] like Figure 6As shown, the operating unit 310 may further include a first gravity compensation mechanism 200, which is connected between the base 12 and the first arm 13 to generate a torque that balances the gravitational torque of the parallelogram mechanism in the first degree of freedom of rotation about the second axis A2. In some embodiments, the first gravity compensation mechanism 20 may include a first rotation mechanism 201 and a first elastic compensation mechanism 202. The first rotation mechanism 201 may include a plurality of rotating parts, which are distributed on the first mounting plate 123 of the base 12 and the first arm 13.

[0072] For ease of understanding, the first rotating mechanism 201 may include a first rotating portion and a second rotating portion. For example, the first rotating portion is disposed on the first mounting plate 123, and the second rotating portion is disposed on the arm turntable 132 of the first arm 13. The first elastic compensation mechanism 202 is coupled between the body turntable of the base 12 and the first rotating mechanism 201 to generate a torque in the first degree of freedom that balances the gravitational torque of the parallelogram mechanism, allowing the user to easily drag the operating part 310 in the first degree of freedom. In some embodiments, the endoscopic surgical robot may also include a master operating table and a slave operating device. The slave operating device includes two robotic arms and catheter instruments detachably mounted at the ends of the two robotic arms. The catheter instruments include an inner catheter instrument and an outer catheter instrument. The outer catheter instrument is mounted on one of the two robotic arms, and the inner catheter instrument is mounted on the other. During operation, the outer and inner catheter instruments face the same direction, and the inner catheter instrument is inserted into the outer catheter instrument in the same direction for use.

[0073] In some embodiments, the surgical robot also includes a controller. For example, the controller may be deployed on the master console 300. Alternatively, the controller may be deployed on the slave operating device 100 (or 200). Or, the controller may be deployed in the cloud. For another example, the controller may include a first controller and a second controller, with the first controller deployed on the master console 300 and the second controller deployed on the slave operating device 100 (or 200); or, the first controller may be deployed in the cloud, and the second controller may be deployed on the master console 300 or the slave operating device 100 (or 200). Yet another example is that the controller includes a first controller, a second controller, and a third controller, with the first controller deployed on the master console 300, the second controller deployed on the slave operating device 100 (or 200), and the third controller deployed in the cloud. The controller includes one or more processors, and the control method of this disclosure can have multiple steps executed in one processor of the controller or in multiple processors of the controller.

[0074] The controller can typically be coupled to any electrical component. For example, the controller can be coupled to the operating unit 310, the display, and the drive arm, respectively. The coupling of the controller to the drive arm includes coupling the controller to any part of the drive arm.

[0075] In single-port laparoscopic surgical robots, the control method disclosed herein is applicable to surgical robots having two or more arms, wherein movement of one arm can cause movement of another arm. One arm is selected from the operating section 310 of the autonomous operating table 300, and the other arm is selected from at least a portion of the drive arm from the operating device 100, such as the manipulator assembly 120, which includes a manipulator 140 and a medical device 150.

[0076] In multi-port laparoscopic surgical robots, the control method disclosed herein is applicable not only to the operation section 310 of the autonomous operating table 300, where one arm is selected from at least a portion of the drive arm of the operating device 200, such as the manipulator assembly 230, including the manipulator 240 and the medical device 250; but also to the two arms respectively selected from two portions of the drive arm of the operating device 200, such as two manipulator assemblies 230, or two adjusting arms 220 and their respective connected manipulator assemblies 230.

[0077] In endoscopic surgical robots, the control method disclosed herein is applicable to two arms, each selected from at least a portion of two robotic arms in the operating device.

[0078] For example, in the aforementioned single-port or multi-port laparoscopic surgical robot, the operating unit 310 of the main operating table 300 can be configured to manipulate (i.e., control) the movement of any part of the drive arm. Specifically, the drive arm can be configured to accept manipulation by the operating unit 310 according to surgical requirements. In some embodiments, the movement of any part of the drive arm can also react on the operating unit 310, causing a corresponding movement of the operating unit 310.

[0079] For example, in the aforementioned multi-port laparoscopic surgical robot, the drive arm from the manipulator 200 can be configured to include two independent arm bodies, where manipulating (e.g., dragging) one arm body will cause a coordinated movement of the other arm body. Such a configuration can also be applied to endoscopic surgical robots, where manipulating one robotic arm will cause a coordinated movement of the other robotic arm.

[0080] In some embodiments, such as in a laparoscopic surgical robot, at least two operating modes can be configured, including a master-slave follow mode and a master-slave alignment mode. The master-slave follow mode is primarily used for intraoperative surgical operations, where the operating unit 310 of the master operating table 300 outputs position and / or posture commands, controlling the end-effectors of at least a portion of the drive arms of the slave operating device 100 (or 200) to adjust their position and / or posture based on these commands. The master-slave alignment mode is primarily used for preoperative preparation to facilitate rapid entry into the master-slave follow mode. In this mode, at least a portion of the drive arms of the slave operating device 100 (or 200) outputs posture commands related to the posture of the end-effectors of that portion of the arm, controlling the operating unit 310 of the master operating table 300 to adjust its posture based on these commands to align with the posture of the aforementioned end-effectors, eliminating the need for manual alignment and facilitating intuitive operation by the operator. The end of at least a portion of the arm body in the drive arm can be configured from any of the components of the drive arm. In one embodiment, the end can generally refer to the end of a medical device, and more specifically, to the end effector of a medical device.

[0081] Whether in the master-slave follow mode or the master-slave alignment mode described above, this disclosure can define the arm that generates commands such as position commands and / or attitude commands through motion as the active arm, and define the arm that moves based on these commands as the driven arm. Based on this, at least one aspect of the control method of this disclosure aims to include, when an operator controls the movement of the driven arm by the active arm, if the movement of the driven arm exceeds its corresponding movement boundary, generating force feedback in the active arm to prompt the operator. This method does not require any force or torque sensors in the driven arm to detect resistance or resistance torque experienced by the driven arm, yet it can still generate appropriate force feedback in the active arm.

[0082] The control method of the surgical robot disclosed herein will be illustrated below from the perspective of the operation mode.

[0083] In some embodiments, the control method of this disclosure is applicable to single-port laparoscopic surgical robots and multi-port laparoscopic surgical robots in master-slave follow mode. In master-slave follow mode, the operating unit typically manipulates the manipulator assembly 120 (or 230) to control the end effector of the medical device 150 (or 250).

[0084] In some embodiments, see Figure 7 The control method includes:

[0085] Step S101: Obtain the first current pose and the first target pose of the end effector of the medical device.

[0086] The current pose is the pose of the medical device's end effector at the current moment, i.e., the actual pose; the target pose, such as the first target pose, is the desired pose of the medical device's end effector at the next moment.

[0087] The drive arm is equipped with sensors that detect the joint variables of each joint component. For example, when the drive arm includes translational joint components, the joint variables include joint displacement values; when the drive arm includes rotational joint components, the joint variables include joint angle values.

[0088] The concept of "pose" is usually described in the operational space, namely the Cartesian space or task space.

[0089] The first current pose can be obtained by sensing joint variables in joint components of manipulators and medical devices, and combining joint variables with positive kinematics.

[0090] Teleoperation of surgical robots can employ incremental pose control. The first target pose of the end effector can be determined based on the first current pose and the first incremental pose of the end effector. The first incremental pose can be determined based on the second incremental pose between the second current pose of the manipulator and the second target pose. Since the second and first current poses are known, considering only variables, it can be simply understood that the first target pose is determined based on the second target pose.

[0091] Incremental pose can include incremental position and incremental attitude. When using incremental pose control, the incremental position can be scaled to ensure surgical safety, while the incremental attitude is not specially processed to maintain attitude tracking.

[0092] Step S102: Detect whether the first current pose has reached the motion boundary, and detect whether the first target pose has exceeded the motion boundary.

[0093] Due to physical or software limitations, the ends of medical devices often have defined motion boundaries.

[0094] If the first current pose reaches the motion boundary and the first target pose exceeds the motion boundary, proceed to step S103; otherwise, proceed to step S107.

[0095] Step S103: Determine the first resistance that is hypothetical to act on the end of the medical device based on the first target pose and the first current pose.

[0096] "Imaginary" means virtual, and also means that it does not actually exist. Therefore, the first resistance is a virtual force, used to hypothetically resist the movement of the medical device's end effector toward the first target pose, that is, it will not actually resist the movement of the medical device's end effector toward the first target pose.

[0097] Step S104: Determine the second resistance that actually acts on the end of the operating part based on the first resistance.

[0098] The second resistance is a real force that actually exists and will act on the end of the operating part to actually resist the movement of the end of the operating part toward the second target pose.

[0099] Step S105: Determine the target joint driving force output by the joint assembly in the desired operating part based on the second resistance.

[0100] In step S105, the target driving force of the joint can be determined using static equations. Static equations are typically expressed as:

[0101] .

[0102] in, Indicates the target driving force of the joint; The transpose of the force Jacobian matrix of the manipulator can be obtained by considering the kinematic Jacobian matrix of the robot. The Jacobian matrix is ​​obtained by transposing it. The joint variables can be determined in real time based on the joint components of the operating unit; This indicates the second resistance level.

[0103] Step S106: The joint assembly in the control operation unit outputs the target driving force of the joint.

[0104] Typically, the joint assembly may include a drive motor as a drive mechanism. The drive motor includes three control modes: position mode, speed mode, and torque mode. In step S106, the torque mode is used to control the joint assembly to output the target driving force, thereby providing force feedback at the operating part and prompting the operator not to move the operating part towards the second target pose.

[0105] Step S107: Determine the target joint variables of the manipulator and the joint components in the medical device based on the first target pose.

[0106] Among these methods, the target joint variables can be determined using inverse kinematics.

[0107] Step S108: Control the joint components in the manipulator and medical device to output the target joint variable.

[0108] In step S108, the position mode control joint assembly outputs target joint variables so that the movement of the end of the medical device is not restricted when the first current pose of the end of the medical device does not reach the motion boundary and / or the first target pose does not exceed the motion boundary.

[0109] The various forces involved in this disclosure, such as the first resistance, the second resistance, and the joint target driving force, can all be generalized forces, which include at least one of force and torque.

[0110] In some embodiments, the motion boundary of the distal end of the medical device includes at least one of the positional motion boundary and the postureal motion boundary of the operating space. Correspondingly, the acquired first current pose includes at least one of the current position and the current posture, and the acquired first target pose includes at least one of the target position and the target posture. The positional out-of-bounds problem and the postureal out-of-bounds problem can be handled separately. For example, positional out-of-bounds refers to one or more positional degrees of freedom of the distal end of the medical device exceeding the positional motion boundary jointly determined by the motion range of all positional degrees of freedom of the manipulator assembly; postureal out-of-bounds refers to one or more postureal degrees of freedom of the distal end of the medical device exceeding the postureal motion boundary jointly determined by the motion range of all postureal degrees of freedom of the manipulator assembly.

[0111] For example, for the position out-of-bounds problem, step S102 above may include detecting whether the current position has reached the position movement boundary and detecting whether the target position has exceeded the position movement boundary.

[0112] Wherein, when the current position reaches the position movement boundary and the target position exceeds the position movement boundary, step S103 may further include controlling the end effector of the medical device to maintain a first current pose. By controlling the end effector of the medical device to maintain a first current pose, damage or safety issues caused by excessive manipulation of the manipulator and / or the medical device can be prevented.

[0113] In some embodiments, the step of controlling the end of the medical device to maintain a first current pose may include controlling the end of the medical device to maintain a current position.

[0114] In some embodiments, when the first current pose further includes the current posture and the target pose further includes the target posture, the above-described step of controlling the end of the medical device to maintain the first current pose may include controlling the end of the medical device to maintain the current position and the current posture.

[0115] In some embodiments, when the current posture has not reached the motion boundary and / or the target posture has not exceeded the motion boundary, the step of controlling the end of the medical device to maintain the first current posture may also include controlling the end of the medical device to maintain the current position and controlling the end of the medical device to move according to the target posture, so that the end of the medical device is always aligned with the posture of the operating part.

[0116] Specifically, step S103, which involves determining the hypothetical first resistance acting on the distal end of the medical device based on the first target pose and the first current pose, may include determining the hypothetical first resistance acting on the distal end of the medical device based on the target position and the current position. This method may include acquiring a motion boundary impedance model of a first position and determining the first resistance based on the target position, the current position, and the motion boundary impedance model of the first position.

[0117] For example, for the attitude out-of-bounds problem, step S102 above may include detecting whether the current attitude has reached the attitude motion boundary and detecting whether the target attitude has exceeded the attitude motion boundary.

[0118] Specifically, when the current posture reaches the posture motion boundary and the target posture exceeds the posture motion boundary, the step of controlling the end effector of the medical device to maintain the first current posture may include controlling the end effector of the medical device to maintain the current posture.

[0119] In some embodiments, when the first current pose further includes the current position and the target pose further includes the target position, the step of controlling the end effector of the medical device to maintain the current pose may include controlling the end effector of the medical device to maintain both the current pose and the current position.

[0120] In some embodiments, when the current posture has not reached the motion boundary and / or the target posture has not exceeded the motion boundary, the step of controlling the end of the medical device to maintain the current posture may also include controlling the end of the medical device to maintain the current posture and follow the target position, and then only the posture needs to be adjusted.

[0121] Specifically, step S103 may include determining a hypothetical first resistance acting on the distal end of the medical device based on the target posture and the current posture. This method may include acquiring a first posture motion boundary impedance model and determining the first resistance based on the target posture, the current posture, and the first posture motion boundary impedance model.

[0122] In the above embodiments, when only the position exceeds the limit, the first resistance is only related to the position; when only the attitude exceeds the limit, the first resistance is only related to the attitude; when both the position and attitude exceed the limit, the first resistance can be a superposition of the resistance related to the position and the resistance related to the attitude.

[0123] In some embodiments, step S104 described above may include a process of converting the first resistance from a first coordinate system at the end of the medical device to a second coordinate system at the end of the operating part. This conversion process may include:

[0124] The first resistance at the tip of the medical device in the first coordinate system is transformed into a first intermediate resistance at the tip of the medical device in the first intermediate coordinate system. This first intermediate resistance is a hypothetical force, or virtual force.

[0125] The second intermediate resistance of the operating part in the second intermediate coordinate system is determined based on the first intermediate resistance of the distal end of the medical device in the first intermediate coordinate system. This second intermediate resistance is an actual force, i.e., a real force.

[0126] The end of the operating part is converted into the second intermediate resistance in the second intermediate coordinate system and then into the second resistance in the second coordinate system.

[0127] For example, the first coordinate system includes the base coordinate system of the medical device, for example, the base coordinate system of the medical device is configured at the end face of the medical device connected to the manipulator at the proximal end; the first intermediate coordinate system includes, for example, the tool coordinate system of the end effector in the imaging device, which is sometimes also called the endoscope coordinate system, for example, the endoscope coordinate system is configured at the distal end face of the end effector in the imaging device; the second intermediate coordinate system includes, for example, the display coordinate system is configured at the display surface of the display; the second coordinate system includes, for example, the base coordinate system of the operating unit, for example, the base coordinate system of the operating unit is configured at the position where the proximal end of the operating unit is connected to the main operating table.

[0128] In master-slave operation mode, the second resistance obtained through the above coordinate transformation process can provide the operator with intuitive force feedback during operation. Here, "intuition" includes the fact that the direction of movement of the end effector observed by the doctor on the monitor is consistent with the direction of movement of the doctor's operating part, and the direction of force on the end effector seen by the doctor's eyes in the operation image displayed on the monitor is basically consistent with the direction of force felt by the doctor's operating part, thus providing a near-intuitive experience in terms of vision and touch.

[0129] In some embodiments, determining the second resistance actually acting on the end of the operating part based on the first resistance may only include the coordinate transformation process described above. That is, the second resistance determined by the first resistance may not require a change in the magnitude of the resistance; in other words, the magnitude of the second resistance may be the same as the magnitude of the first resistance.

[0130] In some embodiments, the magnitude of the second resistance may differ from that of the first resistance; it may include being completely different or partially different. That is, determining the second resistance actually acting on the end of the operating part based on the first resistance may also include a resistance magnitude transformation process. This resistance magnitude transformation process primarily modifies the relatively gradual change in the first resistance into a relatively drastic change in the second resistance, to, for example, enhance the force sensation experienced by the operator.

[0131] In some embodiments, for the position out-of-bounds problem, the process of transforming the magnitude of the resistance may include: obtaining a second position motion boundary impedance model, and determining the second resistance based on the first resistance and the second position motion boundary impedance model.

[0132] Preferably, the second resistance determined by the second position motion boundary impedance model has an improvement in slope and / or rate of change of slope compared to the first resistance determined by the first position motion boundary impedance model. This improvement includes at least a local increase in slope. For example, the improvement may include a local decrease in slope or a local increase in slope.

[0133] For example, the boundary impedance model for the first position motion is expressed by the following formula:

[0134] ,

[0135] In formula (1), Indicates the first resistance; Indicates the spring constant; This indicates the current position of the end effector of the medical device, and that it has reached the positional movement boundary. This indicates the target position of the distal end of the medical device, which extends beyond the positional movement boundary. For example, The value range is between [0, 10].

[0136] For example, the boundary impedance model for the second position motion is expressed as follows:

[0137] (2)

[0138] In formula (2), Indicates the second resistance; Indicates the modulus of the first resistance; Indicates the spring constant; This represents the stiffness coefficient, which reflects the stiffness of the impedance model. This represents the maximum resistance threshold of the second resistance, reflecting the strength of the impedance model. Among them, Indicates the magnitude of the second resistance; This indicates the direction of the second resistance. In formula (2), It cannot be infinitely large. The boundary passes through The constraint can be determined based on the operator's force requirements for the operating part and the maximum driving force that the joint components in the operating part can provide.

[0139] For example, The value range is between [0, 10]. The value range is between [0, 5]. The value range is between [0, 5].

[0140] In some embodiments, for attitude out-of-bounds problems, the process of transforming the magnitude of the drag may include: obtaining a second attitude motion boundary impedance model, and determining the second drag based on the first drag and the second attitude motion boundary impedance model.

[0141] Preferably, the second drag determined by the second attitude motion boundary impedance model has an improvement in slope and / or rate of change of slope compared to the first drag determined by the first attitude motion boundary impedance model. This improvement includes at least a local increase in slope. For example, the improvement may include a local decrease in slope or a local increase in slope.

[0142] For example, the boundary impedance model for the first attitude motion is expressed as follows:

[0143] (3)

[0144] In formula (3), This indicates the first resistance; This indicates the deviation between the target attitude and the current attitude; represents an antisymmetric matrix; V represents the operator that converts an antisymmetric matrix into a vector. Here, the deviation... The following formula can be used to determine this.

[0145] (4)

[0146] In formula (4), Indicates the target's attitude; express The reverse; This indicates the current attitude, and that it has reached the attitude boundary.

[0147] For example, the boundary impedance model for the second attitude motion is expressed as follows:

[0148] (5)

[0149] In formula (5), Indicates the second resistance; Indicates the modulus of the first resistance; Indicates the spring constant; Indicates the stiffness coefficient; This represents the maximum resistance threshold of the second resistance. Wherein, Indicates the magnitude of the second resistance; This indicates the direction of the second resistance. In formula (5), It cannot be infinitely large. The boundary passes through The constraint can be determined based on the operator's force requirements for the operating part and the maximum driving force that the joint components in the operating part can provide.

[0150] For example, The value range is between [0, 10]. The value range is between [0, 5]. The value range is between [0, 1].

[0151] In some embodiments, the second resistance actually acting on the operating part includes a portion of the second resistance determined by the position exceeding the position movement boundary and a portion of the second resistance determined by the attitude exceeding the attitude movement boundary. In step S105 above, the portion of the second resistance determined by the position exceeding the position movement boundary and the portion of the second resistance determined by the attitude exceeding the attitude movement boundary can be superimposed to determine the final second resistance actually acting on the operating part, and then the driving joint force of the joint assembly in the operating part can be determined based on the second resistance obtained by superposition. In step S105 above, a portion of the driving joint force of the joint assembly in the operating part can also be determined based on the portion of the second resistance determined by the position exceeding the position movement boundary and the portion of the driving joint force of the joint assembly in the operating part can also be determined based on the portion of the second resistance determined by the attitude exceeding the attitude movement boundary, and the two portions of driving joint force can be superimposed to obtain the final driving joint force of the joint assembly in the operating part.

[0152] In some embodiments, in robotics, the manipulator space and the joint space can be interchanged. For example, given that the joint variables of the joint components of the manipulator and the medical device are known in the joint space, the pose of the end effector of the medical device in the manipulator space can be determined by combining forward kinematics; given that the pose of the end effector of the medical device in the manipulator space is known, the joint variables of the joint components of the manipulator and the medical device in the joint space can be determined by combining inverse kinematics. It is understood that the first current pose of the end effector in the manipulator space reaching a boundary or the first target pose exceeding a boundary can be mapped to the manipulator and the medical device in the joint space reaching or exceeding the joint motion boundary of one or more joint components. Therefore, this disclosure can also explore the problem of the end effector's pose exceeding the boundary from the perspective of the joint space.

[0153] In some embodiments, for the pose out-of-bounds problem, step S102 above may include:

[0154] The target joint variables of the manipulator and the medical device are determined based on the first target pose, and the current joint variables of the manipulator and the medical device are determined based on the first current pose.

[0155] Identify a target joint assembly from the joint assemblies of the manipulator and medical device that satisfies a first condition, the first condition including that the current joint variable of the target joint assembly reaches the joint motion boundary of the target joint assembly and that the target joint variable exceeds the joint motion boundary of the target joint assembly.

[0156] If there is a target joint component that meets the first condition mentioned above, it indicates that in step S102, a situation was detected where the first current pose reaches the motion boundary and the first target pose exceeds the motion boundary.

[0157] See Figure 8 The above step S103 may include:

[0158] Step S1031: Based on the target joint variables and current joint variables of the target joint component, determine the hypothetical joint driving force acting on the target joint component.

[0159] Step S1032: Assign zero to the joint driving force of non-target joint components in the joint assemblies of the manipulator and medical device that do not meet the first condition.

[0160] Step S1033: Based on the joint driving force of the target joint component and the joint driving force of the non-target joint component, determine the first resistance hypothetically acting on the end of the medical device.

[0161] In one embodiment, the above-mentioned step S1031 implementation method may include obtaining the joint boundary impedance model, and combining the target joint variable of the target joint component, the current joint variable, and the joint boundary impedance model to determine the hypothetical joint driving force acting on the target joint component.

[0162] For example, the joint boundary impedance model can be expressed as follows:

[0163] (6)

[0164] In formula (6), This represents the hypothetical joint driving force acting on the i-th target joint component; Indicates the spring constant; Indicates the stiffness coefficient; Represents the target joint variable of the i-th target joint component; The positive value represents the joint motion boundary of the i-th target joint component (corresponding to the current joint variable when the target joint reaches the joint motion boundary); This represents the negative value of the joint motion boundary of the i-th target joint component (corresponding to the current joint variable when the target joint reaches the joint motion boundary). For example, The value range is between [0, 10]. The value range is between [0, 5].

[0165] In step S1033 above, the joint driving forces of the manipulator and the medical device's joint components can be combined into a force vector, expressed in the following matrix form:

[0166] (7)

[0167] In formula (7), This represents the force vector formed by the combination of joint driving forces of the joint components of a manipulator and medical device.

[0168] Then, by directly deriving the inverse statics solution, the first resistance at the end of the medical device in the operating space can be obtained based on the joint driving force of the joint components of the manipulator and the medical device in the joint space. The solution formula is expressed as:

[0169] (8)

[0170] In formula (8), This indicates the first resistance at the tip of the medical device within the operating space; Represents the inverse of the transpose of the Lijacobi matrix for manipulators and medical devices; The transpose of the force Jacobian matrix representing the manipulator and medical device can be obtained by considering the kinematic Jacobian matrix of the robot. The Jacobian matrix is ​​obtained by transposing it. Joint variables can be determined in real time based on joint components of manipulators and medical devices.

[0171] In some embodiments, while both single-port and multi-port laparoscopic surgical robots require the use of manipulators and medical instruments during surgery, single-port laparoscopic surgical robots generally rely more on the greater degrees of freedom and range of motion provided by the medical instruments themselves to flexibly perform surgery compared to multi-port laparoscopic surgical robots. For example, the medical instruments of single-port laparoscopic surgical robots typically include more joint components than those of multi-port laparoscopic surgical robots, providing greater degrees of freedom and a wider range of motion. Therefore, preferably, when configuring the parameters of the aforementioned motion boundary impedance models, the stiffness and / or strength of the single-port laparoscopic surgical robot can be configured to be correspondingly lower than that of the multi-port laparoscopic surgical robot to provide the operator with better force feedback.

[0172] In some embodiments, at least one of the types of surgical robots and medical devices can be identified. The type of either the surgical robot or the medical device reflects the range of motion of the medical device. The control method of this disclosure can also utilize this feature to automatically recommend or configure parameters of various motion boundary impedance models for different types of surgical robots or medical devices. The control method of this disclosure may further include:

[0173] Identify the type of surgical robot or medical device. Surgical robots include single-port laparoscopic surgical robots and multi-port laparoscopic surgical robots. Medical devices include those suitable for single-port laparoscopic surgical robots and those suitable for multi-port laparoscopic surgical robots.

[0174] When the identified surgical robot is a single-port laparoscopic surgical robot or the identified medical device is suitable for a single-port laparoscopic surgical robot, at least one of a first value of stiffness coefficient and a first value of maximum resistance threshold is recommended or configured for at least one motion boundary impedance model. When the identified surgical robot is a multi-port laparoscopic surgical robot or the identified medical device is suitable for a multi-port laparoscopic surgical robot, at least one of a second value of stiffness coefficient and a second value of maximum resistance threshold is recommended or configured for at least one motion boundary impedance model. The second value of stiffness coefficient is typically greater than the first value of stiffness coefficient, and the second value of maximum resistance threshold is greater than the first value of maximum resistance threshold.

[0175] The method for configuring the parameters of the moving boundary impedance model is applicable, for example, to formulas (2), (5), and (6) above. For example, it can be applied to the determination of the stiffness coefficient and the maximum drag threshold in formulas (2) and (5). For example, it can be applied to the determination of the stiffness coefficient in formula (6).

[0176] For example, for the above formula (2):

[0177] If the identified device is a multi-port laparoscopic surgical robot or a medical device suitable for multi-port laparoscopic surgical robots, the stiffness coefficient... For example, a value of 1.5 could be used, representing the maximum resistance threshold. For example, a value of 3N (or N / m) could be used. If the identified device is a single-port laparoscopic surgical robot or a medical device suitable for single-port laparoscopic surgical robots, the stiffness coefficient... For example, a value of 1.2 could be used, representing the maximum resistance threshold. For example, a value of 0.8 N (or N / m) can be taken. Furthermore, the spring constant of both can take any suitable value, for example, both can be 1.

[0178] For example, for the above formula (5):

[0179] If the identified device is a multi-port laparoscopic surgical robot or a medical device suitable for multi-port laparoscopic surgical robots, the stiffness coefficient... For example, a value of 2 could be taken, representing the maximum resistance threshold. For example, a value of 3 N / m (or N) could be used. If the identified device is a single-port laparoscopic surgical robot or a medical device suitable for single-port laparoscopic surgical robots, the stiffness coefficient... For example, a value of 1.2 could be used, representing the maximum resistance threshold. For example, a value of 0.8 N / m (or N) can be used. Furthermore, the spring constant for both can take any suitable value, for example, both can be 0.01.

[0180] For example, for the above formula (6):

[0181] If the identified device is a multi-port laparoscopic surgical robot or a medical device suitable for multi-port laparoscopic surgical robots, the stiffness coefficient... For example, a value of 2 is possible. If the identified device is a single-port laparoscopic surgical robot or a medical device suitable for single-port laparoscopic surgical robots, the stiffness coefficient... For example, a value of 1.2 can be taken. Furthermore, the spring constants of both... They can all take any suitable value, for example, all can take the value 1.

[0182] In some embodiments, when determining the target joint driving force output by the joint assembly in the desired operating part, step S105 may also superimpose other forces that are expected to actually act on the operating part. These forces may include one or more desired effects, and are also generalized forces.

[0183] In one embodiment, the desired superimposed force may be a damping force to actually dissipate the potential energy accumulated when the second resistance acts on the end of the operating part. See also Figure 9 The above step S105 may further include:

[0184] Step S1051: Obtain the velocity of the end of the operating part, and determine the damping force actually acting on the end of the operating part based on the velocity.

[0185] The velocity is a generalized velocity, which may include at least one of linear velocity and angular velocity. The damping force is a generalized force, which may include at least one of force and torque.

[0186] For example, step S1051 includes acquiring at least one of the linear velocity and angular velocity of the end effector of the operating part; acquiring a damping force model; and determining the damping force by combining at least one of the linear velocity and angular velocity, and the damping force model. Wherein, when the movement of the end effector of the operating part includes a change in position, the acquired velocity includes the linear velocity; when the movement of the end effector of the operating part includes a change in attitude, the acquired velocity includes the angular velocity. In one embodiment, the damping force model can be expressed as the following formula:

[0187] (9)

[0188] In formula (9), This indicates the end damping force of the operating part; Indicates the angular velocity at the end of the operating part; Indicates the linear velocity at the end of the operating part; Indicates the angular velocity damping coefficient; Indicates linear velocity; Let represent the transpose of the kinematic Jacobian matrix of the operating unit, wherein the kinematic Jacobian matrix represents... The joint variables of the first joint component of the operating unit are determined in real time.

[0189] Step S1052: Based on the second resistance and damping force, determine the target joint driving force output by the joint assembly of the desired operating part.

[0190] When the joint target driving force of the joint component outputs the associated damping force in the control unit in step S106, it can prevent rapid rebound or back-and-forth oscillation that may occur when the control unit is disengaged from the operator's control, which helps to ensure the safety of the operation, because such rapid rebound or back-and-forth oscillation may cause undesirable movement of the medical device during master-slave follow control.

[0191] In some embodiments, the forces to be superimposed may further include a joint-compensating driving force output by the joint assembly in the desired operating section, which is used to balance the forces generated by the internal load in the operating section. See also Figure 10 Step S105 above may include:

[0192] Step S1054: Obtain the joint compensation driving force output by the joint assembly in the desired operating part.

[0193] Step S1055: Based on the second resistance and the joint compensation driving force, determine the joint target driving force output by the joint assembly in the desired operating part.

[0194] Specifically, the joint compensation driving force can be obtained by solving the inverse statics equation to obtain the force acting on the end of the operating part due to the internal load. Then, the second resistance and the force can be used to obtain the joint target driving force using the statics equation. Alternatively, the second resistance can be used to obtain the first joint target driving force using the statics equation, and the force can be used to obtain the second joint target driving force using the statics equation. Then, the first joint target driving force and the second joint target driving force can be superimposed to obtain the actual desired joint target driving force. Of course, the aforementioned step S1052 can also be processed in the same way to obtain the desired joint target driving force.

[0195] In some embodiments, the joint compensation driving force may include at least one of gravity compensation force and friction compensation force. For example, both may be included simultaneously. By controlling the joint assembly of the operating unit to output a joint target driving force that correlates gravity and friction, the operator can manipulate the operating unit more easily and flexibly.

[0196] In some embodiments, the operating unit further includes a handle and a spring compensation mechanism. The handle is connected to the nearest joint assembly of the first joint assembly, and the spring compensation mechanism provides a spring force to compensate for gravity on the operating unit, enabling the operator to overcome the gravitational torque caused by gravity with little or no force when manipulating the operating unit. However, because the end position of the operating unit changes in real time, the gravitational torque caused by gravity also changes in real time, and the fixed output spring force of the spring compensation mechanism is usually insufficient to compensate for the gravitational torque in real time. To ensure the operator's sense of force when manipulating the operating unit, a spring compensation force can be output through the joint assembly in the operating unit to counteract the effect of changes in gravitational torque. Therefore, the joint compensation driving force may include a spring compensation force.

[0197] In some embodiments, the joint compensation driving force includes gravity compensation force, friction compensation force, and elastic compensation force. See also Figure 11 The above step S1054 may include:

[0198] Step S1057: Obtain the position and velocity of each joint component in the operating unit.

[0199] Step S1058: Determine the gravity compensation force and elastic compensation force of each joint component based on the position of each joint component in the operating part, and determine the friction compensation force of each joint component based on the speed of each joint component in the operating part.

[0200] Step S1059: Based on the gravity compensation force, elastic compensation force and friction compensation force of each joint component in the operating unit, determine the joint compensation driving force to be output by each joint component in the operating unit.

[0201] For example, the joint compensation driving force of the corresponding joint component can be obtained by superimposing the gravity compensation force, elasticity compensation force and friction compensation force of each joint component in the operating part.

[0202] In some embodiments, each joint component in the operating part has its own corresponding compensation model, which includes a gravity compensation model, an elastic force compensation model, and a friction force compensation model. The compensation models of different joint components are relatively independent of each other.

[0203] In some embodiments, the gravity compensation force of each joint assembly is determined based on the position of each joint assembly in the operating part, including:

[0204] Obtain the gravity compensation parameters corresponding to the gravity compensation model of each joint component in the operating unit;

[0205] The position and gravity compensation parameters of each joint component in the operating unit are used as inputs to the gravity compensation model corresponding to each joint component to determine the gravity compensation force of each joint component.

[0206] In some embodiments, the elastic compensation force of each joint component is determined based on the position of each joint component in the operating part, including:

[0207] Obtain the elastic compensation parameters corresponding to the elastic compensation model of each joint component in the operating part;

[0208] The position and elasticity compensation parameters of each joint component in the operating unit are used as inputs to the elasticity compensation model corresponding to each joint component, and the elasticity compensation force of each joint component is determined.

[0209] In some embodiments, the frictional compensation force of each joint assembly is determined based on the speed of each joint assembly in the operating part, including:

[0210] Based on the speed of each joint component in the operating section, determine the linear velocity and / or angular velocity of the end effector of the operating section;

[0211] The frictional compensation force of each joint component is determined based on the speed of each joint component and the linear velocity and / or angular velocity of the end of the operating part.

[0212] In one embodiment, determining the linear velocity and / or angular velocity of the end effector of the operating part based on the velocities of each joint component in the operating part includes:

[0213] The speeds of each joint component in the operating unit are filtered. For example, filtering the speeds of each joint component in the operating unit includes:

[0214] Obtain the absolute value of the velocity of each joint component in the operating unit at the current moment;

[0215] When the absolute value of the velocity of the joint component is greater than or equal to a preset threshold, the sign of the velocity of the joint component is consistent with the sign of the velocity at the current moment; when the absolute value of the velocity of the joint component is less than the preset threshold, the sign of the velocity of the joint component is adjusted to be consistent with the sign of the velocity at the previous moment.

[0216] The filtered velocity of each joint component is determined based on the sign and absolute value of the velocity.

[0217] In one embodiment, determining the linear velocity and / or angular velocity of the end effector of the operating part based on the velocities of each joint component in the operating part includes:

[0218] The Jacobian matrix of the operator's attitude at the current moment is determined based on the position of each joint component in the operator.

[0219] Based on the Jacobian matrix and the velocities of each joint component, determine the linear velocity and / or angular velocity of the end effector of the operating part.

[0220] In one embodiment, the frictional compensation force of each joint assembly is determined based on the velocity of each joint in the operating part and the linear velocity and / or angular velocity of the end of the operating part, including:

[0221] Determine the target compensation force for friction based on the speed of the joint components;

[0222] The first adjustment coefficient for each joint assembly is determined based on the linear velocity and / or angular velocity of the end of the operating part;

[0223] The friction compensation force is determined based on the first adjustment coefficient and the target friction compensation force.

[0224] In one embodiment, the first adjustment coefficient is linearly or non-linearly related to the linear velocity modulus of the end of the operating part, or the first adjustment coefficient is linearly or non-linearly related to the linear velocity and angular velocity modulus of the end of the operating part.

[0225] In one embodiment, the joint assembly in the operating unit includes a plurality of position joint assemblies for adjusting the end position of the operating unit and a plurality of attitude joint assemblies for adjusting the end attitude of the operating unit. The first adjustment coefficient of the plurality of position joint assemblies is associated with the linear velocity of the end of the operating unit, and the first adjustment coefficient of the plurality of attitude joint assemblies is associated with the angular velocity of the end of the operating unit.

[0226] In one embodiment, when the linear velocity at the end of the operating unit is less than a first threshold, the first adjustment coefficient is a first fixed value; when the linear velocity at the end of the operating unit is greater than the first threshold but less than a second threshold, the first adjustment coefficient increases as the linear velocity at the end of the operating unit increases; and when the linear velocity at the end of the operating unit is greater than the second threshold, the first adjustment coefficient is a second fixed value.

[0227] In one embodiment, determining the target frictional compensation force for each joint component includes:

[0228] Obtain the friction compensation parameters corresponding to the friction compensation model of each joint component;

[0229] Based on the speed and friction compensation parameters of each joint component, the target friction compensation force for each joint component is determined.

[0230] In some embodiments, step S105 may further include:

[0231] Obtain the master-slave mapping control relationship between the operating unit and the medical device;

[0232] The target driving force of the joint component output in the desired operating part is determined based on the master-slave mapping control relationship and the second resistance.

[0233] The master-slave mapping relationship includes at least one of the following: position-to-position mapping, position-to-attitude mapping, attitude-to-attitude mapping, and attitude-to-position mapping. For example, consider the case of attitude out of bounds:

[0234] When the master-slave mapping relationship is a posture-to-posture mapping relationship, the hypothetical force acting on the end of the medical device is the torque in the generalized force, and the force actually acting between the ends of the operating part is also the torque in the generalized force.

[0235] When the master-slave mapping relationship is a mapping relationship from posture to position, the hypothetical force acting on the end of the medical device is the torque in the generalized force, and the actual force acting between the ends of the operating part is also the force in the generalized force.

[0236] By determining the target driving force of the joint based on the master-slave mapping relationship, it is possible to ensure the accurate conversion of generalized force between the medical device and the operating unit, thereby ensuring a good experience for the operator when manipulating the operating unit.

[0237] In some embodiments, in master-slave follow mode, this disclosure also provides another control method, which is equally applicable to single-port laparoscopic surgical robots and multi-port laparoscopic surgical robots.

[0238] In some embodiments, see Figure 12 The control methods disclosed herein include:

[0239] Step S201: Obtain the first target posture of the end effector of the medical device.

[0240] Attitude control is usually also incremental control, so the first target attitude is determined based on the second target attitude reached by the end of the acquired mobile operator.

[0241] The medical device includes a linkage, a wrist joint assembly, and an end effector connected in sequence. The end effector is the end of the medical device.

[0242] In both single-port and multi-port laparoscopic surgical robots, the rotation of the linkage typically provides the end effector with a degree of rotational freedom; the wrist joint assembly provides multiple degrees of freedom for the end effector, including at least one of yaw and pitch degrees of freedom, such as both. Understandably, the end effector's apical degrees of freedom can be provided jointly by the linkage and the wrist joint assembly, depending on the medical device. In both single-port and multi-port laparoscopic surgical robots, if the manipulator's motion affects the end effector's posture, the end effector's apical degrees of freedom can be provided jointly by the manipulator and the medical device; conversely, if the manipulator's motion does not affect the end effector's posture, the end effector's apical degrees of freedom can be provided solely by the medical device. For example, in... Figure 1 In the single-port laparoscopic robot shown, the attitude degrees of freedom of the end effector are provided only by the linkage and wrist joint assembly in the medical device; for example, in... Figure 3 In the multi-port laparoscopic robot shown, since the manipulator can pitch and yaw around a remote center of motion, the attitude freedom of the end effector can be provided by the manipulator, the linkage and wrist joint assembly in the medical device.

[0243] Step S202: Determine the target orientation of the end effector and the target rotation angle of the link based on the first target attitude.

[0244] An end effector has a pointing direction, which is related to the coordinate axis of its rotational degree of freedom, i.e., the axis of rotation. For example, assuming the end effector has three degrees of freedom: yaw, pitch, and rotation, a Cartesian coordinate system can be constructed on the end effector. The X-axis is the rotational axis of the yaw degree of freedom, the Y-axis is the rotational axis of the pitch degree of freedom, and the Z-axis is the rotational axis of the rotational degree of freedom. The pointing direction is defined as the direction of the Z-axis. Of course, if the end effector includes either yaw or pitch degrees of freedom, as well as rotation, the pointing direction can also be defined as the direction of the Z-axis.

[0245] Orientation is typically determined based on the postural degrees of freedom of the wrist joint assembly in a medical device. Orientation includes initial orientation and target orientation. Initial orientation refers to the orientation of the end effector in its initial state, which can refer to the orientation when the links, wrist joint assembly, and end effector in the medical device are aligned in a straight line. Target orientation refers to the orientation of the end effector in its first target posture. Methods for determining the target orientation include:

[0246] Based on the first target posture, target joint variables of the joint components in the manipulator and medical device are determined. Then, the target orientation of the end effector is determined based on the target joint variables of the joint components in the wrist joint assembly related to the yaw and pitch degrees of freedom. For example, when the wrist joint assembly has only a yaw or pitch degree of freedom, the target orientation of the end effector is determined based on the target joint variables associated with the yaw or pitch degree of freedom. As another example, when the wrist joint assembly has both yaw and pitch degrees of freedom, the target orientation of the end effector is determined based on the target joint variables associated with the yaw and pitch degrees of freedom.

[0247] Step S203: Detect whether the deviation angle between the target direction and the initial direction exceeds the pointing motion boundary of the end effector.

[0248] The pointing motion boundary can refer to the maximum deviation angle between the current pointing and the initial pointing. The current pointing usually refers to the actual pointing of the end effector, and the deviation angle between the maximum current pointing and the initial pointing is the maximum deviation angle. The maximum current pointing refers to the current pointing determined by the maximum range of motion that the end effector can achieve based on the pose degrees of freedom of the wrist joint assembly.

[0249] Since the initial orientation is known, when the wrist joint assembly includes only yaw or pitch degrees of freedom, the maximum deviation angle is only related to the maximum current orientation determined by the yaw or pitch degrees of freedom.

[0250] Similarly, since the initial orientation is known, when the wrist joint assembly includes both yaw and pitch degrees of freedom, the maximum deviation angle can be jointly determined based on a first maximum deviation angle between the yaw degree of freedom and the initial orientation, and a second maximum deviation angle between the pitch degree of freedom and the initial orientation. The first maximum deviation angle is related to the maximum current orientation determined separately for the yaw degree of freedom, and the second maximum deviation angle is related to the maximum current orientation determined separately for the yaw degree of freedom. The first maximum deviation angle may be the same as or different from the second deviation angle, depending primarily on the structural design of the wrist joint assembly. For example, when the first maximum deviation angle is the same as the second maximum deviation angle, either the first maximum deviation angle or the second maximum deviation angle can be configured as the maximum deviation angle. For example, when the first maximum deviation angle is different from the second maximum deviation angle, the smaller of the first and second maximum deviation angles can be configured as the maximum deviation angle. For example, the first maximum deviation angle is 80° and the second maximum deviation angle is 90°. Usually, the first maximum deviation angle, such as 80° in this case, is configured as the maximum deviation angle to point to the motion boundary. This can ensure that the pointing motion boundaries corresponding to different degrees of freedom of attitude have the same direction, so that the impedance boundary model has the advantage of regularity. In the subsequent determination of the first resistance torque based on step S204, the calculation can be simplified.

[0251] Step S204: When the deviation angle between the target direction and the initial direction exceeds the pointing motion boundary of the end effector, determine the direction of the hypothetical first resistance acting on the end effector.

[0252] The first resistance is used to hypothetically resist the pitch and / or yaw degrees of freedom of the end effector in the first target attitude.

[0253] Of course, when the deviation angle between the target direction and the initial direction exceeds the pointing motion boundary of the end effector of the medical device, the end effector can rotate toward the target direction without being subject to the imagined resistance acting on the end effector in the pitch and / or yaw degrees of freedom.

[0254] In step S204, the movement of the joint components in the manipulator and medical device may also be controlled so that the deviation angle between the pointing of the end effector and the initial pointing reaches the pointing movement boundary and is maintained at this pointing.

[0255] Step S205: Determine the first resistance based on the direction, deviation angle, and direction of the motion boundary of the first resistance.

[0256] Step S206: Determine the first target resistance that actually acts on the end of the operating part based on the first resistance.

[0257] The first target resistance is used to actually resist the pitch and / or yaw degrees of freedom motion of the end of the operating unit in the second target attitude.

[0258] Step S207: Determine the first joint target driving torque output by the joint assembly in the desired operating part based on the first target resistance.

[0259] Step S208: The joint assembly in the control operation unit outputs the first joint target driving torque.

[0260] Through step S208, force feedback can be achieved in the operating part, thereby prompting the operator not to move the operating part to the second target posture, so as to actually resist the movement of the end of the operating part, such as the pitch and / or yaw degrees of freedom in the second target posture.

[0261] Step S209: Detect whether the target's rotation angle exceeds the motion boundary of the rotation degree of freedom.

[0262] The motion boundary of the rotation degree of freedom refers to the range of motion (rotation) of the rotation degree of freedom.

[0263] Step S210: When the target rotation angle exceeds the motion boundary of the rotation degree of freedom, determine the direction of the hypothetical second resistance acting on the end effector.

[0264] The second resistance is used to hypothetically resist the rotational degree of freedom of the end effector in the first target orientation.

[0265] Of course, when the target rotation angle does not exceed the motion boundary of the rotation degree of freedom, the linkage can be controlled to rotate the target rotation angle without being subject to the hypothetical resistance of the end effector in the rotation degree of freedom.

[0266] In step S210, the movement of the linkage in the medical device can also be controlled so that the rotation angle of the linkage reaches the motion boundary of the degree of freedom of rotation and maintains the rotation angle at this time.

[0267] Step S211: Determine the second resistance based on the direction of the second resistance, the target rotation angle, and the motion boundary.

[0268] Step S212: Determine the second target resistance that actually acts on the end of the operating part based on the second resistance.

[0269] The second target resistance is used to actually resist the rotational degree of freedom movement of the end of the operating unit in the second target attitude.

[0270] Steps S206 and S212 may include a conversion process for converting the corresponding torque from the first coordinate system at the end of the medical device to the second coordinate system at the end of the operating part. This conversion process can be referenced to the conversion of the first resistance described above, and will not be repeated here.

[0271] Step S213: Determine the second joint target driving torque output by the joint assembly in the desired operating part based on the second target resistance.

[0272] In steps S207 and S213, the target driving force of the corresponding joint can be determined by means of static equations.

[0273] Step S214: The joint assembly in the control operation unit outputs the target driving torque of the second joint.

[0274] Through step S214, force feedback can be achieved in the operating part, thereby prompting the operator not to move the operating part to generate the second target posture, so as to actually resist the movement of the end of the operating part, for example, to the rotational degree of freedom in the second posture.

[0275] In the above embodiment, in step S203, if the deviation angle between the target pointing and the initial pointing is detected to be less than the pointing motion boundary of the end effector, step S215 is executed.

[0276] Step S215: Control the end effector to move to the target direction.

[0277] In the above embodiment, in step S209, if the target rotation angle is detected to be within the motion boundary of the rotation degree of freedom, step S216 is executed.

[0278] Step S216: Control the linkage movement to achieve the target rotation angle.

[0279] Steps S203-S208 and S209-S214 described above can be executed sequentially or in parallel. For example, these two steps can be executed in parallel.

[0280] In other embodiments, a first target resistance and a second target resistance can be superimposed, and the final joint target driving force output by the joint component in the desired operating part can be determined based on the superimposed two target resistances. Alternatively, the first joint target driving force and the second joint target driving force can be superimposed as the final joint target driving force output by the joint component in the desired operating part. Then, controlling the joint component in the operating part to output the final joint target driving force can also achieve the same effect.

[0281] All of the aforementioned resistance and driving forces can be generalized forces. In terms of attitude, these resistance and driving forces are usually torques.

[0282] In some embodiments, the distal end of the medical device includes a positional movement boundary within the operating space. See also... Figure 13 The control method disclosed herein may further include:

[0283] Step S220: Obtain the first target position of the end effector of the medical device, and obtain the current position of the end effector of the medical device.

[0284] Position control is typically incremental control, so the first target position can be determined based on the second target position reached by the end of the moving manipulator.

[0285] Step S221: Detect whether the current position has reached the position movement boundary, and detect whether the first target position has exceeded the position movement boundary.

[0286] An end effector includes the positional motion boundary in the operating space.

[0287] If the current position reaches the movement boundary and the first target position exceeds the position movement boundary, proceed to step S222; otherwise, proceed to step S226.

[0288] Step S222: Determine the first resistance that is hypothetical to act on the end of the medical device based on the first target position and the current position.

[0289] Step S223: Determine the second resistance that actually acts on the end of the operating part based on the first resistance.

[0290] The first and second resistances are forces in a generalized sense. The first resistance is used to hypothetically resist the movement of the distal end of the medical device toward the first target position, while the second resistance is used to actually resist the movement of the distal end of the operating part toward the second target position.

[0291] Similarly, step S223 may include at least the aforementioned coordinate system transformation process for converting forces, which will not be described in detail here.

[0292] Step S224: Determine the target joint driving force output by the joint assembly in the desired operating part based on the second resistance.

[0293] Step S225: The joint assembly in the control unit outputs the target driving force of the joint.

[0294] In step S225, the torque mode control joint assembly outputs the target driving force of the joint to realize force feedback in the operating part, thereby prompting the operator not to move the operating part to the position corresponding to the position command.

[0295] Step S226: Determine the target joint variables of the joint components in the manipulator and medical device based on the first target position.

[0296] Step S227: Control the joint components in the manipulator and medical device to output the target joint variable.

[0297] In step S227, the position mode control joint assembly outputs target joint variables so that the movement of the end of the medical device in the positional degrees of freedom is not restricted when the current position of the end of the medical device does not reach the positional movement boundary and / or the first target position does not exceed the positional movement boundary.

[0298] In some embodiments, step S222 above may also include controlling the end of the medical device to maintain its current position.

[0299] In some embodiments, step S222, namely determining the first resistance acting on the end of the medical device based on the first target position and the current position, can be obtained with reference to the preceding text, and can be exemplarily obtained with reference to the aforementioned formula (1). In some embodiments, step S224, namely determining the second resistance actually acting on the end of the operating part based on the first resistance, can also be obtained with reference to the preceding text. For example, this step may also include a resistance magnitude transformation process, which can be exemplarily obtained with reference to the aforementioned formula (2).

[0300] In step S204 above, the direction of the first resistance can be determined, for example, based on either the target direction or the current direction of the end effector, as well as the initial direction. The current direction refers to the direction of the end effector in the current attitude. For example, the direction of the first resistance can be determined based on the target direction and the initial direction. For instance, a first vector can be obtained by cross-multiplying the target direction and the initial direction to obtain the magnitude of the first vector, and the direction of the first resistance can be determined based on the first vector and its magnitude. For instance, the formula for expressing the first vector can be as follows:

[0301] (10)

[0302] In formula (10), Represents the first vector; Indicates the target direction; Indicates the initial pointer. This indicates the direction of the first resistance. In step S205 above, a first pointing motion boundary impedance model associated with the direction of the first resistance can be obtained, and then the first resistance can be determined based on the deviation angle, the pointing motion boundary, and the first pointing motion boundary impedance model. This first pointing motion boundary impedance model can be expressed, for example, as:

[0303] (11)

[0304] Formula (11), The middle represents the first resistance; Indicates the spring constant; Indicates the deviation angle; Indicates pointing to the boundary of motion; This represents the maximum resistance torque threshold of the first resistance. Indicates the direction of the first resistance.

[0305] Step S210 above can, for example, obtain a second vector based on the sign function of the associated target rotation angle and the target direction, obtain the magnitude of the second vector, and determine the direction of the second drag force based on the second vector and its magnitude. For example, the formula for expressing the second vector can be as follows:

[0306] (12)

[0307] In formula (12), Represents the second vector; Represents a symbolic function; Indicates the target's rotation angle; Indicates the target direction. This indicates the direction of the second resistance.

[0308] Step S211 above obtains the second pointing motion boundary impedance model associated with the second resistance direction, and then determines the second resistance based on the target rotation angle, the motion boundary, and the second pointing motion boundary impedance model. This second pointing motion boundary impedance model can be expressed, for example, as:

[0309] (13)

[0310] In formula (13), Indicates the second resistance; Represents the absolute value of the target's rotation angle; The boundary of motion representing the degree of freedom of rotation; Indicates the spring constant; This represents the maximum resistance torque threshold for the second resistance. Indicates the direction of the second resistance.

[0311] In some embodiments, in steps S207, S213, or S224 above, other forces expected to actually act on the operating part may be superimposed. These forces may include one or more desired effects, and are also generalized forces. For example, the forces to be superimposed include, but are not limited to, damping forces, and / or joint compensation driving forces used to balance the forces generated by the internal load in the operating part. For example, the joint compensation driving force may include at least one of gravity compensation force, friction compensation force, and elastic compensation force. For these forces that may be superimposed, please refer to the preceding text; they will not be repeated here. The forces actually acting on the operating part can be superimposed and analyzed together to form the joint driving force output by the joint assembly of the operating part, and then output. Alternatively, the forces actually acting on the operating part can be separately analyzed to form the joint driving force output by the joint assembly of the operating part, superimposed, and then output together.

[0312] In some embodiments, the control method disclosed herein is particularly suitable for application scenarios where the rotational degree of freedom has a rotational range greater than or equal to 360°, and the rotational range of the pitch and / or yaw degrees of freedom is less than 360°.

[0313] In some embodiments, in a single-port laparoscopic surgical robot, see further... Figure 1The trocar 400 is located at the end of the housing 130, which houses multiple manipulators 140. Multiple medical instruments 150 are inserted into the patient's body through multiple airtight channels provided by the same trocar 400 to perform surgery. Because the manipulators 140 and medical instruments 150 provide sufficient degrees of freedom—"sufficient" meaning capable of performing the corresponding surgical procedure—the trocar 400's position can usually be determined according to the target surgical procedure during preoperative positioning. The trocar 400 is then locked in this position using software control, and its position and orientation cannot be changed while locked, thus reducing the stress that may be generated by the movement of the trocar 400 and its potential harm to the patient.

[0314] For example, surgical procedures can be broadly categorized into otolaryngological (ENT) surgeries, prostate surgeries, nephrological surgeries, gastrointestinal surgeries, hepatobiliary surgeries, thoracic surgeries, gynecological surgeries, and cardiac surgeries. Further, surgical procedures can be categorized into subcategories; taking hepatobiliary surgeries as an example, these include liver transplantation, hepatectomy, cholecystectomy, pancreaticoduodenectomy, and splenectomy. Examples of target surgical procedures can be derived from these categories.

[0315] For the different surgical procedures illustrated above, it may be permissible to perform multiple major types of surgical procedures through the same incision or natural cavity of the organism. For example, it may be permissible to perform nephrological surgery, gastrointestinal surgery, and hepatobiliary surgery through the same incision or natural cavity. It may also be permissible to perform multiple minor types of surgical procedures within the same major type through the same incision or natural cavity of the organism. For example, it may be permissible to perform liver transplantation, hepatectomy, cholecystectomy, pancreaticoduodenectomy, and splenectomy through the same incision or natural cavity of the organism within hepatobiliary surgery.

[0316] Since the trocar 400 is connected to the patient, changing its position is generally not permitted. However, if necessary, movement of the trocar 400 around a remote center of motion may be allowed, meaning movement that maintains the position of the trocar 400 while changing its orientation is permitted. This remote center of motion can typically be configured on the trocar 400, for example, as... Figure 14 As shown, the remote motion center (RC) is configured at the position where the trocar 400 connects to the incision or natural cavity opening, such that the trocar 400 does not damage the incision or natural cavity opening when it moves around the remote motion center.

[0317] The motion boundary of the end effector is related to the range of motion of the joint components in the manipulator 140 and the medical device 150; in this sense, the motion boundary is not adjustable in the operating space. The motion boundary of the end effector is also related to the orientation (i.e., posture) of the trocar 400; in this sense, the motion boundary is adjustable in the operating space.

[0318] For the sake of brevity, Figure 15 The manipulator 140 and medical device 150 are deliberately omitted, with only the trocar 400 shown for illustration. Different surgical procedures are often associated with different orientations of the trocar 400. Orientation can usually be adjusted by controlling the movement of the trocar 400 around a remote center of motion to meet the switching needs between different surgical procedures associated with the same incision or natural cavity. The main purpose of switching surgical procedures or adjusting the orientation of the trocar 400 is to ensure that the movement boundary of the end effector meets the surgical operation requirements of the target procedure, rather than excessively hindering the surgical operation. It is evident that the orientation of the trocar 400 and the surgical procedure are usually related; a change in the orientation of the trocar 400 may cause a change in the surgical procedure, or vice versa; the two can influence each other.

[0319] For example, please continue reading Figure 15 The motion boundaries of multiple different end effectors inserted into the same puncture device 400 can be configured to have a common motion boundary. This common motion boundary can, for example, be the intersection region of the motion boundaries of multiple different end effectors; for instance, this common motion boundary can be constrained into a cylindrical space, which can be seen in, for example, [reference needed]. Figure 15 The dashed closed shape in the figure. This same motion boundary is exemplarily associated with the orientation axis of the puncture device 400, for example, the central axis of the cylindrical space, which is sometimes also called the RC axis or central axis.

[0320] In some embodiments, the controller can also predict the orientation of the trocar 400 based on the doctor's manipulation of the manipulator assembly 120 via the operating unit 310 (equivalent to predicting the surgical procedure). Therefore, as... Figure 16 As shown, the controller can also be configured to perform:

[0321] Step S21: Obtain first information that the end effector exceeds its motion boundary during the operation performed by the doctor under the current orientation of the puncture instrument.

[0322] The motion boundary here mainly refers to the positional motion boundary in the operating space. Whether the target surgical procedure can be successfully performed is mainly related to the positional motion boundary of the end effector. In other words, whether the end effector can reach the necessary area in terms of position is the key to whether the surgery can be successfully performed.

[0323] The first piece of information includes one or more of the following: boundary crossing location, boundary crossing count, and boundary crossing time. The boundary crossing count includes the number of boundary crossings associated with each boundary crossing location and / or the total number of boundary crossings. The total number of boundary crossings can be obtained directly by accumulating the count for each boundary crossing, or by summing the number of boundary crossings at each boundary crossing location. The boundary crossing time includes the dwell time of the end effector associated with the boundary crossing location, including the single boundary crossing time at the boundary crossing location and / or the total boundary crossing time, wherein the total boundary crossing time includes the total boundary crossing time at one boundary crossing location and / or the total boundary crossing time across all boundary crossing locations.

[0324] Step S22: Predict the target orientation of the puncture device based on the acquired first information.

[0325] Different orientations of the trocar 400 are usually associated with different surgical procedures, and the target orientation is usually different from the current orientation.

[0326] In some embodiments, predicting the target orientation of the puncture device based on the acquired first information includes: predicting the target orientation of the puncture device based on the acquired first information in response to acquiring a first instruction.

[0327] In some embodiments, the first instruction may include a first instruction processed and obtained by the controller, i.e., the prediction of the orientation of the trocar 400 is automatically triggered. For example, during surgery performed with the trocar 400 in its current orientation, the controller may determine, based on first information, a first segment in which the end effector frequently crosses its motion boundary. Once this first segment can be determined, the controller can obtain the first instruction. This first segment includes a region consisting of one or more points on the boundary. Since the positions of these points can be determined kinematically, the first segment, as a set of points, also covers a region whose positions are determinable.

[0328] For example, the boundary can be pre-divided into multiple segments. The determination of the first segment involves determining whether the number of boundary violations among the statistically divided segments reaches a threshold. If the threshold is reached, the corresponding segment is designated as the first segment. Since more than one segment may reach the threshold during the procedure, the first segment may include more than one segment, such as one, two, or more. For instance, assuming the motion boundary (e.g., a boundary associated with the xy-plane) is circular, it can be divided into more than two segments, such as 2, 3, 4, 5, 6…120 segments, or even more. These segments can be configured as equally divided, partially divided, or unequally divided; for example, they can be configured as equally divided segments. Generally, the more segments there are, the more beneficial it is for accurately predicting the target orientation of the trocar 400. Figure 17 As shown, for example, the boundary can be pre-divided into segments 1 to 8. The end effector has exceeded the boundary in all segments 1 to 8. Statistically, segment 1 has the most boundary exceedances. Therefore, it can be determined that... Figure 18 The section 1 shown is the first section.

[0329] In addition, the determination of the first segment may also include determining whether the time of exceeding the boundary in the statistically divided segments reaches a threshold. If the threshold is reached, the corresponding segment is determined as the first segment.

[0330] For example, it is also possible to determine the first segment without first dividing the boundary into multiple segments. This involves generating a normal distribution curve on the motion boundary, associated with the boundary crossing position and the number of boundary crossings, when the total number of boundary crossings within a certain period reaches a first threshold. Based on this normal distribution curve, a target interval where the boundary crossing probability reaches a second threshold is determined. The first segment can then be determined using the positions on the motion boundary associated with the endpoints of this target interval. Furthermore, after determining the target interval, the number of boundary crossings within the target interval can be counted. Only when the number of boundary crossings reaches a third threshold is the first segment determined using the positions on the motion boundary associated with the endpoints of the target interval. Figure 19 As shown, it is not necessary to divide the boundary into segments beforehand. First, determine the densest interval of the boundary as the target interval, and then use the endpoints of the target interval to determine the boundary. Figure 20 The first segment on the boundary shown.

[0331] Alternatively, the determination of the first segment may also include not having to divide the boundary into multiple segments in advance. The determination of the first segment includes generating a normal distribution curve on the motion boundary that is associated with the over-boundary position and over-boundary time when the total over-boundary time reaches a first threshold within a certain period, and determining the target interval where the over-boundary probability reaches a second threshold based on the normal distribution curve. The first segment can then be determined by using the position on the motion boundary associated with the endpoint of the target interval.

[0332] In addition, after the target interval is determined, the number of times or time of exceeding the limit within the target interval can be counted. When the number of times or time of exceeding the limit reaches the third threshold, the position on the motion boundary associated with the endpoint of the target interval is used to determine the first segment.

[0333] In some embodiments, the first instruction may include a first instruction input by the physician to the controller via an input device, i.e., the prediction of the orientation of the puncture device 400 may be manually triggered. The physician may actively input the first instruction in any way. Exemplarily, the input device for inputting the first instruction includes at least one of the following devices, for example, a touch screen coupled to the controller, a voice recognition device coupled to the controller, an operating unit 310 coupled to the controller, a foot pedal coupled to the controller, a motion recognition device (such as a gesture recognition device) coupled to the controller, an EEG recognition device coupled to the controller, and other devices capable of input.

[0334] In some embodiments, predicting the target orientation of the puncture device based on the acquired first information includes: determining a first segment based on the first information, and predicting the target orientation of the puncture device based on the first segment.

[0335] The methods for determining the first segment include those described above, which will not be repeated here.

[0336] For example, predicting the target orientation of the trocar based on the first segment includes: matching surgical procedures from multiple surgical procedures whose associated end effector's motion boundary is located on one side of the first segment in the transboundary direction as the target surgical procedure; and then determining the target orientation of the trocar based on the target surgical procedure. Such a target surgical procedure, the end effector's motion boundary, or the trocar's target orientation can basically reflect the surgeon's expectation of the end effector's motion boundary adjustment direction, that is, it can basically reflect the surgeon's expectation of the trocar's adjustment direction. For example, as... Figure 21 As shown, assuming that the motion boundaries of the end effector associated with multiple surgical procedures include the motion boundaries A to E of the end effector, if the motion boundaries A, B, and C of the end effector are located on one side of the first segment in the super-boundary direction, and the motion boundaries D and E of the end effector are not located on one side of the first segment in the super-boundary direction, then the orientation of the trocar associated with the motion boundaries A, B, and C of the end effector can be matched as the target orientation.

[0337] Since the orientation of the trocar, the surgical procedure, and the motion boundary of the end effector are all interconnected, the surgical procedure associated with the motion boundaries A, B, and C of the end effector can be matched as the target surgical procedure, and / or, the motion boundaries A, B, and C of the end effector can be matched as the target motion boundaries.

[0338] For example, predicting the target orientation of the trocar based on the first segment includes: matching surgical procedures from multiple surgical procedures whose associated end effector motion boundaries at least partially cover the first segment as target surgical procedures, and then determining the target orientation of the trocar based on the target surgical procedures. For example, a surgical procedure that only needs to cover a portion of the points in the first segment can be used as the target surgical procedure. Another example is a surgical procedure where the proportion of points covering the first segment reaches a first threshold can be used as the target surgical procedure. The predicted target surgical procedure may include more than one surgical procedure. For example, such as... Figure 22 As shown, assuming multiple surgical procedures include the motion boundaries A' to D' of the end effector associated with the surgical procedures, if the proportion of motion boundary A' covering the first segment reaches 100%, the proportion of motion boundary B' covering the first segment reaches 80%, the proportion of motion boundary C' covering the first segment reaches 40%, and the proportion of motion boundary D' covering the first segment is 0%, for example, the surgical procedures associated with motion boundaries A', B', and C' can all be used as target surgical procedures. Or, for example, the surgical procedures with a coverage ratio greater than 50% can be used as target surgical procedures. In this case, only the surgical procedures associated with motion boundaries A' and B' can be used as target surgical procedures among the surgical procedures associated with motion boundaries A' to C'.

[0339] For example, to more accurately predict the target orientation of the trocar 400, predicting the target orientation of the trocar based on the first segment may include: acquiring the first segment; acquiring an operational image captured by an imaging instrument; identifying organs in the operational image; and combining the first segment and the identified organs to predict the target orientation of the trocar. For example, organs include the heart, liver, spleen, lungs, stomach, gallbladder, pancreas, kidneys, bladder, large intestine, and duodenum. Organs also include smaller features, such as the lobes of the liver. Figure 23 As shown, when identifying the liver lobe, kidney, and duodenum from an operational image, the orientation of the trocar corresponding to the liver lobe and kidney can be predicted as the target orientation by combining the first segment.

[0340] In some embodiments, combining the first segment and the identified organ to predict the target orientation of the trocar includes:

[0341] From multiple surgical procedures, a first surgical procedure associated with the first segment is matched; from the first surgical procedure, a second surgical procedure whose associated end effector motion boundary is associated with the identified organ is matched as the target surgical procedure.

[0342] For example, matching a first surgical procedure associated with a first segment from multiple surgical procedures includes: matching a surgical procedure whose associated end effector's motion boundary is located in the transboundary direction on one side of the first segment as the first surgical procedure.

[0343] For example, matching a first surgical procedure associated with a first segment from multiple surgical procedures includes: matching a surgical procedure whose corresponding end effector's motion boundary can at least partially cover the first segment as the first surgical procedure.

[0344] The second surgical procedure, which is associated with the motion boundary of the end effector in the first surgical procedure and the identified organ, is selected as the target surgical procedure. In other words, surgical procedures associated with organs that are not identified are excluded, and the remaining surgical procedures are used as the second surgical procedure. An organ not being identified includes the organ not being in the operation image (i.e., not within the field of view of the imaging instruments) and / or the organ being in the operation image but not identified due to incomplete features or other factors. Furthermore, by combining image recognition with the screening of the first surgical procedure, the prediction accuracy of the second surgical procedure can be improved, and it can better reflect the surgeon's operational intentions.

[0345] In some embodiments, after predicting the target surgical procedure and / or the target orientation of the trocar, the controller can be configured to recommend the target surgical procedure and / or the target orientation of the trocar to the physician. For example, it can generate, for instance, the target orientation of the trocar. Figure 24 The user interface shown presents relevant recommendations to doctors. Figure 24 The interface shown recommends target surgical procedures and / or target orientations of the trocar related to the liver lobe to the physician. The physician can then manually adjust the orientation of the trocar based on the recommended target surgical procedure and / or trocar orientation. This adjustment includes adjusting the trocar's orientation axis (sometimes called the central axis or RC axis) to substantially match the target surgical procedure and / or trocar orientation requirements for the trocar's orientation axis.

[0346] There are several ways for the controller to recommend the target surgical procedure and / or the target orientation of the trocar to the surgeon. For example, a voice device or display device coupled to the controller can be set up to play information related to the target orientation of the second surgical procedure and / or the trocar, and / or to display such information. Recommending the target surgical procedure is generally easier for the surgeon to understand. Of course, for surgeons skilled in using surgical robotic systems, directly recommending the target orientation of the trocar is also acceptable.

[0347] For example, the voice device includes a speaker or headphones that are independent of the main control panel 200 and the slave control device 100. Alternatively, the voice device may include a speaker integrated into the main control panel 200 and / or the slave control device 100.

[0348] For example, the display device includes a display that is independent of the master console 200 and the slave operating device 100, such as a display on an image cart coupled to the master console 200 and the slave operating device 100. Alternatively, the display device may include a display integrated into the master console 200 and / or the slave operating device 100.

[0349] In some embodiments, after predicting the target orientation of the trocar, the controller can be configured to: control the movement of the joint components in the main arm according to the target orientation to move the trocar around a remote center of motion and bring the trocar to the target orientation. Bringing the trocar to the target orientation includes bringing the orientation axis of the trocar to the target orientation. Exemplarily, the target orientation of the trocar includes a target orientation based on the main arm's base coordinate system.

[0350] In some embodiments, after the target surgical procedure is predicted, the controller can be configured to: acquire the orientation of the trocar associated with the target surgical procedure as the target orientation, control the movement of the main arm joint assembly according to the target orientation so that the trocar moves around the remote center of motion and the orientation of the trocar reaches the target orientation.

[0351] The control of the movement of the joint components in the main arm according to the target orientation includes: converting the target orientation into joint variables of the joint components in the main arm through inverse kinematics; and then controlling the movement of the joint components in the main arm according to the corresponding joint variables so that the orientation of the puncture instrument reaches the target orientation.

[0352] In some embodiments, when there are multiple second surgical procedures as the target surgical procedure, the final determination of the target surgical procedure may include various implementation methods, wherein the final determination of the target surgical procedure includes selecting one of the second surgical procedures as the target surgical procedure.

[0353] For example, the final determination of the target surgical procedure may include the doctor's selection of a second surgical procedure. For instance, this selection can be made through voice recognition, such as identifying information associated with the second surgical procedure, including its name or number, and then determining that second surgical procedure as the target procedure. Alternatively, information associated with the second surgical procedure can be generated and displayed on a screen interface, and a corresponding second surgical procedure can be selected as the target procedure by touching or squeezing a corresponding input device such as a touchscreen, button, or foot pedal. Of course, other methods are also possible, such as using an electroencephalogram (EEG) recognition device for brainwave recognition, whereby the second surgical procedure is determined as the target procedure when information associated with it is recognized.

[0354] For example, the final determination of the target surgical procedure may include the surgical robot system, such as the controller, automatically selecting a second surgical procedure. For instance, the surgical procedure with the highest correlation among the second surgical procedures may be chosen as the target surgical procedure by default. When the second surgical procedure includes only one procedure, that procedure is unique and therefore has the highest correlation. As another example, assuming that the closer the motion boundary associated with the end effector is to the first segment, the higher the correlation, the second surgical procedure whose center of the motion boundary associated with the end effector is closest to the aforementioned first segment may be chosen as the target surgical procedure to reduce the range of motion when the trocar is adjusted.

[0355] In some embodiments, controlling the movement of the articular components of the main arm according to the target orientation includes: controlling the movement of the articular components of the main arm according to the target orientation after a delay time has been reached. The delay time can be configured, for example, from 0 to 120 seconds. For example, when the delay time is configured to 0 seconds, the movement of the articular components of the main arm can be controlled immediately to adjust the orientation of the trocar once the target surgical procedure and / or the target orientation of the trocar is determined. For example, when the delay time is configured to 30 seconds, the movement of the articular components of the main arm can be controlled to adjust the orientation of the trocar after the target surgical procedure and / or the target orientation of the trocar is determined and after a delay of 30 seconds has been reached. This delay time configuration allows the physician sufficient time to re-determine the target surgical procedure. Of course, after the target surgical procedure and / or the target orientation of the trocar is determined but before the delay time has been reached, the controller can immediately control the movement of the main arm without the delay time having expired after receiving a confirmation command from the physician via interactive means to immediately adjust the orientation of the trocar.

[0356] In some embodiments, when the end effector frequently overshoots the boundary, automatic or manual adjustment of the trocar orientation can be configured even in scenarios where the target surgical procedure and / or target orientation is not required or cannot be predicted.

[0357] In some embodiments, such as Figure 25 As shown, the controller can be configured to perform:

[0358] Step S31: Obtain first information that the end effector exceeds its motion boundary during the surgeon's operation under the current orientation of the puncture instrument.

[0359] The first piece of information includes one or more of the following: boundary location, boundary number, and boundary time.

[0360] Step S32: Determine the target center point based on the acquired first information.

[0361] Determining the target center location based on the acquired first information includes: determining the position of a feature point on a defined first segment as the target center point. This feature point may include, for example, the center point of the first segment. The determination of the first segment can include any of the methods described above, and will not be elaborated upon here.

[0362] Step S33: Control the movement of the main arm joint assembly to move the puncture device around the remote motion center and align the orientation of the puncture device with the target center point.

[0363] The movement of the puncture device around the remote center of motion includes rotational movement of the puncture device around the remote center of motion; typically, it only moves in the degrees of freedom of posture. The orientation of the puncture device to the target center point includes the orientation axis of the puncture device passing through the target center point.

[0364] Step S33 includes: obtaining the target orientation to which the orientation axis of the puncture device is expected to move; determining the target joint variable of the joint component in the main arm based on the target orientation; and then controlling the movement of the corresponding joint component in the main arm based on the target joint variable to make the puncture device perform RC movement and align the orientation of the puncture device with the target center point.

[0365] For example, the target orientation can be obtained as follows: The controller is configured to perform the following: acquire the orientation of the line connecting the remote motion center and the target center point as the target orientation. This target orientation can, for example, be the target orientation in the base coordinate system of the main arm.

[0366] The alignment of the puncture device's orientation axis with the target center point includes the puncture device's orientation axis coinciding with the line connecting the remote motion center and the target center point. The target center point's position in the base coordinate system can be determined based on the position of the target center point in the image-guided end effector coordinate system (sometimes called the endoscope coordinate system), and the transformation relationship between the endoscope coordinate system and the main arm's base coordinate system, thus aiding in obtaining the target orientation.

[0367] For example, the target joint variables described above can be obtained as follows: The controller is configured to execute: determine the target joint variables of the joint components in the main arm by combining target orientation and inverse kinematics.

[0368] like Figure 26 As shown, the dashed motion boundary image model represents the orientation of the puncture device 400 before adjustment, while the solid motion boundary image model represents the orientation of the puncture device 400 adjusted to align with the target center point.

[0369] Through the above steps S31 to S33, the orientation of the trocar 400 can be automatically adjusted, preventing the end effector from frequently exceeding the movement boundary during the doctor's operation under the current orientation of the trocar 400. Furthermore, by continuously repeating the above steps S31 to S33, the orientation of the trocar 400 can be continuously adjusted to ultimately achieve the orientation of the trocar 400 desired by the doctor.

[0370] In some embodiments, such as Figure 27 As shown, the controller can be configured to perform:

[0371] Step S41: Obtain first information that the end effector exceeds its motion boundary during the surgeon's operation under the current orientation of the puncture instrument.

[0372] The first piece of information includes one or more of the following: boundary location, boundary number, and boundary time.

[0373] Step S42: Determine the first segment based on the first information, and determine the target organ in the operation image by combining the first segment and the operation image.

[0374] For the sake of brevity, please refer to the preceding text for a description of the first section.

[0375] For example, determining the target organ in the manipulation image by combining the first segment and the manipulation image may include:

[0376] All organs within the operation image are identified, and the target organ is determined from the identified organs based on the first segment. That is, all organs within the operation image are first identified, and then organs related to the first segment are selected as target organs from the identified organs. Determining the target organ based on the first segment mainly involves assigning a reasonable range to determine the target organ; for example, this range might include the area of ​​the first segment located away from the orientation axis of the current trocar, and the corresponding organ is determined as the target organ based on this range.

[0377] For example, determining the target organ in the operation image by combining the first segment and the operation image may also include:

[0378] Organs within the manipulated image that are associated with the first segment are identified as target organs. That is, instead of identifying all organs within the manipulated image, organs directly associated with the first segment are identified as target organs. This reduces the amount of image processing required, thereby increasing the speed of image processing.

[0379] Step S43: Determine the target center point based on the identified target organ.

[0380] For example, the geometric center of the target organ can be determined as the target center point based on its contour information.

[0381] Step S44: Control the movement of the main arm joint assembly to move the puncture device around the remote motion center and align the orientation of the puncture device with the target center point.

[0382] Step S44 includes: obtaining the target orientation of the expected orientation axis movement of the puncture device, determining the target joint variable of the joint assembly in the main arm based on the target orientation, and then controlling the movement of the corresponding joint assembly in the main arm based on the target joint variable so that the puncture device performs RC movement and its orientation is aligned with the target center point.

[0383] For example, the target orientation can also be obtained by acquiring the orientation of the line connecting the remote center of motion and the target center point. In some embodiments, the target organ may include more than one, and therefore the target center point may include more than one. The controller is configured to determine one of the multiple target center points as the target center point based on the interaction instructions between the doctor and the surgical robot system. Of course, the controller can also determine one of the multiple target center points as the target center point by default according to a preset rule. The preset rule includes, for example, acquiring multiple target orientations between the remote center of motion and multiple target center points, and determining the target center point associated with the one with the smallest difference between the current orientation of the trocar and the multiple target orientations as the target center point.

[0384] like Figure 28 As shown, when the target organ includes the liver lobe, the center of the liver lobe is taken as the target center point. The dashed motion boundary image model represents the orientation of the puncture device 400 before adjustment, while the solid motion boundary image model represents the orientation of the puncture device 400 adjusted to align with the target center point.

[0385] Through the above steps S41 to S44, the orientation axis of the trocar 400 can be aligned with the target center point associated with the target organ, so as to facilitate the surgery on the target organ.

[0386] It is worth noting that when the orientation axis of the puncture device 400 is manually or automatically adjusted to move around the remote motion center, it is preferable to consider the potential safety risks that the end effector may pose when the medical device 150 moves in sync with the movement of the puncture device 400.

[0387] To minimize the aforementioned safety risks, for example, multiple medical devices 150 passing through the trocar 400 may be retracted to a safe position before the orientation axis of the trocar 400 is adjusted to move around the remote center of motion. Alternatively, while adjusting the orientation axis of the trocar 400 around the remote center of motion, the joint components in the manipulator assembly 120 may be controlled to move in coordination to maintain the position or orientation of the end effector in response to changes in orientation of the trocar 400 around the remote center of motion.

[0388] Continue reading Figure 14 In the medical device 150, since the end effector 154 of the surgical instrument 153 is more invasive to the patient than the end effector 152 of the imaging instrument 151, in some embodiments, when adjusting the orientation axis of the trocar 400 to move around the remote center of motion, only the joint components in the manipulator assembly 120 associated with the surgical instrument 153 are controlled to move in coordination to maintain position and / or pose. At the same time, since the imaging instrument 151 is not controlled, the movement of the imaging instrument 151 following the trocar 400 can generate a new field of view to facilitate observation of changes in the orientation of the trocar 400.

[0389] Of course, in other embodiments, if it is necessary to maintain the current surgical field of view, the joint components in the manipulator assembly 120 associated with the surgical instrument 153 can be controlled to move in coordination to maintain position and / or pose, and the joint components in the manipulator assembly 120 associated with the imaging instrument 151 can be controlled to move in coordination to maintain position and / or pose.

[0390] In some embodiments, the target object manipulated by the operating unit can also be changed based on the doctor's manipulation of the manipulator component. The controller can be configured to perform:

[0391] During surgery performed by the surgeon with the trocar in its current orientation, when the number of times the end effector exceeds its movement boundary within a certain period reaches a target threshold, the first operating mode is switched to the second operating mode. Here, the first operating mode is the aforementioned master-slave follow-up control mode, which includes the manipulation of the manipulator assembly by the operating unit. The second operating mode is the trocar control mode, which includes the manipulation of the main arm by the operating unit, i.e., the manipulation of the trocar connected to the end of the main arm. More preferably, the manipulation of the trocar involves the manipulation of the trocar's remote center of motion. For example, the manipulation of the trocar by the operating unit includes manipulating the trocar to move around the remote center of motion.

[0392] This region can include a segment on a predefined motion boundary. It can also include a segment from multiple pre-divided regions where the number of times the boundary is exceeded reaches a set threshold. Alternatively, it can include a segment determined using methods such as statistically analyzing the normal distribution of the number and location of boundary exceedances during a surgical procedure. (Continue reading...) Figure 17 For example, within a certain period, such as 10 seconds, when the number of times the end effector exceeds the boundary of segment 1 of its motion boundary reaches a target threshold, such as 3 times, the first operation mode is switched to the second operation mode.

[0393] Furthermore, by switching operating modes, doctors can easily and quickly adjust the orientation of the trocar's axis using the operating unit without leaving the operating position. In particular, because there is clear force feedback when the end effector exceeds the motion boundary, it clearly reflects the doctor's intention to switch operating modes, providing a good user experience.

[0394] For example, in the second operating mode, the orientation of the puncture device can be configured to change in accordance with the orientation of the operating part. For instance, the orientation of the puncture device changes in the first orientation degree of freedom (e.g., yaw degree of freedom) in accordance with the first orientation degree of freedom (e.g., yaw degree of freedom) of the operating part; the orientation of the puncture device changes in the first orientation degree of freedom (e.g., pitch degree of freedom) in accordance with the first orientation degree of freedom (e.g., pitch degree of freedom) of the operating part; and the orientation of the puncture device changes in the first orientation degree of freedom (e.g., roll degree of freedom) in accordance with the first orientation degree of freedom (e.g., roll degree of freedom) of the operating part.

[0395] For example, in the second operating mode, the orientation of the puncture device can also be configured to change according to the position of the operating part. For example, the movement of the operating part in the first position degree of freedom (such as the horizontal degree of freedom) can be converted into the movement of the puncture device's orientation in the first orientation degree of freedom (such as the yaw degree of freedom), the movement of the operating part in the second position degree of freedom (such as the vertical degree of freedom) can be converted into the movement of the puncture device's orientation in the second orientation degree of freedom (such as the pitch degree of freedom), and the movement of the operating part in the third position degree of freedom (such as the forward and backward degree of freedom) can be converted into the movement of the puncture device's orientation in the third orientation degree of freedom (such as the roll degree of freedom).

[0396] In the above embodiments, the controller typically needs to receive a command when it is desired to adjust the orientation of the trocar. This command can be input by the physician through interaction (such as voice, gestures, brainwaves, etc.), or the controller can generate the command, for example, by configuring a delay time and then generating it after the delay time expires. In response to receiving the command, the controller controls the movement of the joint components in the main arm to adjust the orientation of the trocar.

[0397] In the above embodiments, by adjusting the orientation of the puncture device, the operating space of the end effector of the medical device relative to the reference coordinate system of the main arm can be greatly improved, avoiding the inconvenience caused by the frequent occurrence of out-of-bounds problems.

[0398] In some embodiments, the controller may be configured to generate an image model of at least the first segment of motion boundary of the end effector as determined above, which frequently exceeds the boundary, and display it on any of the displays described above, for example, see [reference needed]. Figure 18 , Figures 20-24 Any of the image models shown. By viewing the image model displayed on the monitor, the doctor can clearly understand their operation and / or determine their desired operational intent. In other embodiments, the controller may also be configured to generate an image model of the motion boundary of the end effector and highlight a determined first segment, including through differences in color, brightness, lines (including line type and thickness), flicker, etc. In some embodiments, to avoid affecting the doctor's observation of the operation image, the controller may be configured not to display the aforementioned image model when the first segment cannot be determined.

[0399] In some embodiments, to facilitate the physician's understanding of the predicted target surgical procedure and / or the target orientation of the trocar, the controller can be configured to highlight feature points such as the center point of the identified organ and / or the determined first segment on the operation image, including highlighting the outline of the organ and highlighting the target center point on the organ and / or the first segment, so as to help the physician determine his operation intention based on the information displayed in the auxiliary image.

[0400] In some embodiments, the image model described above also includes a first portion of the manipulator assembly 120 being manipulated, which exemplarily includes an end effector. Of course, the first portion may also include other connecting components (composed of multiple joint components) including an end effector, such as those in the medical device 150. The end effector can appear in the image model in various ways, such as as an arrow, a circle, or an icon that is nearly identical in structure to the end effector. The position of the end effector in the image model can be calculated using forward kinematics. By generating an image model including the end effector, physicians can gain insight into the end effector and its boundaries, which is particularly beneficial when switching operating modes.

[0401] For example, when a doctor wants to adjust the orientation of the puncture device, combining the visual feedback mentioned above with the force feedback from the operating part when it is beyond the limits can help the doctor clarify their operating intentions and reduce the possibility of misoperation.

[0402] In some embodiments, master-slave alignment primarily refers to aligning the orientation of the end effector of the manipulator in the display coordinate system with the orientation of the end effector of the surgical instrument in the endoscopic coordinate system of the imaging instrument, which has a master-slave mapping relationship. The master-slave mapping relationship includes a communication connection between a manipulator and a surgical instrument, where movement of the manipulator can cause movement of the surgical instrument, or vice versa. In master-slave alignment mode, since multiple manipulators are housed in the same housing in a single-port laparoscopic surgical robot, and medical instruments mounted on the manipulators pass through the same trocar, the relative orientation relationship between the medical instruments is maintained when the orientation of the housing housing multiple manipulators is changed. Therefore, the orientation of the end effector of the surgical instrument in the end effector of the imaging instrument will not change. Thus, the single-port laparoscopic surgical robot does not require re-aligning the orientation of the end effector of the manipulator in the display coordinate system with the orientation of the end effector of the surgical instrument in the endoscopic coordinate system due to changes in the orientation of the housing.

[0403] In multi-port laparoscopic surgical robots, multiple manipulators are independently configured. Medical instruments mounted on these manipulators pass through different trocars. Changing the pose of any manipulator can alter the pose of the end effector of the surgical instrument in the end effector of the imaging instrument. Therefore, multi-port laparoscopic surgical robots require realignment of the manipulator's pose in the monitor coordinate system with the surgical instrument's pose in the endoscope coordinate system due to changes in manipulator pose. Whether in single-port or multi-port laparoscopic surgical robots, the manipulators are typically located outside the patient's body. Changes in the end effector's pose are usually influenced by the coordinated movement (sometimes called linkage) of the manipulators and the joint components within the medical instruments. In multi-port laparoscopic surgical robots, for example, in the event of a collision between manipulators, if the relative poses between the manipulators located outside the patient's body need to be manually re-planned, the end effector's pose in the monitor coordinate system can be configured to align with the end effector's pose in the endoscope coordinate system to maintain master-slave alignment, facilitating rapid transition to master-slave follow mode. Manually planning the relative pose relationships between manipulators is typically achieved by the operator applying force to the manipulators, thereby driving them to move around a remote center of motion. The movement of the manipulators around the remote center of motion usually includes at least one of pitch and yaw degrees of freedom.

[0404] Therefore, this disclosure can also provide a control method applicable to achieving a master-slave alignment mode in a multi-port laparoscopic surgical robot. Exemplarily, the slave operating device includes a first manipulator and a second manipulator. The first manipulator is used to mount and drive a first medical device, and the second manipulator is used to mount and drive a second medical device. Of the first and second medical devices, one is a surgical instrument and the other is an imaging instrument. Exemplarily, the operator drags the first manipulator to readjust the relative pose between the first and second manipulators, and configures an operating unit to establish a master-slave mapping relationship with the first manipulator and the first medical device mounted thereon, i.e., configures the operating unit to manipulate the first manipulator and the first medical device.

[0405] In some embodiments, see Figure 29 The control method includes:

[0406] Step S301: Obtain the current attitude of the end effector of the operator and the attitude of the first target.

[0407] The movement of the end effector is typically caused by the movement of the manipulator. In some embodiments, the target pose to which the end effector is expected to reach can be obtained, which can be described in the base coordinate system of the surgical robot; then, the target pose is parsed to obtain the target joint variables of the joint components in the manipulator; finally, the target pose of the end effector is determined by combining the target joint variables of the joint components in the manipulator and the current joint variables of the joint components in the medical device. At this time, the target pose of the end effector can still be described in the base coordinate system of the surgical robot. In the master-slave alignment mode, regardless of whether the medical device in the manipulator being adjusted is a surgical instrument or an imaging instrument, the pose of the end effector of the surgical instrument is changing in the endoscopic coordinate system, and the pose of the end of the manipulator is aligned with the pose of the end of the surgical instrument. Therefore, assuming that the medical device in the first manipulator being moved is a surgical instrument, the target pose of the end effector of the surgical instrument in the base coordinate system can be converted to the (second) target pose in the endoscope coordinate system; further assuming that the medical device in the first manipulator being moved is an imaging instrument, although the first manipulator is being moved, the surgical instrument mounted on the second manipulator is actually moved relative to the imaging instrument. In this case, the current pose of the end effector of the surgical instrument in the second manipulator in the base coordinate system can also be converted to the (second) target pose in the endoscope coordinate system based on the target pose of the end effector of the imaging instrument in the first manipulator in the base coordinate system.

[0408] In master-slave alignment mode, the target pose of the end effector of the manipulator usually needs to be aligned with the target pose of the end effector of the surgical instrument. That is, the first target pose of the manipulator in the display coordinate system can be determined based on the second target pose of the end effector of the surgical instrument in the endoscope coordinate system, without having to discuss whether the first manipulator that is moved among multiple manipulators is a surgical instrument or an imaging instrument.

[0409] In this disclosure, the pose (including position and orientation) of the end effector can be described in the endoscope coordinate system, or it can be transformed into a pose described in the endoscope coordinate system. The pose (including position and orientation) of the end effector of the manipulator can be described in the display coordinate system, or it can be transformed into a pose described in the display coordinate system.

[0410] Step S302: Detect whether the current posture has reached the motion boundary of the end of the operating unit, and detect whether the first target posture has exceeded the motion boundary.

[0411] The motion boundary refers to the attitude motion boundary of the end of the actuator, which is determined by the range of motion of all joint components in the actuator that are associated with the attitude degree of freedom.

[0412] If the current pose reaches the motion boundary and the first target pose exceeds the motion boundary, proceed to step S303; otherwise, proceed to step S307.

[0413] Step S303: Determine the hypothetical first resistance acting on the operating unit based on the first target attitude and the current attitude.

[0414] The first resistance is a virtual force used to hypothetically resist the movement of the end of the operating unit toward the first target posture; that is, it does not actually resist the movement of the end of the operating unit toward the first target posture.

[0415] In step S303, the end effector of the operating unit can be controlled to maintain its current posture.

[0416] Step S304: Determine the second resistance that actually acts on the end of the first manipulator based on the first resistance.

[0417] The end of a manipulator can refer to any part of the manipulator, such as specifying the part where a fixed force is applied as the end of the manipulator. For example, the end of a manipulator can be the part at the end of all joint components in the manipulator used to fix the applied force.

[0418] The second resistance is a real force that actually exists and will act on the end of the first manipulator to actually resist the movement of the end of the first manipulator toward the target position of the end of the first manipulator.

[0419] Step S305: Determine the target joint driving force to be output by the joint assembly in the first manipulator based on the second resistance.

[0420] Step S306: Control the joint component in the first manipulator assembly to output the joint target driving force.

[0421] By controlling the joint components in the first manipulator to output the joint target driving force, force feedback is achieved in the first manipulator, thereby prompting the operator not to move the first manipulator to the posture that generates the posture command.

[0422] Step S307: Determine the target joint variables of the joint components in the operating unit based on the first target posture.

[0423] Step S308: The joint assembly in the control operation unit outputs the target joint variable.

[0424] If the current posture has not reached the motion boundary and / or the first target posture has not exceeded the motion boundary, the posture of the end effector of the operating unit can follow the posture of the end effector of the first medical device, thereby maintaining the posture alignment of the two, which facilitates the subsequent rapid switching into master-slave control mode to perform surgery.

[0425] The various forces involved here, such as the first resistance, the second resistance, and the joint target driving force, can all be generalized forces, which include at least one of force and torque.

[0426] In some embodiments, in the master-slave alignment mode, since it is desired to provide force feedback to the operating part performing the following motion beyond the motion boundary of the end effector via the first manipulator, corresponding torque control is required for the joint components in the first manipulator. However, the first number of true degrees of freedom affecting the attitude of the end effector in the operating part may differ from the second number of true degrees of freedom affecting the attitude of the end effector in the first manipulator. For example, in... Figure 3 In the multi-port laparoscopic surgical robot shown, the true degrees of freedom affecting the attitude of the end effector in the manipulator include three: rotation, pitch, and yaw. In contrast, the true degrees of freedom affecting the attitude of the end effector in the first manipulator only include two: pitch and yaw. In other words, the degrees of freedom affecting attitude can also be called the true attitude degrees of freedom.

[0427] Of course, since the end effector includes a linkage and wrist joint assembly, it can still achieve the same degrees of freedom of rotation, pitch and yaw as the operating unit.

[0428] In some embodiments, to compensate for the lack of actual degrees of freedom affecting the attitude of the end effector in the first manipulator, it can be assumed that the first manipulator also includes a third number of hypothetical degrees of freedom to facilitate the determination of the joint target driving force in step S305 above. The third number is equal to the difference between the first number and the second number.

[0429] In some embodiments, the step of determining the target joint driving force output by the joint assembly in the first manipulator based on the second resistance includes: determining the target joint driving force output by the joint assembly in the first manipulator based on the second resistance and the (second number) actual degrees of freedom and the (third number) hypothetical degrees of freedom.

[0430] Correspondingly, the step of controlling the joint component in the first manipulator assembly to output the target joint driving force includes: controlling the (second number) real degrees of freedom of the joint component in the first manipulator assembly to output the target joint driving force. Here, hypothetical degrees of freedom only participate in determining the target joint driving force and do not participate in the output of the target joint driving force.

[0431] Although the first manipulator lacks the corresponding true degrees of freedom, by controlling the true degrees of freedom of the joint components in the first manipulator to output the joint target driving force, it is still possible to clearly prompt the operator not to move the first manipulator to the posture that generates the posture command.

[0432] In some embodiments, in step S305 above, other forces that are expected to actually act on the end effector of the first manipulator may be superimposed. These forces may include one or more desired effects, and are also generalized forces. For example, the forces to be superimposed include, but are not limited to, damping forces, and / or joint compensation driving forces used to balance forces generated by internal loads in the first manipulator. For example, the joint compensation driving force may include at least one of gravity compensation forces and friction compensation forces. For details on these potentially superimposed forces, please refer to the preceding text; they will not be repeated here.

[0433] In one embodiment, the forces to be superimposed include damping forces. Step S305 above includes:

[0434] The velocity of the end of the first manipulator is obtained, and the damping force actually acting on the end of the first manipulator is determined based on the velocity. This damping force is used to actually consume the potential energy accumulated when the second resistance acts on the end of the first manipulator.

[0435] Based on the second resistance and damping force, the target joint driving force expected to be output by the joint components in the first manipulation is determined.

[0436] In master-slave alignment mode, since the motion of the first manipulator around the remote center of motion is a type of pitch and / or yaw motion, the velocity is angular velocity. Further, the steps of obtaining the velocity of the end effector of the first manipulator and determining the damping force actually acting on the end effector of the first manipulator based on the velocity include:

[0437] Obtain the angular velocity of the end effector of the first manipulator;

[0438] Obtain the damping force model;

[0439] The damping force is determined by combining the angular velocity and damping force model.

[0440] For example, the damping force model can be expressed as the following formula:

[0441] (14)

[0442] In formula (14), For damping force; The angular velocity at the end of the first manipulator; The angular velocity damping coefficient; is the transpose of the Jacobian matrix of the first manipulator, wherein the Jacobian matrix is ​​determined in real time based on the joint variables of the joint components in the first manipulator.

[0443] In some embodiments, step S303 may include: when the current posture reaches the posture motion boundary and the first target posture exceeds the posture motion boundary, obtaining a first posture motion boundary impedance model, and determining a first resistance based on the deviation between the first target posture and the current posture, and the first posture motion boundary impedance model. The first posture motion boundary impedance model can be expressed, for example, as the formula (3) above, and will not be repeated here.

[0444] In some embodiments, step S304 may include: obtaining a second attitude motion boundary impedance model, determining a second resistance based on the first resistance and the second attitude motion boundary impedance model, wherein the second resistance determined by the second attitude motion boundary impedance model has an improvement in slope and / or rate of change of slope compared to the first resistance determined by the first attitude motion boundary impedance model. This second attitude motion boundary impedance model can be expressed, for example, as formula (4) above, and will not be repeated here.

[0445] In some embodiments, step S302 can also be implemented based on joint space angle considerations, and this step may include:

[0446] The target joint variables of the joint components in the operating unit are determined based on the first target posture, and the current joint variables of the joint components in the operating unit are determined based on the current posture.

[0447] A target joint component that satisfies a first condition is determined from the joint components in the operating unit. The first condition includes that the current joint variable of the target joint component reaches the joint motion boundary of the target joint component and that the target joint variable exceeds the joint motion boundary of the target joint component.

[0448] If there is a target joint component that meets the first condition mentioned above, it indicates that the current posture has reached the motion boundary in step S302, and the first target posture has exceeded the motion boundary.

[0449] See Figure 30 The above step S303 may include:

[0450] Step S3031: Based on the target joint variables and current joint variables of the target joint component, determine the hypothetical joint driving force acting on the target joint component.

[0451] Step S3032: Assign zero to the joint driving force of non-target joint components in the joint assembly of the operating unit that do not meet the first condition.

[0452] Step S3033: Based on the joint driving force of the target joint assembly and the joint driving force of the non-target joint assembly, determine the first resistance that is hypothetical to act on the end of the operating part.

[0453] In one embodiment, the above-mentioned step S3031 implementation method may include obtaining a joint boundary impedance model and, in combination with the target joint variable of the target joint component, the current joint variable, and the joint boundary impedance model, determining the hypothetical joint driving force acting on the target joint component. This joint boundary impedance model can, for example, be expressed as the formula (6) above, which will not be repeated here.

[0454] In step S3033 above, the joint driving force of the joint assembly in the operating part can be combined into a force vector and expressed in the matrix form as described in formula (7), which will not be repeated here.

[0455] Then, by directly obtaining the inverse solution of statics, the first resistance at the end of the first manipulator in the operating space can be obtained based on the joint driving force of the joint components in the operating part in the joint space.

[0456] In one embodiment, this disclosure also provides a controller for a surgical robot. For example... Figure 31 As shown, the controller may include: a processor 501, a communications interface 502, a memory 503, and a communications bus 504.

[0457] The processor 501, communication interface 502, and memory 503 communicate with each other through the communication bus 504.

[0458] The communication interface 502 is used to communicate with other network elements such as various sensors, motors, solenoid valves, or other clients or servers.

[0459] The processor 501 is used to execute program 505, which can specifically perform the relevant steps in the above method embodiments.

[0460] Specifically, program 505 may include program code that includes computer operation instructions.

[0461] The processor 501 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), one or more integrated circuits configured to implement embodiments of the present disclosure, or a graphics processing unit (GPU). The controller includes one or more processors, which may be processors of the same type, such as one or more CPUs or one or more GPUs; or they may be processors of different types, such as one or more CPUs and one or more GPUs.

[0462] Memory 503 is used to store program 505. Memory 503 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0463] Specifically, program 505 can be used to cause processor 501 to execute the control method as described in any of the above embodiments.

[0464] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0465] The embodiments described above are merely illustrative of several implementations of this disclosure, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this disclosure, and these all fall within the protection scope of this disclosure. Therefore, the protection scope of this patent should be determined by the appended claims.

Claims

1. A surgical robot, characterized in that, include: A first manipulator, which is equipped with and drives a first medical device, has a first joint assembly; The second manipulator is equipped with and drives the second medical device, wherein one of the first medical device and the second medical device is a surgical instrument and the other is an imaging device; The main control panel includes an operating unit and a display. The operating unit is configured to respond to the movement of the first manipulator relative to the second manipulator to a target pose, resulting in the end of the surgical instrument moving to a second target pose in the endoscopic coordinate system of the imaging instrument and to a first target pose in the display coordinate system of the display. and The controller, coupled to the first actuator, the second actuator, and the operating unit, is configured to: Obtain the first target pose of the end of the operating unit, and obtain the current pose of the end of the operating unit in the display coordinate system; When the current posture reaches the motion boundary of the end of the operating part and the first target posture exceeds the motion boundary, a first resistance that is hypothetically acting on the end of the operating part is determined based on the first target posture and the current posture, and a second resistance that is actually acting on the end of the first manipulator is determined based on the first resistance. The first resistance is used to hypothetically resist the movement of the end of the operating part toward the first target posture, and the second resistance is used to actually resist the movement of the end of the first manipulator toward the target posture. Based on the second resistance, the desired joint target driving force output by the first joint assembly is determined, and the first joint assembly is controlled to output the joint target driving force to achieve force feedback in the first manipulator; The first resistance, the second resistance, and the joint target driving force are generalized forces; The step of determining the target joint driving force to be output by the first joint assembly based on the second resistance includes: The velocity of the end of the first manipulator is obtained, and the damping force actually acting on the end of the first manipulator is determined based on the velocity. The damping force is used to actually consume the potential energy accumulated when the second resistance acts on the end of the first manipulator. Based on the second resistance and the damping force, determine the target joint driving force that the first joint assembly is expected to output; The speed is a generalized speed, and the damping force is a generalized force.

2. The surgical robot according to claim 1, characterized in that, The operating part includes a first number of actual degrees of freedom that actually affect the posture of the end effector of the operating part. The first joint assembly includes a second number of actual degrees of freedom that actually affect the posture of the end effector of the first medical device. The first number is greater than the second number. The first joint assembly also includes a third number of hypothetical degrees of freedom that hypothetically affect the posture of the end effector of the first medical device. The third number is equal to the difference between the first number and the second number. The step of determining the desired joint target driving force output by the first joint assembly based on the second resistance and controlling the first joint assembly to output the joint target driving force includes: The target joint driving force is output based on the actual degree of freedom and the hypothetical degree of freedom of the first joint assembly, determined by the second resistance. The target driving force of the joint is output by controlling the true degrees of freedom of the first joint component.

3. The surgical robot according to claim 2, characterized in that, The first quantity is three, and the second quantity is two.

4. The surgical robot according to claim 2, characterized in that, The operating unit includes rotational degree of freedom, yaw degree of freedom, and pitch degree of freedom, and the first manipulator includes yaw degree of freedom and pitch degree of freedom.

5. The surgical robot according to claim 1, characterized in that, The step of obtaining the velocity of the end effector of the first manipulator and determining the damping force actually acting on the end effector of the first manipulator based on the velocity includes: Obtain the angular velocity of the end effector of the first manipulator; Obtain the damping force model; The damping force is determined by combining the angular velocity and the damping force model.

6. The surgical robot according to claim 5, characterized in that, The damping force model is expressed by the following formula: , The angular velocity of the end effector of the first manipulator; The angular velocity damping coefficient; It is the transpose of the Jacobian matrix of the first manipulator, wherein the Jacobian matrix is ​​determined in real time based on the joint variables of the first joint assembly.

7. The surgical robot according to claim 1, characterized in that, The step of determining the desired target driving force of the first joint assembly based on the second resistance includes: Obtain the joint compensation driving force that the first joint assembly is expected to output to balance the force caused by the internal load; Based on the second resistance and the joint compensation driving force, determine the target joint driving force that the first joint assembly is expected to output; The joint compensation driving force is a generalized force, and the force caused by the internal load includes at least one of gravity and friction.

8. The surgical robot according to claim 1, characterized in that, The motion boundary includes the posture motion boundary in the operating space. The step of determining a hypothetical first resistance acting on the end of the operating part based on the first target posture and the current posture, and determining a second resistance actually acting on the end of the first manipulator based on the first resistance, includes: When the current posture reaches the posture motion boundary and the first target posture exceeds the posture motion boundary, a first posture motion boundary impedance model is obtained, and the first resistance is determined based on the deviation between the first target posture and the current posture and the first posture motion boundary impedance model. Obtain a second attitude motion boundary impedance model, and determine the second resistance based on the first resistance and the second attitude motion boundary impedance model. The second resistance determined by the second attitude motion boundary impedance model has an improvement in slope and / or rate of change of slope compared with the first resistance determined by the first attitude motion boundary impedance model.

Images