Surgical robotic system

Abstract

Embodiments of the present application provide a surgical robot system, which includes a plurality of operation modes, the plurality of operation modes including at least a follow mode and a camera mode, in the follow mode, a pose of a surgical instrument is controlled by a first action of an input device to follow a pose of the input device; an operation mode is switched from the follow mode to the camera mode according to an endoscope mode action and the first action; in the camera mode, a position and a pose of an endoscope are adjusted by a second action of the input device, wherein the second action changes the position of the input device without changing the pose of the input device.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical treatment, and in particular, to a surgical robot. BACKGROUND

[0002] Minimally invasive medical technology refers to a medical treatment method of performing surgery or biopsy in the body cavity of a human body by using a laparoscope, a thoracoscope and other modern medical instruments and related equipment. Compared with a traditional surgical method, minimally invasive medical technology has advantages of small trauma, light pain, fast recovery, alleviation of discomfort of a patient and reduction of harmful side effects.

[0003] With the progress of science and technology, minimally invasive medical surgical robot technology has gradually matured and is widely used. A minimally invasive surgical medical robot usually includes a master control console and a slave operating device. A doctor controls the slave operating device by using an input device of the master control console. The slave operating device is used to respond to a control command sent by the master control console and perform a corresponding surgical operation. An instrument is connected with a driving device of the slave operating device and is used to perform a surgical operation. A distal end of the instrument includes an end device used to perform a surgical operation and a joint assembly connected with the end device and capable of multiple degrees of freedom movement.

[0004] During pre-operation and intra-operation, a surgeon needs to detect an environment in a patient's body by using an endoscope of a surgical robot. There is no good solution for how to use multiple endoscopic operation modes to facilitate the surgeon to detect the environment in the patient's body and reduce training time and reduce misoperation. SUMMARY

[0005] Therefore, in a first aspect, the present application provides a surgical robot, which includes:

[0006] a plurality of driving devices configured to drive a surgical instrument and an endoscope;

[0007] an input device configured to manipulate the plurality of driving devices in a plurality of operation modes, the plurality of operation modes including at least a follow mode, a camera mode, a repositioning mode and an adjustment mode;

[0008] a control device configured to:

[0009] switch the operation mode from the follow mode to the adjustment mode in response to the entering endoscope mode action; in the follow mode, a first driving device of the plurality of driving devices is controlled to move according to a first action of the input device to drive the surgical instrument to follow the movement of the input device;

[0010] switching the operation mode from the repositioning mode to the camera mode in response to an entering endoscope mode action and a first action of the input device, in the camera mode, a second driving device of the plurality of driving devices is controlled to move according to a second action of the input device to adjust a position and an attitude of the endoscope;

[0011] switching the operation mode from the repositioning mode to the camera mode in response to an entering endoscope mode action and a first action of the input device, in the camera mode, a second driving device of the plurality of driving devices is controlled to move according to a second action of the input device to adjust a position and an attitude of the endoscope;

[0012] In one embodiment, the second action changes a position of the input device without changing an attitude of the input device.

[0013] In one embodiment, the surgical robot system further comprises a mechanical arm having a plurality of joints, the plurality of driving devices are connected to a distal end of the mechanical arm, in the adjustment mode, the control device controls the mechanical arm to move according to the second action of the input device to rotate the endoscope around a remote center of motion to adjust a position and an attitude of a distal end of the endoscope while maintaining a position and an attitude of a distal end of the surgical instrument unchanged.

[0014] In one embodiment, in the repositioning mode, the control device controls the mechanical arm to move according to the second action of the input device to adjust a position and an attitude of the surgical instrument and the endoscope simultaneously.

[0015] In one embodiment, in the camera mode, the adjustment mode and the repositioning mode, the first action does not cause the plurality of driving devices and the joints of the mechanical arm to move.

[0016] In one embodiment, first and second motion boundaries for constraining the motion of the mechanical arm are pre-stored, after the operation mode is switched from the adjustment mode to the repositioning mode, the control device synchronously switches a motion range of the mechanical arm from the first motion boundary to the second motion boundary.

[0017] In one embodiment, the first motion boundary is determined according to a motion range of a joint assembly of the surgical instrument, the second motion boundary is determined according to a motion range of the plurality of joints of the mechanical arm.

[0018] In one embodiment, the control device calculates a first target position of the robotic arm according to the second motion, and when the first target position exceeds the first motion boundary or the second motion boundary, the control device restricts further motion of the robotic arm and generates an impedance force command that hinders the input device from continuing to move, to generate an impedance force on the input device that hinders the input device from continuing to move.

[0019] In one embodiment, the impedance force increases as the distance between the first target position and the first motion boundary or the second motion boundary increases.

[0020] In one embodiment, the control device calculates a second target position of the robotic arm according to the second motion, and when the second target position is closer to the first motion boundary or the second motion boundary than the first target position, the control device releases the restriction on the motion of the robotic arm and stops generating the impedance force command.

[0021] In one embodiment, the endoscope includes a camera, a wrist joint connected between the camera and a parallel joint, the motion of the wrist joint changes the position and the pose of the camera, the motion of the parallel joint changes the position of the camera and keeps its pose unchanged, and the second motion manipulates the motion of the wrist joint and does not manipulate the motion of the parallel joint. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 A top view of a surgical robot system according to an embodiment of the present application arranged in an operating room;

[0023] Figure 2A A schematic diagram of a master control console of a surgical robot system according to an embodiment of the present application;

[0024] Figure 2B A schematic diagram of a slave operating device of a surgical robot system according to an embodiment of the present application;

[0025] Figure 3A A schematic diagram of a surgical tool according to an embodiment of the present application;

[0026] Figure 3B A schematic diagram of the internal structure of a tool holding device according to an embodiment of the present application;

[0027] Figure 4A A schematic diagram of a slave operating device according to an embodiment of the present application;

[0028] Figure 4B A schematic diagram of a kinematic model of a slave operating device according to an embodiment of the present application;

[0029] Figure 5 Kinematic model diagram of a surgical tool for one embodiment of the present application;

[0030] Figure 6 Diagram of multiple surgical tools passing through a cannula for one embodiment of the present application;

[0031] Figure 7A Diagram of the relationship between the coordinate system of an input device and the coordinate system of a display device for one embodiment of the present application;

[0032] Figure 7B Diagram of the relationship between the coordinate system of a surgical tool tip and the coordinate system of an endoscope tip for one embodiment of the present application;

[0033] Figure 8 Flowchart of a follow mode control method for one embodiment of the present application;

[0034] Figure 9 Flowchart of an adjustment mode control method for one embodiment of the present application;

[0035] Figure 10 Flowchart of an endoscope straightening control method for one embodiment of the present application;

[0036] Figure 11 Flowchart of a control method for switching between multiple operation modes for one embodiment of the present application. DETAILED DESCRIPTION

[0037] For the purpose of promoting an understanding of the principles of the application, reference will now be made to the embodiments illustrated in the drawings. There is shown in the drawings, several embodiments of the application. It is readily appreciated that the drawings are not to scale and not all aspects of the application are to be shown to the same scale, that the variations can be made and that subtleties of the application can be grasped from the following disclosure.

[0038] It should be noted that when an element as a "set on" another element, it can be directly on the other element or there can be an intervening element. When an element is considered to be "connected" to another element, it can be directly connected to the other element or there can be an intervening element, and it can also mean that the two elements are connected by signal interaction. When an element is considered to be "coupled" to another element, it can be directly coupled to the other element or there can be an intervening element, and it can also mean that the two elements are connected by signal interaction. The terms "vertical", "horizontal", "left", "right", "above", "below", and similar expressions as used herein are used for illustration only and do not indicate the only orientation in which the device can be used or operated. It is understood that these spatially relative terms are intended to encompass different orientations of the device in its operation or use, for example, if the device is inverted or flipped over. As such, an element described as "above" or "below" another element can be oriented above or below the other element in use or operation. Thus, the example term "above" can include both above and below orientations.

[0039] The terms "distal" and "proximal" are used herein as orientation terms that are conventional in the field of interventional medical devices, where "distal" means away from the surgeon during a procedure and "proximal" means closer to the surgeon during a procedure. The term "a plurality" includes two or more.

[0040] The term "instrument" is used herein to describe a medical device that is used for insertion into a patient's body and for performing a surgical or diagnostic procedure, and which includes an end effector that can be a surgical tool for performing a surgical procedure, such as a biopsy needle, an electrocautery, a forceps, a stapler, a scissor, an imaging device (e.g., an endoscope or an ultrasound probe), and the like. Some of the instruments used in the embodiments of the present application further include an articulating member (e.g., a joint assembly) that provides the end effector with one or more mechanical degrees of freedom for manipulating the position and orientation of the end effector. Further, the end effector includes functional mechanical degrees of freedom, such as opening and closing forceps. The instrument can also include a memory that can be updated by a surgical system, whereby the memory can provide one-way or two-way communication between the instrument and one or more system elements.

[0041] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0042] A surgical robot system according to an embodiment of the present application is shown in FIG. 1. The surgical robot system includes a master console 20 and a slave operating device 10, the master console 20 is communicatively connected to the slave operating device 10, a surgeon S can remotely operate the slave operating device 10 on the master console 20. The master console 20 is configured to send control signals to the slave operating device 10 according to the operation of the surgeon S and display the images acquired by the slave operating device 10, the surgeon S can observe the three-dimensional images of the patient's body provided by the image system through the master console 10, and the surgeon S can control the slave operating device 10 to perform related operations (such as performing surgery or acquiring images of the patient's body) with an immersive feeling by observing the three-dimensional images of the patient's body. Figure 1

[0043] The slave operating device 10 includes a control device, a mechanical arm 11 and a tool holding device 12, the control device can be arranged in the base of the slave operating device 10 or arranged on the mechanical arm 11, in an embodiment, the control device is used to control the joint movement of the mechanical arm 11 and the movement of the driving device in the tool holding device 12. A plurality of surgical tools can be installed on the tool holding device 12, and the driving device of the tool holding device 12 is used to drive the surgical tools to act to perform various surgeries.

[0044] In an embodiment, the surgical robot system further includes a gas blowing device, a lumen set (not shown in the figure) and a sleeve 13, the lumen set fluidly connects the sleeve 13 with the gas blowing device. The sleeve 13 is connected to the distal end of the tool holding device 12, and the sleeve is inserted into the body cavity of the patient P lying on the operating table T, and the distal end device of the plurality of surgical tools or the camera at the distal end of the endoscope is inserted into the body cavity of the patient P through the sleeve 13 to perform the operation related operations or acquire the environment images of the patient P.

[0045] ​In one embodiment, the surgeon S controls the working mode of the gas insufflation device through the master console 10, such as injecting gas in the gas source into the body cavity of the patient P to form an artificial pneumoperitoneum, or sucking gas from the body cavity of the patient P. The assistant A installs the surgical tool 40 to the tool holding device 12 or replaces the surgical tool 40 from the tool holding device 12 according to the surgical situation. The surgeon S, the assistant A and the anesthetist B form the basic surgical team. The surgical tool 40 can be an electric cauter, a forceps, a stapler, an ultrasonic knife or other surgical tool for performing surgical operations, or an image device (such as an endoscope) for acquiring images or other surgical tools.

[0046] The master console 10 is also remotely connected with the electronic equipment cart 30, and the electronic equipment cart 30 is remotely connected with the slave operating device 10. The electronic equipment cart 30 can include energy generating devices, image signal processing devices, and the above-mentioned gas insufflation devices, etc. In this embodiment, the master console 10, the slave operating device 10 and the electronic equipment cart 30 remotely communicate through wired Ethernet communication mode, but the remote communication is not limited to wired Ethernet communication, and can be other wired modes, such as but not limited to serial port, CAN, RS485, RS232, USB, SPI, etc., or wireless communication modes, such as but not limited to 5G, WiFi, NB, Zigbee, Bluetooth, RFID, etc.

[0047] In one embodiment, as shown in Figure 2A The master console 20 includes a display device 21, an armrest 22, an input device 23, an observation device 24 and a control signal processing system 25, wherein the display device 21 is used to display the images acquired by the above-mentioned image system. The armrest 22 is used to place the arm and / or hand of the doctor S (such as the surgeon S) so that the doctor S can more comfortably operate the input device 23, and the observation device 24 is used to observe the images displayed by the display device. According to actual needs, the armrest can also be omitted; or the observation device 24 can be omitted, at this time it can be directly observed. The doctor S operates the surgical tool of the slave operating device 10 by operating the input device 23, and the control signal processing system of the master console 20 processes the input signal of the input device 23 and then sends a control command to the slave operating device. The slave operating device 10 responds to the control command of the master console 20 and performs corresponding operations. In some embodiments, the control signal processing system 25 can also be arranged in the slave operating device 10, such as arranged in the base of the slave operating device 10. The control signal processing system 25 can be the same device as the above-mentioned control device.

[0048] Surgical robotic systems typically also include an imaging system (not shown) that enables the surgeon S to view the surgical site from outside the patient's body. This imaging system typically includes a surgical tool 40 with video image acquisition capabilities (e.g., an image acquisition function) and one or more video display devices for displaying the acquired images. Generally, the surgical tool 40 with image acquisition capabilities includes optics of one or more imaging sensors (e.g., CCD or CMOS sensors) that acquire images of the patient's body. These one or more imaging sensors can be positioned distal to the surgical tool 40 with image acquisition capabilities, and the signals generated by these sensors can be transmitted via cable or wirelessly for processing and display on the video display device.

[0049] In one embodiment, such as Figure 2B As shown, the robotic arm 11 of the surgical robot system, which is operated by the device 10, includes a base 110, a column 120 connected to the base 110, and a large arm 130, a forearm 140, and a vertical arm 150 connected in sequence. The robotic arm also includes multiple joints J1-J5 for connecting the column 120, the large arm 160, the forearm 140, and the vertical arm 150. Specifically, the column 120 includes a support column 121 and a lifting column 122. The support column 121 is fixedly connected to the base 110, and the lifting column 122 is connected to the support column 121 through a first joint J1. The first joint J1 is a linear motion joint, and the lifting column 122 can move linearly along the axis 101 of the first joint J1 to change the height of the portion of the robotic arm 11 connected to the distal end of the column 120. The lifting column 122 is connected to the upper arm 130 via the second joint J2. The upper arm 130 is connected to the lower arm 140 via the third joint J3. The lower arm 140 is connected to the vertical arm 150 via the fourth joint J4. The second joint J2, the third joint J3, and the fourth joint J4 are all rotary joints, and the rotation axes 102, 103, and 104 of these three rotary joints are all perpendicular to the horizontal plane. The vertical arm 150 is connected to the holding device 112 via the fifth joint J5. The axis 105 of the fifth joint J5 is perpendicular to the axes 101-104.

[0050] The control device 160 is configured to control multiple joints J1-J5 in linkage to achieve various positions of the entire robotic arm 11, adjust the position and posture of the holding device 112, and realize the rotational movement of the holding device 112 around its remote motion center 116 at its far end. The control device 160 can be set in the base 110 or in the main control console 20.

[0051] In one embodiment, the instrument holder 112 further comprises a sleeve 115 removably coupled to the instrument holder 112 via the docking device 114, the central axis 106 of the instrument holder 112 substantially coincides with an axis 118 of the sleeve 115, and the instrument holder 112 rotates the sleeve 115 about the remote center of motion 116, and since the remote center of motion 116 is located at the incision 117, the sleeve 115 does not damage the patient P when the sleeve 115 rotates about the remote center of motion 116.

[0052] In one embodiment, the operating apparatus 10 further comprises a control panel 170 disposed on the support column 121, the control panel 170 comprises at least one switch 171 for inputting a positioning instruction to the control device 160, and the control device 160 controls the movement of the robotic arm 11 to quickly achieve a variety of predetermined poses of the robotic arm 11, such as a pose for arranging a sterile drape, in response to the action of the switch 171.

[0053] In one embodiment, the instrument holder 112 can be loaded with a plurality of surgical tools 40, and the plurality of surgical tools 40 enter the body through the same sleeve 115 from the incision 117. As shown in Figure 3A The surgical tool 40 comprises an instrument box 41, a long shaft 42, a joint assembly 43, and an end device 44, and the surgical tool 40 is removably mounted on a driving system of the instrument holder 112 of the operating apparatus 10, the instrument box 41 has a transmission device (not shown) therein, the transmission device comprises a plurality of transmission units (e.g., winches), the plurality of transmission units are connected to the joint assembly 43 and the end device 44 via a plurality of cables, and the plurality of transmission units are respectively coupled to a plurality of actuators (e.g., motors) in the driving system and are driven by the actuators. The plurality of actuators receive control instructions from the control device and drive the end device 44 to move according to the control instructions by driving the transmission units to rotate and thereby control the cables to be wound / tensioned. The end device 44 can perform a plurality of Cartesian degrees of freedom actions via the joint assembly 43, such as translational movement (including lateral movement and / or longitudinal movement) for changing the position of the end device 44 and pitch, yaw, and roll movement for changing the attitude of the end device 44, and it can be understood that the translational movement and the pitch, the translational movement, the yaw, and the roll movement can be independently moved or simultaneously moved. The end device 44 is used to perform operations related to surgical operations, and the end device 44 can be an electric cauter, a forceps, a stapler, a scissors, an ultrasonic knife, a camera, an imaging device, etc. according to the needs of the surgical operation, wherein the camera or the imaging device is used to obtain internal images of the human body.

[0054] In one embodiment, as Figure 3BAs shown, the drive system of the instrument holder 112 includes at least a first drive device 2201 and a second drive device 2202. A plurality of surgical tools of the surgical instrument 310 is detachably mounted on the first drive device 2201, and a plurality of surgical tools of the endoscope 320 is detachably mounted on the second drive device 2202. A plurality of actuators in the first drive device 2201 is coupled to transmission devices in an instrument box 311 of the surgical instrument 310, and the first drive device 2201 is configured to drive the joint assembly 313 of the surgical instrument 310 to perform a plurality of degrees of freedom movements and a rotation movement of the long shaft 312. A plurality of actuators in the second drive device 2202 is coupled to transmission devices in an instrument box 321 of the endoscope 320, and the second drive device 2202 is configured to drive the joint assembly 323 of the endoscope 320 to perform a plurality of degrees of freedom movements and a rotation movement of the long shaft 322.

[0055] In one embodiment, the drive system of the instrument holder 112 further includes a first feed drive device 2211 and a second feed drive device 2212. The first feed drive device 2211 is configured to drive the first drive device 2201 to perform a linear movement along a guide rail 2213, thereby driving the surgical instrument 310 to perform a feed movement between the proximal end and the distal end. The second feed drive device 2212 is configured to drive the second drive device 2202 to perform a linear movement along a guide rail 2214, thereby driving the endoscope 320 to perform a linear movement.

[0056] Figure 4A is a schematic diagram of a kinematic model of the slave manipulator 500 according to one embodiment, Figure 4B is a schematic diagram of a kinematic model of the slave manipulator 500 according to one embodiment, Figure 4A As shown, the kinematic model 500 includes kinematic information associated with a plurality of active devices. The kinematic information is based on known kinematic models of the links (e.g., the column, the large arm, the small arm, and the vertical arm) and the joints of the slave manipulator 200. The kinematic information is further based on information associated with the positions and poses of the joints of the slave manipulator 200, which are measured by using one or more position sensors (e.g., encoders) to measure the linear positions of the prismatic joints and the rotational positions of the revolute joints. The kinematic model 500 includes a plurality of coordinate systems and coordinate system transformation relationships (e.g., homogeneous transformations of coordinate systems) for transforming the positions and poses of the target object from a description in one coordinate system to a description in another coordinate system. The kinematic model 500 can use one or more of the coordinate systems and the coordinate system transformation relationships involved to establish kinematic relationships between adjacent or non-adjacent coordinate systems through forward or inverse transformations. In some embodiments, the kinematic model 500 is used to model the kinematic relationships of the slave manipulator 200 in Figure 4A .

[0057] The kinematic model 500 includes a base coordinate system 510 for kinematically modeling the positions and poses of the joints of the slave manipulator 200, which is established on the base 210 and remains relatively stationary with respect to the base of the slave manipulator 200. In some embodiments, the base coordinate system 510 is used as a reference point for modeling the kinematic relationships between the joints of the slave manipulator 200. Once the base coordinate system 510 is determined, the origin and the orientation of the coordinate axes of the base coordinate system 510 are also determined with respect to the geometry of the base 210. In addition, to facilitate the intuitive description of the kinematic information of the joints, the Z-axis of the base coordinate system 510 can be oriented perpendicular to the ground or floor surface on which the slave manipulator 200 is placed.

[0058] The kinematic model 500 further includes a column coordinate system 520 for modeling the position and pose of the lifting column 240, which is established on the lifting column 240 and remains relatively stationary with respect to the lifting column 240. In some embodiments, to facilitate the convenience and speed of establishing the coordinate transformation relationship between adjacent coordinate systems, the Z-axis of the column coordinate system 520 can be coincident with the central axis 202 of the lifting column 240, and the X-axis and Y-axis of the column coordinate system 520 can be parallel to the coordinate axes of the base coordinate system 510, while satisfying the rules for establishing coordinate systems. The origin of the column coordinate system 520 can be determined at the intersection of the central axis 202 of the lifting column 240 and the horizontal center plane 201 of the boom 250. In some embodiments, one or more sensors can be used to measure the translational position of the lifting column 240 with respect to the fixed support column 220, and further used to determine the coordinate transformation relationship 515 between the column coordinate system 520 and the base coordinate system 510.

[0059] The kinematic model 500 further includes a base coordinate system 510, which is established at the origin of the base 210. In some embodiments, the base coordinate system 510 can be established at the intersection of the horizontal center plane 201 and the vertical center axis 202 of the base 210, for describing the motion information of the base 210. Based on the rules of establishing coordinate system, the Z-axis of the base coordinate system 510 can coincide with the vertical center axis 202 of the base 210. In some embodiments, the rotational position of the base 210 relative to the ground can be measured using one or more sensors, and further be used to determine the coordinate system transformation relationship 515 between the base coordinate system 510 and the ground coordinate system 500. Based on the coordinate system transformation relationship 515, the kinematic model between the base coordinate system 510 and the ground coordinate system 500 can be established, and further be used to describe the motion information of the base 210 in the ground coordinate system 500. In some embodiments, based on the known motion information of the base 210 in the ground coordinate system 500, the kinematic inverse transformation based on the coordinate system transformation relationship 515 can be used to determine the motion information of the joints of the base 210.

[0060] The kinematic model 500 further includes a base coordinate system 510, which is established at the origin of the base 210. In some embodiments, the base coordinate system 510 can be established at the intersection of the horizontal center plane 201 and the vertical center axis 202 of the base 210, for describing the motion information of the base 210. Based on the rules of establishing coordinate system, the Z-axis of the base coordinate system 510 can coincide with the vertical center axis 202 of the base 210. In some embodiments, the rotational position of the base 210 relative to the ground can be measured using one or more sensors, and further be used to determine the coordinate system transformation relationship 515 between the base coordinate system 510 and the ground coordinate system 500. Based on the coordinate system transformation relationship 515, the kinematic model between the base coordinate system 510 and the ground coordinate system 500 can be established, and further be used to describe the motion information of the base 210 in the ground coordinate system 500. In some embodiments, based on the known motion information of the base 210 in the ground coordinate system 500, the kinematic inverse transformation based on the coordinate system transformation relationship 515 can be used to determine the motion information of the joints of the base 210.

[0061] The kinematics model 500 further includes a vertical arm coordinate system 550, which is established at the end of the vertical arm 270. In some embodiments, the vertical arm coordinate system 550 can be established at the intersection of the central axis 204 of the vertical arm 270 and the rotation axis 206 of the fifth joint J5, for describing the motion information of the vertical arm 270. On the basis of satisfying the coordinate system establishment rules, the Z-axis of the vertical arm coordinate system 550 can coincide with the rotation axis 206 of the fifth joint J5. In some embodiments, the rotational position of the vertical arm 270 relative to the forearm 260 can be measured using one or more sensors, and further used to determine the coordinate system transformation relationship 545 between the vertical arm coordinate system 550 and the forearm coordinate system 540. In some embodiments, based on the combination of the coordinate system transformation relationships 515-545, a kinematics model between the coordinate systems 550 and 510 can be established, and further can be used to describe the motion information of the position and attitude of the vertical arm 270 in the base coordinate system 510. In some embodiments, based on the known motion information of the position and attitude of the vertical arm 270 in the base coordinate system 510, the kinematics inverse transformation can be used to determine the motion information of the joints of the lifting column 240, the forearm 250, the forearm 260 and the vertical arm 270 based on the coordinate system transformation relationships 515-545.

[0062] The kinematics model 500 further includes a tool holding device coordinate system 560, which is established at the end of the tool holding device 280. In some embodiments, the tool holding device coordinate system 560 can be established at the end of the tool holding device 280 and on the vertical center plane 207 of the tool holding device 280 (for example, on the docking portion of the end of the tool holding device), for describing the motion information of the tool holding device 280. In some embodiments, the rotational position of the tool holding device 280 relative to the vertical arm 270 can be measured using one or more sensors, and further used to determine the coordinate system transformation relationship 555 between the tool holding device coordinate system 560 and the vertical arm coordinate system 550. In some embodiments, based on the combination of the coordinate system transformation relationships 515-555, a kinematics model between the coordinate systems 560 and 510 can be established, and further can be used to describe the motion information of the position and attitude of the tool holding device 280 in the base coordinate system 510. In some embodiments, based on the known motion information of the position and attitude of the tool holding device 280 in the base coordinate system 510, the kinematics inverse transformation can be used to determine the motion information of the joints of the robotic arm 211 based on the coordinate system transformation relationships 515-555, and to accurately control the motion of the end point of the robotic arm 211 of the operating device 200.

[0063] The kinematic model 500 further includes a remote motion center coordinate system 570 for the robotic arm 211, the origin of which coincides with the remote motion center 116. In some embodiments, the coordinate system transformation relationship 565 between the remote motion center coordinate system 570 and the holding device coordinate system 560 can be determined through the mechanism geometry on the holding device 280. Furthermore, based on the combination of coordinate system transformation relationships 515 to 565, a coordinate transformation relationship 575 between coordinate systems 570 and 510 can be established, which can be further used to describe the positional motion information of the remote motion center 116 of the robotic arm 211 in the base coordinate system 510 of the trolley base and the postureal motion information of the holding device 280. In some embodiments, given the positional motion information of the remote motion center 116 in the base coordinate system 510 and the postureal motion information of the holding device 280, the motion information of each joint of the robotic arm 211 can be determined through inverse kinematic transformation based on the coordinate transformation relationship 575, thereby enabling the hand-held dragging holding device 280 to perform translational and rotational movements in the workspace.

[0064] Figure 5 This is a simplified kinematic model diagram of multiple surgical tools 40 for a surgical robot according to an embodiment of this application. Figure 5 As shown, multiple surgical tools 40 include surgical instruments and endoscopes. To accurately control the motion of the surgical tools, a base coordinate system 610 is typically established for each surgical tool 40, an axial translation coordinate system 620 is established for the translational degrees of freedom along the central axis 106 of the holding device 112, and a rotation coordinate system 630 is established for the rotational degrees of freedom of the surgical tools around their respective major axes. Furthermore, as... Figure 6 As shown, parallel joint coordinate system 640, wrist joint coordinate system 650, and distal coordinate system 660 are established for the other motion degrees of freedom of the distal end of the surgical tool 40, respectively. The parallel joint coordinate system 640 is established on the proximal base of the proximal joint 4221 of the parallel joint 422.

[0065] In some embodiments, based on coordinate system transformation relationships 615-655, a kinematic model relationship 665 can be established between the end-effector coordinate system 660 and the surgical tool base coordinate system 610, and can be further used to describe the pose and motion information of the end-effector of the surgical tool or the endoscope image in the base coordinate system 610. In some embodiments, based on the known pose and motion information of the end-effector coordinate system 660 in the surgical tool base coordinate system 610, the inverse transformation of the coordinate system transformation relationship 665 can be solved to calculate the motion information of each joint of the surgical tool. In some embodiments, given the known fixed kinematic relationships between the various surgical tool base coordinate systems 610, the pose information of the end-effector of the surgical instrument in the endoscope image can be obtained based on the coordinate system transformation relationship 665, thereby enabling master-slave control operation of the surgical instrument based on the master-slave motion control model. In some embodiments, a kinematic relationship 675 can be established between the surgical tool base coordinate system 610 and the remote motion center coordinate system 560 of the patient surgical platform, thereby establishing a connection between the surgical tool 40 and the robotic arm 211.

[0066] In one embodiment, such as Figure 6 As shown, multiple surgical instruments include surgical instruments 410, 430 and an endoscope 420. The surgical instruments 410, 430 and the endoscope 420 pass together through a cannula 115. In this embodiment, surgical instrument 410 has an end-effector 411 that is a bipolar cauterization forceps; surgical instrument 420 has an end-effector 424 that is a camera; and surgical instrument 430 has an end-effector 424 that is a needle holder. The two surgical instruments 410, 430 and the endoscope 420 have the same articulated assembly. In other embodiments, there may be other numbers of surgical instruments, such as four. In other embodiments, the articulated assemblies of surgical instruments 410, 420, and 430 may have different configurations to achieve different degrees of freedom of movement, depending on the requirements.

[0067] In one embodiment, the surgical instrument 420 includes a proximal instrument cartridge ( Figure 6 (Not shown), long axis 421, joint assembly, and camera 424. The joint assembly includes a parallel joint 422 and a wrist joint 423. The parallel joint 422 includes a proximal joint 4211, a middle segment 4223, and a distal joint 4222. The proximal joint 4211 and the distal joint 4222 move simultaneously to change the position of the camera 424, but can maintain the camera 424's posture. For a detailed explanation of the movement principle of the parallel joint 422, please refer to Chinese patent applications CN202111604327.6 (Instrument with Parallel Joints, Surgical Robot) and CN202111604322.3 (Instrument with Parallel Joints, Surgical Robot). Movement of the wrist joint 423 can change the position and posture of the camera 424 to explore the internal environment.

[0068] In some embodiments, the master console 20 has four different operating modes to operate the surgical tools 410, 420, 430, and the first mode among the four operating modes is a follow mode, as shown in Figure 7A A simplified schematic diagram of the motion relationship of the input device 23 of an embodiment of the present application is shown. A Cartesian space coordinate system ML is established at the handle position of the left input device 23L, a Cartesian space coordinate system MR is established at the handle position of the right input device 23R, and a Cartesian display screen coordinate system MB is established at the display device 21. Based on the mechanical structure design, the left input device 23L and the right input device 23R both have six degrees of freedom of motion, and the six degrees of freedom of motion of the left input device 23L include three translational degrees of freedom along the coordinate axes X ML , Y ML , Z ML of the coordinate system ML and three rotational degrees of freedom about the coordinate axes X ML , Y ML , Z ML ; similarly, the right input device 23R also has three translational degrees of freedom along the coordinate axes X MR , Y MR , Z MR and three rotational degrees of freedom about the coordinate axes X MR , Y MR , Z MR . Through the kinematic relationship between the coordinate systems ML, MR of the input device 23 and the coordinate system MB of the display device, the six degrees of freedom of motion information of the surgeon S's hand can be converted to the display screen coordinate system MB through motion, so that the motion information of the surgeon S's hand can be stored and recorded in the master console.

[0069] Figure 7B A simplified schematic diagram of the motion relationship of the surgical tool of the surgical robot system of an embodiment of the present application is shown. As shown in Figure 6 and Figure 7BAs shown, coordinate system 415 is established at the end of surgical instrument 410, and coordinate system 435 is established at the end of surgical tool 430. Based on the master-slave motion mapping relationship between the input device motion and the surgical tool motion, the coordinate system ML of left input device 23L can be mapped to the coordinate system 414 at the end of surgical instrument 410, the coordinate system MR of right input device 23R can be mapped to the coordinate system 415 at the end of surgical tool 430, and the picture coordinate system MB of display device 21 can be mapped to the picture coordinate system 660 of endoscope 420. Thus, the motion of the coordinate system 415 at the end of surgical instrument 410 in the picture coordinate system 660 of endoscope 420 and the motion of the coordinate system 425 at the end of surgical tool 420 in the picture coordinate system 660 of endoscope 420 can be obtained. Based on the motion conversion relationship of surgical tools 410 and 430, the motion amount of the joint assembly of the surgical tool can be calculated to complete the motion control of the corresponding motor.

[0070] Figure 8 FIG. 1 is a simplified flowchart of a follow-up mode control method of a surgical robot system according to an embodiment of the present application. In process 1111, the surgeon S activates the master-slave alignment by input device 23, thereby activating the master-slave operation function between the master control console and the patient slave operating device. In process 1112, the surgeon S controls the motion of left input device 23L and / or right input device 23R, and based on the coordinate system conversion relationship in Figure 7A , the actual motion information of left input device 23L and right input device 23R is converted into the picture coordinate system MB of display device 21. Based on the master-slave motion control model, the picture coordinate system MB of display device 21 is equivalent to the picture coordinate system 660 of endoscope 420 of the slave operating device, the master-slave motion mapping and data transmission of input device 23 are performed through the master-slave communication bus. After the control device of the slave operating device obtains the motion control instruction, the kinematic model is calculated based on the pre-stored kinematic model to execute the motion calculation of the joint assembly of surgical tools 410 and 430, thereby realizing the real-time follow-up motion of surgical tools 410 and 430 with left input device 23L and right input device 23R.

[0071] In one embodiment, the four operation modes of the surgical robot system further include an adjustment mode. Referring again to Figure 6 , in the adjustment mode, the surgeon S can control the rotation of tool holding device 112 and 280 around remote center of motion 116 by controlling input device 23, so that the endoscope 420 picture can realize four degrees of freedom motion, which includes that the endoscope 420 can rotate around the x m axis and y mThe shaft performs two degrees of freedom rotation movement, which realizes the change of the field of view range of the endoscope 420 in the up-down and left-right directions, thereby facilitating the doctor S to continuously operate the surgical instruments at different surgical sites and improving the smoothness of the surgical operation. In addition, along the direction of the center axis 118 of the sleeve 115, the endoscope 420 can perform a forward-backward movement degree of freedom movement, that is, the endoscope 420 moves linearly along the sleeve axis 118 towards the proximal end or the distal end, so that the observable field of view range of the endoscope picture can be adjusted. This degree of freedom movement helps to adjust the surgical instruments outside the field of view range into the field of view range at any time so that the doctor S can observe, thereby reducing the risk of surgical operation. During the operation, the endoscope picture can be controlled to perform a rotation degree of freedom movement around the picture axis, so that the direction of the field of view of the picture can be adjusted clockwise or counterclockwise. During the degree of freedom movement, the pose of the picture axis in the remote motion center coordinate system 570 remains unchanged, and at the same time, the position of the center of the endoscope picture in the coordinate system 570 remains unchanged.

[0072] In one embodiment, the adjustment mode further includes that the endoscope 420 rotates around the axis z v of the coordinate system 660, so that the endoscope picture rotates, and when the endoscope 420 rotates around the axis z v of the coordinate system 660, the control device controls the parallel joint 422 and the wrist joint 423 to move jointly, so that the pose of the parallel joint 422 and the wrist joint 423 remains unchanged.

[0073] In the adjustment mode, the endoscope 420 and the surgical instruments 410, 430 move jointly, so that when the endoscope 420 rotates around the remote motion center 116, the tips of the surgical instruments 410, 430 remain unchanged. In one embodiment, the control method 300 of the adjustment mode is as shown in Figure 9 Process 761, based on the motion control input instruction of the input device 23, the four degrees of freedom movement control of the endoscope picture described above can be performed. During the movement control process of the endoscope 420, the pose of the tip device of the surgical instruments 410, 430 in the remote motion center coordinate system 570 always remains unchanged, and the joint assemblies of the surgical instruments 410, 420 perform joint movement control process 762 according to the kinematics model. In the process 762, the mechanical arm, the first driving device 2201 and the first feed driving device 2211 move jointly, so that when the endoscope 430 rotates around the remote center 116, the position and the attitude of the tip device of the surgical instruments 410, 420 remain unchanged, thereby enabling the surgical instruments 410, 430 to still allow the endoscope 420 picture to perform multi-degree of freedom movement when clamping the tissue, and the surgical instruments 410, 430 can maintain the pulling force on the tissue unchanged during the movement. In some embodiments, all the surgical instruments installed on the instrument holding device and in the activated state participate in the joint movement control process.

[0074] The surgical instruments 410, 420 and the robotic arm perform joint motion, and the process 763 determines in real time whether the motion of the robotic arm reaches the first motion boundary position. If the output of the process 763 is "no", the process 769 is performed, and the robotic arm drives the endoscope to continue the motion. If the output of the process 763 is "yes", the process 764 is performed, and the boundary impedance model is triggered. In some embodiments, the first motion boundary position is defined as the pose of the endoscope 430 or the pose of the robotic arm when the surgical instruments 410, 420 coupled thereto reach a set arbitrary joint limit position. In the process 764, the endoscope reaches the boundary position, and the boundary impedance model set by the software is triggered. At this time, the joint assemblies of the surgical instruments 410, 430 and the endoscope 420 and the motion joints of the robotic arm 200 stop the continuous motion in the direction. When the boundary impedance model of the endoscope 430 motion is triggered, if the input device continues the motion in the direction, the boundary impedance force of the endoscope motion will be executed by the force feedback process 765 transmitted from the operating device to the input device and the robotic arm, and the boundary impedance force feedback effect is achieved in the hand of the physician S.

[0075] In some embodiments, the deviation between the input instruction for controlling the endoscope motion and the actual motion position of the endoscope has a certain correlation with the size of the boundary impedance force. As the position deviation increases, the effect of the boundary impedance force in the hand of the physician S also gradually increases. In some embodiments, the force feedback function of the input device is allowed to be set to be opened and / or closed by the physician S on the physician operating table or by the auxiliary personnel on the patient surgery platform. In some embodiments, the opening and / or closing setting of the master force feedback function of the physician operating table is allowed to be performed by mechanical buttons and / or touch screens and / or voice systems, etc. When the process 764 is triggered, a prompt information that the master motion reaches the boundary is generated. In some embodiments, the prompt information is allowed to be in the form of sound and / or icon and / or number and / or progress bar and / or other forms, and the output source of the prompt information is allowed to be the physician operating table and / or the patient surgery platform and / or the image trolley and / or other devices. When the endoscope 430 motion reaches the boundary position, the physician S needs to perform the process 767 to make a judgment whether to change the motion direction of the master-slave operation. If the output of the process 767 is "no", the process 768 is triggered, and the robotic arm 211 stops the motion. If the output of the process 767 is "yes", the process 769 is triggered, and the robotic arm 211 is allowed to continue the motion control. The boundary impedance model is automatically closed, and the impedance boundary force of the master of the physician operating table gradually decreases to zero.

[0076] Referring again to Figure 6, in the adjustment model, during the front and back moving freedom motion of the endoscope 420, when the distance between the proximal end of the proximal joint 4221 of the parallel joint 422 and the remote center of motion 116 is greater than the distance between the sleeve port 115a and the remote center of motion 116, the parallel joint 422 will not collide with the sleeve port when the endoscope moves proximally along the central axis 118 of the sleeve 115 from outside the sleeve 115 to approach the sleeve port 115a, on the contrary, when the distance between the proximal end of the proximal joint 4221 of the parallel joint 422 and the remote center of motion 116 is less than the distance between the sleeve port 115a and the remote center of motion 116, the parallel joint 422 will collide with the sleeve port 115a. In order to solve the collision phenomenon between the endoscope 420 and the sleeve port 115a, the endoscope 420 performs a straightening action command during the front and back translation motion along the central axis 118 of the sleeve, that is, when the shoulder and elbow base of the endoscope 430 moves from outside the sleeve to approach the sleeve port, the angle α between the parallel joint 422 and the central axis 118 of the sleeve 115, and the angle β between the wrist joint 423 and the central axis 118 gradually decrease to zero and remain unchanged.

[0077] In one embodiment, in order to provide the doctor S with the advantageous information prompt of the endoscope 420 in the above straightening action process, a virtual region Z1 is provided near the sleeve 115, the virtual region Z1 includes a first boundary S1 located at the proximal end and a second boundary S2 located at the distal end, and a spring impedance force model is established between the first boundary S1 and the second boundary S2 positions, and the control shaft generates a force command to the input device 23 according to the spring impedance force model, so as to generate an impedance force on the joint of the input device 23 which hinders the input device 23 from continuing to move, and the doctor S can know the straightening state of the endoscope 420 by feeling the impedance force.

[0078] In one embodiment, the first boundary S1 is arranged at the position of the sleeve port 115a or the position distal to the sleeve port 115a, the second boundary S2 is arranged between the sleeve port 115a and the remote center of motion 116, the distance h between the first boundary S1 and the second boundary S2 is the straightening process distance, and the straightening speed of the endoscope 420 and the distance h are allowed to establish a linear or nonlinear relationship model, that is, the smaller the distance h, the faster the straightening speed of the endoscope, and vice versa. The value of h remains unchanged after the initial setting of the system is completed.

[0079] The positions of the first boundary S1 and the second boundary S2 are the zero position positions of the trigger type spring impedance force model, that is, when the input device is at the positions of the first boundary S1 and the second boundary S2, the impedance force output by the input device according to the spring impedance force model is zero.

[0080] In one embodiment, the virtual region Z1 further comprises an intermediate boundary S3 between the first boundary S1 and the second boundary S2, when the proximal end of the proximal joint 4221 reaches the intermediate boundary S3, the impedance force output by the input device 23 reaches a maximum value, and the impedance force gradually increases as the parallel joint coordinate system 640 approaches the intermediate boundary S3 from the first boundary S1 and the second boundary S2, and gradually decreases as the parallel joint coordinate system 640 approaches the first boundary S1 and the second boundary S2 from the intermediate boundary S3.

[0081] Specifically, during the process of the input device 23 controlling the endoscope 420 to translate along the cannula central axis 118 to the proximal end, when the parallel joint coordinate system 640 contacts the first boundary S1, i.e., coincides with the first boundary S1, the spring-like impedance force model is triggered, and as the parallel joint coordinate system 640 gradually approaches S1 from S1, the angle β between the parallel joint 422 and the cannula central axis 118 gradually decreases until it is zero, and the impedance force gradually increases from zero to a maximum value at the intermediate boundary S3 and then gradually decreases to zero at the second boundary S2. In one embodiment, the intermediate boundary S3 is at the virtual region h / 2. At this time, the endoscope 420 becomes straightened. In one embodiment, the judgment of the parallel joint coordinate system 640 contacting the first boundary S1, the second boundary S2, or the intermediate boundary S3 can be determined by comparing the position relationship between the parallel joint coordinate system 640 and the coordinate system established at the first boundary S1, the second boundary S2, or the intermediate boundary S3.

[0082] During the process of the input device 23 controlling the endoscope 420 to translate along the cannula central axis 118 to the distal end from inside the cannula 115, when the coordinate system 640 starts to perform the motion of gradually approaching the cannula port 115a from inside the cannula and proximal to the second boundary S2, if the coordinate system 640 contacts the second boundary S2, the straightening action instruction is triggered, and the endoscope 420 is allowed to remain in the straightened state until the coordinate system 640 reaches the outside of the cannula beyond the S1S2 range, and during the process of the coordinate system 640 from the second boundary S2 to the intermediate boundary S3, the impedance force output by the input device 23 gradually increases, and during the process of the coordinate system 640 from the intermediate boundary S3 to the first boundary S1, the impedance force output by the input device 23 gradually decreases until it is zero.

[0083] In some embodiments, during the process of the endoscope 420 moving from the distal end to the proximal end, when the coordinate system 640 contacts the first boundary S1, the control device stores the pose of the joint assembly of the endoscope 420 at this time in the memory. During the process of the endoscope 420 moving from the proximal end to the distal end, when the coordinate system 640 contacts the first boundary S1 again, the control device reads the previously stored pose of the joint assembly of the endoscope 420 and restores the endoscope 420 to the pose.

[0084] Figure 10is a simplified flow chart of the endoscope straightening control method of an embodiment of the present application. As shown in Figure 10 In process 810, the endoscope is controlled to perform a translational movement along the sheath axis direction by input device 23, and at the same time, process 820 is performed, when parallel joint base coordinate system 640 moves from the outside of the distal end of sheath 115 to the first boundary S1 position or moves from the inside of the proximal end of sheath 115 to the second boundary S2 position, the straightening function is triggered, and at the same time, process 830 is performed, the S1 position or S2 position at the time of triggering is recorded, if the first boundary S1 position is triggered, the posture of endoscope 420 at the time is also recorded, and the movement deviation 840 of coordinate system 640 in virtual area Z1 from the reference zero position is calculated with the triggering position as the reference zero position, during which, process 850 is performed at the same time, the postures of wrist joint and parallel joint are changed, that is, the endoscope starts to perform the straightening action or restores the posture before the straightening action.

[0085] In process 860, the control device calculates the force command according to the spring-like model output and transmits the force command to the master console, after receiving the impedance force input instruction, the master console inputs the instruction into the inverse kinematics model of the established input device 23 in the screen of display device 21, performs kinematics calculation process 870, and further obtains the joint movement instruction value of the input device, and performs process 880, each joint movement is according to the movement instruction to output the impedance force, and the force feedback effect is realized to prompt the doctor S the current state of endoscope 420.

[0086] In one embodiment, the plurality of operation modes of the surgical robot system further includes a repositioning mode, which has some similarities with the above-mentioned adjustment mode, but the difference is that in the repositioning mode, surgical instruments 410, 430 are not linked with endoscope 420. Again referring to Figure 6In the repositioning mode, the physician S controls the rotation of the sleeve 115 around the remote center of motion 116 by operating the input device, and the surgical instruments 410, 430 and the endoscope 420 rotate around the remote center of motion 116 with the sleeve 115, thereby adjusting the position and orientation of the surgical instruments 410, 430 and the endoscope 420 as a whole. The input device 23 can also control the forward and backward movement of the surgical instruments 410, 430 and the endoscope 420 as a whole, i.e. the movement of the surgical instruments 410, 430 and the endoscope 420 as a whole towards the proximal end or the distal end. During the forward and backward movement of the surgical instruments 410, 430 and the endoscope 420 as a whole, the first joint of the parallel joint of any of the surgical instruments 410, 430 and the endoscope 420 moving from the distal end to the proximal end contacts the first boundary S1 of the virtual region Z1, or moving from the proximal end to the distal end contacts the second boundary S1, which triggers the above-mentioned spring-like impedance force model, and the input device 23 outputs the impedance force. In an embodiment, the spring-like impedance force model is triggered independently for each of the surgical instruments 410, 430 and the endoscope 420, and the impedance forces are accumulated at the input device 23, so that the physician S can feel how many surgical tools are in the straightened state.

[0087] In an embodiment, after the operating mode of the surgical robot system is switched from the adjustment mode to the repositioning mode, the motion boundary of the robotic arm is also switched from the first motion boundary to the second motion boundary synchronously, wherein the second motion boundary is determined according to the limit position of the joint of the robotic arm.

[0088] In an embodiment, the plurality of operating modes of the surgical robot system described above further includes a camera mode, in which the endoscope 420 is controlled to move independently by the input device 23, while the remote center of motion 116 and the surgical instruments 410, 430 remain stationary.

[0089] In an embodiment, the camera mode includes a first camera mode and a second camera mode. In the first camera mode, the input device 23 independently manipulates the parallel joint 422 of the endoscope 420 to move, thereby changing the position of the camera 424. In an embodiment, in the first camera mode, the input device 23 independently manipulates the wrist joint 423 to move while keeping the parallel joint 422 stationary, and the movement of the wrist joint 423 includes the rotation of the wrist joint around the coordinate axis y w and the coordinate axis x w of the coordinate system 650, thereby changing the position and orientation of the camera 424.

[0090] In one embodiment, in the first camera mode, the input device 23 manipulates the parallel joint 422 and the first feed drive device 2201 in linkage to change the position of the distal end face 424a of the camera 424 and maintain the distance from the center point P1 of the distal end face 424a of the camera 424 to the first plane M1 unchanged, the first plane M1 passing through the remote center of motion 116 and being perpendicular to the long shaft axis 421a and the cannula center axis 118. In one embodiment, the joint assembly includes a linear motion joint, the parallel joint 422 and the parallel joint are in linkage to change the position of the distal end face 424a of the camera 424 and maintain the distance from the center point P1 of the distal end face 424a of the camera 424 to the first plane M1 unchanged.

[0091] In one embodiment, in the first camera mode, the input device 23 manipulates the parallel joint 422 and the wrist joint 423 to move together, when the parallel joint 422 moves to the physical boundary, the wrist joint 423 starts to rotate, or when the wrist joint 423 moves to the physical boundary, the parallel joint 422 starts to rotate, so that the working space of the camera 424 is larger.

[0092] Referring to Figure 6 In one embodiment, in the second camera mode, according to the input of the input device 23, the control device can control the wrist joint 423, the parallel joint 422 and the second feed drive device in linkage, so that the distal end face 424a of the camera 424 rotates around the end face coordinate system 660 and maintains the position of the end face center point P1 relative to the remote center of motion 116 unchanged, wherein the end face center point P1 and the origin of the coordinate system 660 coincide.

[0093] Specifically, the control device controls the wrist joint 423 to rotate around the coordinate axis x w of the coordinate system 650, the parallel joint 422 to rotate around the coordinate axis x b of the coordinate system 640, and the second feed drive device to drive the linear motion of the long shaft 421 along its axis 421a, so that the end face 424a of the endoscope 422 rotates around the first axis x v of the coordinate system 660 and maintains the position of the center point P1 relative to the remote center 116 unchanged. The control device controls the wrist joint 423 to rotate around the coordinate axis y w of the coordinate system 650, the parallel joint 422 to rotate around the coordinate axis y b of the coordinate system 640, and the second feed drive device to drive the linear motion of the long shaft 421 along its axis 421a, so that the end face 424a of the endoscope 422 rotates around the second axis y v of the coordinate system 660 and maintains the position of the center point P1 relative to the remote center 116 unchanged. The second camera mode can be suitable for detecting the surrounding environment in a narrow surgical space.

[0094] In one embodiment, the articulation assembly of the endoscope 430 further comprises a linear motion joint (not shown in the figure), which moves the long shaft 421 along the axis 421a, thereby achieving the feed motion of the camera 424 advancing and retreating. The camera end face 424a rotates around the first axis x v or the second axis y v When the camera end face 424a rotates around the first axis x

[0095] In one embodiment, the present application also provides a method 5000 for switching between the above-mentioned various operating modes, as shown in the figure, in process 5111, after the surgical robot system is started, it is usually in the default standby state, and the operating device and the master console are in the state of keeping the current position without movement, waiting for the doctor S to operate and use, at the same time, the operating device and the input device of the doctor master console are in the initial state, which is usually the fixed pose set by the system. Figure 11

[0096] In process 5112, the method detects whether the doctor S is in place at the master console 20, if it is detected that the doctor S is in place, it can enter process 5113, otherwise it returns to process 5111 to detect whether the doctor S is in place. The detection can be realized by infrared, distance sensor detection, camera or pressure sensor, etc.

[0097] ​In process 5113, the method determines the operation mode selection action performed by the surgeon S, if it is determined that the surgeon S performs the action of selecting to enter the follow mode, in process 5114, the method sets the surgical robot system to move in the follow mode; if it is determined that the surgeon S is selecting to enter the endoscope mode, in process 5210, the surgical robot system is set to move in the endoscope mode, wherein the actions of selecting to enter the follow mode and the endoscope mode are different, in an embodiment, the action of selecting to enter the follow mode is that the surgeon S simultaneously presses the left input device 23L and the right input device 23R, or simultaneously rotates the left input device 23L and the right input device 23R. In an embodiment, the action of selecting to enter the endoscope mode is that the surgeon S steps on the pedal, or presses the button, or voice prompts, or other inputs. It can be understood that the actions of selecting to enter the follow mode and the endoscope mode can also be other actions, as long as the actions of selecting to enter the follow mode and the endoscope mode are different. In the present embodiment, the endoscope mode includes the adjustment mode, the repositioning mode and the camera mode described above, and in some embodiments, the endoscope further includes other modes of operating the endoscope, such as an endoscope cleaning mode, a fluorescence mode, etc.

[0098] In process 5115, the method determines whether the surgeon S has performed the action of entering the endoscope mode, for example, the surgeon S steps on the pedal, or presses the button, or voice prompts, or other inputs, if "yes", process 5210 switches the operation mode of the surgical robot system from the follow mode to the endoscope mode, if "no", process 5116 is entered, in process 5116, the method determines whether to exit the follow mode, if "yes", the method loops back to process 5111, if it is determined that "no", the surgical robot system remains in the follow mode. In some embodiments, whether to exit the follow mode is that the surgeon S disconnects the master-slave control between the master console 20 and the slave operating device 10.

[0099] In an embodiment, in process 5210, the method sets the endoscope mode of the surgical robot system to the default endoscope mode, in the present embodiment, the default endoscope mode is the adjustment mode described above.

[0100] In process 5211, the method determines whether the surgical robot system exits the endoscope mode, if "yes", process 5217 is entered, in process 5217, the method determines whether the surgeon S is in place, if the surgeon S is in place, the method sets the surgical robot system to the follow mode, if "no", the method loops back to process 5111. The action of exiting the endoscope mode can be releasing the pedal, or releasing the button, or voice prompts, or other inputs.

[0101] If the determination in the process 5211 is "No", the method proceeds to a process 5212 in which the method determines whether the input device 23 has performed an endoscope mode switching action. If the determination in the process 5212 is "No", the method keeps the surgical robotic system in the default endoscope mode; if the determination in the process 5212 is "Yes", the method switches the surgical robotic system from the default endoscope to a selected endoscope mode, which in the present embodiment includes one of the above-mentioned repositioning mode and camera mode, but is different from the default endoscope mode. In some embodiments, the selected endoscope mode further includes other modes, such as an endoscope cleaning mode, a fluorescence mode, etc.

[0102] In one embodiment, in the process 5212, the endoscope mode switching action includes at least a first action, wherein the first action controls the first drive device 2202 to move in the follow mode, so as to control the end device of the surgical instrument 410 or the surgical instrument 430 to follow the movement of the input device 23, but the first action does not cause the movement of one of the surgical instruments 410, 430 and the endoscope 420 in the endoscope mode (e.g. the adjustment mode, the repositioning mode, the camera mode), i.e. in the endoscope mode, the first action does not cause the movement of the first drive device 2201, the second drive device 2202, the feed drive devices 2213, 2214 and the mechanical arm.

[0103] In one embodiment, referring to Figure 7A , in response to the entering endoscope mode action and the first action, the method switches the surgical robotic system from the follow mode to the camera mode, the first action being the rotation of the end of the right input device 23R about the axis Y MR in the counterclockwise direction; in response to the entering endoscope mode action and the action opposite to the first action, the method switches the surgical robotic system from the follow mode to the repositioning mode, the action opposite to the first action being the rotation of the end of the right input device 23R about the axis Y MR in the clockwise direction.

[0104] In one embodiment, in the endoscope mode (e.g. adjustment mode, repositioning mode, camera mode), the input device 23 controls the movement of the robotic arm 11, 211, surgical instrument 410, endoscope 420 and related driving system by the second action, which is the up-down, left-right and forward-backward three degrees of freedom movement of the left input device 23L and the right input device 23R, the second action only changes the position of the input device 23L and the right input device 23R without changing its posture. In one embodiment, the second action controls the operation of the repositioning mode, camera mode and adjustment mode with the movement of the midpoint O1 of the line connecting the origin of the left input device coordinate system ML and the origin of the right input device coordinate system MR in the display screen coordinate system MB. When the left input device 23L moves upward / downward and the right input device 23R moves downward / upward, the camera 424 is controlled to rotate around the third axis z of the coordinate system 660 v . .

[0105] In one embodiment, if the endoscope mode is selected as the camera mode, the control device 160 controls the second driving device 2201 to control the movement of the joint assembly of the endoscope 430 based on the second action. The second action includes the second action of the left input device 23L and the second action of the right input device 23R, the second action of the left input device 23L controls the movement of the endoscope 430 in the first camera mode, and the second action of the right input device 23R controls the movement of the endoscope 430 in the second camera mode. .

[0106] In one embodiment, the first action is the action of the input device 23 only changing the posture, and the second action is the related action of the input device 23 only changing the position, since the second action is used to control the endoscope 430 in the endoscope mode, regardless of the adjustment mode, the repositioning mode or the camera mode, the doctor S does not need to remember multiple operation modes, and the misoperation is reduced. .

[0107] In process 5215, the method determines whether to exit the current endoscope mode, if it is determined to be "yes", it returns to process 5217, if it is determined to be "no", it loops back to the default endoscope mode. The action of exiting the current endoscope mode can be releasing the pedal, or releasing the button, or voice prompt and other inputs. .

[0108] In process 5216, the method determines whether to exit the current endoscope mode, if it is determined to be "yes", it returns to process 5217, if it is determined to be "no", it loops back to the selected endoscope mode. .

[0109] In one embodiment, the above switching method is performed by the control device 160. The control device calculates a first target position of the robot arm according to the second motion, and when the first target position exceeds the first motion boundary or the second motion boundary, the control device generates an impedance force command for the input device to generate an impedance force on the input device to impede the input device from continuing the motion. The impedance force command increases as the distance between the first target position and the first motion boundary or the second motion boundary increases.

[0110] In one embodiment, after the first target position exceeds the first motion boundary or the second motion boundary, if the input device continues the second motion, the control device calculates a second target position of the robot arm according to the second motion, and when the second target position is closer to the first motion boundary or the second motion boundary than the first target position, the control device releases the constraint on the motion of the robot arm and stops generating the impedance force command.

[0111] The technical features of the above embodiments can be combined in any manner. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described, but it should be understood that any combination of the technical features is within the scope of the present disclosure as long as the combination does not result in contradictions.

[0112] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be noted that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present application, and these are within the scope of the present application. Therefore, the scope of the patent of the present application should be subject to the appended claims.

Claims

1. A surgical robotic system, characterized by, Comprising: a robot arm having a plurality of joints; a plurality of driving devices for driving a surgical instrument and an endoscope; an input device for manipulating the plurality of driving devices in a plurality of operation modes, the plurality of operation modes including at least a follow mode, a camera mode, a repositioning mode; a control device configured to: in the follow mode, control a first driving device of the plurality of driving devices to move in accordance with a first action of the input device, so that a pose of the surgical instrument follows a pose of the input device; in response to an enter endoscope mode action and the first action, switch the operation mode from the follow mode to a camera mode, in which a second driving device of the plurality of driving devices is controlled to move in accordance with a second action of the input device, so as to adjust a position and a pose of the endoscope, and to rotate a distal end face of a camera of the endoscope around an end face coordinate system, wherein the second action changes the position of the input device without changing the pose of the input device; in response to an action opposite to the enter endoscope mode action and the first action, switch the operation mode from the follow mode to the repositioning mode.

2. The surgical robotic system of claim 1, wherein, The surgical robot system further comprises a cannula through which the surgical instrument and the endoscope are inserted together, and the plurality of operation modes further comprises an adjustment mode, in response to the enter endoscope mode action, the operation mode is switched from the follow mode to the adjustment mode.

3. The surgical robotic system of claim 2, wherein, The plurality of driving devices are connected to a distal end of the robot arm, and in the adjustment mode, the control device controls the robot arm to move in accordance with the second action of the input device, so as to rotate the endoscope around a remote center of motion, to adjust the position and the pose of a distal end of the endoscope, and to maintain the position and the pose of a distal end of the surgical instrument unchanged.

4. The surgical robotic system of claim 3, wherein, In the repositioning mode, the control device controls the robot arm to move in accordance with the second action of the input device, to simultaneously adjust the position and the pose of the surgical instrument and the endoscope.

5. The surgical robotic system of claim 4, wherein, In the camera mode, the adjustment mode and the repositioning mode, the first action does not cause the plurality of driving devices and the joints of the robot arm to move.

6. The surgical robotic system of claim 5, wherein, Pre-stored first and second motion boundaries for constraining the motion of the robot arm are further included, and after the operation mode is switched from the adjustment mode to the repositioning mode, the control device synchronously switches the motion range of the robot arm from the first motion boundary to the second motion boundary.

7. The surgical robotic system of claim 6, wherein, The first motion boundary is determined according to the motion range of a joint assembly of the surgical instrument, and the second motion boundary is determined according to the motion range of the plurality of joints of the robot arm.

8. The surgical robotic system of claim 7, wherein, The control device calculates a first target position of the robot arm in accordance with the second action, and when the first target position exceeds the first motion boundary or the second motion boundary, the control device constrains the robot arm from further moving, and generates an impedance force command that hinders the input device from continuing to move, to generate an impedance force on the input device that hinders the input device from continuing to move.

9. The surgical robotic system of claim 8, wherein, The impedance force increases as a distance between the first target position and the first motion boundary or the second motion boundary increases.

10. The surgical robotic system of claim 8, wherein, The control device calculates a second target position of the robot arm according to the second action, and if the second target position is closer to the first motion boundary or the second motion boundary than the first target position, the control device removes the constraint on the movement of the robot arm and stops generating the impedance force command.

Images