Dual control of a mechanical surgical arm

Abstract

Initially operate a surgical system in a reverse bending mode, the surgical system including: an articulated robotic arm having a plurality of arm joints; first and second user input devices for controlling the arm; and a surgical end effector at a distal end of the arm, wherein the first user input device is activated to direct the bending and rotation of only a single given arm joint and wherein the second user input device is deactivated. When in the reverse bending mode, reverse bend a distal portion of the arm by the bending and rotation of the single given arm joint. In the case of reverse bending the arm, transition the system to a surgical mode, which enables the second user input device to bend and rotate any or all of the arm joints to move the surgical end effector to perform one or more surgical actions.

Description

[0001] Cross - reference to related applications

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 944,351, filed on December 5, 2019, which is hereby incorporated by reference in its entirety. Technical Field

[0003] The present invention relates to a surgical system for performing surgery and methods of using such systems, and more particularly, to controlling the bending and rotation of portions of an articulated robotic arm using multiple operating modes and input devices. Background Art

[0004] This section is intended to introduce to the reader various aspects of the technology that may be related to various aspects of the present invention, which are described and / or claimed hereinafter. This discussion is believed to be helpful in providing background information to facilitate a better understanding of the various aspects of the present invention. Accordingly, it should be understood that these statements are to be read from this perspective and not as an admission of prior art.

[0005] It is well known that minimally invasive surgery has many benefits. Instruments for such surgery typically have a surgical end - effector located at the distal end of an articulated surgical arm (preferably having a minimum diameter), which is inserted through a small opening (e.g., a body wall incision, a natural orifice) to reach the surgical site. In some cases, the surgical instrument can pass through a cannula, and an endoscope can be used to provide an image of the surgical site.

[0006] To achieve convenience, accuracy, and the health of the subject, surgical instruments that utilize the end - effector (e.g., surgical tools for tissue fusion or cutting, or measurement tools) have been developed. In some cases, the articulated surgical arm has one or more curved portions that are remotely controlled using various input devices (e.g., hand and foot controls) to ultimately control the position of the end - effector and change its orientation with reference to the longitudinal axis of the surgical arm. In some cases, the surgical arm is capable of bending in a reverse direction relative to the longitudinal axis of the surgical arm.

[0007] Clinical studies have shown that minimally invasive transvaginal gynecological surgery is superior to abdominal surgery. Advantages include postoperative recovery time, morbidity, infection, mortality, complications, blood loss, and patient satisfaction. To date, the policy of the American College of Obstetricians and Gynecologists (ACOG) states that transvaginal gynecological surgery is preferred whenever feasible. To enable medical devices to access transvaginally for gynecological surgery, an articulated surgical arm needs to bend to a reverse - bent position.

[0008] The current state of the art lacks devices and methods that can provide optimal control elements and usage methods to ensure the performance of the initial reverse bending step in a risk - mitigating manner, where the risk - mitigating method does not affect the ergonomic comfort and convenience provided by the input device optimized for heavy surgical procedures. Therefore, there is a need for a solution that is suitable for limiting risks during the reverse bending of a surgical arm and providing maximum freedom of movement during the surgical procedure without compromising the surgeon's ergonomics. Summary of the Invention

[0009] According to an embodiment, a method for operating a surgical system is disclosed. The surgical system includes (i) an articulated robotic arm that includes a plurality of arm joints, and (ii) first and second user input devices for controlling the arm, and (iii) a surgical end - effector at the distal end of the arm. The method includes: (a) starting the operation of the surgical system in a reverse - bending mode, where with respect to the bending and rotation of the arm joints: (i) the first user input device is activated to direct the bending and rotation of only a given arm joint among the arm joints; and (ii) the second user input device is deactivated; (b) while in the reverse - bending mode, in response to an electronic control output from the first user input device, reverse - bend a distal portion of the articulated robotic arm by bending and rotating the given arm joint among the arm joints so that the end - effector is in a reverse - bending operating position; (c) transitioning the surgical system from the reverse - bending mode to a surgical mode to enable the second user input device with respect to the bending and rotation of at least one arm joint in the arm other than the given arm joint in the arm joints; and (d) while in the surgical mode, in accordance with the respective degrees of freedom of each arm joint, effect the bending and rotation of at least two of the arm joints in response to an electronic control output from the second user input device, thereby moving the surgical end - effector to perform one or more surgical actions.

[0010] In some embodiments, the surgical system may further include a control circuit that effectively restricts the actuation of arm joints other than a single arm joint when the surgical system is in the reverse - bending mode.

[0011] In some embodiments, the transition may include calibrating the input device with respect to at least one of the position and orientation of the end - effector or the distal portion of the arm.

[0012] In some embodiments, the first input device may be configured to control the actuation of a single arm joint and / or not configured to control the actuation of arm joints other than a single arm joint.

[0013] In some embodiments, transitioning to the surgical mode may be responsive to and / or dependent on detecting that the arm is in a reverse - bending position.

[0014] A surgical system according to an embodiment includes: (a) an articulated robotic arm including a plurality of arm joints and a surgical end effector at a distal end of the arm; and (b) first and second user input devices for controlling the arm, wherein the surgical system is configured to operate asynchronously in the following modes: in (i) a reverse bending mode, wherein a distal portion of the articulated robotic arm is operable to reverse bend in response to an electronic control output from the first user input device to place the end effector in a reverse bending operation position, and in (ii) a surgical mode, wherein at least two arm joints are operable to bend and rotate in response to an electronic control output from the second user input device so as to thereby move the surgical end effector to perform one or more surgical actions, such that: (A) when in the reverse bending mode, with respect to the bending and rotation of the arm joints, the first user input device is activated to direct the bending and rotation of only a given arm joint among the arm joints and the second user input device is deactivated, and (B) when in the surgical mode, according to the respective degrees of freedom of each arm joint, the second user input device is enabled with respect to the bending and rotation of at least one arm joint among the arms other than the given arm joint among the arm joints.

[0015] In some embodiments, the surgical system may further include a control circuit that effectively constrains the actuation of the arm joints other than a single arm joint when the surgical system is in the reverse bending mode.

[0016] In some embodiments, the surgical system may be configured such that the transition includes calibrating the input device with respect to at least one of the position and orientation of the end effector or the distal portion of the arm.

[0017] In some embodiments, the first input device may be configured to control the actuation of a single arm joint and / or not be configured to control the actuation of arm joints other than a single arm joint.

[0018] In some embodiments, the surgical system may be configured such that the transition to the surgical mode may be responsive to and / or dependent on detecting that the arm is in a reverse bending position.

[0019] According to an embodiment, a method of operating a surgical system is disclosed. According to the method, the surgical system includes (i) a user input device, and (ii) an articulated robotic arm including a plurality of arm joints and a surgical end effector at a distal end of the arm. The method includes: (a) initiating operation of the surgical system in a reverse bend mode, wherein with respect to bending and rotation of the arm joints, the user input device is activated to direct bending and rotation of only a given arm joint among the arm joints; (b) when in the reverse bend mode, in response to an electronic control output from the user input device, reverse bend a distal portion of the articulated robotic arm by bending and rotating the given arm joint among the arm joints to place the end effector in a reverse bend operating position; (c) transitioning the surgical system from the reverse bend mode to a surgical mode to enable the user input device with respect to bending and rotation of at least one arm joint among the arm joints other than the given arm joint in the arm; and (d) when in the surgical mode, effect bending and rotation of at least two of the arm joints in response to an electronic control output from the user input device according to the respective degrees of freedom of each arm joint, thereby moving the surgical end effector to perform one or more surgical actions.

[0020] In some embodiments of the method, the surgical system may further include a control circuit that effectively constrains actuation of the arm joints other than the given arm joint in the arm when the surgical system is in the reverse bend mode. In some such embodiments, the constraint may be effected by deactivating actuation of the arm joints other than the given arm joint in the arm.

[0021] In some embodiments, the user input device may control actuation of multiple arm joints in both the reverse bend mode and the surgical mode.

[0022] In some embodiments, the user input device may be prevented from generating or transmitting a control output that would control actuation of the arm joints other than the given arm joint in the arm.

[0023] In some embodiments, transitioning to the surgical mode may be responsive to and / or dependent on detecting that the arm is in a reverse bend position.

[0024] In some embodiments, the transition may include calibrating the user input device with respect to at least one of a position and an orientation of the end effector or a distal portion of the arm.

[0025] In some embodiments, the method may further include unbending a distal end of the arm after operating in the second mode to place the arm in an unbent position.

[0026] According to an embodiment, a surgical system is disclosed, comprising: (a) a user input device; and (b) an articulated robotic arm, the articulated robotic arm comprising (i) a plurality of arm joints, and (ii) a surgical end effector at a distal end of the arm, wherein the surgical system is configured to operate asynchronously in the following modes: in (A) a reverse bending mode, wherein a distal portion of the articulated robotic arm bends in reverse in response to an electronic control output from the user input device to place the end effector in a reverse bending operating position, and in (B) a surgical mode, wherein at least two of the arm joints bend and rotate in response to an electronic control output from the user input device so as to thereby move the surgical end effector to perform one or more surgical actions, such that: (A) when in the reverse bending mode, with respect to the bending and rotation of the arm joints, the user input device is activated to direct the bending and rotation of only a given arm joint among the arm joints, and (B) when in the surgical mode, according to the respective degrees of freedom of each arm joint, the user input device is enabled with respect to the bending and rotation of at least one arm joint among the arms other than the given arm joint among the arm joints.

[0027] In some embodiments, the surgical system may additionally include a control circuit that effectively constrains the actuation of the arm joints other than the given arm joint among the arm joints when the surgical system is in the reverse bending mode. In some such embodiments, the constraint may be effected by deactivating the actuation of the arm joints other than the given arm joint among the arm joints.

[0028] In some embodiments, the user input device may effectively control the actuation of a plurality of arm joints in both the reverse bending mode and the surgical mode.

[0029] In some embodiments, the surgical system may be configured such that the user input device may be prevented from generating or transmitting a control output that would control the actuation of the arm joints other than the given arm joint among the arm joints.

[0030] In some embodiments, the surgical system may be configured such that the transition to the surgical mode may be responsive to and / or dependent on detecting that the arm is in the reverse bending position.

[0031] In some embodiments, the surgical system may be configured such that the transition may include calibrating the user input device with respect to at least one of the position and orientation of the end effector or the distal portion of the arm.

[0032] In some embodiments, the surgical system may additionally be configured to, after operating in the second mode, effect unbending of the distal end of the arm to place the arm in an unbended position.

[0033] According to an embodiment, a method of operating a surgical system is disclosed, the surgical system including (i) an articulated robotic arm including a surgical end effector and a plurality of arm joints at its distal end, and (ii) an array of input devices of one or more user input devices, wherein the arm joints are configured to bend and rotate in response to electronic control outputs from one or more of the user input devices of the array of input devices. The method includes: (a) starting operation of the surgical system in a first operating mode defined relative to a given single arm joint of the arm joints, wherein the first operating mode prevents actuation of any arm joint of the arm that is not the given single arm joint, and permits control of the actuation of the single arm joint to cause bending and rotation of the single arm joint; (b) when the surgical system is in the first operating mode, (i) in response to a control signal generated by one or more of the user input devices of the array of input devices, bending and rotating the single arm joint to bend the distal end of the arm in a reverse direction such that the end effector is in a reverse bending operating position, and (ii) monitoring the state of the robotic arm to detect whether the arm is in the reverse bending position. The method further includes: (c) in response to and depending on detecting that the arm is in the reverse bending position, transitioning the operation of the surgical system from the first operating mode to a second operating mode in which the system is capable of controlling the bending and rotation of at least one arm joint excluding the first mode according to the respective degrees of freedom of each arm joint; and (d) operating the surgical system in the second operating mode to perform a surgical action using the end effector.

[0034] In some embodiments, the surgical system may further include a control circuit that effectively constrains actuation of arm joints other than the single arm joint when the surgical system is in the first operating mode. In some such methods, the constraint may be effected by deactivating actuation of arm joints other than the single arm joint of the arm. In some such embodiments, generation or transmission of control outputs by the input devices that may control actuation of arm joints other than the single arm joint of the arm may be prevented. In some such embodiments, the constraint may include disabling the ability of a first input device. In some such embodiments, (i) when the surgical system is in the first operating mode, a first input device of the array of input devices controls actuation of the single arm joint, and when the surgical system is in the second operating mode, a second input device controls actuation of the plurality of arm joints, and / or (ii) the constraint may be effected by providing a first input device configured to control actuation of the single arm joint and not configured to control arm joints other than the single arm joint.

[0035] In some embodiments, the transition may include calibrating the input device relative to at least one of the position and orientation of the end effector or the distal portion of the arm.

[0036] In some embodiments, when the surgical system is in a first operating mode, a first input device of the input device array can control the actuation of a single arm joint, and / or when the surgical system is in a second operating mode, a second input device of the input device array can control the actuation of multiple arm joints.

[0037] In some embodiments, the transition can include calibrating the second input device relative to at least one of the position and orientation of the end effector or the distal portion of the arm.

[0038] In some embodiments, a single user input device can control the actuation of multiple arm joints in both the first and second operating modes.

[0039] In some embodiments of the method, the surgical system can further include a console that includes a display screen, and at least one user input device of the input device array is disposed on or near the display screen.

[0040] In some embodiments, an additional user input device for actuating the linear advancement and retraction of the arm can be disposed on, in the same position as, or in proximity to at least one user input device of the input device array.

[0041] In some embodiments, the reverse bend operating position can be at or near the surgical work site.

[0042] In some embodiments, operation in the second mode can position the arm in a reverse bend position.

[0043] In some embodiments, the method can further include, after operating in the second mode: unbending the distal end of the arm to position the arm in an unbent position.

[0044] According to an embodiment, a surgical system for use with a surgical end effector and configured to operate asynchronously in a first operating mode and a second operating mode includes: (a) an array of input devices of one or more user input devices; and (b) an articulated robotic arm including a surgical end effector at its distal end and a plurality of arm joints configured to bend and rotate in response to control signals generated by one or more of the input devices of the array of input devices, wherein: (i) the first operating mode is defined with respect to a given single one of the arm joints, wherein the first operating mode prevents actuation of any of the arm joints in the robotic arm that is not the given single arm joint and permits control of the actuation of the single arm joint to cause bending and rotation of the single arm joint, (ii) the system is configured to, in response to an electronic control output from one or more of the user input devices of the array of input devices, reverse-bend the distal end of the robotic arm by actuating the single arm joint to cause bending and rotation of the single arm joint while in the first operating mode so that the surgical end effector is in a reverse-bent operating position, (iii) the second operating mode is defined with respect to the plurality of arm joints, wherein the second operating mode enables control of the actuation of at least one of the arm joints excluding the first mode according to the respective degrees of freedom of each arm joint, and (iv) the system is configured to transition from the first operating mode to the second operating mode in response to and depending on detecting that the robotic arm is in a reverse-bent position and, while in the second operating mode, perform a surgical action using the end effector.

[0045] In some embodiments, the surgical system may further include a control circuit that effectively constrains the actuation of the arm joints other than the single arm joint when the surgical system is in the first operating mode. In some such embodiments, the constraint may be effected by deactivating the actuation of the arm joints in the robotic arm other than the single arm joint. In some such embodiments, generation or transmission of control outputs by the input devices that may control the actuation of the arm joints in the robotic arm other than the single arm joint may be prevented. In some such embodiments, the constraint may include the ability to deactivate a first input device. In some such embodiments, (i) a first input device of the array of input devices may control the actuation of the single arm joint when the surgical system is in the first operating mode and a second input device may control the actuation of the plurality of arm joints when the surgical system is in the second operating mode, and / or (ii) the constraint may be effected by providing a first input device configured to control the actuation of the single arm joint and not configured to control the actuation of the arm joints other than the single arm joint.

[0046] In some embodiments, the system may be configured such that the transition may include calibrating the input devices with respect to at least one of the position and orientation of the end effector or the distal portion of the robotic arm.

[0047] In some embodiments, when the surgical system is in a first operating mode, a first input device of the input device array can effectively control the actuation of a single arm joint, and / or when the surgical system is in a second operating mode, a second input device of the input device array can effectively control the actuation of multiple arm joints.

[0048] In some embodiments, the system can be configured such that the transition can include calibrating the second input device relative to at least one of the position and orientation of the end effector or the distal portion of the arm.

[0049] In some embodiments, a single user input device can effectively control the actuation of multiple arm joints in both the first operating mode and the second operating mode.

[0050] In some embodiments, the surgical system can additionally include a console that includes a display screen, and at least one user input device of the input device array is disposed on or near the display screen.

[0051] In some embodiments, an additional user input device for actuating the linear advancement and retraction of the arm can be disposed on, in the same position as, or in proximity to at least one user input device of the input device array.

[0052] In some embodiments, the reverse bending operating position can be at or near the surgical work site.

[0053] In some embodiments, operation in the second mode can place the arm in a reverse bending position.

[0054] In some embodiments, the surgical system may additionally be configured to, after operating in the second mode, effect uncurving of the distal end of the arm to place the arm in an uncurved position. According to an embodiment, a method of operating a surgical system is disclosed, the surgical system including (i) an articulated robotic arm including a plurality of arm joints and a surgical end effector at the distal end of the arm, and (ii) an array of input devices for controlling one or more user input devices of the arm. The method includes: (a) in response to an electronic control output from the array of input devices, reverse-curve a distal portion of the articulated robotic arm by bending and rotating a given arm joint of the arm joints to place the end effector in a reverse-curved operating position without bending or rotating any arm joints other than the given arm joint of the arm joints; and (b) in response to and depending on detecting that the arm is in the reverse-curved position, effect bending and rotation of at least two of the arm joints in accordance with the respective degrees of freedom of each arm joint in response to an electronic control output from the array of input devices, thereby moving the surgical end effector to perform one or more surgical actions.

[0055] In some embodiments of the method, the surgical system may additionally include a control circuit that effectively constrains actuation of arm joints other than the given arm joint of the arm joints during reverse-curving.

[0056] In some embodiments, the method may additionally include, after performing one or more surgical actions, effecting uncurving of the distal end of the arm to place the arm in an uncurved position.

[0057] According to an embodiment, a surgical system includes: (a) an articulated robotic arm including (i) a plurality of arm joints and (ii) a surgical end effector at the distal end of the arm, and (b) an array of input devices for controlling one or more user input devices of the arm, wherein the surgical system is configured to: (i) in response to an electronic control output from the array of input devices, reverse-curve a distal portion of the articulated robotic arm by bending and rotating a given arm joint of the arm joints to place the end effector in a reverse-curved operating position without bending or rotating any arm joints other than the given arm joint of the arm joints, and (ii) in response to and depending on detecting that the arm is in the reverse-curved position, and in response to an electronic control output from the array of input devices, effect bending and rotation of at least two of the arm joints in accordance with the respective degrees of freedom of each arm joint, thereby moving the surgical end effector to perform one or more surgical actions.

[0058] In some embodiments, the surgical system may additionally include a control circuit that effectively constrains actuation of arm joints other than the given arm joint of the arm joints during reverse-curving.

[0059] In some embodiments of the method, the surgical system may additionally include control circuitry that effectively restricts actuation of the arm joints other than a given arm joint in the arm during reverse bending.

[0060] In some embodiments, the method may additionally include, after performing one or more surgical maneuvers, unbending the distal end of the arm to place the arm in an unbent position.

[0061] According to an embodiment, a method of using a surgical system is disclosed. The method includes: (a) providing an articulated robotic arm having a surgical end effector at its distal end, the arm including a plurality of arm segments serially connected by a corresponding plurality of arm joints configured to bend and rotate in response to an electronic control output from a user input device, wherein the providing causes the end effector to be displaced; (b) maneuvering the end effector to a reverse bending operating position while operating in a first input mode, in which displacement of the input device or a displaceable portion thereof is converted into a velocity of (i) bending of the arm joints and (ii) rotation of the arm joints; (c) transitioning from operating in the first input mode to operating in a second input mode in response to and depending on detecting that the end effector is in the reverse bending operating position, in which displacement of the input device or a displaceable portion thereof is converted into a corresponding displacement of at least one arm segment; and (d) performing a surgical maneuver using the end effector after the transition and while operating in the second input mode.

[0062] In some embodiments, a single user input device may be used in both the first input mode and the second input mode.

[0063] In some embodiments, a first input device may be used in the first input mode, and / or a second input device may be used in the second input mode.

[0064] In some embodiments, an additional user input device may be used to actuate linear advancement and retraction of the arm.

[0065] In some embodiments, the reverse bending operating position may be at or near the surgical work site.

[0066] According to an embodiment, a surgical system for use with a surgical end effector includes: (a) an articulated robotic arm having a surgical end effector at its distal end, the arm including a plurality of arm segments serially connected by a corresponding plurality of arm joints configured to bend and rotate in response to an electronic control output from a user input device; and (b) an array of one or more input devices for controlling the arm, wherein the surgical system is configured to: (i) operate asynchronously in the following operating modes: in (A) a first input mode, wherein displacement of the input device or its displaceable portion is converted into a speed of bending and rotation of the arm joints, and in (B) a second input mode, wherein displacement of the input device or its displaceable portion is converted into a corresponding displacement of at least one arm segment; (ii) when operating in the first input mode, maneuver the end effector to a reverse bending operating position during a first phase of end effector displacement; and (iii) when operating in the second input mode, perform a surgical action using the end effector.

[0067] In some embodiments, the system may be configured such that a single user input device may be used in both the first input mode and the second input mode.

[0068] In some embodiments, the system may be configured such that a first input device is used in the first input mode, and / or a second input device is used in the second input mode.

[0069] In some embodiments, the system may be configured such that an additional user input device may be used to actuate linear advancement and retraction of the arm.

[0070] In some embodiments, the reverse bending operating position may be at or near the surgical work site. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The present invention will now be further described by way of example with reference to the accompanying drawings, in which the dimensions of the components and features shown in the figures are chosen for convenience and clarity of presentation and are not necessarily drawn to scale. In the drawings:

[0072] Figure 1 is a simplified illustration of a surgical system according to an embodiment of the present invention.

[0073] Figure 2A is a schematic perspective view of a surgical system including a robotic surgical arm according to an embodiment of the present invention.

[0074] Figure 2B shows a distal portion of a robotic surgical arm according to an embodiment of the present invention.

[0075] Figure 3A-C shows the distal portion of a robotic surgical arm in various bent and reverse bent positions according to an embodiment of the present invention.

[0076] Figure 4 and Figure 5 shows the distal portion of a robotic surgical arm having multiple arm curve shapes according to an embodiment of the present invention.

[0077] Figure 6 shows a flowchart of a method for operating a surgical robotic arm using two operating modes according to an embodiment of the present invention.

[0078] Figure 7 is a schematic diagram of a console for a surgical system according to an embodiment of the present invention, with an input device positioned nearby.

[0079] Figure 8 is a schematic diagram of a user input device according to an embodiment of the present invention.

[0080] Figure 9 is a diagram showing an exemplary scheme for using an input device to control the bending and rotation of an arm joint according to an embodiment of the present invention.

[0081] Figure 10 is a schematic diagram of an articulated user input device according to an embodiment of the present invention.

[0082] Figure 11 is according to an embodiment of the present invention Figure 10 a schematic diagram of a handle member of a user input device.

[0083] Figure 12A -B shows an exemplary graphical aid for aligning the corresponding positions of a robotic surgical arm and an articulated user input device according to an embodiment of the present invention.

[0084] Figure 13 shows an example screen display for aligning the corresponding positions of a robotic surgical arm and an articulated user input device according to an embodiment of the present invention.

[0085] Figure 14A -E shows time-sequential images according to an embodiment of the present invention, the time-sequential images showing exemplary control of a surgical robotic arm using multiple different input devices.

[0086] Figure 15 shows a block diagram of a surgical system according to an embodiment of the present invention.

[0087] Figure 16A flowchart of a method for operating a surgical system in two different operating modes according to an embodiment of the present invention is shown.

[0088] Figure 17A A flowchart of a method for dual control of a surgical arm according to an embodiment of the present invention is shown.

[0089] Figure 17B and Figure 17C is a schematic diagram of a console including a dual control member according to an embodiment of the present invention.

[0090] Figure 18 A flowchart of a method for controlling one or more surgical arms using a haptic handle according to an embodiment of the present invention is shown. Detailed Description

[0091] The present invention has been described herein by way of example only with reference to the drawings. Now, with specific reference to the drawings in detail, it should be emphasized that the details shown are only examples and are used for an illustrative discussion of the preferred embodiments of the present invention, and are presented in order to provide the most useful and easily understood description of the principles and concepts of the present invention. In this regard, no attempt has been made to show the structural details of the present invention in more detail than is necessary for a basic understanding of the present invention, and the description with the drawings enables those skilled in the art to clearly know how to embody several forms of the present invention in practice. Throughout the drawings, the same reference numerals are generally used to denote the same elements.

[0092] The embodiments disclosed herein relate to controlling one or more surgical robotic arms, i.e., articulated robotic arms, using a variety of different operating modes and / or a variety of different input devices.

[0093] Whenever the term 'arm' is used herein or in the appended claims, it refers to an articulated robotic arm that is part of a surgical system or an electrosurgical system and is used to perform or assist in performing surgical (including electrosurgical) actions within a human subject. When not specified, 'a surgical action' may include any medical or surgical-related or diagnostic action taken within the human body, including but not exhaustive: cutting tissue, dissecting tissue, manipulating tissue, suturing tissue, contracting tissue, fusing tissue, taking measurements and imaging. It may be desirable for the size and / or shape of the surgical arm to be designed to be suitable for insertion into the human body. For example, the size and / or shape of the arm may be designed to be inserted through a laparoscopic port and / or for performing laparoscopic surgery. For example, the size and / or shape of the arm may be designed to be inserted through natural body orifices such as the vagina, anus, trachea, esophagus, and ear canal.

[0094] The arm may include an end effector, which is used herein to represent a tool or device that is used in combination with surgery, electro-surgery, diagnosis, or imaging when deployed in a human body. The end effector may be provided as part of the arm, i.e., already installed, mechanically connected, and / or integrated with the power and communication transmission means of the arm; in some embodiments, the arm and the end effector may be provided separately for assembly and / or integration into the work unit before or even during surgery (i.e., before insertion into the subject's body). In any case, terms such as 'arm including an end effector' and 'arm configured to be used with an end effector' should be understood as equivalent for the purposes of the present disclosure and the appended claims.

[0095] As used herein, an 'input device' or equivalently a 'user input device' can be any device capable of receiving user input, i.e., input received from a user of the surgical system. Input devices can include, for example but not exhaustively: buttons, switches, toggle switches, wheels, knobs, small levers such as thumbsticks (or called nipple-type parts), and joysticks (whether articulated or not). Disclosing a particular device type of any specific input device is not intended to exclude replacing the specific input device with other types of input devices. The input device can be standalone, grouped on a single control component or a small number of control components, provided on another input device, or located in the same place, such as on or near a console, display screen, etc. The user can use one or more fingers, thumbs, hands, or feet to operate the input device. Additionally or alternatively but not limited to, the input device can be eye or hand movement operated, voice operated, or controlled by facial expressions.

[0096] The arm and the input device, as well as other aspects and features of the present invention, can be understood in conjunction with any teachings of the co-pending U.S. Patent Application Serial No. 16 / 121,704, filed on September 5, 2018, and published as U.S. Patent Publication US20190000574A1, which is hereby incorporated by reference in its entirety herein.

[0097] 'Handle' or equivalently 'handle component' is generally used herein to describe a manually operated user input device or the manually operated part of a user input device, and in some embodiments to describe a user input device or a part thereof that is grasped by the hand or fingers. The drawings and accompanying descriptions of the handle and the manually operated user input device in the present disclosure are provided as illustrative examples, and these drawings and accompanying descriptions should not be understood as limiting the scope, connectivity, and functionality of the embodiments related to the handle / handle component design.

[0098] For the present disclosure, a'module' and / or 'circuit' or 'electronic circuit' and / or 'control circuit' and / or component and / or unit and / or controller and / or module and / or sensor and / or detector may include any combination of analog and / or digital circuits and / or software / computer-readable code modules and / or firmware and / or hardware elements, including but not limited to digital computers, CPUs, volatile or non-volatile memories, field-programmable logic array (FPLA) elements, hard-wired logic elements, field-programmable gate array (FPGA) elements, and application-specific integrated circuit (ASIC) elements. Any instruction set architecture may be used, including but not limited to reduced instruction set computer (RISC) architecture and / or complex instruction set computer (CISC) architecture.

[0099] In different embodiments, any combination of analog and / or digital circuits and / or software / computer-readable code modules and / or firmware and / or hardware elements may be used to perform any computational or analytical process, including but not limited to digital computers, CPUs, volatile or non-volatile memories, field-programmable logic array (FPLA) elements, hard-wired logic elements, field-programmable gate array (FPGA) elements, and application-specific integrated circuit (ASIC) elements. Any instruction set architecture may be used, including but not limited to reduced instruction set computer (RISC) architecture and / or complex instruction set computer (CISC) architecture.

[0100] Now referring to the drawings, Figure 1 a schematic diagram of a surgical system 100 according to an embodiment is shown. Figure 1 The system 100 includes two surgical robotic arms 102. In other examples of surgical systems, a single surgical arm is provided. In other examples, more than two (e.g., 3 or 4) surgical arms are provided. The size and / or shape of the surgical robotic arms 102 are preferably designed to be inserted into a human body or patient 106. Each of the surgical robotic arms 102 is actuated by a corresponding motor unit 108. In this simplified illustrative example, the surgical arms 102 and / or the motor units 108 are supported by a patient support 116 attached to, for example, a bed, but may also be supported by a patient-side cart or any other suitable device.

[0101] In an embodiment where the surgical system is used for electro-surgery, the arms 102 and the motor units 108 may be powered by a high-frequency electrosurgical knife 112. As is known in the electro-surgery field, the high-frequency electrosurgical knife supplies a high-frequency (e.g., radio-frequency) alternating-polarity current. The high-frequency electrosurgical knife 112 may be configured to supply different frequencies and / or power levels suitable for, for example, cutting and / or coagulating and / or sealing and / or drying and / or cauterizing tissue. Electrical power is supplied to the motor unit 108 via one or more cables 114 configured to transmit radio-frequency electrosurgical power.

[0102] The movement of the surgical arm 102 is controlled by the console 118. The movement is in response to signals generated by one or more input devices. The console 118 includes a plurality of user interfaces, including one or more of the following: an input device, such as the input device arm 120, wherein the console is configured to generate control signals based on the movement of the input device arm 120; a display screen 128, which is configured to receive user input and / or display system status information or imaging, such as of the surgical area, for example to display images collected by a camera inserted into the patient 106 using one of the surgical arms 102, or to display arm position and orientation; and one or more additional user interfaces 130 (such as buttons, switches, etc.).

[0103] The console 118 includes a processor (not shown), which is configured to receive signals from user input and send control signals to the motor unit 108 and / or the high-frequency electrosurgical unit 112. The foot pedal 126 and / or the high-frequency electrosurgical unit 112 includes a processor (not shown), which is configured to receive control signals (such as generated by a user pressing a portion of the foot pedal 126) to change the electrical power supplied to the motor unit 108 based on the control signals. The foot pedal control signal does not necessarily pass through the control unit processor.

[0104] As will be explained in more detail below, in one control mode, the movement of the input device arm 120 controls the movement of the corresponding surgical device arm 102. The user 124 can position and / or move the input arm 120 by grasping the input device arm handle 127. The input arm is a form of input device and is shown here for illustrative purposes. In other embodiments, other types or forms of input devices may be used.

[0105] Now refer to Figure 2A -B, the arm unit 104 includes a proximal end, which is shaped to be received by the motor unit 108; and a distal end, wherein an end effector 174, such as the multi-claw gripper shown (shown only as a non-limiting illustrative example), is attached to the arm 102.

[0106] The use of the relative terms 'proximal' and 'distal' is consistent with the arrows shown in Figure 2A and is used in this manner throughout the present disclosure and the appended claims: As shown, the distal end of the arm 102 that houses the end effector 174 is the end farthest from the motor unit 108 and is the first part of the arm to be inserted into the human subject 19. Thus, the proximal end is the end opposite the distal end and closest to the motor unit 108. As used herein, the term 'distal portion' refers to any portion of the arm 102 that includes the distal end / tip (optionally including the end effector 174) and is less than half the length of the arm 102. In Figure 2BIn [the figure], the bendable portion 170 of the arm 102, i.e., the portion including one or more bendable joints, is positioned closer to the distal end along the length of the arm. The bendable portion 170 may include a series of'stacked linkages' 199 that achieve the flexibility of the outer contour / surface of the arm 102; Figure 2B An example of a plurality of stacked linkages 199 in the bendable portion 170 of the arm 102 is shown.

[0107] As used herein, 'operating mode' or its equivalent'mode' (which may be used in combination with various non-limiting descriptive terms, e.g., 'backward bending mode','surgical operating mode', etc.) refers to an operating regime imposed on the use of a surgical system or an arm by hardware, firmware, or software design or by the control circuitry of the surgical system or otherwise. For clarification: the word 'operating' in 'operating mode' means 'working' or 'functioning' and describes, for example, the operation of an arm and does not mean the performance of a surgery, i.e., the 'operating mode' may or may not happen to include the performance of a surgery. The imposed operating regime may include, but is not limited to, constraining or not constraining certain actions or certain parts of the surgical system that perform one or more actions, and constraining or not constraining may include and / or may be equivalent to enabling or disabling, locking or unlocking, and excluding or allowing or similar words and phrases. In some embodiments herein, a mode is dedicated to or involves achieving a single goal, e.g., bending the distal portion of the arm backward, and may be limited to one or more specific time periods. In other embodiments, a mode may include an infinite number of goals and actions and an infinite or undefined time period.

[0108] As described further below, the operation of a surgical system can be differentiated between different modes in various ways. For example, the differentiation can be based on (but is not limited to): having different input devices (or a plurality of input devices) dedicated to each mode; having various constraints or limitations on specific arm movements and / or specific arm joints and segments; having different conversion schemes for the displacement of the input device or control elements of the input device to arm movement, e.g., displacement to speed versus displacement to displacement, regardless of whether the conversion from user input to arm movement is robotic / semi-autonomous or remotely controlled; and whether the operation of the input device directly or indirectly resolves (through the mechanical and electronic devices of the surgical system) the bending and rotation of the arm joints by resolving the displacement of the arm segments, and resolving the displacement of the arm segments causes the necessary bending and rotation to displace the arm segments as indicated. These differentiations can be used in combination, and they can vary by arm in a multi-arm system. In some embodiments, the differentiation can change when the system is in a given mode.

[0109] Some of the distinctions between modes can be implemented in more than one way. As a non-limiting example, if the distinction between modes involves constraining or limiting certain arm movements (e.g., flexion and rotation of certain joints) or only allowing certain arm movements, the distinction can be achieved by using different input devices for each mode, or by using one input device in both modes, but enforcing software or hardware constraints on the input device that can be switched by the user or enforced by the system. Whether the implementation requires one or more input devices, a software-implemented or hardware-implemented solution can actively allow actuation of a single given arm joint, or can actively block or prevent actuation of any arm joint (of the same arm) that is not a single arm joint.

[0110] It should be noted that wherever in this document actuation or movement of a single 'arm' (as opposed to the plural 'arms') is discussed, this is merely for convenience and is not intended to indicate whether the second arm (or other multiple arms) is actuated or moved in the same way or otherwise. Each arm can be controlled and actuated independently of any other arm by a corresponding input device. On the other hand, when a single 'arm' is disclosed as being constrained or limited in terms of actuation or movement, e.g., when in an operating mode characterized by such a limitation or constraint, the limitation or constraint can equally apply to both / all arms of the surgical system. However, in some embodiments, the constraint or limitation can be applied to one or more given arms, while other arms are not constrained or limited at all, or are not constrained or limited in the same way.

[0111] In embodiments that employ two different operating modes, the first operating mode is typically used when introducing or withdrawing and / or navigating one or more surgical arms into or out of the body (or from a first point to a second point within the body), and more specifically, when introducing or withdrawing and / or navigating one or more surgical arms toward or away from a target surgical site. The second operating mode is typically used when performing surgical actions (e.g., dissecting tissue, manipulating tissue, suturing tissue, taking measurements, imaging, etc.). 'Navigation' in the first mode can include bending one or more arms backward to place at least a portion or the distal portion of the arm in a backward-bent position, or equivalently, placing the end effector in a backward-bent operating position (working position).

[0112] A well-defined transition or'switch' from one operating mode to another may be required. In some embodiments, the transition includes switching from one input device (or multiple input devices) to another. In other embodiments, the transition entirely involves changing the control aspects that distinguish between one mode and another, where the tasks assigned to the two modes can be accomplished by a single input device or by the same input device. The transition can be initiated and managed by the surgical system, or can be initiated by the user.

[0113] In a non - limiting example of transitioning from a first mode to a second mode, the transition can include ending the constraints or limitations imposed during operation in the first mode, such as restricting the bending and rotation of joints to a given single arm joint of any particular arm, such as the elbow joint. Additionally or alternatively, the transition can include enabling the actuation of arm joints that were deactivated (or not specifically enabled) during operation in the first mode, i.e., in addition to the single given joint that was enabled in the first mode. Enabling can include making the joint able to move unconstrained according to the respective degrees of freedom of each joint. For example, if a given arm joint is only configured for rotation and not bending, enabling will make it available only for rotation. Additionally or alternatively, the transition can enable the actuation of all arm joints. Additionally or alternatively, the transition can change the processing of the control output from a user input device from a displacement - velocity conversion to a displacement - displacement conversion. In another non - limiting example, the transition can be automatically implemented by a control circuit in response to an event, which can be implemented based on a user input such as pressing a button or turning a switch, or it can be initiated by noticing that the user has stopped using the first input device and started using a second input device. In another non - limiting example, the transition is an intermediate transition based on the user interface.

[0114] In a non - limiting example of transitioning from a second mode to a first mode, the transition can include restoring the constraints or limitations removed during the transition to second - mode operation, such as restricting the bending and rotation of joints to a given single arm joint of any particular arm, such as the elbow joint. Additionally or alternatively, the transition can include deactivating the actuation of arm joints that were enabled for second - mode operation. Additionally or alternatively, the transition can change the processing of the control output from a user input device from a displacement - displacement conversion back to a displacement - velocity conversion. In another non - limiting example, the transition can be implemented based on a user input such as pressing a button or turning a switch, or it can be initiated by noticing that the user has stopped using the input device dedicated to second - mode operation and started using a different input device dedicated to first - mode operation.

[0115] In an embodiment, and particularly in an embodiment where the input device employed in the second mode is avatar - like and converts input arm displacements into surgical arm displacements, transitioning from a first operating mode to a second operating mode can include alignment calibration. In the alignment calibration Figures 12A to 12B discussed in further detail below, the orientation of the surgical arms with respect to each other is modified, including the 'internal' orientation of arm elements (e.g., arm joints and / or arm segments), to match the'shape' or 'curve' that describes the corresponding orientation of the input arm that will take over the actuation control of the surgical arms when transitioning to the second operating mode. In the case of transitioning back from the second operating mode to the first operating mode, there may not be a corresponding alignment calibration.

[0116] The conversion of user input into arm movement (bending and rotation) can be accomplished in a variety of ways. For example, the displacement (or displacement force) of an input device or a control element of an input device can be converted into the speed of arm movement (e.g., velocity, rate, angular velocity). In a first mode that includes insertion / withdrawal and reverse bending (or reverse bending / non-bending), it may be desirable to use this type of conversion. These are limited and in some embodiments, less precise movements may be beneficial to perform not in a teleoperation or avatar mode, but in a simpler robotic or semi-autonomous mode. In contrast, the movements required in a second mode dedicated to performing various surgical maneuvers may be more beneficial to a controlled displacement-displacement conversion, where the displacement of an input device or a control element of an input device is appropriately converted into the displacement of an arm segment and indirectly into the rotation and bending of a joint. For example, in the second mode, it may be desirable to ensure that the articulation of an articulable input device corresponds, in terms of internal orientation (i.e., segment-to-segment orientation), to the surgical arm, such that the displacement-to-displacement conversion is more intuitive, ergonomic, and precise.

[0117] In the first mode, it may be desirable to constrain the arm movement to a single given arm joint that bends and / or rotates the arm (or each arm). This can be appropriately achieved by configuring the surgical system to receive input from an input device used in the first mode, the input directly addressing the bending and rotation of a specific arm joint, the specific arm joint being, for example, a single given arm joint that allows actuation (in the operating mode). In other words, the user is 'controlling' the arm joint itself through the input device and a suitable control circuit. The movements expected to be actuated in this case are specifically the bending and / or rotation of the arm joint. Whether the actual arm movement is performed semi-autonomously in response to the user's control input or not, the user knows that she is controlling the actuation of the joint. The accompanying shift and reorientation of the end effector may be the expected result of controlling the joint.

[0118] In contrast, in the second mode and particularly in embodiments where the input device employed in the second mode is similar to an avatar and converts input arm displacement into surgical arm displacement, it may be necessary for the surgical system to receive and process input that directly addresses the displacement of an arm segment. The surgical system then indirectly addresses the bending and rotation of the arm joints by controlling the bending and rotation of the arm joints to the extent required to achieve the desired displacement and reorientation of the arm segment. In other words, the user is 'controlling' the displacement and reorientation of the arm segment (or the end effector, which for the purposes of this discussion acts like any arm segment since controlling its position and orientation is the user's goal), and the control circuit of the surgical system uses this information to determine the necessary bending and rotation of each affected arm joint. In an embodiment, the user can manipulate the avatar-like input arm into a shape or configuration that anticipates or drives the anticipated shape of the surgical arm after manipulation.

[0119] In an exemplary first operating mode, movement of the surgical arm can be at least partially restricted or constrained, and certain types of movement can be excluded while other types of movement are permitted. For clarity, 'movement' of the arm can include displacement and / or reorientation of any part of the arm (e.g., one or more segment components of the arm).

[0120] According to a specific implementation where the arm includes three actuatable joints, the terms 'elbow joint', 'wrist joint', and'shoulder joint' as used herein refer to specific joints of the robotic arm. In this case, the joint closest to the distal actuator is referred to as the 'wrist joint', the middle joint among the three is referred to as the 'elbow joint', and the most proximal joint is referred to as the'shoulder joint'. In some embodiments, the wrist joint is limited to rotation, i.e., not configured to bend. The various joints are shown in the Figure 3A below.

[0121] Unless otherwise specified and as used in the context of arm joints (i.e., joints of the arm) in this disclosure and the appended claims, the term 'joint' means any actuatable component capable of causing bending (e.g., planar bending) and / or rotation. It should be noted that an articulatable / avatar-like input arm (a type of input device) can also have joints, and they are referred to as 'joint components' of the input device. Generally, an arm segment is a non-actuatable (with respect to bending / rotation) component of the arm that can be serially connected by actuatable arm joints, where the term'serially connected' simply means that a joint is inserted between two consecutive segment components. The joints can be actuated, for example, mechanically and / or electronically, to bend one arm segment (and each part of the arm distally disposed therefrom) relative to the base arm segment ( Figure 3A 2181 in) or another (adjacent) arm segment, and / or to rotate one arm segment (and each part of the arm distally disposed therefrom) relative to the base arm segment or another arm segment. In some embodiments, a joint includes multiple components, and in some examples, can include both components (or sub-assemblies) that facilitate bending and components (or sub-assemblies) that facilitate rotation. For ease of reading, the combination of these components is referred to herein collectively as a joint or an arm joint.

[0122] In an embodiment, arm movement can be restricted based on the type of articulation (e.g., rotation versus bending), movement speed, which parts of the surgical arm can move, etc. Movement of the surgical arm during the first operating mode can be restricted to movement of a given single arm joint - only the elbow joint - (including, for example, bending and / or rotation of the elbow joint) and linear movement of the surgical arm as a single unit (including, for example, linearly advancing and retracting the arm). It may be desirable to restrict arm movement during the first operating mode to facilitate introducing a surgical arm of minimal volume through a narrow path to a target surgical site.

[0123] It may also be desirable to limit arm movement during the first operating mode to facilitate bending of the arm within a minimum volume, thereby avoiding collisions and potential tissue damage within the human body. Figure 3A FIG. 3 is a simplified schematic side view of a surgical robotic arm 102 in different configurations explained herein for illustrative purposes. The dashed line 2177 represents an obstacle, such as patient tissue. The movement of the arm 102 during reverse bending is preferably controlled to prevent the arm 102 (and in particular the end effector 174) from contacting or colliding with the obstacle 2177. Three scenarios, labeled A, B, and C respectively, are shown in FIG. 3. The arm 102 includes a proximal segment 2181 and a distal bendable portion 170. The bendable portion 2179 includes a shoulder joint 2101 and an elbow joint 2103, which are labeled differently as 2101a, 2101bc, 2103a, 2103b, and 2103c to indicate with which one or more scenarios (A, B, or C) the example joints are associated. For example, the joint 2103a is the elbow joint position in scenario A. Thus, scenario A involves only the bending of the elbow joint 2103a (the shoulder joint 2101a remains unactuated and unbended), and it can be specifically seen that the bending of only the elbow joint 2101a does not result in a collision between the end effector 174 and the obstacle 2177. On the other hand, in scenario B where only the shoulder joint 2101b is bent, a collision occurs between the end effector 174 and the obstacle 2177. The difference in the collision / non - collision results between scenarios A and B is because the portion of the arm 102 away from the elbow joint 2103 is shorter than the portion of the arm away from the shoulder joint 2101. Figure 3A The wrist joint 2105 of the arm 102 in FIG. 3 is not involved in any of the three scenarios because it is designed or configured to rotate but not bend in the non - limiting example of FIG. 3. In scenario C, which is a continuation of scenario A, it can be seen that bending the shoulder joint 2101c after bending the elbow joint 2103a (now 2103c) can help to continue to avoid a collision between the end effector 174 and the obstacle 2177. Figure 3A FIG. 4 shows examples of the elbow joint 2103 bending from an unbent orientation relative to the proximal arm base segment 2181 to various bent - after orientations ranging from less than 90° to greater than 180°.

[0124] Figure 3B FIG. 4 shows examples of the elbow joint 2103 bending from an unbent orientation relative to the proximal arm base segment 2181 to various bent - after orientations ranging from less than 90° to greater than 180°.

[0125] In an embodiment, the flexion range of motion of the elbow joint is >90°, >120°, >140°, >160°, >180°, >190°, >200° or about 210° ± 10°. In some embodiments, the end effector 174 is capable of being positioned relative to the base 2181 of the arm 102 at >90°, >120°, >140°, >160°, >180°, >190°, >200° or about 210° ± 10°. In some embodiments, the end effector 174 is capable of being parallel to the base 2181 of the arm 102 or alternatively reaching the base 2181 of the arm 102 when fully flexed. Additionally, the range of motion of the elbow rotation joint should be at least 200°, at least 250°, at least 300°, at least 310°, at least 320°, at least 330°, at least 350° or about 360°. Figure 3C The arm 102 is shown, where the flexion of the elbow joint 2103 rotates more than 180° from the Figure 3B unflexed orientation shown in, such that the arm 102 is in a reverse flexion configuration or equivalently, a reverse flexion position, and the end effector 174 is transferred to the reverse flexion operating position.

[0126] In some embodiments, the reverse flexion of the surgical arm can be accomplished automatically, i.e., by arranging the arm to respond to a single or limited number of electronically controlled outputs by flexing and / or rotating a single arm joint until a pre-programmed reverse flexion position of the arm and / or end effector at the distal end of the arm is achieved.

[0127] Figure 4 The arm 102 is shown in a front perspective view, where the elbow joint 2103 is flexed and then rotated similarly to the Figure 3B elbow joint 2103a in Case A of, i.e., Figure 4 the elbow joint 2103 in is flexed and rotated. The rotation of any arm joint can be independent of the flexion of the same arm joint. In some embodiments, the flexion and rotation can be simultaneous, and in other embodiments, a constraint on non-simultaneity can be enforced, for example, by hardware design or by a software component of the control circuit that controls the joint actuation.

[0128] Figure 5 The arm 102 is shown in a front perspective view, where the elbow joint 2103 is similar to the Figure 4 elbow joint flexion and rotation, and where the shoulder joint 2101 is flexed to form a complex 'S' shape together with the elbow joint 2101. In some embodiments, for example, in the Figure 3B embodiment represented by Case C of, the shoulder joint 2101 is preferably actuated to flex and rotate only after the flexion of the elbow joint 2103 has 'crossed' the obstacle 2177.

[0129] Now refer to Figure 6, which shows a flowchart of a general method for controlling a surgical robotic arm using different operating modes according to some embodiments.

[0130] The processes described herein, for example, can be implemented for various types of surgeries performed at least in part using one or more surgical robotic arms inserted into a patient's body, such as gynecological, laparoscopic, otolaryngology (ear, nose, and throat) surgeries.

[0131] As discussed above, a surgical robotic arm can be operated according to multiple operating modes. Optionally, the operating mode is selected according to the surgical steps to be performed and / or according to the current stage of the surgery.

[0132] The different operating modes can be characterized by different ways of converting a user input (movement of an input device or a control element of the input device) into a corresponding movement of the surgical robotic arm, i.e., by manipulating the user input device. Additionally or alternatively, the different operating modes can be characterized by different constraints on the articulation of the surgical arm, such as restricting the articulation (e.g., bending) of one or more surgical arm joints; restricting the range of movement; or restricting the arm articulation to selected degrees of freedom. Additionally or alternatively, the different operating modes can be characterized by different types of feedback to the user, such as feedback sensed by the user while controlling the surgical arm through one or more input devices.

[0133] In some embodiments, the selection and / or switching between operating modes is controlled by a user, such as a surgeon. Optionally, the selection of the operating mode can be performed through the system's user interface, such as through a touch screen and / or through buttons or other input devices provided on or near another input device, or a console or display screen. Additionally or alternatively, the selection of the operating mode can be automatic, for example, performed by a suitable control circuit, such as a system controller or a processor. In some embodiments, the selection and / or switching of the operating mode is triggered by one or more (and non-exhaustively) of the following and depends on one or more (and non-exhaustively) of the following: identifying the current 'anatomical' position of the surgical arm or the end effector of the arm, such as by using an electromechanical instrument associated with the actuators and / or motors (e.g., 104 or 108) of the arm, such as an encoder or other sensors (not shown), or by accessing and analyzing an image of the arm 102 obtained visually or through image processing by a camera, when identifying the current position of the input device, when performing a specific articulation of the surgical arm, when receiving an indication from one or more position sensors of the surgical arm, at a timing indication, such as by setting a time point at which to change the operating mode.

[0134] According to some embodiments, Figure 6 the flowchart in shows a method of operating a surgical robotic arm using two operating modes.

[0135] The method includes:

[0136] Step S01: Insert the surgical arm in a first operating mode and navigate the surgical arm to a predetermined position.

[0137] In the first operating mode, the surgical arm is introduced into the body and navigated to a selected anatomical location and / or a selected arm position. The surgical arm can be introduced into the body through natural body orifices (e.g., vagina, anus, trachea, esophagus, ear canal) and / or through an incision.

[0138] In some embodiments, in the first operating mode, arm articulation is restricted. For example, one or more arm joints (bending and rotation) can be constrained or blocked. In one example, the arm includes 3 joints: a shoulder joint, an elbow joint, and a wrist joint that rotates but does not bend, and one or two of the joints are prevented from articulating. In a specific example, only the movement of the elbow joint is allowed, such as bending, extension, and / or rotation of the elbow joint, while the shoulder joint and the wrist joint are constrained from moving.

[0139] In the first operating mode, the arm can also be allowed to move linearly as a whole (including only linearly) to, for example, advance or retract in a one-dimensional movement.

[0140] In some embodiments, the restriction of arm movement is mechanically achieved, for example, by one or more locks (e.g., solenoid locks) that affect the actuation of the arm joints. Additionally or alternatively, the restriction of arm movement can be achieved through a suitable circuit, such as by implementing a software control function that limits the degree and / or type of movement.

[0141] In some embodiments, the degree of restriction of arm movement and / or the restriction of certain types of movement are performed based on the current arm position indicated, for example, by one or more position sensors of the arm. In some embodiments, the degree of restriction of arm movement and / or the restriction of certain types of movement are performed based on the current anatomical position visualized, for example, by an optical member (e.g., a camera optionally introduced into the body together with the surgical arm).

[0142] Step S02: Hinge the surgical arm to a basic position in the first operating mode

[0143] Still in the first operating mode, hinge the surgical arm to a basic position. In some embodiments, the basic position includes a reverse-bent position of the arm, for example, when the arm is bent at least 120 degrees, at least 150 degrees, at least 180 degrees, or an intermediate, greater, or smaller angle. In some embodiments, the basic position is a position where the arm is positioned to allow the user to perform surgical actions from a selected orientation (e.g., corresponding to an abdominal orientation) from which the user may be more comfortable or familiar.

[0144] In some embodiments, control of the movement of the surgical arm in the first operating mode includes robotic control. Optionally, manipulation of the input device by the user in the first operating mode involves only limited types of user movement, e.g., limited movement of the input device along defined axes and / or pressing of buttons. In an example, moving the input device along a first defined axis actuates rotation of a selected arm joint (e.g., the elbow); moving the input device along a second defined axis actuates bending of a selected arm joint (e.g., the elbow); pressing one or more buttons actuates linear advancement or retraction of the surgical arm.

[0145] In some embodiments, the manipulation of the input device by the user in the first operating mode is translated into a movement speed of the surgical arm. For example, when the user moves the input device relative to its stationary position, the extent to which the input device moves relative to its stationary position sets the relative movement speed of the surgical arm.

[0146] Step S03 transitions to the second operating mode, and then surgical actions are performed using the surgical arm

[0147] In the second operating mode, the user performs surgical actions through the surgical arm, such as grasping tissue, dissecting tissue, moving tissue, suturing tissue. In some embodiments, once the arm has been moved to a selected base position, e.g., a reverse bent position, the second operating mode commences at the end of step S02.

[0148] In some embodiments, the user switches the input device when transitioning from the first operating mode to the second operating mode. Alternatively, the user uses the same input device for both the first or navigation operating mode and the second or surgical operating mode.

[0149] In some embodiments, in the second operating mode, the movement of the surgical arm is not as restricted as in the first operating mode. In an example, all arm joints (shoulder, elbow, wrist) are allowed to articulate, e.g., bend and / or rotate. In some embodiments, the extent and / or speed of arm movement during the second operating mode is restricted based on safety considerations, e.g., so as not to damage surrounding tissue and not to perform movements at too high a speed that may pose a risk of damage.

[0150] In some embodiments, control of the surgical arm in the second operating mode involves teleoperation control. Optionally, displacement of the input device by the user is mimicked by corresponding displacement of the surgical arm. Optionally, the displacement speed of the input device by the user is reflected in the corresponding movement speed of the surgical arm.

[0151] Step S04 transitions back to the first operating mode, and then the surgical arm is retracted from the body

[0152] According to some embodiments, optionally, when the surgical operation is completed, the arm retracts outwardly from the patient's body. In some embodiments, the retraction is performed in a first operating mode. Optionally, the arm is straightened before and / or during the retraction.

[0153] Retraction in the first operating mode may be advantageous because it limits the extent and / or type of movement of the arm, thereby potentially reducing damage to the tissue surrounding the anatomical passage (e.g., vagina) through which the retracting arm passes.

[0154] Now referring Figure 7 , the console 118 may include a display screen 407, with the input device 405 located nearby - on the opposite side of the screen 407 in a non-limiting example. Figure 7 In the non-limiting example of

[0155] Each of the input devices 405 (in the form of a 'thumbstick') includes a nipple-shaped controller 409 adapted to be manipulated by the user's thumb. In an embodiment, the degree of movement of the nipple-shaped member 409 relative to the central rest position can be converted into the speed of movement of the selected surgical arm, as discussed above. In one example, the farther the nipple-shaped member 409 is pushed away from its central rest position, the higher the final movement speed of the surgical arm. In another example, the greater the force applied to the nipple-shaped member 409, the higher the final movement speed of the surgical arm.

[0156] In some embodiments, when controlling the surgical arm via the thumbstick 405, the movement of one or more surgical arm joints (e.g., shoulder joint, wrist joint) is restricted, and only the bending and / or rotation of the elbow joint is enabled. In some embodiments, linear movement of the surgical arm (as a single unit) is also enabled, such as to advance or retract the arm. In some embodiments, the movement of the nipple-shaped member 409 actuates the bending and / or rotation of the elbow joint. In some embodiments, the linear movement of the arm 102 is actuated by a separate actuator, such as using another pair of input devices (e.g., input devices 406, 408), which are implemented as buttons disposed along the body of the input device (thumbstick assembly) 405 itself in the Figure 7 illustrated example of

[0157] In some embodiments, during use of the thumbstick 405, other input devices such as the avatar input arm 411 that provide, for example, input for a second mode without any constraints of a first mode are locked in a stationary position by a solenoid lock. In some embodiments, the stationary position of the input arm 411 is selected as the reverse-bent position of the surgical arm 102 such that once the surgical arm 102 has been reverse-bent (e.g., using the thumbstick), the user can pick up the avatar input arm 411 and continue the surgery directly. In some embodiments, when the avatar-like input arm 411 is enabled in the second mode, the operation of the thumbstick is disabled.

[0158] In some embodiments, during insertion of the surgical arm 102 into the body of the patient 106, the surgical arm is straight, i.e., not bent. In some cases, the insertion is performed through a cannula. Meanwhile, the avatar input arm is in a stationary position, which can be a locked position or a reverse-bent position. Optionally, after reverse-bending the surgical arm using the thumbstick 405, the surgeon can release the thumbstick 405 and move her hand to the avatar input arm 411. Once the surgeon grasps and optionally lifts the avatar input arm 411, control of the surgical arm 102 can automatically transfer or switch from the first user input device 405 to the second user input device 411, and the surgeon can continue the surgery using the avatar input arm 411.

[0159] In some embodiments, when one or more avatar input arm joints are locked by a solenoid lock, lifting the avatar input arm by the surgeon automatically releases the solenoid lock. Additionally or alternatively, a manual lock of the avatar input arm joint is released, for example, by a sensor that detects the position of the avatar input arm.

[0160] In some embodiments, the system (e.g., the system processor) is configured to identify one or more positions of the input devices, such as the current position of the thumbstick and / or the current position of the avatar input arm, and optionally present the positions on a user interface screen. In some embodiments, position sensors are used to assist in identifying the positions.

[0161] Figure 8 An image showing an example of a thumb-operated input device 501 is presented, the thumb-operated input device including a gripping handle 503 and optionally a textured surface facilitating gripping, such as a surface including ridges 505. One or more control buttons 507 can be disposed along the gripping handle 503. For example, Figure 8 the thumbstick input device 501 includes two control buttons 507 - one for actuating the linear advancement of the surgical arm distally and one for actuating the linear retraction of the arm proximally.

[0162] In some embodiments, the thumb stick 501 includes a nipple-like member 509 that extends, for example, from the proximal end of the gripping handle 503. In some embodiments, the shape and / or size of the nipple-like member 509 is suitable for the user's thumb. In some embodiments, the nipple-like member 509 includes a circular profile. In some embodiments, a circumferential protrusion 511 is formed on the proximal surface of the nipple-like member 509. The circumferential protrusion can help keep the thumb placed on the nipple-like member 509, thereby potentially preventing or reducing the thumb from slipping off the nipple-like member 509.

[0163] In some embodiments, the nipple-like member 509 moves in a spring-like manner. Optionally, after pushing the nipple-like member 509 away from its initial rest position (e.g., a central position where the nipple-like member 509 is centered and aligned with the long axis 513 of the thumb stick), the nipple-like member springs back to its central position.

[0164] Figure 9 is a schematic diagram illustrating a control example of the thumb-operated input 501 according to some embodiments. In some embodiments, moving (‘shifting’) the nipple-like member 509 relative to a first axis actuates a first type of movement of the surgical arm, such as bending; and shifting the nipple-like member 509 relative to a second axis actuates a second type of movement of the surgical arm, such as rotation.

[0165] In some embodiments, the thumb stick 501 is used during a first operation mode when at least some joints, such as the shoulder joint 2101 and the wrist joint 2105, are constrained. Optionally, shifting the nipple-like member 509 along the Y-axis produces bending of the elbow joint 2103 of the surgical arm 102; and shifting the nipple-like member 509 along an axis produces rotation of the elbow joint. In some embodiments, simultaneous actuation of bending and rotation can be achieved by pushing the nipple-like member relative to two axes (e.g., diagonally relative to the center).

[0166] Now refer to Figure 10 and Figure 11 , side view and front view images of an exemplary avatar-like input arm 701 are shown, the input arm including a plurality of joints for actuating corresponding movements of the surgical arm joints: As shown, the joints include a shoulder joint 703, an elbow joint 705, and a wrist rotation knob 707 for controlling the wrist joint 2105 of the surgical arm 102.

[0167] In some embodiments, the avatar input arm can include additional input devices, such as buttons or control levers 709, which can control the operation of the surgical tool (as the end effector 174) of the surgical arm 102.

[0168] Set in Figures 10 to 11Other input devices on the exemplary input arm 701 include a pause-resume button 711 and buttons 713, 715 for actuating linear advancement and retraction of the arm 102, respectively. The user may use buttons 713, 715 on the avatar input arm and / or buttons 406, 408 disposed on the thumbstick 405 to actuate linear advancement and / or retraction. Additionally or alternatively, linear movement of the arm may be actuated via a screen interface of the console (e.g., via a touchscreen interface).

[0169] In some embodiments, a set of two avatar input arms are provided for separately controlling the left and right arms. For example, a first avatar input arm 701 may control a first motor unit 108 associated with a first surgical arm 102 (e.g., the 'right' arm), and a second avatar input arm 701 may control a second motor unit 108 associated with a second surgical arm 102 (e.g., the 'left' arm). In some embodiments, any one or all of the input arms 120, 411, and 701 may be the same.

[0170] In the prior art example, both the first mode and the second mode use the same input device, namely an avatar-like input arm. As explained above, the input arm is used to reverse-bend the arm in the restricted mode, at which point a transition is made to fully enable use in the second mode. However, at this point, the input arm or at least one handle member (e.g., Figure 10 the handle member 702) may have flipped up to 180° or greater than 180° depending on the bending angle employed in the reverse-bending. This means that (a) from the surgeon's perspective, the handle 702 is uncomfortably 'inverted', and (b) the coordinate system used by the surgical system is 'backward'. The surgeon's perspective has switched from 'looking distally' to reverse-bend the arm to 'looking proximally' to perform surgery with the arm in the reverse-bent position, and thus does not translate the displacement vector or reorientation arc of the segment member into a corresponding (e.g., parallel) displacement vector or reorientation arc in the same x-y-z space.

[0171] It is now disclosed that when transitioning from a first mode to a second mode, transitioning from a first input device of an array of one or more input devices to a second input device of the array can overcome the aforementioned drawbacks. The handle member 702 of the second input device (avatar-like input arm 701) can be pre-positioned in an orientation that is not 'upside down' from the perspective of the surgeon; reverse bending can be performed using an input device (e.g., input device 501 / 405), the use of which does not involve or require moving the handle member 702 of the second input device, or if it is moved, it will not be moved more than 90°, which may be the 'critical point' of becoming 'upside down' from the perspective of the surgeon. Additionally, the control circuit for controlling the second input device (input arm 701) can be set (e.g., programmed) to use a 'direct' coordinate transformation matrix, where the displacement vector of the segment members of the input arm is transformed into a corresponding (e.g., parallel or at least maintaining the same sign in each of the x, y, and z directions or at least two of the three directions) displacement vector of the corresponding surgical arm segment, and / or where the reorientation arc of the segment members of the input arm is converted into a corresponding reorientation arc of the corresponding surgical arm segment.

[0172] Therefore, it may be desirable to ensure a switch (transition) from the first mode to the second mode, where control of the arm movement is transferred from the first input device (e.g., the 'thumb stick' detailed above) to an avatar-like input arm, thus ensuring ergonomic comfort and convenience based on the optimally oriented handle member and the directly translatable (input arm to surgical arm) coordinate translation matrix.

[0173] It should be noted that an exemplary design of the input device 701 and the handle member 702 is provided for illustrative purposes, and in other examples and in other embodiments, the input device and the handle member can be designed and implemented in different ways. For example, in some embodiments, the handle member can be physically separated from the input device, and the handle member is functionally part of the input device. As another example, in some embodiments, an avatar-like or joystick-like input device consists entirely or almost entirely or mainly of a manual and / or hand-grippable handle member. As another example, in some embodiments, the handle member can incorporate multiple operating functions into its design by including (but not exhaustively) buttons, switches, toggle switches, wheels, knobs, and / or sticks such as thumb sticks to 'combine' multiple input devices and their corresponding functions into what visually appears to be a single input device. Figure 10 The repeated presentation of the input device 701 and the handle member 702 herein is for convenience and ease of understanding and should not be construed as limiting the input device and handle member design.

[0174] Now referring to Figure 12A and Figure 12B。In an embodiment, alignment of an input device, such as an avatar input arm, with the current position of the surgical robotic arm is performed. In an example, alignment is performed when the user switches between different input devices, such as when switching from a thumb-operated input to an avatar input arm. In another example, alignment of the input device with the position of the surgical arm is performed when initializing the system, such as at the start of a surgery or before a surgery. In another example, alignment of the input device with the current position of the surgical robotic arm is performed when resuming control after a pause. Optionally, in the pause mode, movement of the input device does not actuate relative movement of the surgical robotic arm, and when resuming control, it may be necessary to adjust the position of the input arm to match the current position of the surgical robotic arm so as to continue the surgery in a smooth and uninterrupted manner.

[0175] In some embodiments, control is resumed in an automatic or semi-automatic manner. For example, the user performs a selected articulation of the input device (e.g., straightening the elbow joint of the avatar input arm) and resumes control of the surgical arm.

[0176] In Figure 12A and Figure 12B , the relative position of the input device (i.e., the input arm) is represented using a cross-shaped diagram 801. In some embodiments, there are two sets of crosses, one for the shoulder joint and one for the elbow joint. In some embodiments, the crosses represent a specific single joint of the surgical robotic arm (e.g., the shoulder joint, the elbow joint). In some embodiments, each line of the slot represents a different type of articulation. For example, the horizontal line 803 represents rotation of the joint; the vertical line 805 represents flexion of the joint. During use, the user manipulates the input device according to the joint positions indicated by two colored dots 807 at the cross. As the position of the input device (in response to the user's manipulation) becomes closer to the position of the surgical robotic arm, the colored dots move closer to the center of the cross (see Figure 12B ). Optionally, once sufficient alignment between the position of the input device and the position of the surgical arm is obtained, the dots change color, e.g., from red to green, as indicated in Figure 12A -B.

[0177] In an embodiment, Figure 12A and Figure 12B 's control device can be used in alignment calibration as part of a transition from a first mode using a first input device to a second mode using an avatar-like input arm. In these embodiments, the first input device can be used to adjust the position of the surgical arm so as to align it with the fixed position of the input arm.

[0178] Figure 13An example of a screen presented to a user during alignment according to some embodiments. In the example shown, a first input device (e.g., corresponding to the right avatar arm) is shown to be correctly aligned with a first surgical arm, e.g., indicated by checkmarks 809 at the positions of both the elbow joint and the shoulder joint, and / or by a locked lock 811. A second input device (e.g., corresponding to the left arm) is shown in a position where it is not yet aligned with the surgical arm. The joint positions are represented by two cross-shaped diagrams as described above and the lock is unlocked.

[0179] In some embodiments, during alignment, certain articulations and / or functions are deactivated, such as restricting the speed of the surgical arm; deactivating the electrosurgical function; and / or other functions.

[0180] Figure 14A -E shows a set of images depicting exemplary control of a surgical robotic arm using multiple different input devices according to some embodiments. Two arms are shown in Figure 14A a non-limiting example of -E, and in other examples, there may be a single arm or more than 2 arms.

[0181] Figure 14A Depicts the control of the arms during the introduction of two surgical robotic arms 901 into a model 903 simulating entry into the body via the vagina according to some embodiments. In this example, the control of the surgical arm 901 during the advancement of the arm into the body (e.g., through the vaginal cavity) is by thumb-operated input, including, for example, a set of thumbsticks 905 as described above.

[0182] Figure 14B Depicts the manipulation of the surgical arm into a reverse-bent position by the thumbstick 905. In the example shown, the arm is bent (e.g., 120, 150, 180, 210 degrees or intermediate, greater or lesser angles) to obtain the reverse-bent position.

[0183] Figure 14C Depicts the surgical arm in the reverse-bent position. After reverse-bending, in some examples, the user can switch the input device used, for example, by releasing the thumbstick and picking up the avatar input arm. At this time, alignment of the avatar input arm with the current position of the surgical arm can be performed, such as as Figure 12A -B and Figure 13 described.

[0184] Figure 14D and Figure 14E Depicts the use of a set of avatar input arms 907 to manipulate a surgical arm according to some embodiments. It can be seen that the corresponding positions of the surgical arm correspond to the positions of the input arms.

[0185] Figure 15The block diagram shows the articulated arm 102, the input device array 1500, and the detector 1520. Those skilled in the art will understand that not every element that appears in Figure 15 is required in every embodiment. Although the input device array 1500 shown in Figure 15 shows two user input devices 1510A (e.g., thumbstick 501), 1510B (e.g., joystick 701), those skilled in the art will understand that fewer or more user input devices may be provided in different embodiments. In an example, the joystick 701 can be both the first user input device 1510A and the second user input device 1510B; in other words, the first user input device 1510A and the second user input device 1510B can be the same single user input device. In an implementation where the first user input device 1510A and the second user input device 1510B are the same single user input device, it is typically a more general or more flexible device, such as the joystick 701. The term user input device refers to a device for converting the input received by the user into an electronic output and / or signal. Examples of user input devices include, but are not limited to, joysticks, touchscreens, thumbsticks, mice, keyboards, gesture detection devices (e.g., including cameras [not shown]). Unless otherwise specified, the term 'array' refers to one or more.

[0186] As Figure 15 shown in the example of, the articulated arm 102 has one or more objects 1530 mounted at its distal end. The objects 1530 can be provided with the arm 102 or can be added or replaced separately. Examples of such objects 1530 include, but are not limited to, end effectors, such as surgical end effectors. Related surgical tools can include, but are not exhaustive: endoscopes for diagnostic / surgical feedback, surgical end effector tools such as needle holders (e.g., large needle holders, curved needle holders), monopolar and bipolar instruments (e.g., monopolar scissors, bipolar forceps), clip appliers (e.g., large and medium clip appliers), vascular closure devices, graspers or dissectors (e.g., Maryland dissector, Tenacululm forceps, micro forceps, long tip forceps, grasping retractor, fundus grasper, Crocodile grasper, Cadiere forceps), scissors (e.g., Potts scissors, curved scissors), hooks (e.g., electrocautery hook), and scrapers (e.g., electrocautery spatula).

[0187] As discussed elsewhere, in some embodiments, the arm 102 and / or the object 1530 operate in response to an electronic control output from one or more control devices. The terms 'control output' and 'control signal' are used synonymously. In different embodiments, the control output can be sent to the arm 102 and / or its controller via wired and / or wireless communication.

[0188] Figure 15 Detector 1520 is also shown. As discussed elsewhere, in some embodiments, the mode transition from the first operating mode of surgical system 999 to the second operating mode of surgical system 999 is responsive to and / or dependent on the output of detector 1520. Surgical system 999 may be functionally equivalent to Figure 1 the surgical system 100 shown in. In one non-limiting example, detector 1520 detects whether a portion of arm 102 (e.g., the distal portion) is bent in reverse and / or is in a reverse-bent position. In one example, a camera may acquire images of arm 102 and detector 1520, the camera, and the image processing circuitry. In another example, an encoder (not shown) or other electromechanical sensors (e.g., 104 or 108) may be used to monitor and detect to track the orientation of, for example, the joints of arm 102. Other examples may involve magnetic or capacitance detectors, or position detection based on, for example, triangulation using ultrasound and / or light (e.g., time-of-flight).

[0189] As used in this disclosure and the appended claims, 'monitoring' and 'detecting' are actions (and / or functions and / or potential actions and / or capabilities) that may be performed by one or more components of a surgical system, by one or more users, or any combination of system components and human users. The descriptive language used herein with respect to automated or machine monitoring or detecting is meant to be non-limiting, and in any such embodiment, user intervention may be part of the design and / or part of the operation. In some embodiments, user intervention is required for safety reasons. In a non-limiting example, one or more sensors relay information about a component of the surgical arm or one of its components to a display screen on which the user trains and / or positions to monitor the arm curve shape and detect when the arm is in the desired position and orientation without or without automated or semi-automated visual assistance, e.g., when bent in reverse at the surgical site. In another non-limiting example, the monitoring and / or detecting is communicated to the user in a non-visual manner, such as but not exclusively, by an audible notification, by tactile feedback, or by locking a control or input device. Clearly, any of these types of communication may also be combined with visual information.

[0190] Those skilled in the art will also understand that additional elements may be provided in surgical system 999 and that not every component of every element shown in Figure 15 is required in every embodiment.

[0191] Now refer to Figure 16。In step S101, the surgical system 999 operates in a first operating mode. In step S121, the surgical system 999 operates in a second operating mode. As discussed elsewhere, in different embodiments, the first and / or second operating modes may involve specific capabilities and / or specific constraints of one or more input devices. Alternatively or additionally, the first and / or second operating modes may involve specific capabilities and / or specific constraints of one or more elements of the arm 102 and / or the object 1530. Examples of such elements include joints and articulators. Alternatively or additionally, the first and / or second operating modes may involve the relationship between the operation of one or more input devices 1510 and the arm 102 or one or more of its components - for example, whether the configuration of the input device 1510 or its components is converted into the speed or position of the arm 102 or its components.

[0192] When the surgical system 999 operates in the first mode in step S101, one or more operations are performed. In one example, when the surgical system operates in the first mode, the distal portion of the arm 102 is bent in reverse, possibly starting from a non-bent and / or straight position, but not necessarily so. In another example, when the surgical system 999 operates in the first mode, the distal portion of the arm 102 is moved such that its curved shape, for example, a curved shape in 3D or its planar projection, matches a predefined curved shape, such as a curved shape for starting a surgical procedure (e.g., a reverse-bent shape or an 'S' curve shape).

[0193] Step S109 involves monitoring and may be performed concurrently with step S101. For example, step S109 may be performed at least in part by the detector 1520. In some embodiments, the monitoring of step S109 may include determining whether the arm is in a reverse-bent position.

[0194] Step S113 involves a mode transition trigger event - i.e., a detectable event whose detection triggers a mode transition of the surgical system 999 from the first mode of step S101 to the second mode of step S121. An example of step S113 is as follows: In response to and depending on, for example, the detector 1520 detecting that the arm 102 is in a reverse-bent position, i.e., detecting that the arm 102 has transitioned from a non-reverse-bent position to a reverse-bent position, the surgical system 999 transitions from the first operating mode to the second operating mode. As discussed above, the transition may include alignment calibration of the surgical arm 102 with the input arm 701.

[0195] The transition of step S117 may be triggered, for example, by and / or in response to and / or depending on detecting that the arm 102 has been bent in reverse, or that the end effector 1530 is in a reverse-bent position.

[0196] Discussion of the first mode of S101 and the second mode of S121

[0197] In some embodiments, step S101 may provide one or more of the following features, either individually or in combination:

[0198] (A1) The configuration of arm 102 or at least its distal section is controlled by user input device 1510A and can respond to an electronic control output from user input device 1510A. In some embodiments, user input device 1510A may be a first user input device, such as thumbstick 501, and in other embodiments, user input device 1510A may be a single identical user input device 1510A / 1510B, such as joystick 701.

[0199] (A2) When the surgical system is in a first mode (e.g., to perform reverse bending), the configuration of the arm is controlled by the output of a user input device (e.g., thumbstick device 501) such that the magnitude (i.e., compared to the displacement speed, the magnitude of displacement from a base position and / or an initial and / or incoming and / or center position) of the user input device or its displaceable part (e.g., located on tiltable nipple-shaped member 409 or 509) is converted into the speed of bending and / or rotation of an arm joint such as elbow 2103;

[0200] (A3) When the surgical system is in a first mode (e.g., to perform reverse bending), the configuration of the arm is controlled by the output of a user input device such that the position specification (e.g., command) of an element of the user input device selected from a plurality of candidate positions (e.g., defined by a plurality of possible tilt angles of a nipple-shaped member or equivalent) of the element of the user input device designates (e.g., commands) a target movement speed of an element of the arm selected from a plurality of candidate speeds (e.g., the greater the tilt angle of the nipple-shaped member, the greater the speed);

[0201] (A4) When the surgical system is in a first mode, the configuration of the arm is controlled by the output of a user input device such that the user input device has a plurality of positions or configurations, where (i) for a subset of the plurality of positions or configurations, each designates movement in the same bending plane or rotational direction, (ii) the first position or configuration designates / commands a first speed, and (iii) the second position or configuration designates (e.g., commands) a second speed, the second speed exceeding the first speed.

[0202] (A5) When the surgical system is in a first operating mode, the control circuit of the surgical system effectively constrains the actuation of arm joints other than a single arm joint.

[0203] Second operating mode: In some embodiments, step S121 may have one or more of the following features, either individually or in combination:

[0204] (B1)For example, in response to controlling the output from the user input device 1510B, the configuration of the arm 102 or at least its distal section is controlled by the user input device 1510B (e.g., the joystick 701);

[0205] (B2)When the surgical system is in the second mode, e.g., to perform a surgical maneuver, the configuration of the arm 102 is controlled by the output of the user input device 1510B such that the displacement value of a given input device or its displaceable part is converted into a corresponding displacement of at least a part of the arm and / or at least one arm segment of the arm - e.g., in Figure 10 , the displacement of the shoulder element 703 of the joystick 701 can be converted into a corresponding displacement of the shoulder joint 2101 of the arm 102, and / or e.g., in Figure 10 , the displacement of the elbow element 705 of the joystick 701 can be converted into a corresponding displacement of the elbow joint 2103 of the arm 102;

[0206] (B3)When the surgical system (e.g., to perform a surgical operation) is in the second mode, the configuration of the arm is controlled by the output of the user control device such that the displacement value of a given input device's displaceable part specifies (e.g., fully specifies; e.g., commands) the target position (i.e., different from the current primary position) of the arm element and / or the target configuration (e.g., different from the current primary configuration) of the arm.

[0207] First and second operating modes: In some embodiments, steps S101 and S121 can together provide one or more of the following notable features (e.g., any combination):

[0208] (C1)In some embodiments, when the surgical system is in the first operating mode, the first input device 1510A (e.g., the thumbstick) of the input device array 1500 controls the actuation of a single arm joint (e.g., the elbow 2103), and when the surgical system is in the second operating mode, the second input device 1510B (e.g., the joystick) of the input device array 1500 controls the actuation of multiple arm joints (e.g., the elbow 2103 and the shoulder 2101).

[0209] (C2) In some embodiments, the input device array includes a first input device 1510A (e.g., thumbstick 501) and a second input device 1510B (e.g., joystick 701); (ii) the first input device is configured to control the actuation of a single arm joint (e.g., elbow 2103) and is not configured to control the actuation of arm joints other than the single arm joint (e.g., elbow 2103) (e.g., shoulder 2101, etc.); (iii) when in the second operation mode, the second input device controls the actuation of multiple arm joints (e.g., elbow 2103 and shoulder 2101); (iv) when the surgical system is in the first operation mode, the control circuit effectively implements the constraint by allowing the first input device 1510A to control the arm 102 while prohibiting the second input device 1510B from controlling the arm. Thus, the transition from the first operation mode to the second operation mode (e.g., entering step S121) can'switch' control from the first user input device 1510A to the second user input device 1510B.

[0210] (C3) In some embodiments, the user input devices 1510A, 1510B are the same user input device, such as joystick 701 or a similar articulated device, i.e., having segment components and segment joints corresponding to the arm segments and arm joints of the arm 102.

[0211] Now describe Figure 16 a plurality of embodiments. Those skilled in the art will understand that these embodiments need not be mutually exclusive - these embodiments or their features may be combined. In some embodiments, not all features and / or method steps need to be present.

[0212] Figure 16 First Embodiment

[0213] The first embodiment relates to a method of operating a surgical system, the surgical system comprising (i) an array of input devices of one or more user input devices and (ii) an articulated robotic arm, the articulated robotic arm comprising a surgical end effector at its distal end and a plurality of arm joints configured to bend and rotate in response to an electronic control output from the user input device, the method comprising: (a) starting operation of the surgical system in a defined first mode of operation S101 with respect to a given single arm joint (e.g., elbow 2103) of the arm joints, wherein the first mode prevents actuation of any arm joint of the actuated arm that is not the given single arm joint (e.g., including shoulder 2101), and allows actuation of the single arm (e.g., elbow 2103) joint to cause bending and rotation of the single arm joint (e.g., elbow 2103); (b) when the surgical system is in the first mode of operation (e.g., S101), (i) in response to a control signal generated by one or more of the user input devices (e.g., thumbstick 501), bend and rotate the distal end of the arm in reverse by bending the single arm joint (e.g., elbow 2103) to place the end effector 174 in a reverse-bent operating position, and (ii) monitor the state of the robotic arm 102 (e.g., S109) to detect whether the arm is in the reverse-bent position; (c) in response to and depending on (in step S113) detecting that the arm 102 is in the reverse-bent position, transition the operation of the surgical system (e.g., the 'yes branch' from S113 to S121) from the first mode to a second mode, in which the system is enabled to control the bending and rotation of at least one arm joint (e.g., elbow 2103) excluding the first mode according to the respective degrees of freedom of each arm joint; (d) operate the surgical system in the second mode (e.g., at S121) to perform a surgical action using the end effector.

[0214] Figure 16 Second Embodiment

[0215] A method of operating a surgical system, the surgical system including (i) a first user input device 1510A (e.g., thumbstick 501) and a second user input device 1510B (e.g., joystick 701), and (ii) an articulated robotic arm 102 including a plurality of arm joints (e.g., elbow 2103 and shoulder 2101) and a surgical end effector 174 at a distal end of the arm, the method including: (a) initiating operation of the surgical system in a reverse bend mode (e.g., the first mode of S101), wherein with respect to bending and rotation of the arm joints: (i) the first user input device 1510A (e.g., thumbstick 501) is activated to direct bending and rotation of only a given arm joint of the arm joints, and (ii) the second user input device 1510B (e.g., joystick 701) is deactivated; (b) while in the reverse bend mode by bending and rotation of a given arm joint of the arm joints (e.g., elbow 2013), in response to an electronic control output from the first user input device 1510A, bending the distal portion of the articulated robotic arm in reverse to place the end effector in a reverse bend operating position; (c) transitioning the surgical system from the reverse bend mode to a surgical mode to enable the second user input device 1510B with respect to bending and rotation of at least one arm joint of the arm other than the given arm joint of the arm joints (e.g., elbow 2103) (e.g., shoulder 2101); and (d) in the surgical mode, effecting bending and rotation of at least two (e.g., shoulder and elbow) of the arm joints in response to an electronic control output from the second user input device 1510B according to the respective degrees of freedom of each arm joint, thereby moving the surgical end effector to perform one or more surgical actions.

[0216] Figure 16 Third Embodiment

[0217] A method of operating a surgical system, the surgical system including (i) an input device array 1500 of one or more user input devices and (ii) an articulated robotic arm 102, the articulated robotic arm including a plurality of arm joints (e.g., a shoulder 2101 and an elbow 2103), and a surgical end effector 1764 at a distal end of the arm, the method comprising: (a) starting operation of the surgical system in a reverse bending mode (e.g., S101), wherein with respect to bending and rotation of the arm joints, the input device array is activated to direct bending and rotation of only a given arm joint (e.g., the elbow 2103) among the arm joints; (b) while in the reverse bending mode, in response to an electronic control output from the input device array, reversely bend a distal portion of the articulated robotic arm by bending and rotating the given arm joint among the arm joints to place the end effector in a reverse bending operating position; (c) transitioning the surgical system from the reverse bending mode to a surgical mode to enable the input device with respect to bending and rotation of arm joints other than the given arm joint among the arm joints; and (d) while in the surgical mode, in accordance with the respective degrees of freedom of each arm joint, effect bending and rotation of at least two (e.g., at least the elbow 2013 and the shoulder 2101) of the arm joints in response to an electronic control output from the input device array, thereby moving the surgical end effector to perform one or more surgical actions.

[0218] Figure 16 Fourth Embodiment

[0219] A method of operating a surgical system, the surgical system including (i) a user input device (e.g., a joystick 701) and (ii) an articulated robotic arm 102, the articulated robotic arm including a plurality of arm joints, and a surgical end effector at a distal end of the arm, the method comprising: (a) starting operation of the surgical system in a reverse bending mode (e.g., S101), wherein with respect to bending and rotation of the arm joints, the user input device is activated to direct bending and rotation of only a given arm joint (e.g., the elbow) among the arm joints; (b) while in the reverse bending mode, in response to an electronic control output from the user input device, reversely bend a distal portion of the articulated robotic arm by bending and rotating the given arm joint among the arm joints to place the end effector in a reverse bending operating position; (c) transitioning the surgical system from the reverse bending mode to a surgical mode (e.g., 121) to enable the user input device with respect to bending and rotation of at least one arm joint other than the given arm joint in the arm; and (d) while in the surgical mode, in accordance with the respective degrees of freedom of each arm joint, effect bending and rotation of at least two of the arm joints in response to an electronic control output from the user input device, thereby moving the surgical end effector to perform one or more surgical actions.

[0220] Figure 16 The fifth embodiment

[0221] A method of using a surgical system, the method comprising: (a) providing an articulated robotic arm capable of displacing a surgical end effector 174 at its distal end, the arm comprising a plurality of arm segments connected by a corresponding plurality of arm joints (e.g., elbow and shoulder joints) (e.g., in series), the arm joints being configured to bend and rotate in response to an electronic control output from a user input device; (b) maneuvering the end effector to a reverse bending operating position when operating in a first input mode, in which the displacement of an input device (e.g., 1510A, e.g., a thumbstick) or a displaceable portion thereof is converted into a speed of bending and / or rotation of the arm joints; (c) transitioning from operating in the first input mode to operating in a second input mode in response to and depending on detecting that the end effector is in the reverse bending operating position, in which the displacement of an input device (e.g., 1510B, e.g., a joystick) or a displaceable portion thereof is converted into a corresponding displacement of at least one arm segment; and (d) performing a surgical action using the end effector after the transition and when operating in the second input mode.

[0222] The present invention has been described using detailed descriptions of its embodiments, which are provided by way of example and are not intended to limit the scope of the present invention. The described embodiments include different features, not all of which are required in all embodiments of the present invention. Some embodiments of the present invention utilize only some of the features or possible combinations of these features. Variations of the described embodiments of the present invention and embodiments of the present invention including different combinations of the features mentioned in the described embodiments will occur to those skilled in the art to which the present invention pertains.

[0223] Additional discussion

[0224] According to aspects of some embodiments, there is provided a method of controlling one or more surgical robotic arms insertable into a patient's body via one or more input devices, each surgical robotic arm comprising a plurality of movable joints, the method comprising: navigating the surgical robotic arms into the patient's body in a first operating mode; performing a surgical action using the surgical robotic arms in a second operating mode; wherein in the first operating mode, the movement of each of the surgical robotic arms is restricted to the movement of a single joint among the plurality of joints and a linear movement of the surgical robotic arms as a single unit.

[0225] In some embodiments, the navigating includes reverse bending the surgical robotic arms within the patient's body.

[0226] In some embodiments, in a first operating mode, the surgical robotic arm is controlled by a thumb-operated input.

[0227] In some embodiments, in a second operating mode, the surgical arm is controlled by an avatar input arm.

[0228] In some embodiments, during the first and second operating modes, the surgical robotic arm is controlled by a haptic handle.

[0229] In some embodiments, in the first operating mode, the manipulation of the input device by the user is converted into the movement speed of the surgical robotic arm, and in the second operating mode, the displacement of the input device by the user is converted into the relative displacement of the surgical robotic arm.

[0230] In some embodiments, in the second operating mode, a clutch-like mode is enabled to disconnect the control of one or more input devices from the surgical arm.

[0231] In some embodiments, a console for controlling one or more surgical robotic arms is provided, including: a thumb-operated input for controlling the first operating mode; a manual input for controlling the second operating mode; and a screen interface.

[0232] In some embodiments, the thumb-operated input includes a nipple-shaped member engagable by the user's thumb; whereby pushing the nipple-shaped member from a central rest position actuates the relative movement of the surgical robotic arm; and wherein the movement speed is affected by the degree of pushing the nipple-shaped member relative to its rest position.

[0233] In some embodiments, in the second operating mode, the manipulation of the manual input by the user is converted into a similar articulation of the surgical arm.

[0234] According to aspects of some embodiments, a method for performing surgery on a patient is provided, including: introducing one or more surgical arms into the abdominal cavity through the vagina; bending one or more surgical arms to a reverse-bent position, wherein during the introduction and bending, the articulation of one or more surgical arms is limited to linear movement and the movement of only a single arm joint; using one or more surgical arms in the reverse-bent position to perform surgery within the abdominal cavity.

[0235] In some embodiments, wherein during linear movement, the surgical arm moves as a single unit, and wherein the movement of a single arm joint includes the bending and extension of the elbow joint.

[0236] Some general aspects of some embodiments of the present invention relate to controlling one or more surgical robotic arms using a first operating mode for navigating the arm to a selected orientation and position within a patient's body and a second operating mode for performing surgical actions within the selected orientation. In some embodiments, controlling in the first mode involves converting the manipulation of a user (e.g., a surgeon) of an input arm (e.g., an avatar arm, a joystick) into corresponding articulations of the surgical arm, whereby the movement speed of the surgical arm varies as a function of the degree of manipulation of the input arm relative to the rest position of the input arm.

[0237] In some embodiments, controlling in the second mode involves converting the manipulation of the user of the input arm into corresponding positions of the surgical arm, such that the positions are directly set according to the input arm positions (e.g., the same articulations). In some embodiments, the user displacement of the user arm is converted into a relative displacement command for the surgical arm. In some embodiments, the second mode includes different controls, such as a control in which the degree of manipulation of the input arm by the user is proportional to a different similar movement performed by the surgical arm.

[0238] In some embodiments, the movement of the surgical arm in the first mode is restricted, for example, by providing only flexion and / or extension of the elbow joint of the surgical arm and linear movement of the surgical arm as a single unit. Optionally, other joints of the surgical arm (e.g., the shoulder joint, the wrist joint) remain fixed, preferably locked in place. Alternatively, at least partially restricted movement of one or more other joints is allowed.

[0239] In some embodiments, when controlling the surgical arm using the first operating mode, the surgical arm is introduced into the body (e.g., through the vagina), the arm is navigated (e.g., into the abdominal cavity), and optionally the surgical arm is bent in reverse. Potential advantages of constraining the articulations of the surgical arm during arm introduction and / or during reverse bending of the arm may include reducing the bending radius of the surgical arm, thereby reducing the likelihood of encountering surrounding obstacles such as the inner abdominal wall. Another potential advantage of constraining the articulations of the surgical arm during navigation and / or reverse bending processes may include improved control of the surgical arm. In some embodiments, when reversing the relative direction (e.g., up and down), the first operating mode (speed control mode) allows the user to continuously control the movement of the surgical arm even during reverse bending of the arm.

[0240] In some embodiments, a system configured to control a surgical arm in two modes includes a dual control member including two different control sets. In some embodiments, a first control set is for inserting the surgical arm into the body, and a second control set is for performing surgical actions within the body. Alternatively, the two sets are for at least one of the phases (insertion and surgery); alternatively, only one set is for the two phases.

[0241] In some embodiments, the first mode is controlled by a set of thumbsticks. Optionally, the degree to which the nipple-shaped member of each thumbstick is pushed relative to the central rest position of the nipple-shaped member affects the speed at which the arm articulates. Optionally, when the user lifts their thumb, the nipple-shaped member springs back to its rest position. In some embodiments, a second operating mode is controlled using a set of avatar input arms manipulated by the user's hand.

[0242] Additionally or alternatively, a set of haptic handles (including, for example, 2 handles, one for controlling each surgical arm) is used to perform the two operating modes. Optionally, in the first operating mode, the handle is set to provide a counter-resistance to the user's movement in response to moving the arm from its rest position, optionally a spring-like resistance. Optionally, in the second operating mode, the haptic handle is set (e.g., pre-programmed) to provide a selected or varying resistance in response to the user's manipulation.

[0243] Additional discussion

[0244] Now refer to Figure 17A -C.

[0245] In some embodiments, control of one or more surgical arms is achieved through one or more input arms, joysticks, control handles, and / or other members suitable for manipulation by a user (e.g., a surgeon), which are then converted into a matching articulation of the surgical arm.

[0246] In the example described herein, as mentioned in the flowchart of Figure 17A some embodiments include dual control of the surgical arm. In some embodiments, a first user input (in this example, the thumbstick 4005 of Figure 17B is used to introduce the surgical arm into the patient's body, for example, through the vagina, and then to bend the surgical arm (4001) in the reverse direction. In some embodiments, a reverse bend of the surgical arm within the patient's body (e.g., a backward bend) is performed to reduce the area where the surgical arm is located. Optionally, the reverse bend is performed during surgery to avoid obstacles, such as certain organs or parts thereof, such as the inner wall of the abdomen. Optionally, the reverse bend is performed to position the surgical arm in an orientation familiar to laparoscopic surgeons for performing the surgery.

[0247] In some embodiments, a second user input, here in the example in the form of the input arm 4011 (such as an avatar joystick), is then used to perform the remaining surgical procedure (4003).

[0248] In some embodiments, the thumbsticks 4005 are positioned near the console screen 4007, such as on opposite sides of the screen. In some embodiments, each thumbstick 4005 includes a nipple-shaped controller 4009, the shape and size of which are adapted to a user's thumb. In some embodiments, the nipple-shaped member of the thumbstick is in a rest position when centered and is configured to spring back to the rest position when the thumb is released. In some embodiments, the degree of movement of the nipple-shaped member relative to its central rest position determines the final speed of the movement of the surgical arm. For example, the further the nipple-shaped member is pushed away from its central rest position, the higher the speed of movement of the surgical arm (and vice versa - the closer the nipple-shaped member is to its central rest position, the lower the speed of the arm).

[0249] In some embodiments, when controlling the surgical arm via the thumbstick, the movement of one or more surgical arm joints (such as the shoulder joint, wrist joint) is constrained. In some embodiments, all surgical arm joints except the elbow joint are prevented from moving, and only the elbow joint can bend and / or rotate. In some embodiments, linear movement of the surgical arm (as a single unit) is also enabled, such as to advance or retract the arm. In some embodiments, the movement of the nipple-shaped member actuates the bending and / or rotation of the elbow joint. In some embodiments, the linear movement of the arm is actuated by a separate actuator, such as using buttons (such as 4006, 4008) configured along the body of the thumbstick 4005. In the example, button 4006 advances the surgical arm distally (such as into the abdomen); button 4008 retracts the surgical arm proximally.

[0250] In some embodiments, during use of the thumbstick, the input arm 4011 is locked in a rest position, such as by a solenoid lock. In some embodiments, the rest position of the input arm is selected as the reverse bend position. Optionally, this position allows the surgeon to continue the surgery immediately after performing a reverse bend using the thumbstick. In some embodiments, when operating the input arm, the operation of the thumbstick is deactivated.

[0251] When the selected arm joint (e.g., the shoulder joint) remains stationary, the potential advantages of using the thumbstick to navigate into the body and reverse bend the surgical arm may include reducing the bending radius of the surgical arm, thereby reducing the likelihood of encountering surrounding obstacles such as the inner abdominal wall. For example, compared to using the input arm for navigation and reverse bending, another potential advantage of using the thumbstick for navigation and / or reverse bending processes may include improved control of the surgical arm, where the ergonomics of the handle may not be well-suited to support the rotational movements that the surgeon needs to perform when holding the handle for reverse bending.

[0252] In some embodiments, during the introduction of the surgical arm into the body, the surgical arm is straight (optionally provided with a y for insertion through an introducer cannula), while the input arm is in a stationary, locked, reverse-bent position. Optionally, after reverse-bending the surgical arm using the thumbstick, the surgeon releases the thumbstick and moves their hand to the input arm. Once the surgeon grasps the input arm and optionally lifts the input arm, control of the surgical arm is automatically obtained, and the surgeon can continue the surgery using the input arm. In some embodiments, when one or more input arm joints are locked by a solenoid lock, lifting the input arm by the surgeon automatically releases the solenoid lock. Additionally or alternatively, the manual lock of the input arm joint is released, for example, by a sensor that detects the position of the input arm.

[0253] In some embodiments, the system (e.g., the system processor) is configured to identify one or more positions of the input arms, for example, when the input arms are in their stationary positions, and optionally display the current positions to the user.

[0254] Now refer to Figure 18 . In some embodiments, a haptic handle that provides force feedback to the user (a suitable example is the 'omega.7' haptic device available from ForceDimension, Nyon, Switzerland) is used throughout the procedure for controlling the movement and articulation of the surgical arm. In some embodiments, the haptic handle is set to provide counter-resistance to prevent the user from moving the surgical arm in directions that are not supported, such as bending the elbow joint of the surgical arm backward; contacting the joint (e.g., the elbow joint) with different segments of the same arm; and / or others. In some embodiments, the handle is set to provide counter-resistance that varies according to the current anatomical position and / or orientation of the surgical arm. In an example, if the user attempts an unauthorized anatomical area, such as an organ that should be avoided, the resistance may increase.

[0255] In some embodiments, the haptic handle is programmed to operate according to various control modes. Optionally, the control mode is selected according to the current stage of the surgical procedure. In some embodiments, switching between different modes is performed via a screen interface, one or more buttons on the console or the handle, a foot pedal, and / or one or more of the others.

[0256] In some embodiments, during the first stage of the surgery, while introducing the surgical arm into the patient's body and optionally bending the surgical arm in reverse, the haptic handle (5001) is used in a'speed-control' mode. Optionally, in the speed-control mode, the relative movement of the handle with respect to the handle's rest position sets the speed of moving the surgical arm. When the user moves the handle away from the rest position, the speed increases, and vice versa. For example, a movement of the handle to the right of its rest position may cause the arm joint (e.g., the elbow joint) to rotate to the right at a speed determined by the distance of the handle from its rest position. In some embodiments, in the speed-control mode, the haptic handle is set to provide elastic (spring-like) counter-resistance to the user's movement. In some embodiments, in the speed-control mode, a control algorithm is applied to convert the current configuration of the haptic handle into a speed command issued to the actuator (e.g., a motor) of the surgical arm, such as to increase the rotational speed of one or more motor gears.

[0257] Potential advantages of using the speed-control mode while introducing the surgical arm into the body and optionally bending the surgical arm in reverse may include: the directions are opposite (e.g., up / down) during reverse bending, but such changes can be ignored and the movement can continue naturally because the final movement of the surgical arm is restricted while the movement speed changes.

[0258] In some embodiments, during the second stage of the surgery, optionally during the remaining surgery, the haptic handle is set to the 'position-control' mode (5003). Optionally, in the position-control mode, the spatial position of the handle sets the corresponding position of the surgical arm. In the position-control mode, the displacement of the user's haptic handle is converted into a relative displacement command for the surgical arm. In some embodiments, the displacement of the haptic handle is controlled according to an algorithm. In some embodiments, the control is based on a known algorithm (e.g., the inverse Jacobian algorithm). Additionally or alternatively, in some embodiments, the control is based on a custom algorithm. In an example, the custom algorithm is set to scale the user's movement, such as to improve the accuracy of the movement. Such scaling may include magnifying the movement required at the user end by a selected coefficient to produce a similar non-magnified movement of the surgical arm. For example, to move the arm a distance X, the user needs to move the handle A*X (A>1). In another example, an algorithm is selected to filter the signal, such as filtering using a low-pass filter to reduce user hand tremors.

[0259] In some embodiments, in position control mode, a clutch mechanism is provided to allow the user to temporarily disconnect from the surgical arm (such that movement of the input haptic handle no longer controls the surgical arm). Optionally, when disconnected, the user can freely reposition the haptic handle. In an example, the user repositions the haptic handle to a position and / or orientation where the user can more comfortably perform and control the next movement.

[0260] In some embodiments, the degree of resistance sensed by the user in response to movement of the handle is selected and controlled. In an example, a floating mode is set where the user essentially encounters no resistance and can move the handle freely in all directions. Additionally or alternatively, the level of resistance sensed by the user is adjusted, such that the user senses high resistance in response to one movement and low or no resistance in response to another movement.

[0261] In some embodiments, the amount of resistance is controlled based on the anatomical position of the surgical arm. For example, high resistance can be set where an obstacle (such as the abdominal wall) may be found near the surgical arm. In a specific example, if an obstacle is found on the right side of the surgical arm, the user may encounter high resistance in response to moving the handle to the right; if no obstacle is found on the left side of the arm, the user may encounter low or no resistance in response to moving the handle to the left. Optionally, the degree of resistance is defined by system settings, such as generating wall-like resistance, rubber-like resistance, sand-like resistance, and / or others.

[0262] Any feature or combination of features described in this document can be combined with any feature or combination of features described in U.S. Patent Application Serial No. 16 / 121,704, filed September 5, 2018, and published as U.S. Patent Publication US20190000574A1; U.S. Patent Application Serial No. 16 / 377,280, filed April 8, 2019, and published as U.S. Patent Publication US201902314445; U.S. Patent Application Serial No. 15 / 915,237, filed March 8, 2018, and published as U.S. Patent Publication US20180256246A1; and U.S. Patent Application Serial No. 15 / 454,123, filed March 9, 2017, and published as U.S. Patent Publication US20170258539A1; U.S. Patent Application Serial No. 15 / 501,862, filed February 6, 2017, and published as U.S. Patent Publication US20170239005A1; all of which are hereby incorporated by reference as if fully set forth herein.

[0263] In the description and claims of the present disclosure, each of the verbs "comprise", "include" and "have" and their variants is used to indicate that the object of the verb is not necessarily a complete list of components, parts, elements or parts of the subject of the verb. As used herein, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" include plural referents. For example, the term "a marker" or "at least one marker" may include a plurality of markers.

Claims

1. A non-surgical method of operating a surgical system, the surgical system including (i) an articulated robotic arm including a plurality of arm joints, and (ii) a first user input device and a second user input device for controlling the arm, and (iii) a surgical end effector at a distal end of the arm, the method comprises: a. initiating operation of the surgical system in a reverse bending mode, wherein with respect to bending and rotation of the arm joints: (i) the first user input device is activated to direct bending and rotation of only a given arm joint among the arm joints, and wherein while the surgical system is in the reverse bending mode, the first user input device and the second user input device are prevented from generating or transmitting control outputs that would control actuation of arm joints other than the given arm joint among the arm joints; and (ii) deactivating the second user input device; b. while in the reverse bending mode, in response to an electronic control output from the first user input device, reverse bending a distal portion of the articulated robotic arm by bending and rotation of the given arm joint among the arm joints to place the end effector in a reverse bending operating position; c. transitioning the surgical system from the reverse bending mode to an operating mode to enable the second user input device with respect to bending and rotation of at least one of the arm joints other than the given arm joint among the arm joints; and d. while in the operating mode, effecting bending and rotation of at least two of the arm joints in response to an electronic control output from the second user input device according to the respective degrees of freedom of each arm joint, thereby moving the surgical end effector.

2. The method according to claim 1, wherein the surgical system further includes a control circuit that effectively constrains actuation of arm joints other than a single arm joint while the surgical system is in the reverse bending mode.

3. The method according to any one of claims 1 or 2, wherein the transition includes calibrating the input devices with respect to at least one of a position and an orientation of the end effector or the distal portion of the arm.

4. The method according to any one of claims 1 or 2, wherein the first user input device is configured to control actuation of a single arm joint and is not configured to control actuation of arm joints other than the single arm joint.

5. The method according to any one of claims 1 or 2, wherein the transition to the operating mode is responsive to and dependent on detecting that the arm is in a reverse bending position.

6. A surgical system, comprising: a. an articulated robotic arm including a plurality of arm joints and a surgical end effector at a distal end of the arm; and b. a first user input device and a second user input device for controlling the arm, The surgical system is configured to operate asynchronously in the following modes: in (i) a reverse bending mode, wherein a distal portion of the articulated robotic arm is operable to bend in reverse in response to an electronic control output from the first user input device to place the end effector in a reverse bending operating position, and in (ii) an operating mode, wherein at least two of the arm joints are operable to bend and rotate in response to an electronic control output from the second user input device so as to thereby move the surgical end effector such that: A. When in the reverse bending mode, with respect to bending and rotation of the arm joints, the first user input device is activated to direct bending and rotation of only a given arm joint of the arm joints, the surgical system is in the reverse bending mode, preventing the input device array from generating or transmitting a control output that would control actuation of arm joints of the arm other than the given arm joint of the arm joints, and deactivating the second user input device, and B. When in the operating mode, with respect to bending and rotation of at least one of the arm joints of the arm other than the given arm joint of the arm joints, the second user input device is enabled according to the respective degrees of freedom of each arm joint.

7. A non-surgical method of operating a surgical system, the surgical system including (i) an input device array of one or more user input devices, and (ii) an articulated robotic arm including a plurality of arm joints and a surgical end effector at a distal end of the arm, the method comprises: a. Initiating operation of the surgical system in a reverse bending mode, wherein with respect to bending and rotation of the arm joints, the input device array is activated to direct bending and rotation of only a given arm joint of the arm joints; b. When in the reverse bending mode, in response to an electronic control output from the input device array, bending in reverse a distal portion of the articulated robotic arm by bending and rotation of the given arm joint of the arm joints to place the end effector in a reverse bending operating position, and preventing the input device array from generating or transmitting a control output that would control actuation of arm joints of the arm other than the given arm joint of the arm joints; c. Transitioning the surgical system from the reverse bending mode to an operating mode to enable the input device array with respect to bending and rotation of at least one of the arm joints of the arm other than the given arm joint of the arm joints; and d. When in the operating mode, effecting bending and rotation of at least two of the arm joints in response to an electronic control output from the input device array according to the respective degrees of freedom of each arm joint, thereby moving the surgical end effector.

8. The method according to claim 7, wherein the surgical system further includes a control circuit that, when the surgical system is in the reverse bending mode, effectively restricts the actuation of the arm joints other than the given arm joint in the arm joints.

9. The method according to claim 8, wherein the restriction is effected by deactivating the actuation of the arm joints in the arm other than the given arm joint in the arm joints.

10. The method according to any one of claims 7 to 9, wherein the input device array includes a single user input device, and the single user input device controls the actuation of the plurality of arm joints in the reverse bending mode and the operation mode.

11. The method according to any one of claims 7 to 9, wherein the transition to the operation mode is responsive to and depends on detecting that the arm is in the reverse bending position.

12. The method according to any one of claims 7 to 9, wherein the transition includes calibrating the user input device of the input device array with respect to at least one of the position and orientation of the end effector or the distal portion of the arm.

13. The method according to any one of claims 7 to 9, further comprising, after operating in the operation mode: unbending the distal end of the arm to place the arm in an unbended position.

14. A surgical system comprising: a. an input device array of one or more user input devices; and b. an articulated robotic arm including (i) a plurality of arm joints and (ii) a surgical end effector at the distal end of the arm, wherein the surgical system is configured to operate asynchronously in the following modes: in (A) a reverse bending mode, wherein the distal portion of the articulated robotic arm reversely bends in response to an electronic control output from the input device array to place the end effector in a reverse bending operation position, and in (B) an operation mode, wherein at least two of the arm joints bend and rotate in response to an electronic control output from the input device array so as to thereby move the surgical end effector such that: A. when in the reverse bending mode, with respect to the bending and rotation of the arm joints, the input device array is activated to direct the bending and rotation of only the given arm joint in the arm joints and to prevent the input device array from generating or transmitting a control output that would control the actuation of the arm joints in the arm other than the given arm joint in the arm joints, and B. when in the operation mode, the input device array is enabled with respect to the bending and rotation of at least one of the arm joints in the arm other than the given arm joint in the arm joints according to the respective degrees of freedom of each arm joint.

15. A non-surgical method of operating a surgical system, the surgical system comprising (i) an articulated robotic arm including a surgical end effector and a plurality of arm joints at its distal end, and (ii) an array of input devices of one or more user input devices, wherein the arm joints are configured to bend and rotate in response to an electronic control output from one or more of the user input devices of the input device array, the method comprises: a. Starting operation of the surgical system in a first operating mode defined relative to a given single arm joint of the arm joints, wherein the first operating mode prevents actuation of any arm joint of the arm that is not the given single arm joint and allows control of the actuation of the single arm joint to cause bending and rotation of the single arm joint; b. When the surgical system is in the first operating mode, i. Responsive to a control signal generated by one or more of the user input devices of the input device array, reversely bend the distal end of the arm through bending and rotation of the single arm joint to place the end effector in a reverse bending operating position, and ii. Monitor the state of the robotic arm to detect whether the arm is in the reverse bending position, wherein when the surgical system is in the first operating mode, generation or transmission of a control output by the input device array that would control actuation of an arm joint of the arm other than the given single arm joint of the arm joints is prevented; c. Responsive to and depending on detecting that the arm is in the reverse bending position, transition the operation of the surgical system from the first operating mode to a second operating mode in which the system is capable of controlling bending and rotation of at least one arm joint excluding the first mode according to the respective degrees of freedom of each arm joint; and d. Operating the surgical system in the second operating mode.

16. The method of claim 15, wherein the surgical system further comprises a control circuit that effectively constrains actuation of the arm joints other than the single arm joint when the surgical system is in the first operating mode.

17. The method of claim 16, wherein the constraint is effected by deactivating actuation of the arm joints of the arm other than the single arm joint.

18. The method of claim 16, wherein the constraint includes deactivating the ability of a first input device.

19. The method of claim 16, wherein (i) when the surgical system is in the first operating mode, a first input device of the input device array controls actuation of the single arm joint and when the surgical system is in the second operating mode, a second input device controls actuation of the plurality of arm joints, and (ii) the constraint is effected by providing the first input device configured to control actuation of the single arm joint and not configured to control arm joints other than the single arm joint.

20. The method according to any one of claims 15 to 19, wherein said transition comprises calibrating the input device relative to at least one of a position and an orientation of the end effector or a distal portion of the arm.

21. The method according to any one of claims 15 to 19, wherein when the surgical system is in the first operating mode, a first input device of the input device array controls actuation of the single arm joint, and when the surgical system is in the second operating mode, a second input device of the input device array controls actuation of the plurality of arm joints.

22. The method according to claim 21, wherein said transition comprises calibrating the second input device relative to at least one of a position and an orientation of the end effector or a distal portion of the arm.

23. The method according to claim 15, wherein a single user input device controls the actuation of the plurality of arm joints in both the first operating mode and the second operating mode.

24. The method according to any one of claims 15 to 19, wherein the surgical system further comprises a console, the console includes a display screen, and at least one user input device of the input device array is disposed on or near the display screen.

25. The method according to any one of claims 15 to 19, wherein an additional user input device for actuating linear advancement and retraction of the arm is disposed on, at the same position as, or in proximity to at least one user input device of the input device array.

26. The method according to any one of claims 15 to 19, wherein the reverse-bent operating position is at or near a surgical work site.

27. The method according to any one of claims 15 to 19, wherein operating in the second operating mode positions the arm in the reverse-bent position.

28. The method according to any one of claims 15 to 19, further comprising, after operating in the second operating mode: unbending the distal end of the arm to position the arm in an unbent position.

29. A surgical system for use with a surgical end effector and configured to operate asynchronously in a first operating mode and a second operating mode, the system comprising: a. an input device array of one or more user input devices; and b. an articulated robotic arm, the articulated robotic arm includes a surgical end effector at its distal end and a plurality of arm joints configured to bend and rotate in response to control signals generated by one or more input devices of the input device array, wherein: i. The first operation mode is defined with respect to a given single arm joint among the arm joints, wherein the first operation mode prevents actuation of any arm joint in the arm that is not the given single arm joint, and wherein when in the first operation mode, the input device array is prevented from generating or transmitting a control output that would control the actuation of arm joints in the arm other than the given single arm joint in the arm joints and allows control of the actuation of the single arm joint to cause bending and rotation of the single arm joint. ii. The system is configured to, in response to an electronic control output from one or more of the user input devices of the input device array, reverse-bend the distal end of the arm when in the first operation mode by actuating the single arm joint to cause bending and rotation of the single arm joint, so that the surgical end effector is in a reverse-bent operating position. iii. The second operation mode is defined with respect to the plurality of arm joints, wherein the second operation mode is capable of controlling the actuation of at least one arm joint excluding the first mode according to the respective degrees of freedom of each arm joint, and iv. The system is configured to transition from the first operation mode to the second operation mode in response to and depending on detecting that the arm is in a reverse-bent position, and when in the second operation mode, perform an action using the end effector.

30. A non-surgical method of operating a surgical system, the surgical system including (i) an articulated robotic arm including a plurality of arm joints and a surgical end effector at a distal end of the arm, and (ii) an input device array for one or more user input devices for controlling the arm, the method comprises: a. In response to an electronic control output from the input device array, reverse-bend a distal portion of the articulated robotic arm by bending and rotating a given arm joint in the arm joints so that the end effector is in a reverse-bent operating position without bending or rotating any of the arm joints other than the given arm joint in the arm joints, wherein during the reverse-bending, the input device array is prevented from generating or transmitting a control output that would control the actuation of arm joints in the arm other than the given arm joint in the arm joints; and b. In response to and depending on detecting that the arm is in a reverse-bent position, effect bending and rotation of at least two of the arm joints according to the respective degrees of freedom of each arm joint in response to an electronic control output from the input device array, thereby moving the surgical end effector to perform one or more actions.

31. The method according to claim 30, wherein the surgical system further includes a control circuit that effectively constrains the actuation of arm joints other than the given arm joint in the arm joints during the reverse-bending.

32. The method according to any one of claims 30 or 31, further comprising, after performing the one or more surgical maneuvers: uncurving the distal end of the arm to place the arm in an uncurved position.

33. A surgical system comprising: a. An articulated robotic arm, the articulated robotic arm including (i) a plurality of arm joints and (ii) a surgical end effector at the distal end of the arm, and b. An array of input devices for controlling one or more user input devices of the arm, wherein the surgical system is configured to: i. In response to an electronic control output from the array of input devices, reverse-curve a distal portion of the articulated robotic arm by bending and rotating a given arm joint among the arm joints to place the end effector in a reverse-curved operating position, without bending or rotating any of the arm joints other than the given arm joint among the arm joints, wherein during the reverse-curving, generation or transmission of a control output by the array of input devices that would control the actuation of an arm joint other than the given arm joint among the arm joints of the arm is prevented, and ii. In response to and depending on detecting that the arm is in the reverse-curved position, and in response to an electronic control output from the array of input devices, effect bending and rotation of at least two of the arm joints according to the respective degrees of freedom of each arm joint, thereby moving the surgical end effector to perform one or more actions.

Images