Orientation of a user input device for controlling a surgical arm

Abstract

A surgical system comprises: an articulated robotic arm comprising arm segments connected in series by arm joints that bend and rotate; and first and second input devices. The second input device comprises a handle that can be configured to be oriented in any orientation in x-y-z space, and the handle comprises segment components and joint components corresponding to the arm segments and arm joints of the arm. The arm joints are actuatable by the handle joint components and have the same degrees of freedom as the handle joint components. A method of using the surgical system comprises: bending the arm in reverse; transferring control of the arm from the first input device to the second input device; and performing a surgical action during which the handle component is converted through a displacement vector or redirection arc in the x-y-z space to a corresponding displacement vector or corresponding redirection arc of the end effector in the same x-y-z space.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 944,351, filed December 5, 2019, which is incorporated herein by reference in its entirety. Technical Field

[0003] The present invention relates to surgical systems for performing surgery and methods of using such systems, and in particular to controlling the bending and rotation of portions of an articulated robotic arm using multiple operating modes and input devices. Background Art

[0004] This section is intended to introduce the reader to various aspects of the art that may be related to various aspects of the present invention, which are described and / or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present invention. Therefore, it should be understood that these statements are to be read in this light, and not as admissions of prior art.

[0005] It is well known that minimally invasive surgery has many benefits. Instruments used for such surgical procedures typically have a surgical end effector located at the distal end of an articulated surgical arm (preferably with a minimal diameter), which is inserted through a small opening (e.g., a body wall incision, a natural orifice) to reach the surgical site. In some cases, surgical instruments can be passed through a cannula, and an endoscope can be used to provide images of the surgical site.

[0006] To achieve convenience, accuracy, and patient health, surgical instruments utilizing end effectors (e.g., surgical tools for tissue fusion or cutting, or measuring tools) have been developed. In some cases, an articulated surgical arm has one or more flexures that are remotely controlled using various input devices (e.g., hand and foot controls) to ultimately control the position of the end effector and change its orientation with respect to the longitudinal axis of the surgical arm. In some cases, the surgical arm is capable of bending in opposite directions relative to the longitudinal axis of the surgical arm.

[0007] Clinical studies have shown that minimally invasive vaginal gynecologic surgery is superior to abdominal surgery. Advantages include postoperative recovery time, morbidity, infection, mortality, complications, blood loss, and patient satisfaction. Today, the American College of Gynecologists (ACOG) policy states that vaginal gynecologic surgery is preferred whenever feasible. To enable transvaginal access for gynecologic surgery, the articulated surgical arm must bend into a reverse flexion position.

[0008] The current state of the art lacks devices and methods that can provide optimal control elements and methods of use to ensure that the initial reverse bending step is performed in a risk-mitigated manner that does not compromise the ergonomic comfort and convenience provided by an input device optimized for demanding surgical procedures. Therefore, there is a need for a solution that is suitable for limiting risk during reverse bending of a surgical arm and providing maximum freedom of movement during the surgical procedure without compromising the surgeon's ergonomics. Summary of the Invention

[0009] A method of operating a surgical system is disclosed, the surgical system comprising (i) an articulated robotic arm having a surgical end effector at a distal end thereof, the arm comprising a plurality of arm segments connected in series by arm joints, the arm joints having respective degrees of freedom and configured to bend and rotate, and (ii) first and second input devices, the second input device comprising a handle component configured to be oriented in any one of a plurality of selectable orientations in xyz space, the handle component comprising respective segment components and joint components corresponding to the arm segments and the arm joints, each of the respective arm joints being actuatable by the corresponding joint component and having the same degrees of freedom as the corresponding joint component, the method comprising: (a ) in response to an electronic control output from a first input device, the electronic control output being effective to regulate the bending and rotation of the arm joints to transfer the end effector to the reverse bending operative position; (b) after reverse bending the arm, transferring control of the arm from the first input device to the second input device; and (c) after the transfer, performing a surgical action with the end effector by reverse bending the handle component to shift and redirect the segment components and the joint component respectively, thereby causing corresponding shifting and redirecting of the corresponding arm segments and arm joints of the reverse bending arm, the orientation of the handle component being so as to convert the displacement vector or redirection arc of the handle component through the xyz space into the corresponding displacement vector or corresponding redirection arc of the end effector in the same xyz space.

[0010] In some embodiments, performing the reverse bending step using the first input device may not include, i.e., may exclude, reorienting the handle member from the selected orientation in xyz space. In some embodiments, performing the reverse bending step using the first input device may not require reorienting the handle member from the selected orientation in xyz space. In some embodiments, performing the reverse bending step using the first input device may not require reorienting the handle member more than 90° from the selected orientation in xyz space.

[0011] In some embodiments, the surgical system may further include control circuitry effective to cause the transition.

[0012] In some embodiments, the first input device may be deactivated after the transition. In some embodiments, the first input device may be disconnected from the arm after the transition.

[0013] In some embodiments, the surgical system may include or additionally include a user input device for linear advancement and retraction of the actuating arm. This user input device may be an additional device on another user input device, such as a button or a nipple-type piece.

[0014] According to an embodiment of the present invention, a surgical system for use with a surgical end effector includes: (a) an articulated robotic arm having a surgical end effector at its distal end, the arm including a plurality of arm segments connected in series by arm joints, the arm joints having corresponding degrees of freedom and configured to bend and rotate, and (b) a first input device configured to transmit electronic control outputs in response to changes in position for controlling corresponding bending and rotation rates of the arm joints to reverse bend the distal end of the arm and the end effector to a reverse bending operating position; and (c) a second input device including a handle component configured to be in any one of a plurality of selectable orientations in xyz space. In an upper orientation, the handle component includes corresponding segment components and joint components corresponding to the arm segments and arm joints of the arm, respectively, each of the corresponding arm joints can be actuatable by the corresponding joint component and has the same degrees of freedom as the corresponding joint component, wherein: (i) the surgical system is configured to transition from controlling the arm with a first input device to controlling the arm with a second input device after reverse bending the arm; and (ii) after the transition, the reverse bending of the handle component to respectively shift and redirect the segment components and the joint components effectively causes corresponding shifting and redirecting of the corresponding arm segments and arm joints of the reverse bending arm, and the orientation of the handle component is so as to convert the displacement vector or redirection arc of the handle component through the xyz space into a corresponding displacement vector or corresponding redirection arc of the end effector in the same xyz space.

[0015] In some embodiments, performing the reverse bend using the first input device may not include, i.e., may exclude, reorienting the handle member from the selected orientation in xyz space. In some embodiments, performing the reverse bend step using the first input device may not require reorienting the handle member from the selected orientation in xyz space. In some embodiments, performing the reverse bend step using the first input device may not require reorienting the handle member more than 90° from the selected orientation in xyz space.

[0016] In some embodiments, the surgical system may further include control circuitry effective to cause the transition.

[0017] In some embodiments, the first input device may be configured to be deactivated after the transition. In some embodiments, the first input device may be configured to be disconnected from the arm after the transition.

[0018] In some embodiments, the surgical system may include a user input device for actuating linear advancement and retraction of the arm.

[0019] According to an embodiment, a method for operating a surgical system is disclosed, wherein the surgical system includes (i) an articulated robotic arm having a surgical end effector at its distal end, the arm including a plurality of arm joints with corresponding degrees of freedom, and (ii) an input device array of one or more user input devices, the input devices being configured to control the bending and rotation of the arm joints, the method comprising: (a) in response to an electronic control output from a first user input device, reversely bending the distal end of the arm to transfer the end effector to a reverse bending operating position, using a first coordinate transformation matrix to convert the user input into the corresponding bending and rotation of the arm joints; (b) in response to and depending on detecting that the end effector is in the reverse bending operating position, transitioning to a second coordinate transformation matrix based on the reverse bending position of the end effector; and (c) after the transition and in response to the electronic control output from the second user input device, performing a surgical activity using the end effector, using a second coordinate transformation matrix to convert the user input into the corresponding bending and rotation of the arm joints.

[0020] In some embodiments, the first and second input devices may be the same input device, and / or the first and second coordinate transformation matrices may not be the same 3D coordinate transformation matrix.

[0021] In some embodiments, the first and second input devices may not be the same input device, and / or the first and second coordinate transformation matrices may not be the same 3D coordinate transformation matrix.

[0022] In some embodiments, prior to the transition and during the reverse bend, a proximal displacement of a first user input device, or a portion thereof, can be converted into a proximal displacement of the end effector, and / or after the transition and with the end effector in the reverse bend position, a proximal displacement of a second user input device, or a portion thereof, can be converted into a distal displacement of the end effector.

[0023] In some embodiments, the surgical system may further include control circuitry effective to cause the transition.

[0024] In some embodiments, the first input device may be deactivated after the transition. In some embodiments, the first input device may be disconnected from the arm after the transition.

[0025] According to an embodiment, a surgical system for use with a surgical end effector includes: (a) an array of one or more input devices; and (b) an articulated robotic arm having a surgical end effector at its distal end, the arm including multiple arm segments connected by multiple arm joints, the arm joints being configured to bend and rotate in response to control signals generated by the input devices, wherein the surgical system is configured to: (i) in response to an electronic control output from a first user input device, reverse bend the distal end of the arm to transfer the end effector to a reverse bend operating position, employing a first coordinate transformation matrix to convert the user input into corresponding bending and rotation of the arm joints, (ii) in response to and depending on detecting that the current orientation of the end effector corresponds to the current orientation of a second user input device, transition to a second coordinate transformation matrix based on the current orientation of the end effector, and (iii) after the transition and in response to the electronic control output from the second user input device, perform a surgical activity using the end effector, employing a second coordinate transformation matrix to convert the user input into corresponding bending and rotation of the arm joints.

[0026] In some embodiments, the first and second input devices may be the same input device, and / or the first and second coordinate transformation matrices may not be the same 3D coordinate transformation matrix.

[0027] In some embodiments, the first and second input devices may not be the same input device, and / or the first and second coordinate transformation matrices may not be the same 3D coordinate transformation matrix.

[0028] In some embodiments, the surgical system can be configured such that, prior to transition and during reverse bending, a proximal displacement of a first user input device, or a portion thereof, is converted into a proximal displacement of the end effector, and / or, after transition and with the end effector in a reverse bending position, a proximal displacement of a second user input device, or a portion thereof, is converted into a distal displacement of the end effector.

[0029] In some embodiments, the surgical system may further include control circuitry effective to cause the transition.

[0030] In some embodiments, the first input device may be configured to be deactivated after the transition. In some embodiments, the first input device may be configured to be disconnected from the arm after the transition.

[0031] According to an embodiment, a method of operating a surgical system is disclosed, the surgical system comprising (i) a given user input device and (ii) an articulated robotic arm comprising a surgical end effector at its distal end and a plurality of arm joints, the arm joints being configured to bend and rotate in response to electronic control output from the given user input device. The method comprises: (a) commencing operation of the surgical system in a first operating mode; (b) while the surgical system is in the first operating mode, (i) modifying a curvilinear shape of the arm by at least one of mechanical bending and mechanical rotation of one or more arm joints of the articulated robotic arm, and (ii) monitoring a shape state of the robotic arm to detect whether at least a portion thereof has a curvilinear shape that matches a current primary curvilinear shape defined by the given user input device. The method further comprises: (c) in response to and dependent on detecting that the curvilinear shape of the robotic arm matches the curvilinear shape of the given user input device, transitioning operation of the surgical system from the first mode to a second mode; and (d) operating the surgical system in the second mode such that the output of the given user input device modifies the configuration of the arm or a segment or element thereof to perform a surgical action using the end effector.

[0032] In some embodiments, the transition may include handing over user control of the configuration of the arm from a user input device that is not the given user input device to the given user input device.

[0033] In some embodiments, the first mode can: (i) be defined with respect to a proper subset of multiple arm joints; (ii) prevent actuation of any arm joint in the arm that is not a member of the proper subset of arm joints via a control signal from a given user input device; and / or (iii) allow control of actuation of one or more arm joints belonging to the proper subset to cause bending and / or rotation of one or more arm joints of the proper subset.

[0034] In some embodiments, when transitioning from a first mode to a second mode, a given user input device may be enabled to control flexion and rotation of at least one arm joint exclusive of the first mode.

[0035] In some embodiments, the modification of the arm shape can be performed in response to an electronic control signal provided by a given user input device. In some embodiments, the modification of the arm shape can be performed in response to an electronic control signal provided by a user input device other than the given user input device. In some embodiments, the modification of the curvilinear shape of the arm can be performed automatically.

[0036] According to an embodiment, a surgical system includes (i) a given user input device and (ii) an articulated robotic arm, the articulated robotic arm including a surgical end effector at its distal end and a plurality of arm joints, the arm joints being configured to bend and rotate in response to electronic control outputs from the given user input device. The system is configured to begin operation in a first operating mode and, during operation in the first operating mode, modify a curvilinear shape of the arm by at least one of mechanical bending and mechanical rotation of one or more arm joints of the articulated robotic arm. The system is further configured to monitor a shape state of the robotic arm to detect whether at least a portion thereof has a curvilinear shape that matches a current primary curvilinear shape defined by the given user input device. The system is further configured to transition operation of the surgical system from the first mode to a second mode in response to and depending on detecting that the curvilinear shape of the robotic arm matches the curvilinear shape of the given user input device, and the system is further configured to operate in the second mode such that the output of the given user input device modifies the configuration of the arm or a segment or element thereof to perform a surgical action using the end effector.

[0037] In some embodiments, the system may be configured such that the transition may include handing over user control of the configuration of the arm from a user input device that is not the given user input device to the given user input device.

[0038] In some embodiments, the system can be configured such that the first mode: (i) is defined with respect to a proper subset of multiple arm joints; (ii) prevents actuation of any arm joint in the arm that is not a member of the proper subset of arm joints via a control signal from a given user input device; and / or (iii) allows control of actuation of one or more arm joints belonging to the proper subset to cause bending and / or rotation of one or more arm joints of the proper subset.

[0039] In some embodiments, the system can be configured such that, when transitioning from a first mode to a second mode, a given user input device is enabled to control flexion and rotation of at least one arm joint excluding the first mode.

[0040] In some embodiments, the system can be configured such that the modification of the arm shape can be performed in response to an electronic control signal provided by a given user input device. In some embodiments, the system can be configured such that the modification of the arm shape can be performed in response to an electronic control signal provided by a user input device other than the given user input device. In some embodiments, the system can be configured such that the modification of the curvilinear shape of the arm can be performed automatically.

[0041] According to an embodiment, a method of operating a surgical system is disclosed, the surgical system comprising (i) a given user input device and (ii) an articulated robotic arm comprising a surgical end effector at its distal end and a plurality of arm joints, the arm joints being configured to bend and rotate in response to an electronic control output from the given user input device. The method comprises: (a) commencing operation of the surgical system in a first operating mode; (b) while the surgical system is in the first operating mode, (i) modifying a curvilinear shape of the arm by at least one of mechanical bending and mechanical rotation of one or more arm joints of the articulated robotic arm, and (ii) monitoring a shape state of the robotic arm to detect whether at least a portion thereof has a curvilinear shape that matches a predefined curvilinear shape. The method further comprises: (c) in response to and depending on detecting that the curvilinear shape of the robotic arm matches the predefined curvilinear shape, transitioning operation of the surgical system from the first mode to a second mode; and (d) operating the surgical system in the second mode such that the output of the given user input device modifies the configuration of the arm or a segment or element thereof to perform a surgical action using the end effector.

[0042] In some embodiments, detecting whether the curved shape of the robotic arm or a portion thereof matches a predefined curved shape may include at least one of: (i) detecting whether the curved shape of the robotic arm or a portion thereof matches a two-dimensional projection of the predefined curved shape, (ii) detecting whether the two-dimensional projection of the curved shape of the robotic arm or a portion thereof matches the predefined curved shape, and / or (iii) detecting whether the two-dimensional projection of the curved shape of the robotic arm or a portion thereof matches the two-dimensional projection of the predefined curved shape.

[0043] In some embodiments, the predefined curve shape may have one or more local minima or local maxima. In some embodiments, the predefined curve shape may have one or more inflection points. In some embodiments, the predefined curve shape or its two-dimensional projection may be an 'S' curve shape.

[0044] In some embodiments, the transition may include handing over user control of the configuration of the arm from a user input device that is not the given user input device to the given user input device.

[0045] In some embodiments, the first mode can: (i) be defined with respect to a proper subset of multiple arm joints; (ii) prevent actuation of any arm joint in the arm that is not a member of the proper subset of arm joints via a control signal from a given user input device; and / or (iii) allow control of actuation of one or more arm joints belonging to the proper subset to cause bending and / or rotation of one or more arm joints of the proper subset.

[0046] In some embodiments, when transitioning from a first mode to a second mode, a given user input device may be enabled to control flexion and rotation of at least one arm joint exclusive of the first mode.

[0047] In some embodiments, the modification of the arm shape can be performed in response to an electronic control signal provided by a given user input device. In some embodiments, the modification of the arm shape can be performed in response to an electronic control signal provided by a user input device other than the given user input device. In some embodiments, the modification of the curvilinear shape of the arm is performed automatically.

[0048] According to an embodiment, a surgical system includes (i) a given user input device and (ii) an articulated robotic arm, the articulated robotic arm including a surgical end effector at its distal end and a plurality of arm joints, the arm joints being configured to bend and rotate in response to an electronic control output from the given user input device. The system is configured to begin operation in a first operating mode, and when in the first operating mode, the curvilinear shape of the arm is modified by at least one of mechanical bending and mechanical rotation of one or more arm joints of the articulated robotic arm, and the shape state of the robotic arm is monitored to detect whether at least a portion thereof has a curvilinear shape that matches a predefined curvilinear shape. The system is also included such that, in response to and depending on detecting that the curvilinear shape of the robotic arm matches the predefined curvilinear shape, the operation of the surgical system is transitioned from the first mode to the second mode; and (d) the surgical system is configured to operate in the second mode such that the output of the given user input device modifies the configuration of the arm or a segment or element thereof to perform a surgical action using the end effector.

[0049] In some embodiments, the system can be configured such that detecting whether the curved shape of the robotic arm or a part thereof matches a predefined curved shape includes at least one of: (i) detecting whether the curved shape of the robotic arm or a part thereof matches a two-dimensional projection of the predefined curved shape, (ii) detecting whether the two-dimensional projection of the curved shape of the robotic arm or a part thereof matches the predefined curved shape, and / or (iii) detecting whether the two-dimensional projection of the curved shape of the robotic arm or a part thereof matches the two-dimensional projection of the predefined curved shape.

[0050] In some embodiments, the system can be configured such that the predefined curve shape can have one or more local minima or local maxima. In some embodiments, the system can be configured such that the predefined curve shape can have one or more inflection points. In some embodiments, the system can be configured such that the predefined curve shape or its two-dimensional projection can be an 'S' curve shape.

[0051] In some embodiments, the system may be configured such that the transition may include handing over user control of the configuration of the arm from a user input device that is not the given user input device to the given user input device.

[0052] In some embodiments, the system can be configured such that the first mode: (i) is defined with respect to a proper subset of multiple arm joints; (ii) prevents actuation of any arm joint in the arm that is not a member of the proper subset of arm joints via a control signal from a given user input device; and / or (iii) allows control of actuation of one or more arm joints belonging to the proper subset to cause bending and / or rotation of one or more arm joints of the proper subset.

[0053] In some embodiments, the system can be configured such that, when transitioning from a first mode to a second mode, a given user input device can be enabled to control flexion and rotation of at least one arm joint excluding the first mode.

[0054] In some embodiments, the system can be configured such that the modification of the arm shape can be performed in response to an electronic control signal provided by a given user input device. In some embodiments, the system can be configured such that the modification of the arm shape can be performed in response to an electronic control signal provided by a user input device other than the given user input device. In some embodiments, the system can be configured such that the modification of the curvilinear shape of the arm is performed automatically. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The present invention will now be further described by way of example with reference to the accompanying drawings, in which the dimensions of components and features shown are chosen for convenience and clarity of presentation and are not necessarily drawn to scale. In the drawings:

[0056] Figure 1 is a simplified pictorial illustration of a surgical system according to an embodiment of the present invention.

[0057] Figure 2A is a schematic perspective view of a surgical system including a robotic surgical arm, according to an embodiment of the present invention.

[0058] Figure 2B A distal portion of a robotic surgical arm is shown, according to an embodiment of the present invention.

[0059] Figure 3A -C shows the distal portion of the robotic surgical arm in various bent and reverse bent positions, according to an embodiment of the present invention.

[0060] Figure 4 and Figure 5 The distal portion of a robotic surgical arm is shown with various arm curve shapes, according to embodiments of the present invention.

[0061] Figure 6 A flow chart illustrating a method for operating a surgical robotic arm using two operating modes according to an embodiment of the present invention is shown.

[0062] Figure 7 is a schematic diagram of a console for a surgical system with an input device positioned adjacent thereto, according to an embodiment of the present invention.

[0063] Figure 8 is a schematic diagram of a user input device according to an embodiment of the present invention.

[0064] Figure 9 is a diagram illustrating an exemplary scheme for controlling bending and rotation of an arm joint using an input device according to an embodiment of the present invention.

[0065] Figure 10 is a schematic diagram of an articulated user input device according to an embodiment of the present invention.

[0066] Figure 11 According to an embodiment of the present invention Figure 10 Schematic diagram of a handle component of a user input device.

[0067] Figure 12A -B shows an exemplary graphical aid for aligning the respective positions of a robotic surgical arm and an articulated user input device, according to an embodiment of the present invention.

[0068] Figure 13 Example screen displays are shown for aligning the respective positions of a robotic surgical arm and an articulated user input device, according to an embodiment of the present invention.

[0069] Figure 14A -E shows a time-series image illustrating exemplary control of a surgical robot arm using multiple different input devices, according to an embodiment of the present invention.

[0070] Figure 15 A block diagram of a surgical system according to an embodiment of the present invention is shown.

[0071] Figure 16 A flow chart illustrating a method for operating a surgical system in two different operating modes, according to an embodiment of the present invention.

[0072] Figure 17 is a schematic diagram of a user input device oriented in a first given xyz space according to an embodiment of the present invention.

[0073] Figure 18 According to an embodiment of the present invention Figure 17 A schematic diagram of a portion of a surgical arm oriented in a first given xyz space.

[0074] Figure 19 is a schematic diagram of a user input device having a handle member according to an embodiment of the present invention, wherein the handle member is repositioned to be oriented in a direction other than Figure 17 The first given xyz space is oriented in xyz space.

[0075] Figure 20 According to an embodiment of the present invention Figure 17 First given xyz space reverse bending Figure 18 Schematic diagram of the surgical arm.

[0076] Figure 21 According to an embodiment of the present invention Figure 17 - given an orientation in xyz space Figure 19 Schematic diagram of a user input device.

[0077] Figure 22A A user input device having a given curved shape according to an embodiment of the present invention is schematically shown.

[0078] Figure 22B A surgical arm is schematically shown for the first time, having a Figure 22A The given curve shape does not match the curve shape.

[0079] Figure 22C A second schematic diagram shows a surgical arm having a Figure 22A The curve shape that matches the given curve shape.

[0080] Figure 23A A flow chart illustrating a method for dual control of a surgical arm according to an embodiment of the present invention is shown.

[0081] Figure 23B and Figure 23C is a schematic diagram of a console including dual control members according to an embodiment of the present invention.

[0082] Figure 24 is a flow chart of a method of controlling one or more surgical arms using a haptic handle, according to an embodiment of the present invention. DETAILED DESCRIPTION

[0083] The present invention has been described herein with reference to the accompanying drawings, by way of example only. With specific reference now to the accompanying drawings in detail, it is emphasized that the details shown are by way of example only and for illustrative discussion of preferred embodiments of the invention, and are presented in order to provide what is believed to be the most useful and understandable description of the principles and conceptual aspects of the invention. In this regard, no attempt is made to illustrate the structural details of the invention in more detail than is necessary for a basic understanding of the invention, and the description with the accompanying drawings makes it clear to those skilled in the art how several forms of the invention may be embodied in practice. Throughout the drawings, the same reference symbols are generally used to represent the same elements.

[0084]

[0014] Embodiments disclosed herein relate to controlling one or more surgical robotic arms, ie, articulated robotic arms, using a variety of different operating modes and / or a variety of different input devices.

[0085] Whenever 'arm' is used herein or in the appended claims, it refers to an articulated robotic arm that is part of a surgical system or electrosurgical system and is used to perform or assist in performing surgical (including electrosurgical) actions within the body of a human subject. Where not specified, 'surgical actions' may include any medical or procedure-related or diagnostic action taken within the human body, including, but not exhaustive: cutting tissue, dissecting tissue, manipulating tissue, suturing tissue, contracting tissue, fusing tissue, taking measurements, and imaging. It may be desirable that the size and / or shape of the surgical arm be designed to be suitable for insertion into the human body. For example, the size and / or shape of the arm may be designed to be inserted through a laparoscopic port and / or used to perform laparoscopic surgery. For example, the size and / or shape of the arm may be designed to be inserted through a natural body orifice, such as the vagina, anus, trachea, esophagus, ear canal.

[0086] The arm may include an end effector, which is used herein to refer to a tool or device used in conjunction with surgery, electrosurgery, diagnosis, or imaging when deployed in the human body. The end effector may be provided as part of the arm, i.e., already mounted, mechanically connected, and / or integrated with the power and communication transmission means of the arm; in some embodiments, the arm and end effector may be provided separately for assembly and / or integration into a work unit prior to or even during surgery (i.e., prior to insertion into the body of a subject). In any case, terms such as 'an arm including an end effector' and 'and an arm configured for use with the end effector' should be understood to be equivalent for the purposes of this disclosure and the appended claims.

[0087] As used herein, an 'input device' or equivalently a 'user input device' may be any device capable of receiving user input, i.e., input received from a user of the surgical system. Input devices may include, for example, but not exclusively: buttons, switches, toggle switches, wheels, knobs, small levers such as thumbsticks (or nipple-type pieces), and joysticks (whether or not articulated). Disclosure of a particular device type for any particular input device is not intended to exclude substitution of the particular input device with other types of input devices. Input devices may be stand-alone, grouped on a single control component or a small number of control components, provided on another input device, or co-located, such as on or near a console, display screen, etc. A user may operate an input device using one or more fingers, thumbs, hands, or feet. Additionally or alternatively, but not limited to, an input device may be eye or hand motion operated, voice operated, or controlled by facial expressions.

[0088] The arm and input device, as well as other aspects and features of the present invention, may be understood in conjunction with any teachings of co-pending U.S. patent application serial number 16 / 121,704, filed on September 5, 2018, and published as U.S. Patent Publication US20190000574A1, which is hereby incorporated by reference in its entirety.

[0089] 'Handle' or equivalently 'handle component' is generally used herein to describe a manually operated user input device or a manually operated portion of a user input device, and in some embodiments, to describe a hand-gripped or finger-gripped user input device or a portion thereof. The drawings and accompanying descriptions of handles and manually operated user input devices in this disclosure are provided as illustrative examples, and these drawings and accompanying descriptions should not be construed as limiting the scope, connectivity, and functionality of the embodiments related to the handle / handle component design.

[0090] For the purposes of the present disclosure, a 'module' and / or 'circuit' or 'electronic circuit' and / or 'control circuit' and / or element and / or unit and / or controller and / or module and / or sensor and / or detector may include any combination of analog and / or digital circuits and / or software / computer readable code modules and / or firmware and / or hardware elements, including but not limited to digital computers, CPUs, volatile or non-volatile memories, field programmable logic array (FPGA) elements, hardwired logic elements, field programmable gate array (FPGA) elements, and application specific integrated circuit (ASIC) elements. Any instruction set architecture may be used, including but not limited to a reduced instruction set computer (RISC) architecture and / or a complex instruction set computer (CISC) architecture.

[0091] In various embodiments, any computational or analytical process may be performed using any combination of analog and / or digital circuits and / or software / computer readable code modules and / or firmware and / or hardware elements, including but not limited to digital computers, CPUs, volatile or non-volatile memory, field programmable logic array (FPGA) elements, hardwired logic elements, field programmable gate array (FPGA) elements, and application specific integrated circuit (ASIC) elements. Any instruction set architecture may be used, including but not limited to a reduced instruction set computer (RISC) architecture and / or a complex instruction set computer (CISC) architecture.

[0092] Referring now to the accompanying drawings, Figure 1 A schematic diagram of a surgical system 100 is shown, according to an embodiment. Figure 1 The system 100 includes two surgical robotic arms 102. In other examples of surgical systems, a single surgical arm is provided. In other examples, more than two (e.g., 3 or 4) surgical arms are provided. The size and / or shape of the surgical robotic arms 102 are preferably designed to be inserted into a human body or patient 106. Each of the surgical robotic arms 102 is actuated by a corresponding motor unit 108. In this simplified illustrative example, the surgical arms 102 and / or motor units 108 are supported by a patient support 116 attached to, for example, a bed, but may also be supported by a patient side cart or any other suitable device.

[0093] In embodiments where the surgical system is used for electrosurgery, power can be supplied to the arm 102 and the motor unit 108 by a high-frequency electrosurgery unit 112. As is known in the art of electrosurgery, a high-frequency electrosurgery unit supplies a high-frequency (e.g., radiofrequency) alternating polarity current. The high-frequency electrosurgery unit 112 can be configured to supply different frequencies and / or power levels suitable for, for example, cutting and / or coagulating and / or sealing and / or drying and / or cauterizing tissue. Power is supplied to the motor unit 108 via one or more cables 114 configured to deliver radiofrequency electrosurgery power.

[0094] Movement of the surgical arm 102 is controlled by a console 118. Movement is in response to signals generated by one or more input devices. The console 118 includes a plurality of user interfaces, including one or more of the following: an input device, such as an input device arm 120, wherein the console is configured to generate control signals based on movement of the input device arm 120; a display screen 128 configured to receive user input and / or display, for example, system status information or imaging of the surgical area, such as to display images collected by a camera inserted into the patient 106 using one of the surgical arms 102, or to display arm position and orientation; and one or more additional user interfaces 130 (e.g., buttons, switches, etc.).

[0095] The console 118 includes a processor (not shown) configured to receive signals from user input and send control signals to the motor unit 108 and / or the high-frequency electrosurgical unit 112. The foot pedal 126 and / or the high-frequency electrosurgical unit 112 include a processor (not shown) configured to receive a control signal (e.g., generated by a user depressing a portion of the foot pedal 126) to vary the electrical power supplied to the motor unit 108 based on the control signal. The foot pedal control signal does not necessarily pass through the control unit processor.

[0096] As will be explained in more detail below, in one control mode, movement of the input device arm 120 controls movement of the corresponding surgical device arm 102. The user 124 can position and / or move the input arm 120 by grasping the input device arm handle 127. The input arm is one form of input device and is shown here for illustrative purposes. In other embodiments, other types or forms of input devices may be used.

[0097] Now refer to Figure 2A -B, the arm unit 104 includes a proximal end shaped to be received by the motor unit 108 and a distal end where an end effector 174, such as a multi-jaw grasper as shown (shown as a non-limiting illustrative example only), is attached to the arm 102.

[0098] The relative terms 'proximal' and 'distal' are used in relation to Figure 2A 108 and is used in this manner throughout this disclosure and the appended claims: As shown, the distal end of the arm 102, where the end effector 174 is located, is the end farthest from the motor unit 108 and is the first portion of the arm to be inserted into the human subject 19. Thus, the proximal end is the end opposite the distal end that is closest to the motor unit 108. As used herein, the term 'distal portion' refers to any portion of the arm 102 that includes the distal end / tip (optionally including the end effector 174) and is less than half the length of the arm 102. Figure 2B In the embodiment of the present invention, the flexible portion 170 of the arm 102, i.e., the portion containing one or more flexible joints, is located closer to the distal end along the length of the arm. The flexible portion 170 may include a series of 'stacked links' 199 that achieve flexibility in the outer contour / surface of the arm 102; Figure 2B An example of a plurality of stacked links 199 in the bendable portion 170 of the arm 102 is shown.

[0099] As used herein, 'operating mode' or its equivalent 'mode' (which may be used in conjunction with various non-limiting descriptive terms, e.g., 'reverse bending mode', 'surgical operating mode', etc.) refers to an operating regime imposed on the use of a surgical system or arm by hardware, firmware, or software design, or by the control circuitry of the surgical system, or otherwise. To clarify: the word 'operating' in 'operating mode' means 'work' or 'function', and describes, for example, the operation of an arm, and does not imply the performance of a procedure, i.e., 'operating mode' may happen to include (but not necessarily) the performance of a procedure. The operating regime imposed may include, but is not exhaustive, constraining or unconstraining certain actions or portions of the surgical system to perform one or more actions, and constraining or unconstraining may include and / or may be equivalent to enabling or disabling, locking or unlocking, and excluding or allowing or similar words and phrases. In some embodiments herein, a mode is dedicated to or directed to achieving a single goal, such as reversing bending of the distal portion of the arm, and may be limited to one or more specific time periods. In other embodiments, a mode may encompass an unlimited number of goals and actions and unlimited or undefined time periods.

[0100] As further described below, the operation of the surgical system can be differentiated between different modes in various ways. For example, the distinctions can be based on (but not exhaustively): having a different input device (or multiple input devices) dedicated to each mode; having various constraints or restrictions on specific arm movements and / or specific arm joints and segments; having different conversion schemes for the input device or the control element of the input device to the displacement of the arm movement, for example, displacement to velocity versus displacement to displacement, regardless of whether the conversion from user input to arm movement is robotic / semi-autonomous or teleoperated; and whether the operation of the input device solves (through the mechanical and electronic equipment of the surgical system) the bending and rotation of the arm joint directly or indirectly by solving the displacement of the arm segment, which in turn causes the necessary bending and rotation to occur to shift the arm segment as instructed. These distinctions can be used in combination, and they can vary from arm to arm in a multi-arm system. In some embodiments, the distinction can change when the system is in a given mode.

[0101] Some distinctions between modes can be achieved in more than one way. As a non-limiting example, if the distinction between modes involves constraining or restricting certain arm movements (e.g., bending and rotation of certain joints) or allowing only certain arm movements, the distinction can be achieved by using a different input device for each mode, or by using one input device in both modes but enforcing software or hardware constraints on the one input device that can be switched by the user or enforced by the system. Regardless of whether the implementation requires one or more input devices, the software-implemented or hardware-implemented solution can actively allow actuation of a single given arm joint, or can actively block or prevent actuation of any arm joint (of the same arm) that is not a single arm joint.

[0102] It should be noted that wherever actuation or movement of a singular 'arm' (as opposed to plural 'arms') is discussed herein, this is for convenience only and is not intended to indicate whether a second arm (or other multiple arms) is actuated or moved similarly or otherwise. Each arm can be controlled and actuated independently of any other arm via a corresponding input device. On the other hand, when an 'arm' (singular) is disclosed as being constrained or limited in actuation or movement, such as when in an operating mode characterized by such a constraint or constraint, the constraint or constraint may apply equally to both / all arms of the surgical system. However, in some embodiments, a constraint or constraint may apply to one or more given arms, while other arms are not constrained or limited at all, or are not constrained or limited in the same manner.

[0103] In embodiments employing two distinct modes of operation, a first mode of operation is typically used when introducing, withdrawing, and / or navigating one or more surgical arms into or from a body (or from a first point to a second point within the body), and more specifically, introducing, withdrawing, and / or navigating one or more surgical arms toward or away from a target surgical site. The second mode of operation is typically used when performing surgical actions (e.g., dissecting tissue, manipulating tissue, suturing tissue, taking measurements, imaging, etc.). The first mode of 'navigation' can include bending one or more arms backward to place at least a portion of the arm or a distal portion thereof in a backward bent position, or equivalently, placing the end effector in a backward bent operational position (working position).

[0104] A well-defined transition or 'switch' from one mode of operation to another may be required. In some embodiments, the transition comprises switching from one input device (or multiple input devices) to another. In other embodiments, the transition involves simply changing the control aspect that distinguishes between one mode and another, where the tasks assigned to both modes can be accomplished by a single input device or by the same input device. The transition can be initiated and managed by the surgical system, or can be initiated by the user.

[0105] In a non-limiting example of transitioning from a first mode to a second mode, the transition can include ending constraints or restrictions imposed during operation in the first mode, such as limiting joint flexion and rotation to a given single arm joint, such as the elbow joint, of any particular arm. Additionally or alternatively, the transition can include enabling actuation of arm joints that were disabled (or not specifically enabled) during operation in the first mode, i.e., other than a single given joint enabled in the first mode. Enabling can include enabling joints to move unconstrained according to each joint's respective degrees of freedom. For example, if a given arm joint is configured only for rotation and not flexion, enabling it for rotation only. Additionally or alternatively, the transition can enable actuation of all arm joints. Additionally or alternatively, the transition can change the processing of control outputs from a user input device from displacement-to-velocity conversion to displacement-to-displacement conversion. In another non-limiting example, the transition can be automatically implemented by the control circuitry in response to an event, can be implemented based on user input such as pressing a button or rotating a switch, or can be implemented by noticing that the user has stopped using the first input device and started using the second input device. In another non-limiting example, the transition is an intermediate transition based on the user interface.

[0106] In a non-limiting example regarding transitioning from the second mode to the first mode, the transition can include restoring a constraint or restriction that was removed during the transition to the second mode of operation, such as limiting joint flexion and rotation to a given single arm joint of any particular arm, such as the elbow joint. Additionally or alternatively, the transition can include deactivating actuation of the arm joints enabled for the second mode of operation. Additionally or alternatively, the transition can change the processing of the control output from the user input device from a displacement-displacement conversion back to a displacement-velocity conversion. In further non-limiting examples, the transition can be implemented based on user input, such as pressing a button or rotating a switch, or it can be implanted by noticing that the user has stopped using an input device dedicated to the second mode of operation and begun using a different input device dedicated to the first mode of operation.

[0107] In embodiments, and particularly embodiments where the input device employed in the second mode resembles an avatar and converts input arm displacements into surgical arm displacements, transitioning from the first mode of operation to the second mode of operation may include alignment calibration. FIG. 12A to FIG. 12B In the alignment calibration discussed in further detail, the orientation of the surgical arms relative to each other, including the 'internal' orientation of the arm elements (e.g., arm joints and / or arm segments), is modified to match the 'shape' or 'curve' describing the corresponding orientation of the input arm that will take over actuation control of the surgical arm upon transitioning to the second operating mode. In the case of transitioning from the second operating mode back to the first operating mode, there is not necessarily a corresponding alignment calibration.

[0108] The conversion of user input to arm motion (bending and rotation) can be accomplished in a variety of ways. For example, the displacement (or displacement force) of the input device or the control element of the input device can be converted into the speed (e.g., speed, rate, angular velocity) of the arm motion. In the first mode including insertion / extraction and reverse bending (or reverse bending / no bending), it may be desirable to use this type of conversion. These are limited and in some embodiments, less precise motions may be advantageous not to be performed in teleoperation or avatar mode, but to be performed in a simpler robot or semi-autonomous mode. In contrast, the motion required in the second mode dedicated to performing various surgical actions may be more conducive to controlled displacement-displacement conversion, wherein the displacement of the input device or the control element of the input device is appropriately converted into the displacement of the arm segment and indirectly converted into the rotation and bending of the joint. For example, in the second mode, it may be desirable to ensure that the articulation of the articulated input device corresponds to the surgical arm in internal orientation (i.e., segment to segment orientation) so that the conversion of displacement to displacement is more intuitive, more ergonomic and more precise.

[0109] In the first mode, it may be desirable to constrain the arm motion to bending and / or rotating a single given arm joint of the arm (or each arm). This may be suitably achieved by configuring the surgical system to receive input from the input device used in the first mode, said input directly addressing the bending and rotation of a specific arm joint, e.g. the single given arm joint that is permitted to be actuated (in the operational mode). In other words, the user is 'controlling' the arm joint itself via the input device and suitable control circuitry. The motion that is intended to be actuated in this case is specifically bending and / or rotation of the arm joint. Regardless of whether the actual arm motion is performed semi-autonomously in response to the user's control input, the user is aware that she is controlling the actuation of the joint. Concomitant displacement and reorientation of the end effector may be an intended consequence of controlling the joint.

[0110] In contrast, in the second mode, and particularly in embodiments where the input device employed in the second mode is avatar-like and converts input arm displacements into surgical arm displacements, the surgical system may be required to receive and process inputs that directly address the displacement of the arm segments. The surgical system then indirectly addresses the bending and rotation of the arm joints by controlling the arm joint bending and rotation to the extent necessary to achieve the desired arm segment displacement and reorientation. In other words, the user is 'controlling' the displacement and reorientation of the arm segments (or end effectors, which for the purposes of this discussion act similarly to any arm segment since controlling their position and orientation is the user's goal), and the control circuitry of the surgical system uses this information to determine the necessary bending and rotation of each affected arm joint. In embodiments, the user can manipulate the avatar-like input arm into a shape or configuration that is intended or drives the intended shape of the surgical arm after manipulation.

[0111] In an exemplary first mode of operation, surgical arm movement may be at least partially restricted or constrained, and certain types of movement may be excluded while other types of movement are permitted. For clarity, 'movement' of the arm may include displacement and / or reorientation of any portion of the arm (e.g., one or more segments of the arm).

[0112] The terms 'elbow joint', 'wrist joint' and 'shoulder joint' as used herein refer to specific joints of the robotic arm, depending on the specific implementation in which the arm includes three actuatable joints. In this case, the joint closest to the distal effector is called the 'wrist joint', the middle joint of the three is called the 'elbow joint', and the most proximal joint is called the 'shoulder joint'. In some embodiments, the wrist joint is limited to rotation only, i.e., it is not configured to flex. The various joints are discussed below. Figure 3A Shown in.

[0113] Unless otherwise stated and as used in the present disclosure and appended claims in the context of an arm joint (i.e., a joint of an arm), the term 'joint' is meant to refer to any actuatable component that is capable of causing bending (e.g., planar bending) and / or rotation. It should be noted that an articulated / avatar-like input arm (a type of input device) may also have joints, and they are referred to as 'joint components' of the input device. Typically, arm segments are non-actuatable (with respect to bending / rotation) components of an arm that can be connected in series via actuatable arm joints, where the term 'connected in series' simply means that a joint is inserted between two consecutive segment components. The joints can be actuated, for example, mechanically and / or electronically, to cause one arm segment (and each part of the arm disposed distally therefrom) to move relative to a base arm segment ( Figure 3A 2181 in the figure) or another (adjacent) arm segment, and / or rotate one arm segment (and each portion of the arm disposed distally therefrom) relative to the base arm segment or another arm segment. In some embodiments, the joint includes multiple components, and in some examples, may include both components (or subassemblies) that facilitate bending and components (or subassemblies) that facilitate rotation. For ease of reading, the combination of these components is collectively referred to herein as a joint or arm joint.

[0114] In an embodiment, arm movement can be limited based on articulation type (e.g., rotation vs. flexion), movement speed, which parts of the surgical arm can move, etc. Movement of the surgical arm during the first operating mode can be limited to movement of a given single arm joint—the elbow joint only—(including, for example, flexion and / or rotation of the elbow joint) and linear movement of the surgical arm as a single unit (including, for example, linear advancement and retraction of the arm). It may be desirable to limit arm movement during the first operating mode to facilitate introduction of a surgical arm with minimal bulk through a narrow pathway to a target surgical site.

[0115] It may also be desirable to limit arm movement during the first mode of operation to facilitate arm flexure within a minimal volume, thereby avoiding collisions and potential tissue damage within the human body. Figure 3A 3 is a simplified schematic side view of the surgical robotic arm 102 in different configurations explained here for illustrative purposes. The dashed line 2177 represents an obstacle, such as patient tissue. The movement of the arm 102 during reverse bending is preferably controlled to prevent the arm 102 (and particularly the end effector 174) from contacting or colliding with the obstacle 2177. Three scenarios are shown in FIG3 , which are labeled A, B, and C, respectively. The arm 102 includes a proximal segment 2181 and a distal bendable portion 170. The bendable portion 2179 includes a shoulder joint 2101 and an elbow joint 2103, which are labeled differently as 2101a, 2101bc, 2103a, 2103b, and 2103c to indicate which scenario or scenarios (A, B, or C) the example of the joint is associated with. For example, joint 2103a is the elbow joint position in scenario A. Thus, scenario A involves bending only the elbow joint 2103a (shoulder joint 2101a remains unactuated and unbent), and in particular, it can be seen that bending only the elbow joint 2101a does not result in a collision between the end effector 174 and the obstacle 2177. On the other hand, in scenario B, where only the shoulder joint 2101b is bent, a collision occurs between the end effector 174 and the obstacle 2177. The difference between the collision / non-collision results of scenarios A and B is because the portion of the arm 102 distal to the elbow joint 2103 is shorter than the portion of the arm distal to the shoulder joint 2101. Figure 3A The wrist joint 2105 of the arm 102 in does not involve any of the three situations because Figure 3A In the non-limiting example of , the shoulder joint 2101c is designed or configured to rotate but not bend. In Scenario C, which is a continuation of Scenario A, it can be seen that bending the shoulder joint 2101c after the elbow joint 2103a (now 2103c) is bent can help continue to avoid collisions between the end effector 174 and the obstacle 2177.

[0116] Figure 3B Examples of the elbow joint 2103 being flexed from an unflexed orientation to various flexed orientations ranging from less than 90° to greater than 180° relative to the proximal arm base segment 2181 are shown.

[0117] In an embodiment, the elbow joint has a flexion range of motion of >90°, >120°, >140°, >160°, >180°, >190°, >200°, or approximately 210°±10°. In some embodiments, the end effector 174 can be positioned relative to the base 2181 of the arm 102 at >90°, >120°, >140°, >160°, >180°, >190°, >200°, or approximately 210°±10°. In some embodiments, the end effector 174 can be parallel to the base 2181 of the arm 102 or alternatively reach the base 2181 of the arm 102 when fully flexed. Additionally, the elbow rotation range of motion should be at least 200°, at least 250°, at least 300°, at least 310°, at least 320°, at least 330°, at least 350°, or approximately 360°. Figure 3C An arm 102 is shown with the elbow joint 2103 bent from Figure 3B The unbent orientation shown in FIG is rotated more than 180°, thereby placing the arm 102 in a reverse-bend configuration or, equivalently, a reverse-bend position, and transferring the end effector 174 to the reverse-bend operative position.

[0118] In some embodiments, reverse bending of the surgical arm can be accomplished automatically, i.e., by arranging the arm to respond to a single or a limited number of electronic control outputs by bending and / or rotating a single arm joint until a preprogrammed reverse bending position of the arm and / or end effector at the distal end of the arm is achieved.

[0119] Figure 4 The arm 102 is shown in a front perspective view with the elbow joint 2103 and Figure 3B The elbow joint 2103a in case A of FIG is similarly flexed and then rotated - i.e., Figure 4 The elbow joint 2103 in the arm can bend and rotate. The rotation of any arm joint can be independent of the bending of the same arm joint. In some embodiments, the bending and rotation can be simultaneous, and in other embodiments, constraints on non-simultaneity can be enforced, for example, by hardware design or by software components of the control circuitry that controls joint actuation.

[0120] Figure 5 The arm 102 is shown in a front perspective view, with the elbow joint 2103 similar to Figure 4 The elbow joint 2101 is bent and rotated, and the shoulder joint 2101 is bent to form a complex 'S' shape together with the elbow joint 2101. Figure 3B In the embodiment represented by Case C, the shoulder joint 2101 is preferably actuated to flex and rotate only after the flexion of the elbow joint 2103 has 'passed' the obstacle 2177.

[0121] Now refer to Figure 6, showing a flowchart of a general method for controlling a surgical robotic arm using different operating modes according to some embodiments.

[0122] Processes such as those described herein may be implemented for various types of surgeries performed at least in part using one or more surgical robotic arms inserted into a patient's body, such as gynecological, laparoscopic, and otolaryngological (ear, nose, and throat) surgeries.

[0123] As discussed above, the surgical robot arm can be operated according to a plurality of operating modes. Optionally, the operating mode is selected according to the surgical step to be performed and / or according to the current stage of the surgical operation.

[0124] Different operating modes can be characterized by different ways of converting user input (movement of the input device or a control element of the input device) into corresponding movements of the surgical robot arm, i.e., by manipulating the user input device. Additionally or alternatively, different operating modes can be characterized by different constraints on the articulation of the surgical arm, such as limiting articulation (e.g., bending) of one or more surgical arm joints; limiting the range of motion; or limiting the arm articulation to selected degrees of freedom. Additionally or alternatively, different operating modes can be characterized by different types of feedback to the user, such as feedback sensed by the user controlling the surgical arm via one or more input devices.

[0125] In some embodiments, the selection and / or switching between operating modes is controlled by a user, such as a surgeon. Alternatively, the selection of the operating mode can be performed by a user interface of the system, such as a touch screen and / or by a button or other input device provided on or near another input device, or a console or display screen. Additionally or alternatively, the selection of the operating mode can be automatic, such as by suitable control circuitry, such as by a system controller or processor. In some embodiments, the selection and / or switching of the operating mode is triggered by and depends on one or more of the following (and not exhaustive): identifying the current 'anatomical' position of the surgical arm or the end effector of the arm, such as by using an electromechanical instrument such as an encoder or other sensor (not shown) associated with the arm's actuators and / or motors (e.g., 104 or 108), or by accessing and analyzing an image of the arm 102 obtained by a camera through image processing or visually, upon identifying the current position of the input device, upon performing a specific articulation of the surgical arm, upon receiving an indication from one or more position sensors of the surgical arm, upon a timing indication, such as by setting a time point at which the operating mode is changed.

[0126] According to some embodiments, Figure 6 The flowchart in describes a method for operating a surgical robot arm using two operating modes.

[0127] The method comprises:

[0128] Step S01: Inserting the surgical arm and navigating the surgical arm to a predetermined position in a first operation mode.

[0129] In a first mode of operation, the surgical arm is introduced into the body and navigated to a selected anatomical location and / or a selected arm position. The surgical arm can be introduced into the body through a natural body orifice (e.g., vagina, anus, trachea, esophagus, ear canal) and / or through an incision.

[0130] In some embodiments, in a first operating mode, the arm is restricted from articulating. For example, one or more arm joints can be constrained or prevented from articulating (flexing and rotating). In one example, the arm includes three joints: a shoulder joint, an elbow joint, and a wrist joint that rotates but does not flex, and one or two of the joints are prevented from articulating. In a specific example, only movement of the elbow joint, such as flexion, extension, and / or rotation of the elbow joint, is allowed, while movement of the shoulder joint and wrist joint is constrained.

[0131] In the first mode of operation, the arm may also be allowed to move linearly (including only linearly) as a whole, for example to advance or retract in a one-dimensional movement.

[0132] In some embodiments, the arm movement is restricted mechanically, for example, by one or more locks (e.g., solenoid locks) that affect the actuation of the arm joints. Additionally or alternatively, the arm movement can be restricted by suitable circuitry, for example, by implementing software control functions that limit the extent and / or type of movement.

[0133] In some embodiments, limiting the extent of arm movement and / or constraining certain types of movement is performed based on the current arm position as indicated, for example, by one or more position sensors of the arm. In some embodiments, limiting the extent of arm movement and / or constraining certain types of movement is performed based on the current anatomical position as visualized, for example, by optical means (e.g., a camera optionally introduced into the body along with the surgical arm).

[0134] Step S02: Articulating the surgical arm to the basic position in the first operating mode

[0135] Still in the first mode of operation, the surgical arm is articulated to a base position. In some embodiments, the base position comprises a reverse bent position of the arm, for example, when the arm is bent at least 120 degrees, at least 150 degrees, at least 180 degrees, or intermediate, greater, or lesser angles. In some embodiments, the base position is a position in which the arm is positioned to allow the user to perform surgical maneuvers from a selected orientation that the user may be more comfortable or familiar with (e.g., corresponding to an abdominal orientation).

[0136] In some embodiments, control of the movement of the surgical arm in the first operating mode includes robotic control. Optionally, user manipulation of the input device in the first operating mode involves only limited types of user movement, such as limited movement of the input device along a defined axis and / or button presses. In an example, moving the input device along a first defined axis actuates rotation of a selected arm joint (e.g., an elbow); moving the input device along a second defined axis actuates flexion of a selected arm joint (e.g., an elbow); and pressing one or more buttons actuates linear advancement or retraction of the surgical arm.

[0137] In some embodiments, user manipulation of the input device in the first operating mode is converted to a speed of movement of the surgical arm. For example, when the user moves the input device relative to a resting position of the input device, the extent to which the input device moves relative to its resting position sets the relative speed of movement of the surgical arm.

[0138] Step S03: Transition to the second operation mode and then use the surgical arm to perform surgical operations.

[0139] In the second operating mode, the user performs surgical actions by the surgical arm, such as grasping tissue, dissecting tissue, moving tissue, suturing tissue. In some embodiments, once the arm has been moved to a selected basic position, such as a reverse bending position, the second operating mode starts when step S02 ends.

[0140] In some embodiments, the user switches input devices when passing from the first operating mode to the second operating mode. Alternatively, the user uses the same input device for both the first or navigation operating mode and the second or surgical operating mode.

[0141] In some embodiments, in the second operating mode, the surgical arm movement is not as restricted as in the first operating mode. In an example, all arm joints (shoulder, elbow, wrist) are allowed to articulate, e.g., flex and / or rotate. In some embodiments, the extent and / or speed of arm movement during the second operating mode is limited based on safety considerations, e.g., to avoid damaging surrounding tissue or to avoid performing movements at speeds that are too high and could risk damage.

[0142] In some embodiments, control of the surgical arm in the second operating mode involves remote operation control. Optionally, displacement of the input device by the user is simulated by a corresponding displacement of the surgical arm. Optionally, the displacement speed of the input device by the user is reflected in the corresponding surgical arm movement speed.

[0143] Step S04 transitions back to the first operating mode and then retracts the surgical arm from the body.

[0144] According to some embodiments, optionally, when the surgical action has been completed, the arm is retracted outwardly from the patient's body. In some embodiments, retraction is performed in the first operating mode. Optionally, before retraction and / or during retraction, the arm is straightened.

[0145] Retracting in the first mode of operation may be advantageous because it limits the extent and / or type of arm movement, thereby potentially reducing damage to tissue surrounding the anatomical passage (e.g., vagina) through which the arm is retracted.

[0146] Now refer to Figure 7 , the console 118 may include a display screen 407 with an input device 405 located adjacent thereto - in Figure 7 are located on opposite sides of screen 407 in the non-limiting example.

[0147] Each of the input devices 405 (in the form of a 'thumbstick') includes a nipple-shaped control 409 adapted to be manipulated by a user's thumb. In an embodiment, the degree of movement of the nipple-shaped member 409 relative to a central resting position can be converted into a selected speed of surgical arm movement, as discussed above. In one example, the further the nipple-shaped member 409 is pushed from its central resting position, the higher the resulting speed of the surgical arm movement. In another example, the greater the force applied to the nipple-shaped member 409, the higher the resulting speed of the surgical arm movement.

[0148] In some embodiments, when the surgical arm is controlled by the thumb stick 405, the movement of one or more surgical arm joints (e.g., shoulder joint, wrist joint) is constrained, and only flexion and / or rotation of the elbow joint is enabled. In some embodiments, linear movement of the surgical arm (as a single entity) is also enabled, for example to advance or retract the arm. In some embodiments, movement of the nipple-shaped piece 409 actuates flexion and / or rotation of the elbow joint. In some embodiments, linear movement of the arm 102 is actuated by a separate actuator, for example using another pair of input devices (e.g., input devices 406, 408), which are implemented as Figure 7 405 itself. In other examples, buttons 406, 408 may be provided independently, or provided on (or closer to) display screen 407. In the example, button 406 advances the surgical arm distally (e.g., in the abdominal direction), and button 408 retracts the surgical arm proximally. In an embodiment, input devices 406, 408 may be used when in a first mode in which flexion or rotation of arm joints other than the elbow joint is prevented because linear motion does not require any flexion or rotation.

[0149] In some embodiments, during use of the thumbstick 405, other input devices, such as the avatar input arm 411, which is provided for use in the second mode without being constrained by any first mode, are locked in a rest position, such as by a solenoid lock. In some embodiments, the rest position of the input arm 411 is selected to be the reverse flex position of the surgical arm 102, so that once the surgical arm 102 has been reverse flexed (e.g., using the thumbstick), the user can pick up the avatar input arm 411 and continue the procedure directly. In some embodiments, when the avatar-like input arm 411 is enabled in the second mode, operation of the thumbstick is disabled.

[0150] In some embodiments, during insertion of the surgical arm 102 into the body of the patient 106, the surgical arm is straight, i.e., unbent. In some cases, insertion is performed through a cannula. Simultaneously, the avatar input arm is in a rest position, which can be a locked position or a reversed bent position. Optionally, after reverse bending the surgical arm using the thumbstick 405, the surgeon can release the thumbstick 405 and move her hand to the avatar input arm 411. Once the surgeon grasps and optionally lifts the avatar input arm 411, control of the surgical arm 102 can be automatically transferred or switched from the first user input device 405 to the second user input device 411, and the surgeon can continue the procedure using the avatar input arm 411.

[0151] In some embodiments, when one or more avatar input arm joints are locked by a solenoid lock, the surgeon automatically releases the solenoid lock by lifting the avatar input arm. Additionally or alternatively, manual locking of the avatar input arm joints can be released, for example, by sensors detecting the position of the avatar input arm.

[0152] In some embodiments, the system (e.g., a system processor) is configured to identify one or more positions of an input device, such as the current position of a thumbstick and / or the current position of an avatar input arm, and optionally present the positions on a user interface screen. In some embodiments, identifying the positions is aided by using position sensors.

[0153] Figure 8 An image showing an example of a thumb-operated input device 501 including a grip handle 503 and optionally a textured surface to facilitate gripping, such as a surface including ridges 505. One or more control buttons 507 may be provided along the grip handle 503. For example, Figure 8 The thumbstick input device 501 includes two control buttons 507 - one for actuating linear advancement of the surgical arm distally and one for actuating linear retraction of the arm proximally.

[0154] In some embodiments, the thumb lever 501 includes a nipple-shaped member 509, which extends, for example, from the proximal end of the grip handle 503. In some embodiments, the nipple-shaped member 509 is shaped and / or sized to fit the user's thumb. In some embodiments, the nipple-shaped member 509 includes a rounded profile. In some embodiments, the proximal surface of the nipple-shaped member 509 is formed with a circumferential protrusion 511. The circumferential protrusion can help keep the thumb in place on the nipple-shaped member 509, thereby potentially preventing or reducing the thumb from slipping off the nipple-shaped member 509.

[0155] In some embodiments, the nipple-shaped piece 509 moves in a spring-like manner. Optionally, after the nipple-shaped piece 509 is pushed away from its initial rest position (e.g., a central position in which the nipple-shaped piece 509 is centrally aligned with the long axis 513 of the thumbstick), the nipple-shaped piece springs back to its central position.

[0156] Figure 9 is a schematic diagram illustrating an example of control of a thumb-operated input 501 according to some embodiments. In some embodiments, moving ('displacing') the nipple-shaped piece 509 relative to a first axis actuates a first type of surgical arm movement, such as bending, and displacing the nipple-shaped piece 509 relative to a second axis actuates a second type of surgical arm movement, such as rotation.

[0157] In some embodiments, the thumbstick 501 is used during the first mode of operation when constraining at least some joints, such as the shoulder joint 2101 and the wrist joint 2105. Optionally, displacing the nipple 509 along the Y axis produces flexion of the elbow joint 2103 of the surgical arm 102; and displacing the nipple 509 along the axis produces rotation of the elbow joint. In some embodiments, simultaneous flexion and rotation actuation can be achieved by pushing the nipple about two axes (e.g., diagonally about the center).

[0158] Now refer to Figure 10 and Figure 11 , showing side and front view images of an exemplary avatar-like input arm 701, which includes multiple joints for actuating corresponding movements of the surgical arm joints: as shown, the joints include a shoulder joint 703, an elbow joint 705 and a wrist rotation knob 707, which is used to control the wrist joint 2105 of the surgical arm 102.

[0159] In some embodiments, the avatar input arm may include additional input devices, such as buttons or joysticks 709, which may control the operation of the surgical tools of the surgical arm 102 (as the end effector 174).

[0160] Set in Figures 10 and 11Other input devices on the exemplary input arm 701 include a pause-resume button 711 and buttons 713, 715 for actuating linear advancement and retraction, respectively, of the arm 102. The user can actuate linear advancement and / or retraction using the buttons 713, 715 on the avatar input arm and / or buttons 406, 408 provided on the thumbstick 405. Additionally or alternatively, linear movement of the arm can be actuated via a screen interface of the console (e.g., via a touch screen interface).

[0161] In some embodiments, a set of two avatar input arms is provided for controlling the left and right arms, respectively. For example, a first avatar input arm 701 can control a first motor unit 108 associated with a first surgical arm 102 (e.g., the 'right' arm), and a second avatar input arm 701 can control a second motor unit 108 associated with a second surgical arm 102 (e.g., the 'left' arm). In some embodiments, any or all of the input arms 120, 411, and 701 can be the same.

[0162] Now refer to Figure 12A and Figure 12B . In an embodiment, alignment of an input device, such as an avatar input arm, with the current position of the surgical robotic arm is performed. In an example, alignment is performed when a user switches between different input devices, such as when switching from a thumb-operated input to an avatar input arm. In another example, alignment of the input device with the position of the surgical arm is performed when the system is initialized, such as at the beginning of an operation or prior to an operation. In another example, alignment of the input device with the current position of the surgical robotic arm is performed when control is resumed after a pause. Optionally, in pause mode, movement of the input device does not actuate relative movement of the surgical robotic arm, and when control is resumed, the position of the input arm may need to be adjusted to match the current position of the surgical robotic arm to allow the operation to continue in a smooth, uninterrupted manner.

[0163] In some embodiments, restored control is obtained in an automatic or semi-automatic manner. For example, the user performs a selected articulation of the input device (e.g., straightening the elbow joint of the avatar input arm) and regains control of the surgical arm.

[0164] exist Figure 12A and Figure 12B, the relative position of the input device (i.e., input arm) is represented using a cross-shaped diagram 801. In some embodiments, there are two sets of crosses, one for the shoulder joint and one for the elbow joint. In some embodiments, the cross represents a specific single joint of the surgical robot arm (e.g., shoulder joint, elbow joint). In some embodiments, each line of the slot represents a different type of articulation, for example, the horizontal line 803 represents the rotation of the joint; the vertical line 805 represents the bending of the joint. During use, the user manipulates the input device according to the joint position indicated by the two colored dots 807 at the cross. As the position of the input device (in response to the user's manipulation) becomes closer to the position of the surgical robot arm, the colored dots move closer to the center of the cross (see Figure 12B ). Optionally, once sufficient alignment between the input device position and the surgical arm position is obtained, the dot changes color, for example from red to green, as shown Figure 12A -B as indicated.

[0165] In an embodiment, Figure 12A and Figure 12B The control device can be used in alignment calibration as part of a transition from a first mode using a first input device to a second mode using an avatar-like input arm. In these embodiments, the first input device can be used to adjust the position of the surgical arm to align with the fixed position of the input arm.

[0166] Figure 13 8 is an example of a screen displayed to a user during alignment, according to some embodiments. In the example shown, a first input device (e.g., corresponding to the right avatar arm) is shown correctly aligned with the first surgical arm, for example, as indicated by check marks 809 at both the elbow and shoulder joints, and / or by a locked lock 811. A second input device (e.g., corresponding to the left arm) is shown in a position where it is not yet aligned with the surgical arm. The joint positions are represented by two crosshairs, for example, as described above, and the lock is unlocked.

[0167] In some embodiments, during alignment, certain articulations and / or functions are disabled, such as limiting the speed of the surgical arm; disabling electrosurgical functions; and / or other functions.

[0168] Figure 14A -E shows a set of images illustrating exemplary control of surgical robotic arms using multiple different input devices, according to some embodiments. Figure 14A -E is shown in the non-limiting example, and in other examples, there may be a single arm or more than 2 arms.

[0169] Figure 14AFIG2 shows the control of two surgical robotic arms 901 during their introduction into a model 903 simulating vaginal entry into the body, according to some embodiments. In this example, control of the surgical arms 901 during advancement of the arms into the body (e.g., through the vaginal canal) is via thumb-operated inputs, including, for example, a set of thumbsticks 905 as described above.

[0170] Figure 14B Manipulation of the surgical arm into a reverse bent position is shown via thumbstick 905. In the example shown, the arm is bent (e.g., 120, 150, 180, 210 degrees, or intermediate, greater, or lesser angles) to obtain the reverse bent position.

[0171] Figure 14C The surgical arm is shown in a reverse bending position. After reverse bending, in some examples, the user can switch the input device being used, for example by releasing the thumbstick and picking up the avatar input arm. At this point, alignment of the avatar input arm with the current position of the surgical arm can be performed, for example, as shown in FIG. Figure 12A -B and Figure 13 Described in .

[0172] Figure 14D and Figure 14E Shown is manipulation of a surgical arm using a set of avatar input arms 907, according to some embodiments. As can be seen, the respective positions of the surgical arms correspond to the positions of the input arms.

[0173] Figure 15 The block diagram of FIG. 1 shows the articulated arm 102, the input device array 1500, and the detector 1520. Those skilled in the art will appreciate that not all input devices are present in the drawings. Figure 15 Each element present in is required in every embodiment. Figure 15The illustrated input device array 1500 shows two user input devices 1510A (e.g., thumbstick 501) and 1510B (e.g., joystick 701), but those skilled in the art will appreciate that fewer or more user input devices may be provided in different embodiments. In the example, joystick 701 may be both the first user input device 1510A and the second user input device 1510B; in other words, the first user input device 1510A and the second user input device 1510B may be the same single user input device. In implementations where the first user input device 1510A and the second user input device 1510B are the same single user input device, it is typically a more versatile or flexible device, such as joystick 701. The term user input device refers to a device for converting input received from a user into an electronic output and / or signal. Examples of user input devices include, but are not limited to, a joystick, a touch screen, a thumbstick, a mouse, a keyboard, and a gesture detection device (e.g., including a camera [not shown]). Unless otherwise specified, the term 'array' refers to one or more items.

[0174] like Figure 15 As shown in the example of , the articulated arm 102 has one or more objects 1530 mounted at its distal end. The objects 1530 can be provided with the arm 102 or can be added or replaced separately. Examples of such objects 1530 include, but are not limited to, end effectors, such as surgical end effectors. Related surgical tools may include, but are not exhaustive: endoscopes for diagnostic / surgical feedback, surgical end effector tools such as needle holders (e.g., large needle holders, curved needle holders), monopolar and bipolar instruments (e.g., monopolar scissors, bipolar forceps), clip appliers (e.g., large and medium clip appliers), vascular closures, graspers or dissectors (e.g., Maryland dissectors, Tenacululm forceps, micro forceps, long-tipped forceps, grasping retractors, fundus graspers, Crocodile graspers, Cadiere forceps), scissors (e.g., Potts scissors, curved scissors), hooks (e.g., electric hooks), and scrapers (e.g., electric spatulas).

[0175] As discussed elsewhere, in some embodiments, the arm 102 and / or object 1530 operates in response to electronic control outputs from one or more control devices. The terms 'control output' and 'control signal' are used synonymously. In various embodiments, the control output can be sent to the arm 102 and / or its controller via wired and / or wireless communication.

[0176] Figure 15Detector 1520 is also shown. As discussed elsewhere, in some embodiments, a mode transition from a first operating mode of the surgical system 999 to a second operating mode of the surgical system 999 is responsive to and / or dependent upon an output of detector 1520. The surgical system 999 may be functionally equivalent to Figure 1 . In one non-limiting example, detector 1520 detects whether a portion of arm 102 (e.g., a distal portion) is reverse bent and / or is in a reverse bent position. In one example, a camera can acquire an image of arm 102 and detector 1520, camera, and image processing circuitry. In another example, an encoder (not shown) or other electromechanical sensors (e.g., 104 or 108) can be used for monitoring and detection to track, for example, the orientation of a joint of arm 102. Other examples can involve magnetic or capacitive detectors, or position detection based on triangulation (e.g., time of flight) using, for example, ultrasound and / or light.

[0177] As used in this disclosure and the appended claims, 'monitoring' and 'detection' are actions (and / or functions and / or potential actions and / or capabilities) that can be performed by one or more components of a surgical system, by one or more users, or any combination of system components and human users. The descriptive language used herein regarding automatic or machine monitoring or detection is intended to be non-limiting, and in any such embodiment, user intervention can be part of the design and / or part of the operation. In some embodiments, user intervention is required for safety reasons. In a non-limiting example, one or more sensors relay information about a surgical arm or one of its components to a display screen on which the user is trained and / or positioned to monitor the arm curve shape and detect when the arm is in the desired position and orientation without or without automatic or semi-automatic visual assistance, such as when it is reversed at the surgical site. In another non-limiting example, monitoring and / or detection is communicated to the user in a non-visual manner, for example, but not exclusively, by auditory notification, by tactile feedback, or by locking a control or input device. Obviously, any of these types of communications can also be combined with visual information.

[0178] Those skilled in the art will also appreciate that additional components may be provided within the surgical system 999 and not within the surgical system 999. Figure 15 Every component of every element shown in is essential in every embodiment.

[0179] Now refer to Figure 16. In step S101, the surgical system 999 operates in a first operating mode. In step S121, the surgical system 999 operates in a second operating mode. As discussed elsewhere, in different embodiments, the first and / or second operating modes may relate to specific capabilities and / or specific constraints of one or more input devices. Alternatively or in addition, the first and / or second operating modes may relate to specific capabilities and / or specific constraints of one or more elements of the arm 102 and / or object 1530. Examples of such elements include joints and articulators. Alternatively or in addition, the first and / or second operating modes may relate to a relationship between the operation of one or more input devices 1510 and the arm 102 or one or more components thereof - for example, whether the configuration of the input device 1510 or its components is converted into a velocity or position of the arm 102 or its components.

[0180] When the surgical system 999 operates in the first mode in step S101, one or more operations are performed. In one example, when the surgical system operates in the first mode, the distal portion of the arm 102 is reverse bent, possibly starting from an unbent and / or straight position, but not necessarily. In another example, when the surgical system 999 operates in the first mode, the distal portion of the arm 102 is moved so that its curved shape, such as a curved shape in 3D or a planar projection thereof, matches a predefined curved shape, such as a curved shape used to start a surgical procedure (e.g., a reverse bent shape or an 'S' curve shape).

[0181] Step S109 involves monitoring and may be performed concurrently with step S101. For example, step S109 may be performed at least in part by detector 1520. In some embodiments, the monitoring of step S109 may include determining whether the arm is in a reversed flexion position.

[0182] Step S113 involves a mode transition trigger event—i.e., a detectable event whose detection triggers a mode transition of the surgical system 999 from the first mode of step S101 to the second mode of step S121. An example of step S113 is as follows: In response to and dependent on, for example, detecting, by detector 1520, that the arm 102 is in a reversed flexed position, i.e., detecting that the arm 102 has transitioned from a non-reverse flexed position to a reversed flexed position, the surgical system 999 transitions from the first operating mode to the second operating mode. As discussed above, the transition may include alignment calibration of the surgical arm 102 with the input arm 701.

[0183] The transition to step S117 may be triggered, for example, by and / or in response to and / or dependent on detecting that the arm 102 has reverse bent, or that the end effector 1530 is in a reverse bent position.

[0184] Now describe Figure 16A plurality of embodiments of the present invention are disclosed. Those skilled in the art will understand that these embodiments need not be mutually exclusive—these embodiments or features thereof may be combined. In some embodiments, not all features and / or method steps need to be present.

[0185] Figure 16 First embodiment

[0186] A first embodiment relates to a method of operating a surgical system, the surgical system comprising (i) an input device array of one or more user input devices and (ii) an articulated robotic arm, the articulated robotic arm comprising a surgical end effector at its distal end and a plurality of arm joints, the arm joints being configured to bend and rotate in response to electronic control outputs from the user input devices, the method comprising: (a) commencing operation of the surgical system in a defined first operating mode S101 relative to a given single arm joint (e.g., elbow 2103) of the arm joints, wherein the first mode prevents actuation of any arm joint (e.g., including shoulder 2101) in the arm other than the given single arm joint and allows control of actuation of a single arm (e.g., elbow 2103) joint to cause bending and rotation of the single arm joint (e.g., elbow 2103); and (b) while the surgical system is in the first operating mode (e.g., S101), (i ) in response to control signals generated by one or more of the user input devices (e.g., the thumb stick 501), reversely bend the distal end of the arm by bending and rotating a single arm joint (e.g., the elbow 2103) so that the end effector 174 is in a reverse bending operating position, and (ii) monitor the state of the robotic arm 102 (e.g., S109) to detect whether the arm is in the reverse bending position; (c) in response to and depending on (in step S113) detecting that the arm 102 is in the reverse bending position, transition the operation of the surgical system (e.g., the 'yes branch' from S113 to S121) from a first mode to a second mode in which the system is enabled to control the bending and rotation of at least one arm joint (e.g., the elbow 2103) excluding the first mode according to the corresponding degrees of freedom of each arm joint; and (d) operate the surgical system in the second mode (e.g., in S121) to perform surgical actions using the end effector.

[0187] Figure 16 Second Implementation Method

[0188] A method of operating a surgical system comprising (i) a first user input device 1510A (e.g., thumb stick 501) and a second user input device 1510B (e.g., joystick 701), and (ii) an articulated robotic arm 102 comprising a plurality of arm joints (e.g., elbow 2103 and shoulder 2101), and a surgical end effector 174 at a distal end of the arm, the method comprising: (a) commencing operation of the surgical system in a reverse bending mode (e.g., the first mode of S101), wherein with respect to bending and rotation of the arm joints: (i) the first user input device 1510A (e.g., thumb stick 501) is activated to guide bending and rotation of only a given one of the arm joints, and (ii) the second user input device 1510B (e.g., joystick 701) is deactivated; (b) at a given one of the arm joints, (c) when the surgical system is in a reverse bending mode due to bending and rotation of a fixed arm joint (e.g., elbow 2013), in response to the electronic control output from the first user input device 1510A, reversely bend the distal portion of the articulated robotic arm so that the end effector is in a reverse bending operating position; (c) converting the surgical system from the reverse bending mode to the surgical mode to enable the second user input device 1510B with respect to at least one arm joint (e.g., shoulder 2101) in the bending and rotation arm other than a given arm joint (e.g., elbow 2103); and (d) in the surgical mode, according to the corresponding degrees of freedom of each arm joint, in response to the electronic control output from the second user input device 1510B, achieve bending and rotation of at least two of the arm joints (e.g., shoulder and elbow), thereby moving the surgical end effector to perform one or more surgical actions.

[0189] Figure 16 The third implementation plan

[0190] A method of operating a surgical system comprising (i) an input device array 1500 of one or more user input devices and (ii) an articulated robotic arm 102 comprising a plurality of arm joints (e.g., a shoulder 2101 and an elbow 2103), and a surgical end effector 1764 at a distal end of the arm, the method comprising: (a) commencing operation of the surgical system in a reverse bending mode (e.g., S101), wherein with respect to bending and rotation of the arm joints, the input device array is activated to direct bending and rotation of only a given one of the arm joints (e.g., the elbow 2103); and (b) while in the reverse bending mode, in response to a command from the input device array. (c) transitioning the surgical system from a reverse bending mode to a surgical mode to enable input devices for bending and rotating arm joints other than a given one of the arm joints; and (d) when in the surgical mode, enabling bending and rotation of at least two of the arm joints (e.g., at least the elbow 2013 and the shoulder 2101) in response to electronic control outputs from the input device array according to respective degrees of freedom of each arm joint, thereby moving the surgical end effector to perform one or more surgical actions.

[0191] Figure 16 The fourth implementation plan

[0192] A method of operating a surgical system comprising (i) a user input device (e.g., a joystick 701) and (ii) an articulated robotic arm 102 comprising a plurality of arm joints and a surgical end effector at a distal end of the arm, the method comprising: (a) commencing operation of the surgical system in a reverse bending mode (e.g., S101), wherein with respect to bending and rotation of the arm joints, the user input device is activated to direct bending and rotation of only a given one of the arm joints (e.g., the elbow); and (b) while in the reverse bending mode, responsive to an electronic control output from the user input device, directing the arm joints to rotate and bend. (c) transitioning the surgical system from a reverse bending mode to a surgical mode (e.g., 121) to enable a user input device with respect to bending and rotation of at least one arm joint in the arm other than a given one of the arm joints; and (d) when in the surgical mode, enabling bending and rotation of at least two of the arm joints in response to an electronic control output from the user input device according to the corresponding degrees of freedom of each arm joint, thereby moving the surgical end effector to perform one or more surgical actions.

[0193] Figure 16 Fifth implementation plan

[0194] A method of using a surgical system, the method comprising: (a) providing an articulated robotic arm capable of displacing a surgical end effector 174 at its distal end, the arm comprising a plurality of arm segments connected (e.g., in series) by corresponding plurality of arm joints (e.g., elbows and shoulders), the arm joints being configured to bend and rotate in response to electronic control output from a user input device; (b) manipulating the end effector to a reverse bend operating position while operating in a first input mode, wherein displacement of the input device (e.g., 1510A, such as a thumb stick) or a displaceable portion thereof is converted into a rate of bending and / or rotation of the arm joints; (c) in response to and dependent on detecting that the end effector is in the reverse bend operating position, transitioning from operating in the first input mode to operating in a second input mode, wherein displacement of the input device (e.g., 1510B, such as a joystick) or a displaceable portion thereof is converted into a corresponding displacement of at least one arm segment; and (d) after the transition and while operating in the second input mode, performing a surgical action using the end effector.

[0195] and Figure 22A -C related Figure 16 specific implementation plan.

[0196] (A) A method of operating a surgical system comprising (i) a given user input device (e.g., an articulated and / or flexible user input device—e.g., a joystick 701) and (ii) an articulated robotic arm 102 comprising a surgical end effector 174 at a distal end thereof, and a plurality of arm joints (e.g., a shoulder 2101 and an elbow 2103) configured (e.g., in at least one operating mode of the surgical system) to respond to electronic control outputs (e.g., and optionally, to a plurality of control elements) from a given (articulated and / or flexible) user input device 1510B (e.g., a joystick 701). The method further comprises: a. initiating operation of the surgical system in a first operating mode (e.g., wherein the movement of the arm is not controlled by the output of the given user input device 1510B and / or is insensitive to the output of the given user input device 1510B); b. while the surgical system is in the first operating mode (e.g., step S101), (i) modifying the curved shape of the arm by mechanical bending and / or by rotation of one or more arm joints of the articulated robotic arm (e.g., from Figure 22B2830 ), and (ii) monitoring the shape state of the robotic arm to detect whether part or all of the arm has a curvilinear shape that matches (e.g., a curve match or a 2D projection thereof) the current primary curvilinear shape (e.g., shape 2810 ) and / or the rest state curve (e.g., shape 2810 ) defined by a (e.g., articulated and / or flexible) given user input device 1510B; c. responsive to and dependent on a positive detection of a match (i.e., a 'positive' detection is a detection that there is a match between the curvilinear shapes defined by the arm and the given user input device) [i.e., at Figure 22B There is no match between 2830 and 2810, but Figure 22C d. in response to the output of a (e.g., articulated and / or flexible) given user input device (e.g., 1510B - e.g., a joystick), operate the surgical system in the second mode (e.g., the (e.g., control) output of the given user input device modifies the configuration of the arm or segment (e.g., a non-distal segment)) or an element of the arm (e.g., any joint thereof) to perform a surgical action using the end effector.

[0197] (B) A method of operating a surgical system comprising (i) a user input device (e.g., an articulated and / or flexible user input device) (e.g., controlled by a user input device having position control) and (ii) an articulated robotic arm comprising a surgical end effector at its distal end and a plurality of arm joints, the arm joints being configured to bend and rotate (e.g., in at least one operating mode of the surgical system) in response to electronic control output from a given (e.g., articulated and / or flexible) user input device (e.g., optionally also in response to output from an additional user input device other than the given user input device), the method comprising: a. commencing operation of the surgical system in a first operating mode (e.g., wherein movement of the arm is not controlled by and / or is insensitive to output from the given user input device); b. when the surgical When the surgical system is in a first operating mode, (i) the curvilinear shape of the arm is modified by mechanical bending and / or rotation of one or more arm joints of an articulated robotic arm, and (ii) the shape state of the robotic arm is monitored to detect whether part or all of the arm has a curvilinear shape that matches (e.g., a curve match or a 2D projection thereof) a predefined curvilinear shape (e.g., having one or more local minima or local maxima; having one or more inflection points - e.g., an S-curve shape); c. in response to and depending on a positive detection of a match, the operation of the surgical system is transitioned from the first mode to the second mode; d. in response to the output of a given user input device (e.g., articulated and / or flexible), the surgical system is operated in the second mode (e.g., the (e.g., control) output of the given user input device modifies the configuration of the arm or segment (e.g., a non-distal segment)) or an element of the arm (e.g., any joint thereof) to perform a surgical action using an end effector.

[0198] and Figure 16 Additional features relevant to specific embodiments:

[0199] One or more of the following (e.g., in any combination) may be provided:

[0200] (1) The transition step of step c includes transferring user control of the arm configuration from a different user input device (e.g., 1510A - e.g., a thumb stick device) other than the given user input device (e.g., 1510B - e.g., a joystick) to the given user input device (e.g., 1510B - e.g., a joystick).

[0201] (2) The method also includes, simultaneously or subsequently and when the surgical system is in the second mode (e.g., step S121), configuring the arm 102 through the output of a user control device (e.g., 1510A - e.g., a thumb stick device) so that the displacement magnitude of a given input device or its displaceable portion is converted into a corresponding displacement of at least one part of the arm and / or at least one arm segment of the arm.

[0202] (3) The method also includes, simultaneously with step (d) (e.g., S121) or after step (d) and when the surgical system is in the second mode, controlling the configuration of the arm via an output of a user control device (e.g., 1510B—e.g., joystick 701) such that a displacement magnitude of a given input device or displaceable portion specifies (e.g., fully specifies; e.g., commands) a target position of the arm element (e.g., different from the current primary position) and / or a target configuration of the arm (e.g., different from the current primary configuration).

[0203] (4) a first mode that: (i) is defined with respect to a proper subset of the plurality of arm joints (e.g., only the elbow 2103); (ii) prevents actuation of one or more arm joints (e.g., the shoulder 2101) in the arm that are not members of the proper subset of joints by control signals from the articulated user input device, and (iii) allows control of actuation of the arm joints (e.g., the elbow 2103) that are members of the proper subset to cause bending and / or rotation of each arm joint of the proper subset;

[0204] (5) When transitioning from the first mode to the second mode, enabling the articulated user input device (e.g., 1510B - e.g., joystick 701) to control the bending and rotation of at least one arm joint (e.g., 703 of joystick 701) excluding the first mode.

[0205] (6) Modification of the shape of arm 102 is performed in response to electronic control signals provided by a given user input device (e.g., 1510B - such as a joystick).

[0206] (7) Modification of the arm shape is performed in response to an electronic control signal provided by a user input device other than a given user input device.

[0207] (8) Automatically perform modification of the arm shape (e.g., automatic reverse bending).

[0208] (9) (i) in a first mode, the arm shape is controlled by the operation of an articulated input device or another input device so that the displacement of the input device or its displaceable part is converted into the speed of bending and rotation of the arm joint; and (ii) in a second mode, the arm shape is controlled by the operation of an articulated input device so that the displacement of the articulated input device or its displaceable part is converted into a corresponding displacement of at least one arm segment.

[0209] Figures 17 to 21 Discussion

[0210] In the example, both the first and second modes use the same input device, namely an avatar-like input arm. As explained above, the input arm is used to reverse the bending arm in the limited mode, and then transitions to fully enable use in the second mode. However, at this time, the input arm or at least one handle component (e.g., Figure 10 The handle component 702 may have been flipped by as much as 180° or more depending on the bend angle employed in the reverse bend. This means that (a) from the surgeon's perspective, the handle 702 is uncomfortably 'upside down,' and (b) the coordinate system used by the surgical system is 'backwards.' The surgeon's perspective has switched from viewing 'distally' to reverse bend the arm to viewing 'proximally' to perform the procedure with the arm in reverse bend, and therefore does not translate the displacement vectors or reorientation arcs of the segment components to corresponding (e.g., parallel) displacement vectors or reorientation arcs in the same xyz space.

[0211] It should be noted that the exemplary designs of the input device 701 and the handle component 702 are provided for illustrative purposes, and in other examples and in other embodiments, the input device and the handle component can be designed and implemented in different ways. For example, in some embodiments, the handle component can be physically separated from the input device, and the handle component is functionally a part of the input device. As another example, in some embodiments, an avatar-like or joystick-like input device is entirely or almost entirely or primarily composed of a manual and / or hand-graspable handle component. As another example, in some embodiments, the handle component can incorporate multiple operating functions into its design by including (but not exhaustively) buttons, switches, toggle switches, wheels, knobs and / or small sticks such as thumb sticks to 'combine' multiple input devices and their corresponding functions into what visually appears to be a single input device. Figure 10 The repeated presentation of the input device 701 and the handle component 702 in this article is for convenience and ease of understanding, and should not be understood as limiting the design of the input device and the handle component.

[0212] It is now disclosed that transitioning from a first input device of an array of one or more input devices to a second input device of the array when transitioning from a first mode to a second mode can overcome the aforementioned disadvantages. The handle member 702 of the second input device (e.g., the avatar's input arm 701) can be pre-positioned in an orientation that is not 'upside down' from the surgeon's perspective; a reverse bend can be performed using an input device (e.g., input device 501 / 405) whose use does not include or require movement of the handle member 702 of the second input device, or if moved, does not move more than 90°, which may be a 'critical point' at becoming 'upside down' from the surgeon's perspective. Furthermore, the control circuitry controlling the second input device (input arm 701) can be configured (e.g., programmed) to use a 'direct' coordinate transformation matrix, wherein displacement vectors of a segment member of the input arm are transformed into corresponding (e.g., parallel or maintaining the same sign in each of the x, y, and z directions or in at least two of the three directions) displacement vectors of a corresponding surgical arm segment, and / or wherein reorientation arcs of a segment member of the input arm are transformed into corresponding reorientation arcs of a corresponding surgical arm segment.

[0213] Therefore, in some embodiments, it may be desirable to ensure a switch (transition) from a first mode to a second mode, wherein control of arm movement is transferred from a first input device (e.g., the 'thumb stick' detailed above) to an avatar-like input arm, ensuring ergonomic comfort and convenience based on an optimally oriented handle assembly and a directly convertible (input arm to surgical arm) coordinate transformation matrix.

[0214] Now refer to Figure 17 , the user input device 701 is oriented in xyz space, said xyz space corresponding to the position of the arm 102 or at least the distal portion thereof in Figure 18 The same xyz space oriented in . Figure 18 The arm 102 is shown unbent with the end effector 174 attached at the distal end of the arm 102, indicating the orientation of the arm at the beginning of or prior to a surgical procedure. Figures 17 and 18 The end effector 174 in FIG. 1 is simple; a leftward displacement of the input device 701 translates to a leftward displacement of the arm 102 (e.g., along the x-axis), forward translates to forward (e.g., along the y-axis), rightward translates to rightward (again, along the x-axis), and backward translates to backward (again, along the y-axis). An upward displacement of the input device 701 translates to an upward displacement of the arm 102 (e.g., along the z-axis) and a downward displacement translates to downward.

[0215] exist Figure 19 and Figure 20In the reverse bending position, the distal portion of the arm 102 is reverse bent, thereby placing the arm (and attached surgical end effector 174) in the reverse bending position. The handle member 702 of the user input device 701 has been rotated to achieve the reverse bending movement of the arm 102. If the surgical mode is to be switched to at this time, the user input device 701 will be in the reverse bending position. Figure 19 The rotation of the xyz coordinate system is positioned in the Figure 20 , the xyz coordinate system of the end effector 174 in the reverse bend of the arm 102 does not match (the coordinate system has been rotated according to the reverse bend of the arm 102 and now matches the reverse bend orientation / position of the end effector 174). NOTE: The xyz coordinate system of the reverse bend arm 102 has been 'flipped' to match the perspective of the surgeon in a typical surgical system. For example, if a camera (e.g., an endoscopic camera at the end of the arm) was not 'flipped', the surgeon would not be able to see the surgical workspace or anything that needed to be seen. However, the flipped handle 702 and the 'flipped' (reverse bend) end effector are no longer oriented in a shared three-dimensional xyz space. As discussed above, there are two options at this point: grasp the device 701 from a different perspective than is normally employed, or perform the surgery such that the motion of the input device 701 (specifically, the motion of the handle component 702) does not translate easily and ergonomically to the surgical arm 102 in the same xyz coordinate system. For example, Figure 19 A leftward displacement of the device 701 (according to its xyz coordinate system) will result in a rightward displacement of the end effector 174 according to its xyz coordinate system, while upward means downward and so on.

[0216] As mentioned previously, the first solution to a 'flip handle' is to use a different user input device for the reverse bending of the arm. This results in e.g. Figure 17 The unflipped handle shown controls the following assumed post-transition surgical mode. Figure 20 Those skilled in the art will appreciate that the xyz coordinate systems appear to be inverted relative to one another, but in fact align the input device 701 and arm 102 so that a leftward displacement (e.g., of the device 701 in the surgeon's hand) translates to a leftward displacement of the end effector 174 (from the surgeon's perspective), and so on, in all directions of the shared xyz coordinate system—unless, for example, the surgeon can grasp the handle assembly from the opposite direction.

[0217] The second solution of 'flipping the handle' is in Figure 21 As mentioned above, it has been flipped, that is, rotated Figure 19The handle component 702 is used to rotate the xyz coordinate system of the input device 701. According to an embodiment, the exemplary surgical system of the second solution is configured to change the orientation of the xyz coordinate system of the input device 701. In other words, the same input device 701 is used in the first reverse bending mode. Figure 17 The xyz space is shown in Figure 19 The input device 701 is rotated in the middle (due to the 'flip handle'), making it difficult to use in the second surgical mode (e.g., left is right, up is down, etc.). Figure 21 The new xyz coordinate system replaces Figure 19 The original but flipped xyz coordinate system of Figure 17 The original xyz coordinate system matches, but now it cannot be matched by Figure 17 The input device is oriented in the middle, but through the input Figure 21 Flip handle input device navigation in medium orientation.

[0218] Now refer to Figure 22A -C, these figures have been discussed above Figure 16 Certain embodiments of the present invention are discussed. Figure 22A An input device 701 is shown having a curved shape 2810. At a first time T1, the corresponding arm 102 has Figure 22B The arm 102 has a curved shape 2830 that does not clearly match the curved shape 2810 of the input device 701 at time = T1. Figure 22A -C) After manipulating the arm 102, the arm 102 bends in the reverse direction before time = T2, as shown in FIG. Figure 22C At this point, the curved shape 2830 of the arm 102 matches the curved shape 2810 of the input device 701. When this match is detected, as previously described in Figure 16 The surgical system described in the discussion of can be transitioned from a first (reverse bending) mode to a second (surgical) mode, wherein the apparatus 701 and the end effector 174 are oriented in the same xyz coordinate system, as shown in FIG. Figures 20 to 21 As shown in .

[0219] Figure 22A -C shows a reverse curved arm as a non-limiting example of a curved shape to be matched between the surgical arm and the user input device. In other examples, the curved shape to be mapped can include any useful curved shape of the surgical arm. Two illustrative examples of useful arm shape curves can be found in the discussion above. Figure 4 and Figure 5 Seen in.

[0220] First additional discussion

[0221] Now refer to Figure 23A -C.

[0222] In some embodiments, control of one or more surgical arms is achieved through one or more input arms, joysticks, control handles, and / or other components suitable for manipulation by a user (e.g., a surgeon), which is then translated into matching articulation of the surgical arm.

[0223] In the examples described herein, as in Figure 23A As mentioned in the flowchart of , some embodiments include dual control of the surgical arm. In some embodiments, the first user input (in this example, Figure 23B The thumb lever 4005 is used to introduce the surgical arm into the patient's body, for example, through the vagina, and then to reverse bend the surgical arm (4001). In some embodiments, the reverse bending of the surgical arm within the patient's body (e.g., bending backward) is performed to reduce the area where the surgical arm is located. Optionally, the reverse bending is performed during surgery to avoid obstacles, such as certain organs or parts thereof, such as the inner wall of the abdomen. Optionally, the reverse bending is performed to position the surgical arm in an orientation familiar to laparoscopic surgeons for performing the surgery.

[0224] In some embodiments, a second user input, in this example in the form of an input arm 4011 (eg, an avatar joystick), is then used to perform the remainder of the surgical procedure (4003).

[0225] In some embodiments, the thumb sticks 4005 are positioned near the console screen 4007, for example, on opposite sides of the screen. In some embodiments, each thumb stick 4005 includes a nipple-shaped controller 4009 that is shaped and sized to fit the user's thumb. In some embodiments, the nipple-shaped piece of the thumb stick is in a rest position when centered and is configured to spring back to the rest position when the thumb is released. In some embodiments, the degree of movement of the nipple-shaped piece relative to its central rest position determines the final speed of movement of the surgical arm. For example, the further the nipple-shaped piece is pushed away from its central rest position, the higher the speed of movement of the surgical arm (and vice versa - the closer the nipple-shaped piece is to its central rest position, the lower the speed of the arm).

[0226] In some embodiments, when the surgical arm is controlled by the thumb stick, the movement of one or more surgical arm joints (e.g., shoulder joint, wrist joint) is constrained. In some embodiments, all surgical arm joints except the elbow joint are prevented from moving, and only the elbow joint can bend and / or rotate. In some embodiments, linear movement of the surgical arm (as a single entity) is also enabled, for example, to advance or retract the arm. In some embodiments, the movement of the nipple-shaped piece actuates the bending and / or rotation of the elbow joint. In some embodiments, the linear movement of the arm is actuated by a separate actuator, for example, using buttons (e.g., 4006, 4008) configured along the body of the thumb stick 4005. In the example, button 4006 advances the surgical arm distally (e.g., into the abdomen); button 4008 retracts the surgical arm proximally.

[0227] In some embodiments, the input arm 4011 is locked in a rest position during use of the thumb lever, for example, by a solenoid lock. In some embodiments, the rest position of the input arm is selected to be a reverse bending position. Optionally, this position allows the surgeon to immediately resume the procedure after reverse bending using the thumb lever. In some embodiments, operation of the thumb lever is disabled while the input arm is being operated.

[0228] A potential advantage of using the thumbstick to navigate into the body and reverse bend the surgical arm while the selected arm joint (e.g., the shoulder joint) remains stationary may include reducing the bending radius of the surgical arm, thereby reducing the likelihood of encountering surrounding obstacles, such as the inner abdominal wall. Another potential advantage of using the thumbstick for the navigation and / or reverse bending process may include improved control of the surgical arm, for example, as compared to using the input arm for navigation and reverse bending, where the ergonomics of the handle may not be well suited to support the rotational movement that the surgeon needs to perform when grasping the handle to reverse bend.

[0229] In some embodiments, during the introduction of the surgical arm into the body, the surgical arm is straight (optionally providing a y for insertion through a cannula) and the input arm is in a stationary, locked, reverse-bend position. Optionally, after reverse-bending the surgical arm using the thumb lever, the surgeon releases the thumb lever and moves their hand to the input arm. Once the surgeon grasps the input arm and optionally lifts the input arm, control of the surgical arm is automatically gained and the surgeon can continue the procedure using the input arm. In some embodiments, when one or more input arm joints are locked by a solenoid lock, the surgeon lifting the input arm automatically releases the solenoid lock. Additionally or alternatively, the manual locking of the input arm joints can be released, for example by a sensor that detects the position of the input arm.

[0230] In some embodiments, the system (eg, a system processor) is configured to identify one or more positions of the input arms, for example, when the input arms are in their rest positions, and optionally display the current positions to a user.

[0231] Now refer to Figure 24 . In some embodiments, a haptic handle that provides force feedback to the user is used throughout the operation (a suitable example is the 'omega.7' haptic device available from ForceDimension of Nyon, Switzerland) for controlling the movement and articulation of the surgical arm. In some embodiments, the haptic handle is configured to provide counter-resistance to prevent the user from moving in a direction not supported by the surgical arm, such as bending the elbow joint of the surgical arm backward; contacting a joint (e.g., the elbow joint) with a different segment of the same arm; and / or other. In some embodiments, the handle is configured to provide counter-resistance that varies according to the current anatomical position and / or orientation of the surgical arm. In an example, the resistance may increase if the user attempts an anatomical area that is not allowed, such as an organ that should be avoided.

[0232] In some embodiments, the tactile handle is programmed to operate according to various control modes. Optionally, the control mode is selected according to the current stage of the surgical procedure. In some embodiments, switching between different modes is performed by one or more of a screen interface, one or more buttons on the console or handle, a foot pedal, and / or other.

[0233] In some embodiments, during the first stage of the operation, the haptic handle (5001) is used in a 'speed-control' mode while the surgical arm is introduced into the patient and, optionally, during the reverse bending of the surgical arm. Optionally, in speed-control mode, the relative movement of the handle relative to the handle's resting position sets the speed at which the surgical arm is moved. When the user moves the handle away from the resting position, the speed increases, and vice versa. For example, movement of the handle to the right of its resting position may cause the arm joint (e.g., elbow joint) to rotate to the right at a speed determined by the distance of the handle from its resting position. In some embodiments, in speed-control mode, the haptic handle is configured to provide an elastic (spring-like) counter-resistance to the user's movement. In some embodiments, in speed-control mode, a control algorithm is applied to convert the current configuration of the haptic handle into a speed command issued to an actuator (e.g., a motor) of the surgical arm, for example, to increase the rotational speed of one or more motor gears.

[0234] Potential advantages of using a speed control mode during introduction of the surgical arm into the body and optionally reverse bending of the surgical arm may include that the direction is reversed (e.g. up / down) during reverse bending, but this change can be ignored and movement can continue naturally because the final movement of the surgical arm is limited while the speed of movement is changed.

[0235] In some embodiments, during the second phase of the surgical procedure, and optionally during the remainder of the procedure, the haptic handle is set to 'position control' mode (5003). Optionally, in position control mode, the spatial position of the handle sets the corresponding position of the surgical arm. In position control mode, the user's displacement of the haptic handle is converted into a relative displacement command for the surgical arm. In some embodiments, the conversion of the haptic handle's displacement is controlled according to an algorithm. In some embodiments, the control is based on a known algorithm (e.g., an inverse Jacobi algorithm). Additionally or alternatively, in some embodiments, the control is based on a custom algorithm. In an example, the custom algorithm is set to scale the user's motion, for example to improve the accuracy of the movement. Such scaling may include amplifying the desired movement on the user's end by a selected factor to produce a similar non-amplified movement of the surgical arm. For example, in order for the arm to move a distance X, the user needs to move the handle by A*X (A>1). In another example, an algorithm is selected to filter the signal, for example, using a low-pass filter to reduce user hand tremors.

[0236] In some embodiments, in position control mode, a clutch mechanism is provided to allow the user to temporarily disconnect from the surgical arm (so that movement of the input haptic handle no longer controls the surgical arm). Optionally, when disconnected, the user can freely reposition the haptic handle. In an example, the user repositions the haptic handle to a position and / or orientation where the user is more comfortable performing and controlling the next movement.

[0237] In some embodiments, the degree of resistance a user experiences in response to movement of the handle is selected and controlled. In one example, a floating mode is provided in which the user encounters substantially no resistance and can freely move the handle in all directions. Additionally or alternatively, the level of resistance a user experiences can be adjusted, for example, so that the user experiences high resistance in response to one movement and low or no resistance in response to another movement.

[0238] In some embodiments, the amount of resistance is controlled based on the anatomical location of the surgical arm. For example, high resistance can be set where an obstacle is found near the surgical arm (e.g., the abdominal wall). In a specific example, if an obstacle is found on the right side of the surgical arm, the user may experience high resistance in response to moving the handle to the right; if no obstacle is found on the left side of the arm, the user may experience low or no resistance in response to moving the handle to the left. Optionally, the degree of resistance is defined by the settings system, such as to generate wall-type resistance, rubber-type resistance, sand-type resistance, and / or other.

[0239] Second additional discussion

[0240] According to an embodiment, a method for operating a surgical system is disclosed, wherein the surgical system includes (i) an articulated robotic arm, the articulated robotic arm including a plurality of arm joints, and (ii) first and second user input devices for controlling the arm, and (iii) a surgical end effector at a distal end of the arm. The method includes: (a) initiating operation of a surgical system in a reverse bending mode, wherein, with respect to bending and rotation of the arm joints: (i) a first user input device is activated to direct the bending and rotation of only a given one of the arm joints; and (ii) a second user input device is deactivated; (b) when in the reverse bending mode, in response to an electronic control output from the first user input device, reversely bending a distal portion of an articulated robotic arm by bending and rotating a given one of the arm joints so that the end effector is in a reverse bending operating position; (c) transitioning the surgical system from the reverse bending mode to a surgical mode to enable the second user input device with respect to bending and rotation of at least one arm joint in the arm other than the given one of the arm joints; and (d) when in the surgical mode, in response to an electronic control output from the second user input device, enabling bending and rotation of at least two of the arm joints according to the corresponding degrees of freedom of each arm joint, thereby moving the surgical end effector to perform one or more surgical actions.

[0241] In some embodiments, the surgical system may additionally include control circuitry effective to constrain actuation of arm joints other than a single arm joint when the surgical system is in the reverse flexion mode.

[0242] In some embodiments, the transitioning can include calibrating at least one of the input device relative to the position and orientation of the distal portion of the end effector or arm.

[0243] In some embodiments, the first input device may be configured for controlling actuation of a single arm joint and / or not configured for controlling actuation of arm joints other than the single arm joint.

[0244] In some embodiments, transitioning to surgical mode can be responsive to and / or dependent upon detecting that the arm is in a reversed flexion position.

[0245] A surgical system according to an embodiment includes: (a) an articulated robotic arm comprising a plurality of arm joints and a surgical end effector at a distal end of the arm; and (b) first and second user input devices for controlling the arm, wherein the surgical system is configured to operate asynchronously in the following modes: (i) a reverse bending mode of operation, wherein the distal portion of the articulated robotic arm is operable to reverse bend in response to an electronic control output from the first user input device to place the end effector in a reverse bending operational position, and (ii) a surgical mode, wherein at least two of the arm joints are operable to reverse bend in response to an electronic control output from the first user input device. The device is operable to bend and rotate in response to an electronic control output from a second user input device so as to thereby move the surgical end effector to perform one or more surgical actions such that: (A) when in the reverse bending mode, with respect to bending and rotation of the arm joints, the first user input device is activated to guide bending and rotation of only a given one of the arm joints and the second user input device is deactivated, and (B) when in the surgical mode, the second user input device is enabled with respect to bending and rotation of at least one arm joint in the arm other than the given one of the arm joints according to the corresponding degree of freedom of each arm joint.

[0246] In some embodiments, the surgical system may additionally include control circuitry effective to constrain actuation of arm joints other than a single arm joint when the surgical system is in the reverse flexion mode.

[0247] In some embodiments, the surgical system can be configured such that the transition includes at least one calibration input device relative to the position and orientation of the distal portion of the end effector or arm.

[0248] In some embodiments, the first input device may be configured for controlling actuation of a single arm joint and / or not configured for controlling actuation of arm joints other than the single arm joint.

[0249] In some embodiments, the surgical system can be configured such that transitioning to a surgical mode can be responsive to and / or dependent upon detecting that the arm is in a reversed flexion position.

[0250] According to an embodiment, a method of operating a surgical system is disclosed. According to the method, the surgical system includes (i) a user input device, and (ii) an articulated robotic arm including a plurality of arm joints and a surgical end effector at a distal end of the arm. The method includes: (a) initiating operation of the surgical system in a reverse bending mode, wherein a user input device is activated to guide the bending and rotation of only a given one of the arm joints with respect to bending and rotation of the arm joints; (b) when in the reverse bending mode, in response to an electronic control output from the user input device, reversely bending a distal portion of the articulated robotic arm by bending and rotating a given one of the arm joints so that the end effector is in a reverse bending operating position; (c) transitioning the surgical system from the reverse bending mode to a surgical mode to enable the user input device with respect to bending and rotation of at least one arm joint in the arm other than the given one of the arm joints; and (d) when in the surgical mode, in response to the electronic control output from the user input device, enabling bending and rotation of at least two of the arm joints according to the corresponding degrees of freedom of each arm joint, thereby moving the surgical end effector to perform one or more surgical actions.

[0251] In some embodiments of the method, the surgical system can further include control circuitry effective to constrain actuation of the arm joints other than a given one of the arm joints when the surgical system is in the reverse flexion mode. In some such embodiments, the constraining can be performed by deactivating actuation of the arm joints other than the given one of the arm joints.

[0252] In some embodiments, a user input device can control actuation of multiple arm joints in both reverse bending mode and surgical mode.

[0253] In some embodiments, the user input device may be prevented from generating or transmitting a control output that would control actuation of arm joints other than a given one of the arm joints.

[0254] In some embodiments, transitioning to surgical mode can be responsive to and / or dependent upon detecting that the arm is in a reversed flexion position.

[0255] In some embodiments, the transitioning can include calibrating the user input device relative to at least one of the position and orientation of the end effector or the distal portion of the arm.

[0256] In some embodiments, the method can further include, after operating in the second mode, unbending the distal end of the arm to place the arm in an unbent position.

[0257] According to an embodiment, a surgical system is disclosed comprising: (a) a user input device; and (b) an articulated robotic arm comprising (i) a plurality of arm joints, and (ii) a surgical end effector at a distal end of the arm, wherein the surgical system is configured to operate asynchronously in the following modes: in (A) a reverse bending mode of operation, wherein the distal portion of the articulated robotic arm reversely bends in response to an electronic control output from the user input device so as to place the end effector in a reverse bending operating position, and in (B) a surgical mode, wherein at least two of the arm joints bend and rotate in response to an electronic control output from the user input device so as to thereby move the surgical end effector to perform one or more surgical actions, such that: (A) when in the reverse bending mode, the user input device is activated to guide the bending and rotation of only a given one of the arm joints with respect to the bending and rotation of the arm joints, and (B) when in the surgical mode, the user input device is enabled with respect to the bending and rotation of at least one arm joint in the arm other than a given one of the arm joints according to the corresponding degree of freedom of each arm joint.

[0258] In some embodiments, the surgical system can further include control circuitry that, when the surgical system is in the reverse flexion mode, effectively constrains actuation of the arm joints other than a given one of the arm joints. In some such embodiments, the constraining can be performed by deactivating actuation of the arm joints other than the given one of the arm joints.

[0259] In some embodiments, the user input device can be effective to control actuation of multiple arm joints in the reverse bending mode and the surgical mode.

[0260] In some embodiments, the surgical system can be configured such that a user input device can be prevented from generating or transmitting a control output that would control actuation of an arm joint other than a given one of the arm joints.

[0261] In some embodiments, the surgical system can be configured such that transitioning to a surgical mode can be responsive to and / or dependent upon detecting that the arm is in a reversed flexion position.

[0262] In some embodiments, the surgical system can be configured such that the transition can include calibrating the user input device relative to at least one of the position and orientation of the end effector or the distal portion of the arm.

[0263] In some embodiments, the surgical system can be further configured to achieve unbending of the distal end of the arm after operating in the second mode, such that the arm is in an unbent position.

[0264] According to an embodiment, a method for operating a surgical system is disclosed, wherein the surgical system includes (i) an articulated robotic arm, the articulated robotic arm including a surgical end effector and a plurality of arm joints at its distal end, and (ii) an input device array of one or more user input devices, wherein the arm joint is configured to bend and rotate in response to electronic control outputs from the one or more user input devices of the input device array. The method includes: (a) commencing operation of the surgical system in a first operating mode defined relative to a given single arm joint in the arm joints, wherein the first operating mode prevents actuation of any arm joint in the arm that is not the given single arm joint and allows actuation of the single arm joint to be controlled to cause bending and rotation of the single arm joint; (b) when the surgical system is in the first operating mode, (i) in response to control signals generated by one or more user input devices in the input device array, reversely bend the distal end of the arm by bending and rotating the single arm joint so that the end effector is in a reverse bending operating position, and (ii) monitoring a state of the robotic arm to detect whether the arm is in the reverse bending position. The method further includes: (c) in response to and depending on detecting that the arm is in a reverse bending position, transitioning the operation of the surgical system from a first operating mode to a second operating mode, in which the system is capable of controlling the rotation of at least one arm joint excluding the first mode based on the corresponding degrees of freedom of each arm joint; and (d) operating the surgical system in the second operating mode to perform surgical actions using the end effector.

[0265] In some embodiments, the surgical system may further include control circuitry that effectively constrains actuation of arm joints other than a single arm joint when the surgical system is in a first operating mode. In some such methods, the constraint may be performed by deactivating actuation of arm joints other than a single arm joint in the arm. In some such embodiments, an input device may be prevented from generating or transmitting a control output that may control actuation of arm joints other than a single arm joint in the arm. In some such embodiments, the constraint may include the ability to deactivate a first input device. In some such embodiments, the constraint may be performed (i) when the surgical system is in the first operating mode, a first input device of the input device array controls actuation of a single arm joint, and when the surgical system is in the second operating mode, a second input device controls actuation of multiple arm joints, and / or (ii) by providing a first input device that is configured to control actuation of a single arm joint and is not configured to control actuation of arm joints other than a single arm joint.

[0266] In some embodiments, the transitioning can include calibrating at least one of the input device relative to the position and orientation of the distal portion of the end effector or arm.

[0267] In some embodiments, a first input device of the input device array can control actuation of a single arm joint when the surgical system is in a first operating mode, and / or a second input device of the input device array can control actuation of multiple arm joints when the surgical system is in a second operating mode.

[0268] In some embodiments, the transitioning can include calibrating the second input device relative to at least one of a position and an orientation of the end effector or the distal portion of the arm.

[0269] In some embodiments, a single user input device can control actuation of multiple arm joints in both the first mode of operation and the second mode of operation.

[0270] In some embodiments of the method, the surgical system can further include a console comprising a display screen, and at least one user input device of the array of input devices is disposed on or near the display screen.

[0271] In some embodiments, an additional user input device for linear advancement and retraction of the actuator arm may be provided on, co-located with, or positioned proximate to at least one user input device of the input device array.

[0272] In some embodiments, the reverse bending operative position may be at or near the surgical work site.

[0273] In some embodiments, operation in the second mode may place the arms in a reverse bent position.

[0274] In some embodiments, the method can further include, after operating in the second mode: unbending the distal end of the arm to place the arm in an unbent position.

[0275] According to an embodiment, a surgical system for use with a surgical end effector and configured to operate asynchronously in a first operating mode and a second operating mode includes: (a) an input device array of one or more user input devices; (b) an articulated robotic arm, the articulated robotic arm including a surgical end effector at its distal end and a plurality of arm joints, the arm joints being configured to bend and rotate in response to control signals generated by one or more input devices of the input device array, wherein: (i) the first operating mode is defined relative to a given single arm joint of the arm joints, wherein the first operating mode prevents actuation of any arm joint in the arm that is not the given single arm joint, and allows actuation of the single arm joint to be controlled to cause bending of the single arm joint; and rotation, (ii) the system is configured to, in response to electronic control outputs from one or more user input devices of the input device array, reversely bend the distal end of the arm when in a first operating mode by actuating a single arm joint to cause bending and rotation of the single arm joint so as to place the surgical end effector in a reverse bending operating position, (iii) a second operating mode is defined relative to the plurality of arm joints, wherein the second operating mode is capable of controlling actuation of at least one arm joint excluding the first mode according to the corresponding degree of freedom of each arm joint, and (iv) the system is configured to transition from the first operating mode to the second operating mode in response to and depending on detecting that the arm is in the reverse bending position and, when in the second operating mode, perform a surgical action using the end effector.

[0276] In some embodiments, the surgical system may further include control circuitry that effectively constrains actuation of arm joints other than a single arm joint when the surgical system is in a first operating mode. In some such embodiments, the constraint may be performed by disabling actuation of arm joints other than a single arm joint in the arm. In some such embodiments, an input device may be prevented from generating or transmitting a control output that may control actuation of arm joints other than a single arm joint in the arm. In some such embodiments, the constraint may include the ability to deactivate a first input device. In some such embodiments, the constraint may be performed (i) when the surgical system is in the first operating mode, a first input device of the input device array controls actuation of a single arm joint, and when the surgical system is in the second operating mode, a second input device controls actuation of multiple arm joints, and / or (ii) by providing a first input device that is configured to control actuation of a single arm joint and is not configured to control actuation of arm joints other than a single arm joint.

[0277] In some embodiments, the system can be configured such that the transition can include calibrating at least one of the input device relative to the position and orientation of the distal portion of the end effector or arm.

[0278] In some embodiments, a first input device of the input device array can effectively control actuation of a single arm joint when the surgical system is in a first operating mode, and / or a second input device of the input device array can effectively control actuation of multiple arm joints when the surgical system is in a second operating mode.

[0279] In some embodiments, the system can be configured such that the transitioning can include calibrating the second input device relative to at least one of the position and orientation of the end effector or the distal portion of the arm.

[0280] In some embodiments, a single user input device can be effective to control actuation of multiple arm joints in both the first mode of operation and the second mode of operation.

[0281] In some embodiments, the surgical system may further include a console comprising a display screen, and at least one user input device of the array of input devices is disposed on or near the display screen.

[0282] In some embodiments, an additional user input device for linear advancement and retraction of the actuator arm may be provided on, co-located with, or positioned proximate to at least one user input device of the input device array.

[0283] In some embodiments, the reverse bending operative position may be at or near the surgical work site.

[0284] In some embodiments, operation in the second mode may place the arms in a reverse bent position.

[0285] In some embodiments, the surgical system can be further configured to achieve, after operating in the second mode, unbending the distal end of the arm so that the arm is in an unbent position. According to an embodiment, a method of operating a surgical system is disclosed, the surgical system comprising (i) an articulated robotic arm comprising a plurality of arm joints and a surgical end effector at the distal end of the arm, and (ii) an input device array for controlling one or more user input devices of the arm. The method comprises: (a) in response to an electronic control output from the input device array, reversely bending the distal portion of the articulated robotic arm by bending and rotating a given arm joint in the arm joints so that the end effector is in a reverse bending operating position without bending or rotating any arm joint other than the given arm joint in the arm joints; (b) in response to and depending on detecting that the arm is in a reverse bending position, achieving bending and rotation of at least two of the arm joints according to the corresponding degrees of freedom of each arm joint in response to the electronic control output from the input device array, thereby moving the surgical end effector to perform one or more surgical actions.

[0286] In some embodiments of the method, the surgical system can further include control circuitry effective to constrain actuation of arm joints other than given ones of the arm joints during reverse bending.

[0287] In some embodiments, the method may further include unbending the distal end of the arm to place the arm in an unbent position after performing the one or more surgical actions.

[0288] According to an embodiment, a surgical system includes: (a) an articulated robotic arm, the articulated robotic arm including (i) multiple arm joints and (ii) a surgical end effector at a distal end of the arm, and (b) an input device array of one or more user input devices for controlling the arm, wherein the surgical system is configured to: (i) in response to electronic control outputs from the input device array, reversely bend the distal portion of the articulated robotic arm by bending and rotating a given arm joint in the arm joints so that the end effector is in a reverse bending operating position without bending or rotating any arm joint other than the given arm joint, and (ii) in response to and depending on detecting that the arm is in the reverse bending position and in response to electronic control outputs from the input device array, achieve bending and rotation of at least two of the arm joints according to the corresponding degrees of freedom of each arm joint, thereby moving the surgical end effector to perform one or more surgical actions.

[0289] In some embodiments, the surgical system may further include control circuitry effective to constrain actuation of arm joints other than given ones of the arm joints during reverse bending.

[0290] In some embodiments of the method, the surgical system can further include control circuitry effective to constrain actuation of arm joints other than given ones of the arm joints during reverse bending.

[0291] In some embodiments, the method may further include unbending the distal end of the arm to place the arm in an unbent position after performing the one or more surgical actions.

[0292] According to an embodiment, a method of using a surgical system is disclosed. The method includes: (a) providing an articulated robotic arm having a surgical end effector at a distal end thereof, the arm comprising a plurality of arm segments connected in series by corresponding plurality of arm joints, the arm joints being configured to bend and rotate in response to an electronic control output from a user input device, wherein the providing causes displacement of the end effector; (b) manipulating the end effector to a reverse bending operating position while operating in a first input mode, in which the displacement of the input device or a displaceable portion thereof is converted into a velocity of at least one of (i) bending of the arm joint and (ii) rotation of the arm joint; (c) in response to and dependent on detecting that the end effector is in the reverse bending operating position, transitioning from operating in the first input mode to operating in a second input mode, in which the displacement of the input device or a displaceable portion thereof is converted into a corresponding displacement of at least one arm segment; and (d) after the transition and while operating in the second input mode, performing a surgical action using the end effector.

[0293] In some embodiments, a single user input device may be used in both the first input mode and the second input mode.

[0294] In some embodiments, the first input device may be used in a first input mode, and / or the second input device may be used in a second input mode.

[0295] In some embodiments, additional user input devices may be used to actuate the linear advancement and retraction of the arm.

[0296] In some embodiments, the reverse bending operative position may be at or near the surgical work site.

[0297] According to an embodiment, a surgical system for use with a surgical end effector includes: (a) an articulated robotic arm having a surgical end effector at its distal end, the arm including a plurality of arm segments connected in series by corresponding plurality of arm joints, the arm joints being configured to bend and rotate in response to electronic control outputs from a user input device; and (b) an array of one or more input devices for controlling the arm, wherein the surgical system is configured to: (i) operate asynchronously in the following operating modes: in (A) a first input mode, wherein displacement of the input device or its displaceable portion is converted into a velocity of bending and rotation of the arm joint, and in (B) a second input mode, wherein displacement of the input device or its displaceable portion is converted into a corresponding displacement of at least one arm segment; (ii) when operating in the first input mode, manipulating the end effector to a reverse bending operating position during a first phase of end effector displacement; and (iii) when operating in the second input mode, performing a surgical action using the end effector.

[0298] In some embodiments, the system may be configured such that a single user input device may be used in both the first input mode and the second input mode.

[0299] In some embodiments, the system may be configured such that a first input device is used in a first input mode, and / or a second input device is used in a second input mode.

[0300] In some embodiments, the system can be configured so that additional user input devices can be used to actuate the linear advancement and retraction of the arm.

[0301] In some embodiments, the reverse bending operative position may be at or near the surgical work site.

[0302] The present invention has been described using detailed descriptions of its embodiments, which are provided by way of example and are not intended to limit the scope of the invention. The described embodiments include different features, not all of which are required in all embodiments of the invention. Some embodiments of the invention utilize only some of the features or possible combinations of these features. Those skilled in the art to which the invention pertains will recognize variations of the described embodiments of the invention as well as embodiments of the invention that include different combinations of the features mentioned in the described embodiments.

[0303] Any feature or combination of features described in this document may be combined with any feature or combination of features described in U.S. patent application serial number 16 / 377,280, filed on April 8, 2019, and published as U.S. Patent Publication US201902314445; U.S. patent application serial number 15 / 915,237, filed on March 8, 2018, and published as U.S. Patent Publication US20180256246A1; and U.S. patent application serial number 15 / 454,123, filed on March 9, 2017, and published as U.S. Patent Publication US20170258539A1; U.S. patent application serial number 15 / 501,862, filed on February 6, 2017, and published as U.S. Patent Publication US20170239005A1; all of which are hereby incorporated by reference herein as if fully set forth in their entirety.

[0304] In the specification and claims of the present disclosure, each of the verbs 'comprise,' 'include,' and 'have,' and their variations, is used to indicate that the object of the verb is not necessarily a complete list of components, parts, elements, or parts of the subject of the verb. As used herein, the singular forms 'a,' 'an,' and 'the' include plural referents unless the context clearly indicates otherwise. For example, the term 'a marker' or 'at least one marker' may include a plurality of markers.

Claims

1. A non-surgical method of operating a surgical system, the surgical system comprising (i) an articulated robotic arm having a surgical end effector at a distal end thereof, the articulated robotic arm including a plurality of arm joints with respective degrees of freedom, and (ii) an input device array of one or more user input devices configured to control bending and rotation of the arm joints, the method comprising: a. in response to an electronic control output from a first user input device, reversely bend the distal end of the articulated robotic arm to transfer the end effector to a reverse bending operating position, thereby converting the user input into bending and rotation of the arm joint in the same direction using a first coordinate transformation matrix; b. in response to and depending on detecting that the end effector is in the reverse bending operating position, transitioning to a second coordinate transformation matrix based on the reverse bending position of the end effector; as well as c. After the transition and in response to the electronic control output from the second user input device, convert the user input into a same-direction bend and rotation of the arm joint using the second coordinate transformation matrix. 2 . The method of claim 1 , wherein the first and second user input devices are the same user input device, and the first and second coordinate transformation matrices are not the same 3D coordinate transformation matrix.

3. The method of claim 1, wherein the first and second user input devices are not the same user input device, and the first and second coordinate transformation matrices are not the same 3D coordinate transformation matrix.

4. The method of any one of claims 1 to 3, wherein before the transition and during the reverse bending, a proximal displacement of the first user input device or a portion thereof is converted into a proximal displacement of the end effector, and after the transition and with the end effector in the reverse bent position, a proximal displacement of the second user input device or a portion thereof is converted into a distal displacement of the end effector.

5. The method of any one of claims 1 to 3, wherein the surgical system further comprises a control circuit effective to cause the transition.

6. A method as claimed in any one of claims 1 to 3, wherein the first user input means is deactivated after the transition.

7. A surgical system for use with a surgical end effector, the system comprising: a. an array of one or more user input devices; as well as b. an articulated robotic arm having a surgical end effector at a distal end thereof, the articulated robotic arm comprising a plurality of arm segments connected by a plurality of arm joints, the arm joints being configured to bend and rotate in response to control signals generated by a user input device, wherein the surgical system is configured to: i. in response to an electronic control output from a first user input device, reversely bend the distal end of the articulated robotic arm to transfer the end effector to a reverse bend operating position, thereby converting the user input into bending and rotation of the arm joint in the same direction using a first coordinate transformation matrix; ii. in response to and dependent on detecting that the current orientation of the end effector corresponds to the current orientation of the second user input device, transitioning to a second coordinate transformation matrix based on the current orientation of the end effector, and iii. After the transition and in response to the electronic control output from the second user input device, performing a surgical action using the end effector, converting the user input into same-direction bending and rotation of the arm joint using the second coordinate transformation matrix.

Images