Robot collision boundary detection

Abstract

The technology relates to determining a region associated with an object to assist in controlling a robotic arm. For example, a system can determine that a robotic arm is positioned adjacent to an object in an environment. The system can determine a region in the environment associated with the object based at least in part on a position of a distal end of the robotic arm. The system can control the robotic arm or another robotic arm to move through the environment based at least in part on the region.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims priority to U.S. Provisional Patent Application No. 63 / 073,860, filed September 2, 2020, entitled "ROBOTIC COLLISION BOUNDARY DETERMINATION," the disclosure of which is incorporated herein by reference in its entirety.

[0002] FIELD OF THE INVENTION TECHNICAL FIELD The present disclosure relates to the field of medical devices and procedures. [Background technology]

[0003] Various medical procedures involve the use of one or more medical devices to examine and / or treat a patient. In some cases, multiple systems / devices are implemented to control the medical devices and perform the procedure on the patient. Improper use of such systems, devices, and / or medical devices can adversely affect the patient's health and / or the effectiveness of the procedure. Summary of the Invention [Means for solving the problem]

[0004] In some embodiments, the present disclosure relates to a system comprising: a robotic system including a first robotic arm configured to couple to a medical device; and a control circuit communicatively coupled to the robotic system, wherein the control circuit is configured to: receive input data indicating that the first robotic arm is positioned adjacent to an object in an environment; determine a region in the environment associated with the object based at least in part on a position of a distal end of the first robotic arm; and control at least one of the first robotic arm or a second robotic arm of the robotic system to move through the environment without moving into the region.

[0005] In some embodiments, the first robotic arm may be configured to operate in an admittance control mode in which user manipulation of the first robotic arm causes movement of the first robotic arm. Input data may be received when the first robotic arm is operating in the admittance control mode. Further, in some embodiments, the control circuitry may be configured to determine a region based at least in part on a position of the robotic system. The region may include the object and exclude the robotic system.

[0006] In some embodiments, the input data indicates that a first robotic arm is positioned adjacent a first edge of the object. The control circuitry may be configured to receive additional input data indicating that a second robotic arm is positioned adjacent a second edge of the object and to determine the region based at least in part on a position of a distal end of the second robotic arm. The control circuitry may be configured to determine the region by determining a first boundary of the region based at least in part on the position of the distal end of the first robotic arm and determining a second boundary of the region based at least in part on the position of the distal end of the second robotic arm.

[0007] In some embodiments, the input data indicates that the first robotic arm is positioned adjacent a first edge of the object. The control circuitry may be configured to receive additional input data indicating that the first robotic arm is positioned adjacent a second edge of the object, and to determine the region based at least in part on a position of the distal end of the first robotic arm when the input data is received and a position of the distal end of the first robotic arm when the additional input data is received.

[0008] In some embodiments, the control circuitry is further configured to display a visual representation of the region, receive adjustment input data including adjustments to the visual representation, and update the region based at least in part on the adjustments to the visual representation. Additionally, in some embodiments, the control circuitry is further configured to set the system to a treatment mode for performing a medical procedure, determine that at least one of the first robotic arm or the second robotic arm experienced a collision, and update the region based on at least one of a position of the distal end of the first robotic arm or a position of the second robotic arm when the collision occurred.

[0009] In some embodiments, the present disclosure relates to a method that includes manually moving a first robotic arm; a control circuit receiving input data indicating that the first robotic arm is positioned adjacent to one or more objects; the control circuit determining a collision area based at least in part on a position of an end of the first robotic arm; and, based at least in part on the collision area, the control circuit controlling movement of at least one of the first robotic arm or a second robotic arm to perform a medical procedure.

[0010] In some embodiments, the input data indicates that the first robotic arm is positioned adjacent a first edge of the one or more objects. The method may further include receiving additional input data indicating that a second robotic arm is positioned adjacent a second edge of the one or more objects. Determining the collision region may further be based at least in part on a position of an end of the second robotic arm. Determining the collision region may include defining a first plane based at least in part on the position of the end of the first robotic arm, defining a second plane based at least in part on the position of the end of the second robotic arm, and determining the region based at least in part on the first plane, the second plane, and an intersection of the first and second planes.

[0011] In some embodiments, the input data indicates that the first robotic arm is positioned adjacent a first edge of the one or more objects. The method may further include receiving additional input data indicating that the first robotic arm is positioned adjacent another edge of the one or more objects. Determining the collision region may include determining the collision region based at least in part on a position of the end of the first robotic arm when the input data is received and a position of the end of the first robotic arm when the additional input data is received. Determining the collision region may include defining a first plane based at least in part on the position of the end of the first robotic arm when the input data is received, defining a second plane based at least in part on the position of the end of the first robotic arm when the additional input data is received, and determining the region based at least in part on the first plane, the second plane, and an intersection of the first and second planes.

[0012] In some embodiments, at least one of the first robotic arm or the second robotic arm is configured to connect to a medical device. The method may further include receiving input control data from an input device regarding movement of the medical device and determining that the input control data is associated with at least one of the first robotic arm or the second robotic arm moving into a collision zone. Controlling the movement of at least one of the first robotic arm or the second robotic arm may include preventing at least one of the first robotic arm or the second robotic arm from moving into the collision zone. The method may further include displaying a notification indicating that the input control data is associated with movement into the collision zone and receiving additional input data indicating whether to proceed into the collision zone. Controlling the movement of at least one of the first robotic arm or the second robotic arm may be based at least in part on the additional input data.

[0013] In some embodiments, the first robotic arm is connected to a robotic system. Receiving input data and controlling movement of at least one of the first robotic arm or the second robotic arm may occur while the robotic system is located in the same parked position. Further, in some embodiments, the end of the first robotic arm may be an end effector end of the first robotic arm.

[0014] In some embodiments, the present disclosure relates to a control system comprising: a communication interface configured to communicate with a first robotic arm; and a control circuit communicatively coupled to the communication interface. The control circuit may be configured to: determine that the first robotic arm is positioned adjacent a first edge of one or more objects in an environment; determine a collision region relative to the environment based at least in part on a position of a distal end of the first robotic arm; and control movement of at least one of the first robotic arm or a second robotic arm based at least in part on the collision region. At least one of the first robotic arm or the second robotic arm may be configured to couple to a medical device.

[0015] In some embodiments, the control circuitry may be further configured to receive, from the input device, input control data for controlling the medical device and determine that the input control data is associated with at least one of the first robotic arm or the second robotic arm moving into a collision region. The control circuitry may be configured to control movement of at least one of the first robotic arm or the second robotic arm by preventing movement of at least one of the first robotic arm or the second robotic arm from moving into the collision area. Furthermore, in some embodiments, the control circuitry is further configured to determine that the second robotic arm is positioned adjacent a second edge of the one or more objects, and the control circuitry is configured to determine the collision region based at least in part on a position of the distal end of the first robotic arm and a position of the distal end of the second robotic arm.

[0016] In some embodiments, the control circuitry is further configured to determine that the first robotic arm is positioned adjacent a second edge of the one or more objects. The control circuitry may be configured to determine a collision region based at least in part on a position of the distal end of the first robotic arm at the first edge of the one or more objects and a position of the distal end of the first robotic arm at the second edge of the one or more objects. Furthermore, in some embodiments, the control circuitry is further configured to display a visual representation of the collision region, receive adjustment input data including adjustments to the visual representation, and update the collision region based at least in part on the adjustments to the visual representation.

[0017] In some embodiments, the control circuitry is further configured to set the control system to a treatment mode to perform a medical treatment; determine that at least one of the first robotic arm or the second robotic arm has experienced a collision; and update the collision region based on at least one of the position of the distal end of the first robotic arm or the position of the second robotic arm when the collision occurred.

[0018] In some embodiments, the present disclosure relates to one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by control circuitry, cause the control circuitry to perform operations including: determining that a first robotic arm is positioned adjacent a first edge of one or more objects in an environment, determining a collision area relative to the environment based at least in part on a position of a distal end of the first robotic arm, and controlling movement of at least one of the first robotic arm or a second robotic arm based at least in part on the collision area. At least one of the first robotic arm or the second robotic arm may be configured to couple to a medical device.

[0019] In some embodiments, the operations may further include determining that the second robotic arm is positioned adjacent a second edge of the one or more objects. Determining the collision area may further be based at least in part on a position of a distal end of the second robotic arm. Determining the collision area may include defining a first plane based at least in part on the position of the distal end of the first robotic arm, defining a second plane based at least in part on the position of the distal end of the second robotic arm, and determining the collision area based at least in part on the first plane, the second plane, and an intersection of the first and second planes.

[0020] In some embodiments, the operations further include determining that the first robotic arm is positioned adjacent a second edge of the one or more objects. Determining the collision area may be based at least in part on a position of the distal end of the first robotic arm at the first edge of the one or more objects and a position of the distal end of the first robotic arm at the second edge of the one or more objects. Determining the collision area may include defining a first plane based at least in part on the position of the distal end of the first robotic arm at the first edge of the one or more objects, defining a second plane based at least in part on the position of the distal end of the first robotic arm at the second edge of the one or more objects, and determining the collision area based at least in part on the first plane, the second plane, and an intersection of the first and second planes.

[0021] In some embodiments, the operations further include receiving input control data from an input device regarding movement of the medical device and determining that the input control data is associated with movement of at least one of the first robotic arm or the second robotic arm moving into a collision area. Controlling the movement of at least one of the first robotic arm or the second robotic arm may include preventing movement of at least one of the first robotic arm or the second robotic arm from moving into the collision area. Further, in some embodiments, determining that the first robotic arm is positioned adjacent a first edge of one or more objects includes receiving the input control data when the first robotic arm is operating in an admittance control mode. The input data may indicate that the first robotic arm is positioned adjacent a first edge of one or more objects.

[0022] For purposes of summarizing this disclosure, certain aspects, advantages, and features have been described. It is to be understood that not necessarily all such advantages may be achieved in accordance with any particular embodiment. Thus, the disclosed embodiments may be practiced in a manner that achieves or optimizes one advantage or group of advantages taught herein without necessarily achieving other advantages that may be taught or suggested herein. [Brief explanation of the drawings]

[0023] Various embodiments are shown in the accompanying drawings for purposes of illustration and should not be construed as limiting the scope of the present disclosure in any way. In addition, various features of different disclosed embodiments may be combined to form further embodiments that are part of the present disclosure. Throughout the drawings, reference numerals may be reused to indicate correspondence between referenced elements. [Figure 1] FIG. 1 illustrates an example medical system for performing various medical procedures in accordance with one or more embodiments. [Figure 2] 2 is a perspective view of an impact region and other aspects of the medical system of FIG. 1 in accordance with one or more embodiments. [Figure 3] FIG. 2 illustrates example details of the control system and robotic system of FIG. 1 in accordance with one or more embodiments. [Figure 4] FIG. 2 illustrates example details of the robotic system of FIG. 1 in accordance with one or more embodiments. [Figure 5] FIG. 1 illustrates a top view of a medical system as a physician moves the robotic system, in accordance with one or more embodiments. [Figure 6] FIG. 6 is a plan view of the medical system of FIG. 5 as a physician positions the robotic arm adjacent to an object, in accordance with one or more embodiments. [Figure 7] FIG. 6 is a plan view of the medical system of FIG. 5 as a physician positions another robotic arm adjacent to an object, in accordance with one or more embodiments. [Figure 8]FIG. 6 is a plan view of the medical system of FIG. 5 in an alternative example in which a physician positions the robotic arm at a further position adjacent to the object, in accordance with one or more embodiments. [Figure 9] FIG. 6 is a plan view of the medical system of FIG. 5 with an impingement area, according to one or more embodiments. [Figure 10] 10A-10C illustrate example interfaces for visualizing and / or configuring collision areas, according to one or more embodiments. [Figure 11] 1 is a flow diagram of an example process for determining a region associated with an object, according to one or more embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0024] The headings provided herein are for convenience only and do not necessarily affect the scope or meaning of the disclosure. While specific embodiments and examples are disclosed below, the subject matter extends beyond the specifically disclosed embodiments to other alternative embodiments and / or uses, as well as to modifications and equivalents thereof. Accordingly, the scope of claims that may arise from this specification is not limited by any of the specific embodiments described below. For example, in any method or process disclosed herein, the acts or operations of the method or process may be performed in any suitable order and are not necessarily limited to any particular disclosed order. Various operations may be described in sequence as multiple separate operations, in a manner that may be helpful in understanding a particular embodiment. However, the order of description should not be construed to imply that these operations are order-dependent. Furthermore, structures, systems, and / or devices described herein may be embodied as integrated or separate components. For purposes of comparing various embodiments, certain aspects and advantages of these embodiments are discussed. Not necessarily all such aspects or advantages are achieved by any particular embodiment. Thus, for example, various embodiments may be implemented in a manner that achieves or optimizes one advantage or group of advantages as taught herein without necessarily achieving other aspects or advantages that may also be taught or suggested herein.

[0025] Certain standard anatomical terms of location may be used herein to refer to animal (i.e., human) anatomical structures with respect to preferred embodiments. While certain spatially relative terms, such as "outer," "inner," "superior," "lower," "below," "upper," "vertical," "horizontal," "top," "bottom," and similar terms, are used herein to describe the spatial relationship of one device / element or anatomical structure to another, it should be understood that these terms are used herein for ease of description to describe positional relationships between elements / structures, as illustrated in the drawings. It should be understood that spatially relative terms are intended to encompass different orientations of elements / structures during use or operation in addition to the orientation shown in the drawings. For example, when an element / structure is described as being "above" another element / structure, it may refer to a position below or beside such other element / structure relative to the intended patient or alternative orientations of the element / structure, and vice versa.

[0026] Overview Medical procedures are often performed on patients in environments that include multiple objects, such as hospital beds, medical equipment, carts, surgical systems, etc. In some embodiments, robotically-assisted medical procedures can be performed in such environments, and robotic tools can enable physicians to perform endoscopic and / or percutaneous procedures. For example, the robotic tools can engage and / or control one or more medical devices to access and / or perform procedures at target sites within the patient. However, because the robotic tools may not be aware of the location of objects in the environment, the robotic tools may be susceptible to collisions with objects. For example, the robotic tools may collide with a hospital bed, a patient on the hospital bed, medical equipment, and / or other objects in the environment while performing a medical procedure. This may cause harm to the patient and / or reduce the effectiveness of the procedure.

[0027] The present disclosure relates to systems, devices, and methods for avoiding collisions with objects in an environment to assist in the performance of a medical procedure. For example, a robotic system may include one or more robotic arms configured to couple to one or more medical devices to perform a procedure. During configuration of the robotic system or at other times, the one or more robotic arms may be positioned adjacent to one or more objects in the environment. For example, a user may manually move the one or more robotic arms to contact one or more edges of the one or more objects. A collision region associated with the one or more objects may then be determined based on the position of the one or more robotic arms, e.g., the distal ends of the robotic arms. The one or more robotic arms and / or associated medical devices may be controlled based on the collision region, e.g., by moving through the environment without moving into the collision region, moving into the collision region upon confirmation by the user, etc. By doing so, the robotic system may avoid collisions with objects in the environment / workspace (due to undesired movement of the robotic arms / medical devices), ultimately preventing harm to patients, improving the effectiveness of the procedure, etc.

[0028] While certain aspects of the present disclosure are described herein in the context of renal, urinary, and / or nephrological procedures, such as kidney stone removal / treatment procedures, it should be understood that such context is provided for convenience and that the concepts disclosed herein may be applied to any suitable medical procedure. For example, the following description may also be applied to other surgical / medical operations or procedures involving the removal of an object from a patient, including any object that can be removed from a treatment site or a patient's body cavity (e.g., esophagus, ureter, intestine, eye, etc.) via percutaneous and / or endoscopic access, such as gallbladder stone removal, lung (pulmonary / transthoracic) tumor biopsy, or cataract removal. However, as previously mentioned, a description of the renal / urinary anatomical structure and associated medical problems and procedures is provided below to aid in explaining the concepts disclosed herein.

[0029] Medical system example FIG. 1 illustrates an example medical system 100 for performing various medical procedures according to aspects of the present disclosure. The medical system 100 includes a robotic system 110 configured to engage with and / or control one or more medical devices (not shown) to perform a procedure on a patient 120. The medical system 100 also includes a control system 130 configured to interface with the robotic system 110, provide information related to the procedure, and / or perform various other operations. For example, the control system 130 may include display(s) 132 that show specific information to assist a physician 140. The medical system 100 may include a table 150 (e.g., a bed) configured to hold the patient 120. Various actions are described herein as being performed by the physician 140. These actions may be performed directly by the physician 140, by a user under the direction of the physician 140, by another user (e.g., a technician), a combination thereof, and / or by any other user.

[0030] The control system 130 can be coupled to the robotic system 110 and can operate in cooperation with the robotic system 110 to perform a medical procedure on the patient 120. For example, the control system 130 can communicate with the robotic system 110 via a wireless or wired connection to control a medical device connected to the robotic system 110, receive image(s) captured by the medical device (e.g., a scope), etc. Additionally or alternatively, the control system 130 can provide fluid to the robotic system 110 via one or more fluid passageways, provide power to the robotic system 110 via one or more electrical connections, provide optics to the robotic system 110 via one or more optical fibers or other components, etc. In some embodiments, the control system 130 can communicate with the medical device and receive sensor data (via the robotic system 110 and / or directly from the medical device). The sensor data can indicate or be used to determine the position and / or orientation of the medical device. Additionally, in some embodiments, control system 130 may communicate with table 150 to position or otherwise control table 150. Also, in some embodiments, control system 130 may communicate with an EM field generator (not illustrated) to control the generation of an EM field around patient 120.

[0031] The robotic system 110 may include one or more robotic arms 112 configured to engage and / or control medical instruments to perform a procedure. For example, the distal end (e.g., end effector) of the robotic arm 112 may be physically connected to a medical instrument that can be inserted and / or navigated within a patient to explore and / or treat a target site. In the example of FIG. 1 , the robotic arm 112 is illustrated without a medical instrument attached. While three robotic arms are illustrated, the robotic system 110 may include any number of robotic arms. Each robotic arm 112 may include multiple arm segments coupled to joints, thereby providing multiple degrees of movement. The robotic system 110 may also be configured to couple to other types of instruments / devices, such as an electromagnetic (EM) field generator, which may be configured to generate an EM field that is detected by a sensor on the medical instrument. The EM field generator may be positioned near the treatment site during a phase of the procedure. The robotic system 110 may be configured in various ways depending on the particular procedure. In the example of FIG. 1, the robotic system 110 also includes display(s) 116 configured to display information and / or receive input.

[0032] The robotic system 110 can be coupled to any component of the medical system 100. In one example, the robotic system 110 is communicatively coupled to the control system 130 to receive control signals from the control system 130 to perform actions such as controlling the robotic arm 112 in a particular manner or manipulating a medical instrument. In another example, the robotic system 110 is configured to receive images (also referred to as image data) depicting the internal anatomical structures of the patient 120 from a scope and / or send the images to the control system 130, which can then be displayed on the display 132. Furthermore, in some embodiments, the robotic system 110 is coupled to components of the medical system 100, such as the control system 130, in a manner such that it can receive fluids, optics, power, etc. from the components.

[0033] In some embodiments, the robotic system 110 can be used to determine an area in the environment associated with an object (sometimes referred to as a "collision area" or "object zone"). For example, the robotic arm 112(A) can be moved to the left edge of the table 150, and the physician 140 can provide an input indicating that the robotic arm 112(A) is positioned adjacent to an object in the environment, as shown in FIG. 1 . The control system 130 and / or the robotic system 110 can then determine the location of the distal end of the robotic arm 112(A). Similarly, the robotic arm 112(B) can be moved to the bottom edge of the table 150, and the physician 140 can provide an input indicating that the robotic arm 112(B) is positioned adjacent to an object. The control system 130 and / or the robotic system 110 can then determine the location of the distal end of the robotic arm 112(B).

[0034] Based on the position of the distal end of the robot arm 112(A) and / or the position of the distal end of the robot arm 112(B), the control system 130 and / or the robot system 110 can determine a collision region 160. For example, a first boundary 162 of the collision region 160 can be based on the position of the distal end of the robot arm 112(A) and the intersection of the first boundary 162 and a second boundary 164. Furthermore, the second boundary 164 can be based on the position of the distal end of the robot arm 112(B) and the intersection of the first boundary 162 and the second boundary 164. The collision region 160 can include an area excluding the robot system 110 (e.g., the distal ends of the robot arms 112(A) and 112(B)). For ease of explanation, the boundaries 162 and 164 are illustrated as extending from the intersection of the boundaries 162 and 164 to the edges of FIG. 1 . However, boundaries 162 and 164 can be of any length.

[0035] The robotic arms 112(A) and 112(B) can be positioned adjacent to the table 150 in a variety of ways. For example, as described in more detail below, the physician 140 can manually move the robotic arms 112(A) and 112(B), such as by selecting a button on the robotic arms to enable an admittance control mode associated with manual movement. Alternatively, or in addition, the physician 140 can use an I / O device (e.g., a controller, a mouse, etc.) associated with the control system 130 / robotic system 110 to provide inputs that cause movement of the robotic arms 112(A) and 112(B). While many examples are described in the context of positioning a first robotic arm and determining the position of the first robotic arm, and then positioning a second robotic arm and determining the position of the second robotic arm, the positions of the first / second robotic arms can be determined at any time, such as after both arms are positioned.

[0036] The control system 130 and / or the robotic system 110 can use the collision region 160 to control the movement of one or more robotic arms 112 and / or instruments coupled to the one or more robotic arms 112. The movement of the one or more robotic arms 112 and / or instruments coupled to the one or more robotic arms 112 can be controlled based on the collision region 160 to avoid collisions with the table 150 and / or other objects within the collision region 160. The robotic arm 112 can generally operate in a robotic control mode, in which the robotic system 110 moves the robotic arm 112 without user manipulation of the robotic arm 112, or in an admittance control mode, in which a user manipulates the robotic arm 112 (e.g., manually moves the robotic arm 112). In either mode of operation, the robotic arm 112 can be controlled to move through an environment based on the collision region 160.

[0037] In some embodiments, the control system 130 and / or the robotic system 110 may move the robotic arm 112 into the collision zone 160 based on an override input from the physician 140. In one example, if the physician 140 provides an input that would normally move the robotic arm 112 into the collision zone 160, the control system 130 may determine that the input will cause such a movement into the collision zone 160 and may notify / alert the physician 140 (e.g., ask the physician 140 whether he or she wants to override the configuration and move into the collision zone 160). The notification / alert may indicate that the physician 140 is requesting a movement that could potentially cause a collision with the robotic arm 112. The physician 140 may then confirm that he or she wants to continue moving into the collision zone 160 (e.g., override the avoidance of the collision zone 160) or request that such a movement not be made (e.g., avoid moving into the collision zone 160). In another example, if the physician 140 attempts to manually move the robotic arm 112 into the collision region 160 (e.g., while operating in admittance control mode), a similar notification / alert can be issued to the physician 140, and the robotic arm 112 can be controlled based on a response from the physician 140.

[0038] Additionally, in some embodiments, the control system 130 and / or the robotic system 110 can control the robotic arm 112 to move through the environment without moving into the collision zone 160. In one example, if the physician 140 provides an input that would normally move the robotic arm 112 into the collision zone 160, the control system 130 can determine that the input will cause such movement into the collision zone 160 and can prevent movement into the collision zone 160. In another example, if the physician 140 attempts to manually move the robotic arm 112 into the collision zone 160 (e.g., while operating in an admittance control mode), the robotic system 110 can prevent such movement, such as by stopping movement of the robotic arm 112 at the boundary of the collision zone 160.

[0039] In some embodiments, the control system 130 and / or the robotic system 110 can control the robotic arm 112 to move within the collision zone 160 (e.g., in either robotic control mode or admittance control mode) using different algorithms than when moving outside the collision zone 160. For example, the robotic arm 112 can be controlled to move slower within the collision zone 160 than outside the collision zone 160, to move with less force within the collision zone 160 than outside the collision zone 160, to move into the collision zone 160 unless such movement is within a predetermined distance to the initial location where the robot arm was positioned to set the collision zone 160, to move within the collision zone 160 with a different amount of resistance than when moving outside the collision zone 160 (e.g., so that the robotic arm 112 feels heavier in the collision zone 106 than outside the collision zone 106 when operating in admittance control mode), etc.

[0040] A collision region can represent an area / space in an environment containing an object, such as an area / space where a collision with a robotic arm / medical device may occur. The boundaries of the collision region can be represented / defined using a virtual surface / plane. While many examples are described in the context of determining a collision region for a table, the technique can be applied to other types of objects. For example, the technique can be used to determine a collision region for other medical devices in the environment, a collision region for a patient, etc. In some embodiments, multiple collision regions are determined for multiple objects in the environment. Also, while many collision regions are described as including two boundaries / surfaces, a collision region can include any number of boundaries / surfaces. For example, the collision region 160 for the table 150 can also include a boundary based on the height of the table 150, thereby allowing the robotic arm 112 to move above the table 150.

[0041] In some embodiments, by allowing a user to manually position the robotic arm 112 adjacent to an object and determining the collision area based on the position of the robotic arm 112, the present technique can intelligently / effectively determine the collision area without relying on other sensors / devices. For example, the present technique can avoid the need to use additional components / sensors other than those included within the medical system 100. Furthermore, in some embodiments, the user can configure the collision area without providing input via a touchscreen, mouse, keyboard, or other type of input device, which can be time-consuming and / or result in an inaccurately defined collision area.

[0042] Medical instruments can include various types of instruments, such as scopes (sometimes referred to as "endoscopes"), catheters, needles, guidewires, lithotriptors, basket retrieval devices, forceps, vacuums, needles, scalpels, imaging probes, graspers, scissors, graspers, needle holders, microdissectors, staple appliers, tackers, suction / irrigation tools, clip appliers, etc. Medical instruments can include direct entry instruments, percutaneous entry instruments, and / or other types of instruments. In some embodiments, medical instruments are steerable devices, while in other embodiments, medical instruments are non-steerable devices. In some embodiments, surgical tools refer to devices configured to puncture or be inserted through a body structure, such as needles, scalpels, guidewires, etc. However, surgical tools can also refer to other types of medical instruments.

[0043] The terms "scope" or "endoscope" are used herein according to their broad and ordinary meaning and can refer to any type of elongated medical instrument having imaging, viewing, and / or capture capabilities and configured to be introduced into any type of organ, cavity, lumen, chamber, and / or space in the body. For example, a scope or endoscope can refer to a ureteroscope (e.g., for accessing the urinary tract), a laparoscope, a nephroscope (e.g., for accessing the kidneys), a bronchoscope (e.g., for accessing the airways such as the bronchi), a colonoscope (e.g., for accessing the colon), an arthroscope (e.g., for accessing joints), a cystoscope (e.g., for accessing the bladder), a borescope, etc. A scope / endoscope may, in some cases, include a rigid or flexible tube and may be sized to pass through an outer sheath, catheter, introducer, or other luminal device, or may be used without such a device. In some embodiments, the scope includes one or more working channels through which additional tools, such as lithotriptors, basket devices, forceps, etc., can be introduced into the treatment site.

[0044] The terms "direct entry" or "direct access" are used herein in accordance with their broad and ordinary meaning and can refer to any entry of instrumentation through a natural or artificial opening in a patient's body. For example, a scope can be referred to as a direct access instrument because it enters a patient's urinary tract via the urethra.

[0045] The terms “percutaneous entry” or “percutaneous access” are used herein according to their broad and ordinary meaning and can refer to the entry of instrumentation, such as by puncture and / or small incision, through a patient's skin and any other body layers necessary to reach a target anatomical location associated with a procedure (e.g., the calyx rete of a kidney). Accordingly, a percutaneous access instrument can refer to a medical instrument, device, or assembly configured to puncture or be inserted through the skin and / or other tissue / anatomical structure, such as a needle, scalpel, guidewire, sheath, shaft, scope, catheter, etc. However, it should be understood that a percutaneous access instrument can refer to other types of medical instruments in the context of this disclosure. In some embodiments, a percutaneous access instrument refers to an instrument / device inserted or implemented by a device that facilitates a puncture and / or small incision through a patient's skin. For example, a catheter can be referred to as a percutaneous access instrument when inserted through a sheath / shaft that has punctured the patient's skin.

[0046] In some embodiments, the medical device includes a sensor (which may be referred to as a position sensor) configured to generate sensor data. In examples, the sensor data may indicate the position and / or orientation of the medical device and / or may be used to determine the position and / or orientation of the medical device. For example, the sensor data may indicate the position and / or orientation of a scope, which may include the roll of the distal end of the scope. The position and orientation of the medical device may be referred to as the pose of the medical device. The sensor may be located at the distal end of the medical device and / or at any other location. In some embodiments, the sensor may provide the sensor data to the control system 130, the robotic system 110, and / or another system / device to perform one or more localization techniques to determine / track the position and / or orientation of the medical device.

[0047] In some embodiments, the sensor can include an electromagnetic (EM) sensor with a coil of conductive material. Here, an EM field generator can provide an EM field that is detected by the sensor EM on the medical device. The magnetic field can induce small currents in the coil of the EM sensor, which can be analyzed to determine the distance and / or angle / orientation between the EM sensor and the EM field generator. Additionally, the sensor can include other types of sensors, such as a camera, a distance sensor, a radar device, a shape-sensing fiber, an accelerometer, a gyroscope, a satellite-based positioning sensor (e.g., a global positioning system (GPS)), a radio frequency transceiver, etc.

[0048] In some embodiments, the medical system 100 also includes an imaging device (not illustrated in FIG. 1 ) that may be integrated into the C-arm and / or configured to image during a procedure, such as in the case of a fluoroscopy-type procedure. The imaging device may be configured to capture / generate one or more images of the patient 120 during the procedure, such as one or more X-ray or CT images. In an example, images from the imaging device may be provided in real time to visualize anatomical structures and / or medical equipment within the patient 120 to assist the physician 140 in performing the procedure. The imaging device may be used to perform fluoroscopy (e.g., with a contrast agent within the patient 120) or another type of imaging technique.

[0049] The various components of the medical system 100 may be communicatively coupled to one another via a network, which may include wireless and / or wired networks. Exemplary networks include one or more personal area networks (PANs), local area networks (LANs), wide area networks (WANs), Internet area networks (IANs), cellular networks, the Internet, etc. Additionally, in some embodiments, the components of the medical system 100 are connected for data communication, fluid / gas exchange, power exchange, etc. via one or more supporting cables, tubes, etc.

[0050] In some embodiments, the medical system 100 can be used to treat kidney stones. Kidney stone disease, also known as urolithiasis, is a medical condition involving the formation of solid pieces of material in the urinary tract, referred to as "kidney stones," "urinary stones," "nephrolithiasis," or "nephrolithiasis." Urinary stones may form and / or be found in the kidneys, ureters, and bladder (referred to as "bladder stones"). Such urinary stones may form as a result of the concentration of minerals in urine, and if such stones reach a size sufficient to obstruct urine flow through the ureter or urethra, they can cause significant abdominal pain. Urinary stones may be formed from calcium, magnesium, ammonia, uric acid, cysteine, and / or other compounds, or combinations thereof.

[0051] Generally, there are several methods for treating patients with kidney stones, including observation, medical treatment (such as expulsion therapy), non-invasive treatment (such as extracorporeal shock wave lithotripsy (ESWL)), and surgical treatment (such as ureteroscopy and percutaneous nephrolithotomy (PCNL)). In surgical approaches (e.g., ureteroscopy and PCNL), a physician accesses the lesion (i.e., the object to be removed, e.g., the stone), breaks the stone into smaller pieces or fragments, and mechanically removes the smaller stone fragments / particles from the kidney.

[0052] To remove urinary stones from the bladder and ureters, a surgeon may insert a ureteroscope through the urethra and into the urinary tract. The ureteroscope typically includes an endoscope at its distal end configured to allow visualization of the urinary tract. The ureteroscope may also include a lithotripsy device for capturing or fragmenting urinary stones. During a ureteroscopy procedure, one physician / technologist may control the position of the ureteroscope, while another physician / technologist may control the lithotripsy device. To remove relatively large stones (i.e., "kidney stones") from the kidney, a physician may use a percutaneous nephrolithotomy ("PCNL") technique. This involves inserting a nephroscope through the skin (i.e., percutaneously) and intervening tissue to provide access to the treatment site for crushing and / or removing the stone.

[0053] In some of the examples described herein, robotic-assisted percutaneous procedures can be performed in connection with various medical procedures, such as kidney stone removal procedures, and robotic tools (e.g., one or more components of medical system 100) can enable a physician / urologist to perform endoscopic (e.g., ureteroscopy) targeted access as well as percutaneous access / procedures. However, the present disclosure is not limited to kidney stone removal and / or robotic-assisted procedures. In some implementations, robotic medical solutions can provide relatively high precision, better control, and / or better eye-hand coordination for certain instruments compared to strictly manual procedures. For example, robotic-assisted percutaneous access to the kidney through some procedures can advantageously enable a urologist to perform both direct-entry endoscopic renal access and percutaneous renal access. While some embodiments of the present disclosure are presented in the context of a catheter, a nephroscope, a ureteroscope, and / or the anatomy of the human kidney, it should be understood that the principles disclosed herein can be implemented in any type of endoscopic / percutaneous procedure or another type of procedure.

[0054] In one exemplary procedure, the medical system 100 can be used to remove a kidney stone from a patient 120. During setup for the procedure, the physician 140 can use the robotic system 110 to determine the impact region 160 in the manner described herein. One or more robotic arms 112 can then be arranged in various configurations / positions around the patient 120 to facilitate the procedure, such as extending outward to reach between the patient's 120 legs (e.g., align with the patient's 120 urethra), positioned near the patient's 120 abdomen, etc. The physician 140 can connect a scope and / or other medical equipment to the robotic arms 112. The physician 140 can interact with the control system 130 (e.g., via I / O device(s)) to cause the robotic system 110 to advance and / or navigate the scope up the urethra, through the bladder, up the ureter, and into the kidney where the stone is located. The control system 130 can provide information about the scope, such as real-time images acquired with the scope, via display(s) 132 to assist the physician 140 in navigating the scope. Once the site of the kidney stone (e.g., in the calyx of the kidney) is reached, the scope can be used to designate / tag a target location for a catheter (which can be connected to another robotic arm 112) to percutaneously access the kidney. To minimize damage to the kidney and / or surrounding anatomical structures, the physician 140 can designate the papilla as the target location for percutaneous entry into the kidney with the catheter. However, other target locations can be designated or determined.

[0055] The physician 140 can also interact with the control system 130 to cause the robotic system 110 to advance and / or navigate the catheter through a percutaneous access pathway to a target location designated by the scope. In some embodiments, a needle or another medical instrument is inserted into the patient 120 to form the percutaneous access pathway. The control system 130 can provide information about the catheter via the display(s) 132 to assist the physician 140 in navigating the catheter. For example, the interface(s) can provide image data from the field of view of the scope. The image data can depict the catheter (e.g., when it is within the field of view of the scope's imaging device).

[0056] Once the scope and / or catheter are positioned at the target location, the physician 140 can use the scope to break up the kidney stone and / or use the catheter to remove small pieces of the kidney stone from the patient 120. For example, the scope can position a tool (e.g., a laser, a resector, etc.) to fragment the kidney stone into small pieces, and the catheter can suction the pieces out of the kidney through a percutaneous access pathway. In examples, the catheter and / or scope can provide irrigation and / or suction to facilitate removal of the kidney stone. For example, the catheter can be coupled to an irrigation and / or suction system.

[0057] During a kidney stone removal procedure, control system 130 and / or robotic system 110 can control one or more robotic arms 112 based on the collision zone 160. For example, if physician 140 provides an input that would normally move robotic arm 112 into collision zone 160, control system 130 can determine that such input will move robotic arm 112 into collision zone 160 and can issue a notification / alert to physician 140 indicating that the input will cause movement into collision zone 160. Physician 140 can instruct whether to proceed with the movement.

[0058] The medical system 100 provides various benefits, such as providing guidance to assist physicians in performing procedures (e.g., instrument tracking, instrument navigation, instrument calibration, etc.), allowing physicians to perform procedures from ergonomic positions without requiring skilled arm movements and / or positions, allowing a single physician to perform procedures using one or more medical devices, avoiding radiation exposure (e.g., associated with fluoroscopy techniques), performing procedures in a single surgical setting, and providing continuous suction for more efficient removal of objects (e.g., removing kidney stones). For example, the medical system 100 provides guidance information to assist physicians in accessing target anatomical features with various medical devices while minimizing bleeding and / or damage to anatomical structures (e.g., critical organs, blood vessels, etc.). Furthermore, the medical system 100 can provide non-radiation-based navigation and / or localization techniques to reduce physician and patient radiation exposure and / or reduce the amount of equipment in the operating room. Furthermore, the medical system 100 can provide distributed functionality between at least the control system 130 and the robotic system 110, which can be independently mobile. Such distribution of functionality and / or mobility can allow the control system 130 and / or robotic system 110 to be placed in a location that is optimal for a particular medical procedure, thereby maximizing the working area around the patient and / or providing an optimized location for the physician to perform the procedure.

[0059] Although various techniques and systems are discussed as being implemented in robotically-assisted procedures (e.g., procedures that at least partially use medical system 100), these techniques and systems may be implemented in other procedures, such as fully robotic medical procedures, human-only procedures (e.g., procedures that do not include a robotic system), etc. For example, medical system 100 may be used to perform a procedure without a physician holding / manipulating medical devices and / or providing direct navigational input to the medical devices (e.g., a fully robotic procedure). That is, each of the medical devices used during a procedure may be held / controlled by a component of medical system 100, such as the robotic arm(s) 112 of robotic system 110.

[0060] FIG. 2 illustrates a perspective view of a collision area 160 and other aspects of the medical system 100 of FIG. 1 in accordance with one or more embodiments. As shown, the collision area 160 includes a first boundary 162 associated with a first edge of the table 150 and a second boundary 164 associated with a second edge of the table 150 (e.g., the feet of the table 150). Here, the boundaries 162 and 164 each include or represent a substantially flat surface that may extend any distance in the X, Y, and / or Z directions. In the example of FIGS. 1 and 2, the collision area 160 encompasses three-dimensional (3D) space. That is, the collision area 160 is a 3D collision area. Here, the collision area 160 encompasses a space that includes the table 150 and most of the patient 120 (excluding a portion of the patient's 120's legs), while excluding a space that includes the robotic system 110. The control system 130 from FIG. 1 is not shown in FIG. 2. Although illustrated with a particular form (e.g., a 3D cube-shaped region with flat surfaces), the collision region 160 can take other forms, such as a 2D collision region, a non-planar surface, other shapes, etc. Additionally, while the robotic arm 112 is shown in various positions in the figures, it should be understood that such configurations are shown for convenience and explanation, and that such robotic arm 112 can have different configurations.

[0061] In many examples herein, the collision area is defined relative to a table and a patient positioned on the table. For example, the control system 130 may establish the collision area 160 based on the physician 140 positioning the robotic arms 112 adjacent to the table 150 and assuming that the patient 120 is positioned on the table 150, as described above. However, the collision area may be defined relative to any number of objects. In some cases, the physician 140 may provide an input indicating the type of object associated with the collision area, such as the type of object adjacent to which the robotic arms 112 are positioned. For example, the physician 140 may position one or more of the robotic arms 112 adjacent to the table 150 and provide an input indicating that one or more robotic arms 112 are positioned adjacent to the table 150. The physician 140 can then position one or more of the robotic arms 112 adjacent to the patient 120 (which may include positioning the robotic arms 112 adjacent to a portion of the patient's 120 leg that extends beyond the table 150) and provide input indicating that the object type is a patient / user. The control system 130 can define separate collision regions for the table 150 and the patient 120, or can determine a collision region that encompasses both the table 150 and the patient 120.

[0062] Control system and robot system examples Figure 3 illustrates example details of the control system 130 and robotic system 110 of Figure 1, and Figure 4 illustrates example details of the robotic system 110 according to one or more embodiments. While certain components of the control system 130 and / or robotic system 110 are illustrated in Figures 3 and / or 4, it should be understood that additional components not shown may be included in embodiments according to the present disclosure. Furthermore, any of the illustrated components may be omitted, replaced, and / or integrated into other devices / systems, such as the table 150, medical equipment, etc.

[0063] 3 , the control system 130 can include one or more of the following components, devices, modules, and / or units (referred to herein as “components”), either separately / individually and / or in combination / collectively: one or more I / O components 302, one or more communication interfaces 304, one or more power supply units 306, and / or one or more mobilization components 308 (e.g., casters or other types of wheels). In some embodiments, the control system 130 can include a housing / enclosure configured and / or dimensioned to house or accommodate at least a portion of one or more of the components of the control system 130. In this example, the control system 130 is illustrated as a cart-based system that is movable using one or more mobilization components 308. In some cases, once the appropriate location is reached, the one or more mobilization components 308 can be secured using wheel locks to hold the control system 130 in place. However, the control system 130 can be implemented as a fixed system, integrated into another system / device, etc.

[0064] The various components of control system 130 may be electrically and / or communicatively coupled using certain connection circuits / devices / features that may or may not be part of control circuitry. For example, the connection feature(s) may include one or more printed circuit boards configured to facilitate mounting and / or interconnection of at least some of the various components / circuits of control system 130. In some embodiments, two or more of the components of control system 130 may be electrically and / or communicatively coupled to one another.

[0065] The one or more I / O components / devices 302 may include various components for receiving input and / or providing output, such as for interfacing with a user to assist in performing a medical procedure. The one or more I / O components 302 may be configured to receive touch, speech, gestures, or any other type of input. In an example, the one or more I / O components 302 may be used to provide input for device / system control, such as to control the robotic system 110, navigate a scope or other medical instrument attached to the robotic system 110, control the table 150, control a fluoroscopy device, etc. For example, the physician 140 may provide input via the I / O components 302, and in response, the control system 130 may send control signals to the robotic system 110 to operate a medical instrument. In an example, the physician 140 may use the same I / O device to control multiple medical instruments (e.g., switch control between instruments).

[0066] As shown, the one or more I / O components 302 may include one or more displays 132 (sometimes referred to as “one or more display devices 132”) configured to display data. The one or more displays 132 may include one or more liquid-crystal displays (LCDs), light-emitting diode (LED) displays, organic LED displays, plasma displays, electronic paper displays, and / or any other type of technology. In some embodiments, the one or more displays 132 include one or more touchscreens configured to receive input and / or display data. Additionally, the one or more I / O components 302 may include one or more I / O devices / controllers 310, which may include a touchpad, a controller (e.g., a handheld controller, a video game-type controller, etc.), a mouse, a keyboard, a wearable device (e.g., an optical head-mounted display), a virtual or augmented reality device (e.g., a head-mounted display), etc. Additionally, the one or more I / O components 302 may include one or more speakers configured to output sound based on an audio signal and / or one or more microphones configured to receive sound and generate an audio signal. In some embodiments, one or more of the I / O components 302 include or are implemented as a console.

[0067] In some embodiments, one or more I / O components 302 can output information related to a procedure. For example, the control system 130 can receive real-time images acquired by the scope and display the real-time images and / or visual representations of the real-time images via the display(s) 132. The display(s) 132 can present interface(s), such as any of the interfaces described herein, which can include image data from the scope and / or another medical device. Additionally or alternatively, the control system 130 can receive signals (e.g., analog, digital, electrical, acoustic / sonic, pneumatic, tactile, hydraulic, etc.) from medical monitors and / or sensors associated with the patient, and the display(s) 132 can present information related to the patient's health or environment. Such information can include, for example, information displayed via a medical monitor, such as heart rate (e.g., ECG, HRV, etc.), blood pressure / blood velocity, muscle biosignals (e.g., EMG), body temperature, blood oxygen saturation (e.g., SpO2), CO2, brain waves (e.g., EEG), environmental temperature, and / or local or core body temperature.

[0068] The one or more communication interfaces 304 can be configured to communicate with one or more devices / sensors / systems. For example, the one or more communication interfaces 304 can send / receive data wirelessly and / or wired over a network. Networks according to embodiments of the present disclosure can include a local area network (LAN), a wide area network (WAN) (e.g., the Internet), a personal area network (PAN), a body area network (BAN), etc. In some embodiments, the one or more communication interfaces 304 can implement wireless technologies such as Bluetooth, Wi-Fi, near field communication (NFC), etc.

[0069] The one or more power supply units 306 can be configured to manage and / or provide power to the control system 130 (and / or, in some cases, the robotic system 110). In some embodiments, the one or more power supply units 306 include one or more batteries, such as lithium-based batteries, lead-acid batteries, alkaline batteries, and / or other types of batteries. That is, the one or more power supply units 306 can include one or more devices and / or circuits configured to provide a power source and / or provide power management functions. Also, in some embodiments, the one or more power supply units 306 include a mains power connector configured to couple to an alternating current (AC) or direct current (DC) mains power source.

[0070] 3 , the control system 130 can include and / or control other components, such as one or more pumps, flow meters, valve controls, and / or fluid access components, to provide controlled irrigation and / or aspiration capabilities to a medical instrument (e.g., a scope), devices positionable through the medical instrument, etc. In some embodiments, irrigation and aspiration capabilities can be delivered directly to the medical instrument via separate cables. Additionally, the control system 130 can include voltage and / or surge protectors designed to provide filtered and / or protected power to another device, such as the robotic system 110, thereby avoiding the placement of power transformers and other auxiliary power components within the robotic system 110, making the robotic system 110 smaller and more mobile.

[0071] In some embodiments, the control system 130 can include support equipment for sensors located throughout the medical system 100. For example, the control system 130 can include optoelectronics for detecting, receiving, and / or processing data received from optical sensors and / or cameras. Such optoelectronics can be used to generate real-time images for display on any number of devices / systems included within the control system 130. Similarly, the control system 130 can include electronic subsystems for receiving and / or processing signals received from located electromagnetic (EM) sensors. In some embodiments, the control system 130 can be used to house and / or position EM field generators for detection by EM sensors in or on the medical device.

[0072] Additionally, in some embodiments, control system 130 can be coupled to robotic system 110, table 150, and / or medical equipment via one or more cables or connections (not shown). In some implementations, support functions from control system 130 can be provided via a single cable, simplifying and cluttering the operating room. In other implementations, certain functions can be combined in separate cable lines and connections. For example, power can be provided via a single power cable, while support for control, optics, fluidics, and / or navigation can be provided via separate cables.

[0073] 3 and 4 , the robotic system 110 generally includes an elongated support structure 310 (also referred to as a “column”), a robotic system base 312, and a console 314 at the top of the column 310. The column 310 can include one or more carriages 316 (also referred to as “arm support members 316”) for supporting the placement of one or more robotic arms 112. The carriages 316 can include individually configurable arm mounts that rotate along a vertical axis to adjust the base of the robotic arms 112 for positioning relative to the patient. The carriages 316 also include a carriage interface 318 that allows the carriage 316 to translate vertically along the column 310. The carriage interface 318 can connect to the column 310 through slots, such as slots 320 located on either side of the column 310 to guide the vertical translation of the carriage 316. The slots 320 can include vertical translation interfaces for positioning and / or maintaining the carriage 316 at various vertical heights relative to the base 312. Vertical translation of carriage 316 allows robotic system 110 to adjust the reach of robotic arm 112 to accommodate different table heights, patient sizes, physician preferences, etc. Similarly, individually configurable arm mounts on carriage 316 allow robotic arm base 322 of robotic arm 112 to be angled in various configurations. Column 310 may contain mechanisms therein, such as gears and / or motors, designed to use vertically aligned lead screws to translate carriage 316 in a mechanized manner in response to control signals generated in response to user input, such as input from an I / O device.

[0074] The base 312 can balance the weight of the column 310, carriage 316, and / or robotic arm 112 on a surface such as a floor. Thus, the base 312 can house heavier components, such as one or more electronics, motors, and power supplies, as well as components that enable and / or stabilize the robotic system 110. For example, the base 312 can include rollable wheels 324 (also referred to as "casters 324" or "mobilization components 324") that allow the robotic system 110 to be moved around a room for a procedure. Once in the appropriate position, the casters 324 can be locked using wheel locks to hold the robotic system 110 in place during a procedure. As shown, the robotic system 110 also includes a handle 326 to assist in steering and / or stabilizing the robotic system 110. In this example, the robotic system 110 is illustrated as a possible mobile, cart-based robot. However, the robotic system 110 can also be implemented as a fixed system, integrated into a table, or the like.

[0075] The robotic arm 112 generally can include a robotic arm base 322 and an end effector 328 separated by a series of linkages 330 connected by a series of joints 332. Each joint 332 can include an independent actuator, and each actuator can include an independently controllable motor. Each independently controllable joint 332 represents an independent degree of freedom available to the robotic arm 112. For example, each arm 112 can have seven joints, thus providing seven degrees of freedom. However, any number of joints can be implemented with any number of degrees of freedom. In examples, multiple joints can provide multiple degrees of freedom, allowing for "redundant" degrees of freedom. Redundant degrees of freedom enable the robotic arm 112 to position its individual end effectors 328 at specific positions, orientations, and / or trajectories in space using different linkage positions and / or joint angles. In some embodiments, the end effectors 328 can be configured to engage with and / or control medical instruments, devices, objects, etc. The freedom of movement of the arm 112 allows the robotic system 110 to position and / or orient medical equipment from a desired point in space and / or allows a physician to move the arm 112 to a clinically convenient position away from the patient for access while avoiding arm collisions.

[0076] Each end effector 328 of the robotic arm 112 may include an instrument device manipulator (IDM), which may be attached using a mechanism changer interface (MCI). In some embodiments, the IDM may be detached and replaced with a different type of IDM. For example, a first type of IDM may operate an endoscope, while a second type of IDM may operate a catheter. Another type of IDM may be configured to hold an electromagnetic field generator. The MCI may include connectors for transmitting air pressure, power, electrical signals, and / or optical signals from the robotic arm 112 to the IDM. The IDM 328 may be configured to manipulate a medical device (e.g., a surgical tool / instrument) using techniques including, for example, a direct drive, a harmonic drive, a gear drive, a belt and pulley drive, a magnetic drive, etc. In some embodiments, the IDM 328 may be attached to each one of the robotic arms 112. The robotic arms 112 are configured to insert or retract each coupled medical device into or from a treatment site.

[0077] In some embodiments, the robotic arm 112 can be configured to control the position, orientation, and / or tip articulation of a medical device (e.g., a sheath and / or reader of a scope). For example, the robotic arm 112 can be configured / configurable to manipulate a scope using an elongated movement member. The elongated movement member can include one or more pull wires (e.g., pull or push wires), cables, fibers, and / or flexible shafts. For example, the robotic arm 112 can be configured to actuate multiple pull wires coupled to the scope to deflect the tip of the scope. The pull wires can comprise any suitable or desirable material, such as metallic and / or non-metallic materials, such as stainless steel, Kevlar, tungsten, carbon fiber, etc. In some embodiments, the scope is configured such that the elongated movement members exhibit nonlinear behavior in response to applied forces. The nonlinear behavior can be based on the stiffness and compressibility of the scope and the variability in sag or stiffness between different elongated movement members.

[0078] As shown, the console 314 is positioned at the top of the column 310 of the robotic system 110. The console 314 may include display(s) 334 to provide a user interface (e.g., a dual-purpose device such as a touchscreen) for receiving user input and / or providing output to provide pre-operative and / or intra-operative data to the physician / user. Potential pre-operative data on the console / display 334 may include pre-operative planning, navigation and mapping data derived from pre-operative computerized tomography (CT) scans, and / or notes from a pre-operative patient interview. Intra-operative data may include optical information provided by tools, sensor and coordinate information from sensors, and vital patient statistics such as respiration, heart rate, and / or pulse. The console 314 may be positioned and tilted to allow the physician to access the console 314 from the opposite side of the column 314 from the arm support member 316. From this position, the physician may view the console 314, the robotic arm 112, and the patient while operating the console 314 from behind the robotic system 110.

[0079] The robotic system 110 may include one or more I / O components / devices 336 to receive input and / or provide output, for example, to interface with a user. The one or more I / O components 336 may be configured to receive touch, speech, gestures, or any other type of input. In an example, the one or more I / O components 336 may be used to provide input for device / system control, for example, to control / configure the robotic system 110. As shown, the one or more I / O components 334 may include one or more displays 334 configured to display data. The one or more displays 334 may include one or more liquid crystal displays (LCDs), light-emitting diode (LED) displays, organic LED displays, plasma displays, electronic paper displays, and / or any other type of technology. In some embodiments, the one or more displays 334 may include one or more touchscreens configured to receive input and / or display data. Further, the one or more I / O components 336 may include one or more I / O devices / controllers 338, which may include a touchpad, a controller, a mouse, a keyboard, a wearable device (e.g., an optical head-mounted display), a virtual or augmented reality device (e.g., a head-mounted display), etc. Further, the one or more I / O components 336 may include one or more speakers configured to output sound based on an audio signal and / or one or more microphones configured to receive sound and generate an audio signal. In some embodiments, the one or more I / O components 336 include or are implemented as the console 314. Further, the one or more I / O components 336 may include one or more physically depressible buttons, such as a button on the distal end of a robotic arm (which may enable an admittance control mode). This is illustrated in more detail with reference to FIG. 6 .

[0080] The various components of the robotic system 110 may be electrically and / or communicatively coupled using certain connection circuits / devices / features that may or may not be part of control circuitry. For example, the connection features may include one or more printed circuit boards configured to facilitate the mounting and / or interconnection of at least some of the various components / circuits of the robotic system 110. In some embodiments, two or more of the components of the robotic system 110 may be electrically and / or communicatively coupled to one another.

[0081] In some embodiments, one or more of the robotic arm 112 and / or robotic system 110 can be configured to operate in an admittance control mode. As used herein, the term “admittance control mode” (or simply “admittance mode”) can refer to a control mode of the robotic arm 112 / robot system 110 in which a user controls the movement of the robotic arm 112 by applying a force. For example, when operating in the admittance control mode, the robotic arm 112 can be moved manually by a user without electronic user controls, such as by grasping the robotic arm 112 and applying a force to it. Thus, a user may have direct control over the position of the robotic arm. The robotic arm 112 can include drive components (e.g., motors / actuators that control the movement of the robotic arm 112) configured to reposition and / or maintain the current pose (e.g., orientation and position) of the robotic arm 112. Thus, to provide the admittance control function, the robotic system 110 / control system 130 can measure the force applied to the robotic arm 112 by the user and use the measured force as an input to actuate one or more of the drive components.

[0082] For example, when the admittance control mode is enabled, the robotic arm 112 can be freely moved by a user by manually manipulating the robotic arm 112 based on a force applied to the robotic arm. For example, a user can grasp the distal end of the robotic arm 112 and apply a force to position the distal end of the robotic arm 112 (and / or other portions of the robotic arm 112) at a desired position. When the admittance control mode is disabled and / or the force applied to the robotic arm 112 is below a threshold, the robotic arm 112 may remain fixed in a position (e.g., preventing manual movement of the robotic arm 112). In some cases of the admittance control mode, such as when positioning the robotic arm 112 at a boundary to determine a collision region, the robotic arm 112 can move in an X, Y, and Z manner without changing the orientation of the end effector of the robotic arm 112 (e.g., the user cannot tilt the robotic arm 112). However, in other embodiments, the orientation of the robotic arm 112 can be changed in the admittance control mode. Accordingly, the robotic system 110 can be configured to receive user input in the form of a force applied directly by the user to the robotic arm 112 while in the admittance control mode.

[0083] The robotic arm 112 / robot system 110 can enter / exit the admittance control mode in various ways. For example, a user can provide input via the robotic system 110 / control system 130 (e.g., an interface, controller, etc.), via a button on the robotic arm 112, or in other ways to enable / disable the admittance control mode. Although the admittance control mode is described in many examples as being enabled / disabled in connection with pressing a button on the robotic arm 112, the admittance control mode can be enabled / disabled in various ways, such as via any type of I / O device.

[0084] The robotic arm 112 may generally exhibit a certain amount of resistance when operating in admittance control mode. The amount of resistance may affect the amount of force required to move the robotic arm 112, to move the robotic arm 112 at a particular speed, to move the robotic arm 112 a particular distance, etc. Thus, the amount of resistance associated with manual movement of the robotic arm 112 may indicate the force returned to the user (e.g., felt by the user) when manually moving the robotic arm 112. In some embodiments, one or more actuators / hardware of the robotic arm 112 may be controlled to configure the amount of resistance to manual movement of the robotic arm 112. For example, motors in the joints of the robotic arm 112 may be controlled based on a resistance parameter / value such that the robotic arm 112 exhibits a particular amount of resistance when moved by the user. In some embodiments, when operating in admittance control mode, one or more parameters such as the force applied by the user to the robotic arm 112, the virtual mass of the robotic arm 112, and / or virtual damping may be used to determine the speed at which the robotic arm 112 is moved. Virtual mass can indicate how heavy the robot arm 112 feels to the user (e.g., acceleration of robot movement), while virtual damping can give the user a sense of resistance (e.g., how fast the robot arm 112 moves).

[0085] The one or more communication interfaces 340 can be configured to communicate with one or more devices / sensors / systems. For example, the one or more communication interfaces 340 can send / receive data wirelessly and / or wired over a network. Networks according to embodiments of the present disclosure can include local area networks (LANs), wide area networks (WANs) (e.g., the Internet), personal area networks (PANs), body area networks (BANs), etc. In some embodiments, the one or more communication interfaces 340 can implement wireless technologies such as Bluetooth, Wi-Fi, near field communication (NFC), etc.

[0086] The one or more power supply units 342 can be configured to manage and / or provide power to the robotic system 110. In some embodiments, the one or more power supply units 342 include one or more batteries, such as lithium-based batteries, lead-acid batteries, alkaline batteries, and / or other types of batteries. That is, the one or more power supply units 342 can include one or more devices and / or circuits configured to provide a power source and / or provide power management functions. Also, in some embodiments, the one or more power supply units 342 include a mains power connector configured to couple to an alternating current (AC) or direct current (DC) mains power source.

[0087] The robotic system 110 may also include one or more actuators / hardware 344 to facilitate movement of the robotic arm 112. Each actuator 344 may include a motor, which may be mounted at a joint or elsewhere within the robotic arm 112 to facilitate movement of the joint and / or connected arm segment / linkage. Additionally, the robotic system 110 may include various other components, such as pneumatics, light sources, etc.

[0088] 3 , the control system 130 and / or the robotic system 110 may include control circuitry 346 and / or data storage / memory 348 configured to perform the functions described herein. For ease of explanation and illustration, the control circuitry 346 and data storage 348 are shown in blocks between the control system 130 and the robotic system 110. It should be understood that in many embodiments, the control system 130 and the robotic system 110 may include separate instances of the control circuitry 346 and data storage 348. That is, the control system 130 may include its own control circuitry and data storage (e.g., for performing processing on the control system 130), while the robotic system 110 may include its own control circuitry and data storage (e.g., for performing processing on the robotic system 110). In many embodiments, any reference herein to control circuitry may refer to circuitry embodied in a robotic system, a control system, or any other component of a medical system, such as any component of the medical system 100 shown in FIG. 1 .

[0089] Although the control circuitry 346 is illustrated as a component separate from the other components of the control system 130 / robotic system 110, it should be understood that any or all of the remaining components of the control system 130 and / or robotic system 110 can be at least partially embodied in the control circuitry 346. For example, the control circuitry 346 can include various devices (active and / or passive), semiconductor materials and / or areas, layers, regions, and / or portions thereof, conductors, leads, vias, connections, etc. One or more of the remaining components of the control system 130 / robotic system 110 and / or portions thereof can be at least partially formed and / or embodied in / by such circuit components / devices.

[0090] As illustrated, data storage device 348 may include collision component 350 configured to facilitate various functions described herein. In some embodiments, collision component 350 may include one or more instructions executable by control circuitry 346 to perform one or more operations. While many embodiments are described in the context of collision component 350 including one or more instructions executable by control circuitry 346, collision component 350 (and / or other components, such as a location component) may be implemented at least in part as one or more hardware logic components, such as one or more application specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), one or more program-specific standard products (ASSPs), one or more complex programmable logic devices (CPLDs), etc.

[0091] The collision component 350 can be configured to determine a collision region relative to the environment. For example, the collision component 350 can perform any of the operations described herein in connection with establishing a collision region associated with an object based on the position of the robotic arm 112 adjacent the object. The collision component 350 can also use the collision region to control the movement of the robotic system 110 and / or a medical device connected to the robotic system 110.

[0092] In some embodiments, the collision component 350 can adjust the collision region during a procedure. For example, if a collision region is determined for an object during procedure setup and the robotic arm contacts additional objects and / or additional edges of the object during the procedure, the collision component 350 knows where the additional collision points are located and can update the previously determined collision region to reflect the additional objects / edges, such as by adding additional boundaries to the collision region or updating the existing boundaries of the collision region.

[0093] Additionally, in some embodiments, the collision component 350 can determine the collision area in other ways. In one example, a user may be asked to draw a bounding box on a user interface displaying information representing the environment, and the collision component 350 can formulate a collision area defined by the user-drawn line. In another example, a user may interact with a user interface to place a box or other shape on information displayed in the interface representing the environment. Similarly, the collision component 350 can formulate a collision area for the box / shape. In yet another example, an imaging device / depth sensor may be positioned at the distal end of the robotic arm to capture one or more images of the environment, and the collision component 350 can process the one or more images (e.g., using image / vision processing techniques) to identify a collision area for one or more objects and / or objects in the environment.

[0094] 3 , in some embodiments, the data storage device 348 includes a localization component configured to perform one or more localization techniques to determine and / or track the position and / or orientation of an object, such as a medical device, connected to the robotic system 110. For example, the localization component can process input data, such as sensor data from the medical device (e.g., EM field sensor data, visual data acquired by an imaging device / depth sensor on the medical device, accelerometer data from an accelerometer on the medical device, gyroscope data from a gyroscope on the medical device, satellite-based positioning data from a satellite-based sensor (e.g., Global Positioning System (GPS)), etc.), robot commands and / or kinematic data for the robotic arm 112, sensor data from shape-sensing fibers (which can provide, e.g., shape data regarding the location / shape of the medical device), model data regarding the patient's anatomy, patient position data, pre-operative data, etc. Based on such processing, the localization component can generate position / orientation data for the medical device. The position / orientation data can indicate the location and / or orientation of the medical device relative to a frame of reference. The frame of reference may be a frame of reference relative to the patient's anatomy, a known object (e.g., an EM field generator), a coordinate system / space, etc. In some embodiments, the position / orientation data may indicate the location and / or orientation of the distal end (and / or, in some cases, the proximal end) of the medical device. The position and orientation of the object may be referred to as the pose of the object.

[0095] In some implementations, the localization component can use electromagnetic tracking to determine the position and / or orientation of an object. For example, the localization component can use real-time EM tracking to determine the real-time location of a medical device in a coordinate system / space, which can be registered to the patient's anatomy. This location can be represented by a pre-operative or other model. In EM tracking, an EM sensor (or tracker) including one or more sensor coils can be embedded in one or more locations and / or orientations within a medical device (e.g., a scope, needle, etc.). The EM sensor can measure variations in an EM field created by one or more static EM field generators positioned at known locations. The location information detected by the EM sensor can be stored as EM data. The localization component can process the EM data to determine the position and / or orientation of an object, such as a medical device. An EM field generator (or transmitter) can be placed near (e.g., within a predetermined distance from) the patient to create a low-intensity magnetic field that can be detected by the EM sensor. The magnetic field can induce small currents in the sensor coil of the EM sensor, which can be analyzed to determine the distance and / or angle between the EM sensor and the EM field generator. These distances and / or orientations can be "recorded" intraoperatively on the patient's anatomy (e.g., a pre-operative model) to determine a geometric transformation that aligns a single location in a coordinate system with a position on the pre-operative model of the patient's anatomy. Once recorded, EM sensors (e.g., an implanted EM tracker) at one or more locations on the medical device (e.g., the distal tip of an endoscope, a needle, etc.) can display the position and / or orientation of the medical device through the patient's anatomy in real time.

[0096] The term "control circuitry" is used herein according to its broad and ordinary meaning and may refer to any collection of the following: one or more processors, processing circuits, processing modules / units, chips, dies (e.g., semiconductor dies including one or more active and / or passive devices and / or connection circuits), microprocessors, microcontrollers, digital signal processors, microcomputers, central processing units, graphics processing units, field programmable gate arrays, programmable logic circuits, state machines (e.g., hardware state machines), logic circuits, analog circuits, digital circuits, and / or any device that manipulates signals (analog and / or digital) based on hard-coding of circuit and / or operational instructions. The control circuitry may further include one or more storage devices, which may be embodied in a single memory device, multiple memory devices, and / or embedded circuitry of a device. Such data storage devices may include read-only memory, random access memory, volatile memory, non-volatile memory, static memory, dynamic memory, flash memory, cache memory, data storage registers, and / or any device that stores digital information. It should be noted that in embodiments where the control circuitry includes a hardware state machine (and / or implements a software state machine) and includes analog, digital, and / or logic circuits, the data storage devices / registers that store any associated operational instructions may be embedded within or external to the circuitry that includes the state machine, analog, digital, and / or logic circuits.

[0097] The term "memory" is used herein according to its broadest and ordinary meaning and can refer to any suitable or desirable type of computer-readable medium, including, for example, one or more volatile, non-volatile, removable, and / or non-removable data storage devices implemented using any technology, layout, and / or data structures / protocols, containing any suitable or desirable computer-readable instructions, data structures, program modules, or other types of data.

[0098] Computer-readable media that can be implemented according to embodiments of the present disclosure include, but are not limited to, phase-change memory, static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage device, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device, or any other non-transitory medium that can be used to store information for access by a computing device. As used in certain contexts herein, computer-readable media generally may not include communication media such as modulated data signals and carrier waves. Accordingly, computer-readable media should be understood to generally refer to non-transitory media.

[0099] Collision area determination example 5-9 illustrate plan views of the medical system 100 of FIG. 1 arranged in several configurations for determining an impaction region, according to one or more embodiments. In these examples, the medical system 100 is positioned in an operating room to remove a kidney stone from a patient 120. In many embodiments, the patient 120 is positioned in a modified supine position, with the patient 120 tilted slightly to the side, to access the back or side of the patient 120 as illustrated. However, the patient 120 can be positioned in other ways, such as supine or prone. While FIGS. 5-9 illustrate the use of the medical system 100 to perform a percutaneous procedure to remove a kidney stone from the patient 120, as previously discussed, the medical system 100 can be used to remove kidney stones in other ways and / or perform other procedures. Various actions are described in FIGS. 5-9 and throughout this disclosure as being performed by a physician 140. It should be understood that these actions may be performed directly by the physician 140, by a user under the direction of the physician 140, by another user (eg, a technician), a combination thereof, and / or by any other user.

[0100] The anatomy of the kidney, as illustrated at least in part in Figures 5-9, is described herein for reference with respect to certain medical procedures related to aspects of the present concepts. The kidneys generally comprise two bean-shaped organs located side by side in the retroperitoneal space. In adults, the kidneys are generally approximately 11 cm long. The kidneys receive blood from paired renal arteries and blood exits into paired renal veins. Each kidney is attached to a ureter, which is a tube that carries excreted urine from the kidney to the bladder. The bladder is attached to the urethra.

[0101] The kidneys are usually located relatively high in the abdominal cavity, at a slightly oblique angle in a retroperitoneal position. Abdominal asymmetry caused by the position of the liver usually results in the right kidney being slightly lower, smaller, and more centrally positioned than the left. Above each kidney is an adrenal gland. The superior portion of the kidney is partially protected by the 11th and 12th ribs. Each kidney, along with its adrenal gland, is surrounded by two layers of fat: perirenal fat, located between the renal fascia and the renal capsule, and pararenal fat, located above the renal fascia.

[0102] The kidneys are responsible for regulating the volume of various body fluid compartments, fluid osmolality, acid-base balance, various electrolyte concentrations, and the removal of toxins. The kidneys perform a filtration function by secreting certain substances and reabsorbing others. Examples of substances secreted in urine are hydrogen, ammonium, potassium, and uric acid. In addition, the kidneys also perform various other functions, such as hormone synthesis.

[0103] The concave area on the kidney's concave border is the renal hilum. At the hilum, the renal artery enters the kidney and the renal vein and ureter exit. The kidney is surrounded by the renal capsule, a tough fibrous tissue. The renal capsule is itself surrounded by perirenal fat, renal fascia, and pararenal fat. The anterior (front) surface of these tissues is the peritoneum, while the posterior (back) surface is the transversalis fascia.

[0104] The functional substance, or parenchyma, of the kidney is divided into two main structures: the outer renal cortex and the inner renal medulla. These structures take the form of multiple cone-shaped renal lobes, each containing a renal cortex surrounding a portion of the medulla called the renal pyramid. Between the renal pyramids are projections of the cortex called the renal columns. The nephron, the urine-producing functional structure of the kidney, spans the cortex and medulla. The initial filtering portion of the nephron is the renal corpuscle, located within the cortex. This is followed by the renal tubules, which run deep from the cortex to the interior of the medullary pyramid. The medullary ray, part of the renal cortex, is a collection of tubules that drain into a single collecting duct.

[0105] The tip, or papilla, of each pyramid empties urine into an individual minor calyx, which then empties into a major calyx, which then empties into the renal pelvis, which then transitions into the ureter. At the hilum, the ureter and renal vein leave the kidney and the renal artery enters. Hilar fat and lymphatic tissue, along with lymph nodes, surround these structures. The hilar fat is adjacent to a fat-filled cavity called the renal sinus. The renal sinus collectively houses the renal pelvis and calyx and separates these structures from the renal medullary tissue.

[0106] 5-9 illustrate various features of the anatomy of a patient 120. For example, the patient 120 includes a kidney 502 fluidly connected to a bladder 504 via a ureter 506 and a urethra 508 fluidly connected to the bladder 504. As shown in the enlarged representation of the kidney 502(A), the kidney 502(A) includes a renal calyx (including a renal calyx 510), a renal papilla (including a renal papilla 512), and a renal pyramid (including a renal pyramid 514). In these examples, a kidney stone 516 is located proximate to the papilla 512. However, the kidney stone 516 may be located elsewhere within the kidney 1502(A). For ease of illustration, the enlarged representation of the kidney 502(A) is not shown in FIGS. 6-9.

[0107] As shown in FIGS. 5 and 6 , to remove a kidney stone 516 in an example percutaneous procedure, the physician 140 can move the robotic system 110 to the side / foot of the table 150 to set up / configure the robotic system 110. Specifically, as illustrated in FIG. 6 , the robotic system 110 can be positioned on the side of the table 150 adjacent to the feet of the patient 120. This can allow the robotic system 110 to be positioned to access the urethra 508 of the patient 120. In the example, the hip joint of the patient 120 is used as a reference point for positioning the robotic system 110. While the robotic system 110 is being moved, the robotic arm 112 can be positioned in a docked manner, similar to that shown in FIG. 5 . However, the robotic arm can be positioned in any manner while the robotic system 110 is being moved.

[0108] Once positioned on the feet of the table 150, the robotic system 110 can be immobilized using wheel locks to hold the control system 130 in place, as shown in FIG. 6 . In an example, the location of the robotic system 110 within the environment is maintained throughout the procedure (e.g., the position of the robotic system 110 on the floor of an operating room). That is, the position of the robotic system 110 relative to the table 150 and / or other devices / medical equipment within the environment can be maintained throughout the procedure. While the robotic system 110 is illustrated as being positioned in a particular location, the robotic system 110 can be positioned in other locations during setup and / or at other times during the procedure.

[0109] With reference to FIG. 6 , the physician 140 may manually move the robotic arm 112(A) to a first edge of the table 150. In this example, the physician 140 may press a button 602 located on a distal end of the robotic arm 112(A) to manually move the robotic arm 112(A) adjacent to the table 150. For example, the physician 140 may press the button 602 to enable an admittance control mode that allows the robotic arm 112(A) to be manually moved. In the example, the physician 140 may move the robotic arm 112(A) (e.g., the admittance control mode is enabled) as long as the button 602 is pressed. However, the admittance control mode may be enabled by other types of input. Additionally, the robotic arm 112(A) may be moved without enabling / disabling the admittance control mode (e.g., the robotic arm 112(A) may always be configured to be moved manually). In some embodiments, the physician 140 can navigate to a setup interface on the control system 130 and / or the robotic system 110, which can instruct the physician 140 to position one or more of the robotic arms 112 adjacent to an object in the environment to detect the surroundings of the robotic system 110.

[0110] In some embodiments, once the robotic arm 112(A) is positioned at a first edge of the table 150, the physician 140 can provide an input indicating that the robotic arm 112(A) is positioned adjacent to an object in the environment. The control system 130 can then receive position information from the robotic system 110 indicating the position of the robotic arm 112(A). Alternatively, or in addition, in some embodiments, the robotic system 110 can notify the control system 130 that the robotic arm 112(A) is positioned adjacent to an object (and send position information regarding the robotic arm 112(A)) when the physician 140 releases the button 602 (and / or after a predetermined time has elapsed since releasing the button 602). In any case, the control system 130 can use the position information regarding the robot arm 112(A) to determine the position of the distal end of the robot arm 112(A), such as the position of the distal end of the robot arm 112(A) within the environment, the position of the distal end of the robot arm 112(A) relative to the rest of the robot system 110, or the position of the distal end of the robot arm 112(A) relative to the control system 130.

[0111] Based on the position of the distal end of the robot arm 112(A), the control system 130 can define a plane / boundary 604 (also referred to as a "first plane 604"), such as a plane tangent to the distal end of the robot arm 112(A). In some embodiments, a component / marking on the end effector / IDM 606 of the robot arm 112(A) can be used as an alignment / reference point relative to the plane (e.g., a component / marking on the robot arm 112(A), a particular face (e.g., the front face) of the end effector 606, etc.). For example, as shown in the close-up of the distal end of the robotic arm 112(A), the end effector 606 of the robotic arm 112(A) can include multiple gears 608 for controlling / articulating the medical device, a reader 610 for reading data from the medical device (e.g., a radio-frequency identification (RFID) reader for reading a serial number from the medical device), fasteners 612 for attaching the medical device to the IDM 606 (e.g., latches for securing the medical device), and markers 614 for aligning with instruments manually attached to the patient (e.g., an access sheath) and / or for defining the front surface of the IDM 606. In the example of FIG. 6 , fasteners 612(A) are used to define a plane tangent to the end effector 606 of the robotic arm 112(A). In some embodiments, portion 606(A) of the end effector 606 can be configured to rotate / spin, such as by a user, when the robotic arm 112(A) is operating in an admittance control mode. For example, physician 140 can move robotic arm 112(A) to a desired position and then rotate top plate 606(A) of IDM 606 to the orientation shown in FIG. 6 to determine the orientation of plane 604. Accordingly, physician 140 can be instructed to align fastener 612(A) (or one of fasteners 612) with the edge of table 150 (e.g., rotate portion 606(A)). However, any component / marking on end effector 606 can be used as a reference point.Additionally, in some embodiments, a plane can be defined relative to a linkage segment of the robot arm 112(A), such as perpendicular to the distal-most linkage segment 616 of the robot arm 112(A) (and / or tangent to the end effector 606). In an example, the control system 130 can define the plane / boundary for the collision region based on information indicative of one or more dimensions of the robot arm 112(A) and / or the end effector 606. This information can be maintained / received by the control system 130.

[0112] 7 illustrates an example in which the robotic arm 112(B) (also referred to as the “second robotic arm 112(B)”) is positioned adjacent to a second edge of the table 150 to determine a plane / boundary 702 (also referred to as the “second plane 702”). Similar to what was described above with respect to the robotic arm 112(A) (also referred to as the “first robotic arm 112(A)”), the physician 140 can manually move the distal end of the second robotic arm 112(B) adjacent to the second edge of the table 150 (e.g., the foot of the table 150). As illustrated, the end effector 704 of the second robotic arm 112(B) can be positioned adjacent to the second edge of the table 150, and the control system 130 can define the second plane 702 based on the position of the end effector 704 of the second robotic arm 112(B). In this example, the second plane 702 is established to be tangent to the end effector 704 at the connecting element 706. The physician 140 may be instructed to position the second robotic arm 112(B) relative to the connecting element 706. However, other components / markings on the second robotic arm 112(B) may be used.

[0113] FIG. 8 illustrates an alternative example of positioning the first robotic arm 112(A) adjacent to the second edge of the table 150 to determine the second plane / boundary 702. Similar to what was described above with respect to FIG. 6, the physician 140 can manually move the distal end of the first robotic arm 112(A) adjacent to the second edge of the table 150. As illustrated, the end effector 604 of the first robotic arm 112(A) can be positioned adjacent to the second edge of the table 150, and the control system 130 can define the second plane 702 based on the position of the end effector 604 of the first robotic arm 112(A). In this example, the second plane 702 is formed tangent to the end effector 604 at the reader 610. However, other components / markings on the first robotic arm 112(A) can be used. In some embodiments, the first robotic arm 112(A) is positioned adjacent to a first edge of the table 150, and then positioned adjacent to a second edge of the table 150.

[0114] In either example, the control system 130 can determine a collision region 902 based on the first plane 604 and the second plane 702, as illustrated in FIG. 9 . For example, the control system 130 can define a first boundary 904 for the collision region 902 based on the first plane 604 and an intersection 906 between the first plane 604 and the second plane 702. The first boundary 904 can extend upward from the intersection 906 with respect to FIG. 9 . Additionally, the control system 130 can define a second boundary 908 for the collision region 902 based on the second plane 702 and an intersection 906 between the first plane 604 and the second plane 702. The second boundary 908 can extend to the right from the intersection 906 with respect to FIG. 9 . Boundaries 904 and 908 can extend any distance from intersection 906 (e.g., encircling an area substantially larger than table 150 and / or patient 120, encircling only table 150, etc.). For example, boundaries 904 and / or 908 can extend a distance associated with a dimension of table 150, such as a known length, height, and / or depth of table 150 (or another object), an average length, height, and / or depth of the table (or another object), etc. Control system 130 can define collision region 902 to exclude robotic arms 112, such as robotic arms 112(A) and 112(B), positioned adjacent table 150. In an example, collision region 902 can include a three-dimensional (3D) shape, such as that illustrated in FIG. 2.

[0115] In some embodiments, once the collision region 902 is determined, the control system 130 can output information representing the collision region 902. For example, the display(s) 132 can present a visualization of the collision region 902 via an interface, such as the interface illustrated in FIG. 10 . The interface can enable the physician 140 to accept and / or adjust / reconfigure the collision region 902, as described in further detail below. The physician 140 can interact with the interface via an I / O device, such as the handheld controller 910, the display(s) 132 (e.g., a touchscreen), or any other I / O device. Once the collision region 902 is accepted, the control system 130 can configure the robotic system 110 to perform a procedure based on the collision region 902, such as by designating the collision region 902 as being associated with one or more objects that may cause a collision.

[0116] The collision region 902 can be used to perform a procedure in a manner that generally attempts to avoid collision with objects in the environment. For example, one or more medical devices can be connected to one or more of the robotic arms 112 of the robotic system 110, and the one or more robotic arms 112 can be positioned in a manner appropriate for the procedure. A physician 140 can facilitate / manage / control the procedure from the convenience of the control system 130 (e.g., with the physician 140 positioned as shown in FIG. 9 ). For example, the physician 140 can interact with an I / O device of the control system 130, such as a handheld controller 910, to provide user input for controlling one or more medical devices attached to the robotic system 110. The base of the robotic system 110 can remain stationary during a procedure, while the robotic arms 112 are moved in different ways to control one or more medical devices.

[0117] During a procedure, the control system 130 can process user input from the physician 140 to control the movement of the robotic arm 112, such that the robotic arm 112 and / or attached medical device generally moves within the environment outside the collision region 902. For example, the robotic arm 112(A) can be controlled to move the robotic arm 112(A) and / or attached medical device within the environment without crossing the first boundary 904 and / or the second boundary 908. In an example, the control system 130 can take into account one or more dimensions / shapes of the medical device and / or the orientation of the medical device on the robotic arm 112(A). In some embodiments, the control system 130 and / or the robotic system 110 can alert the physician 140, such as via the display(s) 132, if user input causes movement into the collision region 902. If desired, the physician 140 can override the settings of the robotic system 110 / control system 130 and move the robotic arms 112 into the collision zone 902. Additionally, in some embodiments, the movement of the robotic arms 112 can be controlled in other ways based on the collision zone 902. In an example, the physician 140 can manually move one or more of the robotic arms 112 into the collision zone 902 or elsewhere in the environment by enabling an admittance control mode.

[0118] 5-9, the robotic arm 112 is positioned adjacent to a table 150 that does not have a medical device attached to it. However, in some embodiments, the robotic arm 112 can be positioned adjacent to an object that has one or more medical devices attached to it. In such embodiments, the control system 130 can use information about reference points on the medical device and / or one or more dimensions of the medical device to define a plane / boundary for the impact area.

[0119] 5-9 illustrate a collision area determined by positioning one or two of the robot arms 112 at two positions adjacent to the table 150. It should be understood that one or more of the robot arms 112 can be positioned at more than two positions (or only one position) adjacent to the table 150 and / or other objects in the environment. Thus, the collision area can have any number of surfaces / planes / points. In one example, a robot arm 112(C) (also referred to as a "third robot arm 112(C)") can be positioned adjacent to the foot of the table 150, similar to the second robot arm 112(B) illustrated in FIG. 9. Here, the third robot arm 112(C) can help accurately define a second boundary 908 of the collision area 902 (e.g., a plane / surface tangent to both robot arms 112(B) and 112(C)). In another example, the third robotic arm 112(C) (and / or any other robotic arms 112) can establish the height of the table 150, the position of the patient 120 (e.g., the legs of the patient 120 extending beyond the bed), etc. In yet another example, the collision region can include a single boundary / plane, such as the region that includes the table 150 and is defined only by plane 604, as illustrated in FIG.

[0120] In some embodiments, the present techniques enable the robotic system 110 to be used at a convenient / desired location in an environment and avoid collisions with objects within the workspace of the robotic system 110 at that location. For example, the robotic system 110 can be positioned anywhere in an environment and used to detect the location of objects within the environment. For ease of explanation, many operations, such as determining the position of the robot arm and determining collision regions, are described in the context of being performed by the control system 130, although such operations may alternatively or additionally be performed by the robotic system 110 and / or another device / system.

[0121] Interface Example 10 illustrates an example interface 1002 for visualizing and / or setting a collision area, according to one or more embodiments. In the example, the interface 1002 may be displayed via the control system 130, the robotic system 110, and / or any other device of the medical system 100. For example, the interface 1002 may be displayed via the control system 130 to enable the physician 140 to view the collision area determined for the environment in which the robotic system 110 is located and / or to provide adjustment input data for reconfiguring the collision area and / or other elements within the environment.

[0122] As shown, the interface 1002 can present a visualization 1004 of the environment in which the robotic system 110 is located. The visualization 1004 can include a visual representation 1006 of the robotic system 110 (e.g., an icon or other user interface element), a visual representation 1008 of a collision region determined for the environment, and a visual representation 1010 of a table 150 located within the environment. The visualization 1004 can be based on position / orientation information of the robotic system 110, information about the position / orientation of the collision region 1008, information about the table 150 (e.g., estimated / actual dimensions of the table 150), and / or any other information about the environment.

[0123] The interface 1002 allows a user to accept and / or set the collision area. For example, a user can select user interface element 1012 to adjust a first boundary 1014 of the collision area 1008 and / or select user interface element 1016 to adjust a second boundary 1018 of the collision area 1008. For example, a user can select and drag interface elements 1012 / 1016 to a desired location to change the position / or orientation of boundary 1014 / 1018, thereby increasing / decreasing the size of collision area 1008 (and the associated collision area relative to the environment), changing the shape of collision area 1008 (and the associated collision area relative to the environment), etc. For example, a user can move second boundary 1018 to extend beyond the bottom edge of table 1010 so that collision area 1008 encompasses a portion of the patient (not illustrated) that extends beyond the bottom edge of table 150. Additionally, in some examples, a user can select and drag collision area 1008 to a different location. In an example, a user can manipulate the collision area 1008, the visual representation 1006 of the robotic system 110, and / or the visual representation 1010 of the table 150 to change any characteristics, such as removing boundaries, repositioning / reorienting the visual representations 1006 / 1008 / 1010 (which can cause the control system 130 / robotic system 110 to update associated position / orientation information relative to the environment), etc. If the collision area 1008 (and / or other elements of the visualization 1004) are acceptable to the user, the user can select button 1020 to accept the settings and configure the control system 130 / robotic system 110 to operate the medical system 100 based on the settings (e.g., associated collision area).

[0124] Flow diagram example FIG. 11 illustrates an example flow diagram of a process 1100 for determining a region associated with an object, according to one or more embodiments. The various actions / acts associated with process 1100 may be performed by control circuitry implemented in any or a combination of the devices / systems described herein, such as control system 130, robotic system 110, table 150, medical equipment, and / or another device. Process 1100 may be performed during setup / configuration of medical system 100 for a procedure, during a procedure, after a procedure, and / or at other times. In one example, process 1100 is performed to configure robotic system 110 for a procedure. Although various blocks are illustrated as being part of process 1100, any of such blocks may be omitted. Additionally, additional blocks may be implemented as part of process 1100. The illustrated order of the blocks is for illustrative purposes only, and the blocks may be implemented in any order. In some embodiments, one or more of the blocks of process 1100 are implemented as executable instructions that, when executed by control circuitry, cause the control circuitry to perform the described functionality / operations. However, one or more of the blocks of process 1100 may be implemented in other ways, such as by another device / system, a user, etc.

[0125] At block 1102, process 1100 may include enabling manual movement of the robotic arm. For example, a user may provide input to set the robotic arm to an admittance control mode in which the robotic arm moves through user manipulation of the robotic arm. The input may be provided in various ways, such as by selecting a button on the robotic arm, providing input through an interface / controller, etc. In some embodiments, the robotic arm may be enabled for manual movement upon entering a setup mode on the robot system, control system, and / or another device / system associated with the robotic arm. Although operation 1102 is illustrated in process 1100, in some embodiments, the robotic arm may include a default / permanent state that enables manual movement of the robotic arm (e.g., not implementing block 1102).

[0126] At block 1104, process 1100 may include determining that the robot arm is positioned adjacent to an object in the environment. For example, the robot arm may determine that it is adjacent to an object when it receives input data (from an I / O device) indicating that it is positioned adjacent to an object, when a user releases a button on the robot arm to disable admittance control mode, when the robot arm remains stationary for a period of time after being moved (and / or after the user releases a button on the robot arm), combinations thereof, and / or other events. In some embodiments, the robot arm may be said to be positioned adjacent to an object when it is in contact with or otherwise positioned proximate to (e.g., within a predetermined distance of) the object.

[0127] At block 1106, process 1100 may include determining a position of the robotic arm. For example, the control system / robotic system may use the position data for the robotic arm to determine a position of an end of the robotic arm, such as a distal end of the robotic arm. The position information may indicate a position of an end effector end of the robotic arm, such as an end configured to couple to a medical device.

[0128] At block 1108, process 1100 may include determining whether the robot arm has moved to another position or determining whether an additional robot arm has moved. If the robot arm has moved to another position or if an additional robot arm has moved to a position, process 1100 may return to block 1104. For example, if the robot arm has moved to a different position, process 1100 may return to block 1104 to determine that the robot arm is positioned adjacent to another edge of the object and determine the position of the robot arm at the other edge of the object at block 1106. Furthermore, if an additional robot arm has moved to a position, process 1100 may return to block 1104 to determine that an additional robot arm is positioned adjacent to the object and determine the position of the additional robot arm at a position adjacent to the object at block 1106. Process 1100 may loop through blocks 1104-1108 any number of times to determine position information associated with any number of reference points relative to the object.

[0129] At block 1110, process 1100 may include determining a region / area within the environment associated with the object. For example, the region (also referred to as a “collision region” or “object region”) may be determined based on the position of the distal end of a first robotic arm at a first reference point (e.g., at a first time), the position of the distal end of a first robotic arm at a second reference point (e.g., at a second time), the position of the distal end of a second robotic arm at a third reference point, the position of the distal end of a second robotic arm at a fourth reference point, etc. The region may be based on position information for any number of robot arms at any number of reference points relative to the object. In some embodiments, the boundaries of the region may be determined based on the positions of the distal ends of the robot arms. In one example, the region may be determined by defining a first plane based on the position of the end of the first robotic arm and defining a second plane based on the position of the end of the second robotic arm and / or the position of the end of the first robotic arm at the second time. Here, the region can be based on a first plane, a second plane, and the intersection of the first and second planes. The region can include the object and / or exclude the robotic system.

[0130] At block 1112, process 1100 may include displaying a visualization of the region and / or allowing a user to update the region. For example, interface data representing the visualization may be sent for display and / or the interface data may be displayed via display(s) associated with the control system / robotic system. Adjustment input data may be received that includes adjustments to the visual representation and / or the region may be updated based on the adjustments to the visual representation (e.g., adjustment data).

[0131] At block 1114, process 1100 may include controlling one or more robotic arms to move based at least in part on the region. For example, a robotic arm of a robotic system may be controlled to move through an environment without moving into the region, to move into the region upon receiving confirmation from a user, etc. For example, the control system / robotic system may receive input control data from an input device regarding movement of a medical device attached to the robotic arm. The control system / robotic system may determine that the input control data is associated with movement of the robotic arm into / through the collision region. The control system / robotic system may display a notification / alert indicating that the input control data is associated with movement into the collision region. The control system / robotic system may then receive input data (e.g., based on user input from a user) indicating whether to proceed into the collision region. The control system / robotic system may move the robotic arm into the collision region or refrain from movement based on the input data. Alternatively, in some cases, the control system / robotic system may prevent movement into the collision region without notifying / alerting the user and / or perform other processing without notifying / alerting the user.

[0132] At block 1116, process 1100 may include updating the region. For example, the control system / robotic system may be set to a treatment mode to perform a medical procedure. During the treatment, if the control system / robotic system determines that the robotic arm experienced a collision, the control system / robotic system may update the region based on the position of the distal end of the robotic arm when the collision occurred. In an example, a visualization of the region may be displayed during the treatment before the region is updated, similar to that described for block 1112.

[0133] In some embodiments, the region can be updated at any time before, during, or after a procedure, which may or may not include displaying a visualization of the region to a user. For example, blocks 1112 and / or 1116 can be performed at any time before, during, and / or after a procedure. In one example, additional objects can be brought into the environment (e.g., additional medical devices can be moved into proximity of the robotic system during a procedure), and the region can be updated to avoid collisions with the additional objects.

[0134] Additionally, in some embodiments, one or more of the blocks of process 1100 can be performed while the robotic system is located in the same standby position, such as a stationary position where one or more wheels for the robotic system are locked.

[0135] Further embodiments Depending on the embodiment, certain acts, events, or functions of any of the algorithms or processes described herein may be performed in a different order, added, merged, or omitted entirely. Thus, in some embodiments, not all of the described acts or events are necessary to the performance of a process.

[0136] In particular, conditional language used herein, such as "can," "could," "might," "may," "eg," and the like, is intended to have its ordinary meaning unless specifically stated otherwise or understood otherwise within the context in which it is used, and is generally intended to convey that certain embodiments include certain features, elements, and / or steps, while other embodiments do not. Thus, such conditional language is generally not intended to suggest that features, elements, and / or steps are required in any way for one or more embodiments, or that one or more embodiments necessarily include logic, with or without input or prompting, for determining whether those features, elements, and / or steps are included in or performed in any particular embodiment. Terms such as "comprising," "including," "having," and the like, are used in their ordinary sense and are used inclusively in a non-limiting manner and do not exclude additional elements, features, acts, operations, etc. Also, when the term "or" is used, for example, to connect a list of elements, the term "or" is used in its inclusive sense (and not its exclusive sense) to mean one, some, or all of the listed elements. Unless specifically stated otherwise, connective language such as the phrase "at least one of X, Y, and Z" is understood in the context as it is commonly used to convey that an item, term, element, etc. can be either X, Y, or Z. Thus, such connective language is not generally intended to imply that a particular embodiment requires that at least one of X, at least one of Y, and at least one of Z, respectively, be present.

[0137] In the foregoing description of the embodiments, it should be understood that various features are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, this method of disclosure should not be interpreted as reflecting an intention that any claim requires more features than are expressly recited in that claim. Moreover, any component, feature, or step illustrated and / or described in a particular embodiment herein may be applied to or used in conjunction with any other embodiment. Moreover, no component, feature, step, or group of components, features, or steps is necessary or essential for each embodiment. Accordingly, it is intended that the scope of the disclosure as disclosed herein and claimed below should not be limited by the specific embodiments described above, but should instead be determined by a fair reading of the following claims.

[0138] It should be understood that certain ordinal terms (e.g., "first" or "second") may be provided for ease of reference and do not necessarily imply any physical characteristics or ordering. Thus, as used herein, ordinal terms (e.g., "first," "second," "third," etc.) used to modify elements such as structures, components, operations, etc., do not necessarily indicate a priority or order of the element relative to any other elements, but rather may generally distinguish the element from other elements having a similar or identical name (apart from the use of the ordinal terminology). Furthermore, as used herein, the indefinite articles ("a" and "an") may indicate "one or more" rather than "one." Furthermore, an action performed "based on" a condition or event may also be performed based on one or more other conditions or events not expressly recited.

[0139] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the example embodiments belong. It is further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and should not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0140] Spatially relative terms such as "outside," "inside," "upper," "lower," "below," "upper," "vertical," "horizontal," and similar terms may be used herein for ease of description to describe the relationship between one element or component and another element or component as illustrated in the figures. It should be understood that spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if a device shown in the figures is inverted, a device positioned "below" or "under" another device may be disposed "above" the other device. Thus, the illustrative term "lower" can include both lower and upper positions. Devices may also be oriented in other directions, and thus spatially relative terms may be interpreted differently depending on the orientation.

[0141] Unless otherwise specified, comparative and / or quantitative terms such as "less," "more," "greater than," etc. are intended to encompass the concept of equality. For example, "less" can mean "less than" in the strict mathematical sense as well as "less than or equal to."

[0142] [Embodiment] (1) A system comprising: a robotic system including a first robotic arm configured to couple to a medical device; a control circuit communicatively coupled to the robotic system, receiving input data indicating that the first robotic arm is positioned adjacent to an object in an environment; determining a region in the environment associated with the object based at least in part on a position of the distal end of the first robotic arm; and controlling at least one of the first robotic arm or the second robotic arm of the robotic system to move within the environment without moving into the area. (2) the first robotic arm is configured to operate in an admittance control mode in which user manipulation of the first robotic arm causes movement of the first robotic arm; 2. The system of claim 1, wherein the input data is received when the first robotic arm is operating in the admittance control mode. (3) the input data indicates that the first robot arm is positioned adjacent to a first edge of the object, and the control circuitry: receiving additional input data indicating that the second robotic arm is positioned adjacent a second edge of the object; and determining the region based at least in part on a position of the distal end of the second robotic arm. (4) The system of embodiment 3, wherein the control circuitry is configured to determine the region by determining a first boundary of the region based at least in part on the position of the distal end of the first robotic arm and determining a second boundary of the region based at least in part on the position of the distal end of the second robotic arm. (5) the input data indicates that the first robot arm is positioned adjacent to a first edge of the object, and the control circuitry: receiving additional input data indicating that the first robotic arm is positioned adjacent a second edge of the object; and determining the region based at least in part on a position of the distal end of the first robotic arm when the input data is received and a position of the distal end of the first robotic arm when the additional input data is received.

[0143] (6) The system of embodiment 1, wherein the control circuit is configured to determine the region based at least in part on the position of the robotic system, the region including the object and excluding the robotic system. (7) The control circuit displaying a visual representation of the region; receiving adjustment input data including adjustments to the visual representation; updating the region based at least in part on the adjustment to the visual representation. (8) The control circuit placing the system in a treatment mode to perform a medical treatment; determining that at least one of the first robotic arm or the second robotic arm has experienced a collision; 2. The system of claim 1, further configured to update the region based on at least one of the position of the distal end of the first robotic arm or the position of the second robotic arm when the collision occurred. (9) A method comprising: allowing the first robotic arm to be manually movable; a control circuit receiving input data indicating that the first robotic arm is positioned adjacent to one or more objects; the control circuitry determining a collision area based at least in part on a position of the end of the first robotic arm; and the control circuitry controls movement of at least one of the first robotic arm or the second robotic arm to perform a medical procedure based at least in part on the collision area. (10) The input data indicates that the first robot arm is positioned adjacent to a first edge of the one or more objects, and the method further comprises: receiving additional input data indicating that the second robotic arm is positioned adjacent a second edge of the one or more objects; The method of claim 9, wherein determining the collision area is further based at least in part on the position of the end of the second robot arm.

[0144] (11) The step of determining the collision area includes: defining a first plane based at least in part on the position of the end of the first robotic arm; defining a second plane based at least in part on the position of the end of the second robotic arm; determining the region based at least in part on the first plane, the second plane, and an intersection of the first plane and the second plane. (12) The input data indicates that the first robot arm is positioned adjacent to a first edge of the one or more objects, and the method further comprises: receiving additional input data indicating that the first robotic arm is positioned adjacent another edge of the one or more objects; 10. The method of claim 9, wherein determining the collision area includes determining the collision area based at least in part on the position of the end of the first robot arm when the input data is received and the position of the end of the first robot arm when the additional input data is received. (13) The step of determining the collision area includes: defining a first plane based at least in part on the position of the end of the first robotic arm upon receiving the input data; defining a second plane based at least in part on the position of the end of the first robotic arm upon receiving the additional input data; determining the region based at least in part on the first plane, the second plane, and an intersection of the first plane and the second plane. (14) At least one of the first robotic arm or the second robotic arm is configured to connect to a medical device, and the method further comprises: receiving input control data relating to movement of the medical device from an input device; determining that the input control data is associated with at least one of the first robot arm or the second robot arm moving into the collision region; 10. The method of claim 9, wherein controlling the movement of at least one of the first robot arm or the second robot arm includes preventing at least one of the first robot arm or the second robot arm from moving into the collision area. (15) displaying a notification indicating that the input control data is associated with movement into the collision region; and receiving additional input data indicating whether to proceed into the collision region; 15. The method of claim 14, wherein controlling the movement of at least one of the first robotic arm or the second robotic arm is based at least in part on the additional input data.

[0145] (16) The method described in embodiment 9, wherein the first robotic arm is connected to a robotic system, and the receiving of the input data and the controlling of the movement of at least one of the first robotic arm or the second robotic arm are performed while the robotic system is located in the same standby position. (17) The method of embodiment 9, wherein the end of the first robotic arm is an end effector end of the first robotic arm. (18) A control system comprising: a communication interface configured to communicate with the first robotic arm; a control circuit communicatively coupled to the communication interface, determining that the first robotic arm is positioned adjacent to a first edge of one or more objects in an environment; determining a collision region relative to the environment based at least in part on a position of the distal end of the first robotic arm; and a control circuit configured to: control movement of at least one of the first robotic arm or the second robotic arm based at least in part on the collision area, wherein at least one of the first robotic arm or the second robotic arm is configured to couple to a medical device. (19) The control circuit receiving input control data from an input device for controlling the medical device; determining that the input control data is associated with at least one of the first robot arm or the second robot arm moving into the collision region; A control system as described in embodiment 18, wherein the control circuit is configured to control movement of at least one of the first robot arm or the second robot arm by preventing at least one of the first robot arm or the second robot arm from moving into the collision area. (20) The control circuitry is further configured to determine that the second robotic arm is positioned adjacent to a second edge of the one or more objects; 19. The control system of claim 18, wherein the control circuit is configured to determine the collision area based at least in part on the position of the distal end of the first robotic arm and the position of the distal end of the second robotic arm.

[0146] (21) The control circuitry is further configured to determine that the first robotic arm is positioned adjacent to a second edge of the one or more objects; A control system as described in embodiment 18, wherein the control circuit is configured to determine the collision area based at least in part on the position of the distal end of the first robot arm at the first edge of the one or more objects and the position of the distal end of the first robot arm at the second edge of the one or more objects. (22) The control circuit displaying a visual representation of the collision area; receiving adjustment input data including adjustments to the visual representation; 20. The control system of claim 18, further configured to: update the collision region based at least in part on the adjustment to the visual representation. (23) The control circuit placing the control system in a treatment mode to perform a medical treatment; determining that at least one of the first robotic arm or the second robotic arm has experienced a collision; 19. The control system of claim 18, further configured to: update the collision area based on at least one of the position of the distal end of the first robot arm or the position of the second robot arm when the collision occurs. (24) One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by control circuitry, cause the control circuitry to: determining that a first robotic arm is positioned adjacent a first edge of one or more objects in an environment; determining a collision area relative to the environment based at least in part on a position of the distal end of the first robotic arm; and controlling movement of at least one of the first robotic arm or the second robotic arm based at least in part on the collision area, wherein at least one of the first robotic arm or the second robotic arm is configured to couple to a medical device. (25) The operation is determining that the second robotic arm is positioned adjacent a second edge of the one or more objects; One or more non-transitory computer-readable media as described in embodiment 24, wherein determining the collision area is further based at least in part on the position of the distal end of the second robotic arm.

[0147] (26) The step of determining the collision area includes: defining a first plane based at least in part on the position of the distal end of the first robotic arm; defining a second plane based at least in part on the position of the distal end of the second robotic arm; and determining the collision area based at least in part on the first plane, the second plane, and an intersection of the first plane and the second plane. (27) The operation is determining that the first robotic arm is positioned adjacent a second edge of the one or more objects; One or more non-transitory computer-readable media as described in embodiment 24, wherein determining the collision area is based at least in part on the position of the distal end of the first robot arm at the first edge of the one or more objects and the position of the distal end of the first robot arm at the second edge of the one or more objects. (28) The step of determining the collision area includes: defining a first plane based at least in part on the position of the distal end of the first robotic arm at the first edge of the one or more objects; defining a second plane based at least in part on the position of the distal end of the first robotic arm at the second edge of the one or more objects; 28. The one or more non-transitory computer-readable media of claim 27, further comprising: determining the collision area based at least in part on the first plane, the second plane, and an intersection of the first plane and the second plane. (29) The operation is receiving input control data relating to movement of the medical device from an input device; determining that the input control data is associated with at least one of the first robotic arm or the second robotic arm moving into the collision area; One or more non-transitory computer-readable media as described in embodiment 24, wherein controlling the movement of at least one of the first robot arm or the second robot arm includes preventing at least one of the first robot arm or the second robot arm from moving into the collision area. (30) The one or more non-transitory computer-readable media of embodiment 24, wherein determining that the first robotic arm is positioned adjacent to the first edge of the one or more objects includes receiving input data indicating that the first robotic arm is positioned adjacent to the first edge of the one or more objects when the first robotic arm is operating in an admittance control mode.

Claims

1. 1. A system comprising: a robotic system including a first robotic arm and a second robotic arm, at least one of the first robotic arm or the second robotic arm configured to couple to a medical device; a control circuit communicatively coupled to the robotic system, receiving input data indicating that the first robotic arm is positioned adjacent a first edge of an object in an environment; receiving additional input data indicating that the second robotic arm is positioned adjacent a second edge of the object; determining a first boundary of a region in the environment associated with the object based at least in part on a position of a distal end of the first robotic arm; determining a second boundary of the region based at least in part on a position of the distal end of the second robotic arm; and controlling at least one of the first robotic arm or the second robotic arm to move within the environment without moving into the area.

2. the first robotic arm is configured to operate in an admittance control mode in which user manipulation of the first robotic arm causes movement of the first robotic arm; The system of claim 1 , wherein the input data is received when the first robotic arm is operating in the admittance control mode.

3. The system of claim 1 , wherein the control circuitry is configured to determine the region based at least in part on a position of the robotic system, the region including the object and excluding the robotic system.

4. The control circuit displaying a visual representation of the region; receiving adjustment input data including adjustments to the visual representation; 10. The system of claim 1, further configured to: update the region based at least in part on the adjustment to the visual representation.

5. The control circuit placing the system in a treatment mode to perform a medical treatment; determining that at least one of the first robotic arm or the second robotic arm has experienced a collision; updating the region based on at least one of a position of the distal end of the first robotic arm or a position of the second robotic arm when the collision occurred.

6. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by control circuitry, cause the control circuitry to: receiving input data indicating that a first robotic arm of a robotic system is positioned adjacent to a first edge of an object in an environment; receiving additional input data indicating that a second robotic arm of the robotic system is positioned adjacent a second edge of the object; determining a first boundary of a region in the environment associated with the object based at least in part on a position of a distal end of the first robotic arm; determining a second boundary of the region based at least in part on a position of the distal end of the second robotic arm; and controlling movement of at least one of the first robotic arm or the second robotic arm to move within the environment without moving into the region, wherein at least one of the first robotic arm or the second robotic arm is configured to couple to a medical device.

7. The first robotic arm is configured to operate in an admittance control mode in which user manipulation of the first robotic arm causes movement of the first robotic arm; The one or more non-transitory computer-readable media of claim 6 , wherein the input data is received when the first robotic arm is operating in the admittance control mode.

8. The operation is determining the region based at least in part on a position of the robotic system; The one or more non-transitory computer-readable media of claim 6 , wherein the region includes the object and excludes the robotic system.

9. The operation is displaying a visual representation of the region; receiving adjustment input data including adjustments to the visual representation; updating the region based at least in part on the adjustment to the visual representation; and The one or more non-transitory computer-readable media of claim 6 , further comprising:

10. The operation is placing the system in a treatment mode to perform a medical treatment; determining that at least one of the first robotic arm or the second robotic arm has experienced a collision; and updating the region based on at least one of a position of the distal end of the first robotic arm or a position of the second robotic arm when the collision occurred.

Images