Positioning an imaging device to view a portion of an instrument during insertion of the instrument

Abstract

A computer-assisted system includes a first repositionable structure configured to support a first instrument comprising an imaging device with a field of view, a second repositionable structure configured to support a second instrument; and a control system. The first repositionable structure a first plurality of links coupled by a first plurality of joints. The control system is configured to, while the first repositionable structure is supporting the first instrument: in response to detecting an indication to track an insertion of the second instrument into a workspace, determine a configuration of the first plurality of joints that poses the field of view to include an intermediate position for a distal portion of the second instrument as the second instrument is inserted into the workspace, and, in response to determining the configuration, command the first plurality of joints toward the configuration.

Description

POSITIONING AN IMAGING DEVICE TO VIEW A PORTION OF AN INSTRUMENT DURING INSERTION OF THE INSTRUMENTRELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 514,396, filed July 19, 2023, and entitled “Positioning an Imaging Device to View a Portion of an Instrument During Insertion of the Instrument,” which is incorporated by reference herein.TECHNICAL FIELD

[0002] The present disclosure relates generally to computer-assisted systems and more particularly to positioning an imaging device to view a portion of an instrument during insertion of the instrument.BACKGROUND

[0003] Some computer-assisted systems include one or more instruments that are articulated in order to perform various procedures. The computer-assisted system can be automated, semi-automated, teleoperated, etc. In a teleoperated example, a human operator manipulates one or more leader input controls to command motion of one or more follower instruments located in a workspace. In some examples, the teleoperated system is configured to support an instrument that includes an imaging device, such as an endoscope or a camera, that enables the operator to observe the workspace. In some instances, the field of view of the imaging device is directed to enable the operator to see the instrument s) as the operator commands motion of the instrum ent(s).

[0004] In some examples, the computer-assisted system includes multiple repositionable structures, such as manipulator arms, where one or more instruments or other devices are mounted to each of the repositionable structures. In a teleoperation example, a first instrument comprising an imaging device is mounted to a first repositionable structure and is directed towards a worksite, and a second instrument, such as a catheter, electrocautery device, cutting device, grasping device, stapler, etc., is mounted to a second repositionable structure. When the second instrument is moved within a workspace to perform a task at the worksite, such as to manipulate specific tissue within the interior anatomy of a patient in a medical example, an operator often observes the movement of the second instrument in one or more images captured by the imaging device. Example medical imaging devices include endoscopes, ultrasound probes, hyperspectral sensors, etc.

[0005] It can be desirable to view the distal portion of the second instrument as the second instrument is introduced into the workspace. In a medical example, the distal portion of the second instrument comprises the end effector. Being able to observe the distal portion as the second instrument is introduced into the workspace helps the operator to understand the location of the distal portion in the workspace and relative to the worksite, and potentially avoid collisions between the distal portion and objects in the workspace (in a medical example, other instruments, patient tissue, etc.), etc. This can help increase the efficiency and effectiveness of operation, reduce operating time for a procedure, reduce likelihood of undesirable motions, etc. Thus, improved techniques for viewing instruments as they are introduced into the workspace are desirable.

[0006] Accordingly, improved techniques for viewing an instrument during insertion through a cannula of a computer-assisted system are desirable.SUMMARY

[0007] Consistent with some embodiments, a computer-assisted system includes a computer-assisted system includes a first repositionable structure configured to support a first instrument, the first instrument comprising an imaging device with a field of view, wherein the first repositionable structure supporting the first instrument comprises a first plurality of links coupled by a first plurality of joints. The computer-assisted system further includes a second repositionable structure configured to support a second instrument. The computer-assisted system further includes a control system comprising one or more processors, the control system communicatively coupled to the first repositionable structure and the second repositionable structure. The control system is configured to, while the first repositionable structure is supporting the first instrument: in response to detecting an indication to track an insertion of the second instrument into a workspace, determine a configuration of the first plurality of joints that poses the field of view to include an intermediate position for a distal portion of the second instrument as the second instrument is inserted into the workspace, and in response to determining the configuration, command the first plurality of joints toward the configuration.

[0008] Consistent with some embodiments, a method comprises, while a first repositionable structure is supporting a first instrument: in response to detecting an indication to track an insertion of the second instrument into a workspace, determining, by a control system, a configuration of a first plurality of joints that poses the field of view to include an intermediate position for a distal portion of a second instrument as a second instrument is inserted into theworkspace, and, in response to determining the configuration, commanding, by the control system, the first plurality of joints toward the configuration.

[0009] Consistent with some embodiments, one or more non-transitory machine-readable media include a plurality of machine-readable instructions which when executed by a control system are adapted to cause the control system to perform any of the methods described herein.BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a diagram of a computer-assisted system in accordance with one or more embodiments.

[0011] Figure 2 is a flow diagram of method steps for manipulating an imaging device when inserting an instrument in a computer-assisted system in accordance with one or more embodiments.

[0012] Figures 3 A-3F illustrate facilitating viewing of an instrument by an imaging device.

[0013] Figures 4A-4G illustrate facilitating viewing an instrument by an imaging device as the instrument is inserted toward a worksite.

[0014] In the figures, elements having the same designations have the same or similar functions.DETAILED DESCRIPTION

[0015] In this description, specific details are set forth describing some embodiments consistent with the present disclosure. Numerous specific details are set forth in order to provide a thorough understanding of the embodiments. It will be apparent, however, to one skilled in the art that some embodiments may be practiced without some or all of these specific details. The specific embodiments disclosed herein are meant to be illustrative but not limiting. One skilled in the art may realize other elements that, although not specifically described here, are within the scope and the spirit of this disclosure. In addition, to avoid unnecessary repetition, one or more features shown and described in association with one embodiment may be incorporated into other embodiments unless specifically described otherwise or if the one or more features would make an embodiment non-functional.

[0016] Further, the terminology in this description is not intended to limit the invention. For example, spatially relative terms-such as “beneath”, “below”, “lower”, “above”, “upper”, “proximal”, “distal”, and the like-may be used to describe the relation of one element orfeature to another element or feature as illustrated in the figures. These spatially relative terms are intended to encompass different positions (i.e., locations) and orientations (i.e., rotational placements) of the elements or their operation in addition to the position and orientation shown in the figures. For example, if the content of one of the figures is turned over, elements described as “below” or “beneath” other elements or features would then be “above” or “over” the other elements or features. A device may be otherwise oriented and the spatially relative descriptors used herein interpreted accordingly. Likewise, descriptions of movement along and around various axes include various special element positions and orientations. In addition, the singular forms “a”, “an”, and “the” are intended to include the plural forms as well, unless the context indicates otherwise. And, the terms “comprises”, “comprising”, “includes”, and the like specify the presence of stated features, steps, operations, elements, and / or components but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups. Components described as coupled may be electrically or mechanically directly coupled, or they may be indirectly coupled via one or more intermediate components.

[0017] Elements described in detail with reference to one embodiment, implementation, system, or module may, whenever practical, be included in other embodiments, implementations, systems, or modules in which they are not specifically shown or described. For example, if an element is described in detail with reference to one embodiment and is not described with reference to a second embodiment, the element may nevertheless be claimed as included in the second embodiment. Thus, to avoid unnecessary repetition in the following description, one or more elements shown and described in association with one embodiment, implementation, or application may be incorporated into other embodiments, implementations, or aspects unless specifically described otherwise, unless the one or more elements would make an embodiment or implementation non-functional, or unless two or more of the elements provide conflicting functions.

[0018] In some instances, well known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

[0019] This disclosure describes various devices, elements, and portions of computer- assisted systems and elements in terms of their state in three-dimensional space. As used herein, the term “position” refers to the location of an element or a portion of an element (e.g., three degrees of translational freedom in a three-dimensional space, such as along Cartesian x- , y-, and z-coordinates). As used herein, the term “orientation” refers to the rotationalplacement of an element or a portion of an element (e.g., three degrees of rotational freedom in three-dimensional space, such as about roll, pitch, and yaw axes, represented in angle-axis, rotation matrix, quaternion representation, and / or the like). As used herein, and for a device with a kinematic series, such as with a repositionable structure with a plurality of links coupled by one or more joints, the term “proximal” refers to a direction toward a base of the kinematic series, and “distal” refers to a direction away from the base along the kinematic series.

[0020] As used herein, the term “pose” refers to the multi-degree of freedom (DOF) spatial position and orientation of a coordinate system of interest attached to a rigid body. In general, a pose includes a pose variable for each of the DOFs in the pose. For example, a full 6-DOF pose for a rigid body in three-dimensional space would include 6 pose variables corresponding to the 3 positional DOFs (e.g„ x, y, and z) and the 3 orientational DOFs (e.g., roll, pitch, and yaw). A 3-DOF position only pose would include only pose variables for the 3 positional DOFs. Similarly, a 3-DOF orientation only pose would include only pose variables for the 3 rotational DOFs. Further, a velocity of the pose captures the change in pose over time (e.g., a first derivative of the pose). For a full 6-DOF pose of a rigid body in three-dimensional space, the velocity would include 3 translational velocities and 3 rotational velocities. Poses with other numbers of DOFs would have a corresponding number of velocities translational and / or rotational velocities.

[0021] Aspects of this disclosure are described in reference to computer-assisted systems, which can include devices that are teleoperated, externally manipulated, autonomous, semiautonomous, and / or the like. Further, aspects of this disclosure are described in terms of an implementation using a teleoperated surgical system, such as the da Vinci® Surgical System commercialized by Intuitive Surgical, Inc. of Sunnyvale, California. Knowledgeable persons will understand, however, that inventive aspects disclosed herein may be embodied and implemented in various ways, including teleoperated and non-teleoperated, and medical and non-medical embodiments and implementations. Implementations on da Vinci® Surgical Systems are merely exemplary and are not to be considered as limiting the scope of the inventive aspects disclosed herein. For example, techniques described with reference to surgical instruments and surgical methods may be used in other contexts. Thus, the instruments, systems, and methods described herein may be used for humans, animals, portions of human or animal anatomy, industrial systems, general robotic, or teleoperated systems. As further examples, the instruments, systems, and methods described herein may beused for non-medical purposes including industrial uses, general robotic uses, sensing or manipulating non-tissue work pieces, cosmetic improvements, imaging of human or animal anatomy, gathering data from human or animal anatomy, setting up or taking down systems, training medical or non-medical personnel, and / or the like. Additional example applications include use for procedures on tissue removed from human or animal anatomies (with or without return to a human or animal anatomy) and for procedures on human or animal cadavers. Further, these techniques can also be used for medical treatment or diagnosis procedures that include, or do not include, surgical aspects.

[0022] Figure 1 is a simplified diagram of an example computer-assisted system 100, according to various embodiments. In some examples, the computer-assisted system 100 is a teleoperated system. In medical examples, computer-assisted system 100 can be a teleoperated medical system such as a surgical system. As shown, computer-assisted system 100 includes a follower device 104 that can be teleoperated by being controlled by one or more leader devices (also called “leader input devices” when designed to accept external input), described in greater detail below. Systems that include a leader device and a follower device are referred to as leader-follower systems, and also sometimes referred to as master-slave systems. Also shown in Figure 1 is an input system that includes a workstation 102 (e.g., a console), and in various embodiments the input system can be in any appropriate form and may or may not include a workstation 102.

[0023] In the example of Figure 1, workstation 102 includes one or more leader input devices 106 that are designed to be contacted and manipulated by an operator 108. For example, workstation 102 can comprise one or more leader input devices 106 for use by the hands, the head, or some other body part(s) of operator 108. Leader input devices 106 in this example are supported by workstation 102 and can be mechanically grounded. In some embodiments, an ergonomic support 110 (e.g., forearm rest) can be provided on which operator 108 can rest his or her forearms. In some examples, operator 108 can perform tasks at a worksite near follower device 104 during a procedure by commanding follower device 104 using leader input devices 106.

[0024] A display unit 112 is also included in workstation 102. Display unit 112 can display images for viewing by operator 108. Display unit 112 can be moved in various degrees of freedom to accommodate the viewing position of operator 108 and / or to provide control functions as another leader input device. In the example of computer-assisted system 100, displayed images can depict a worksite at which operator 108 is performing various tasks bymanipulating leader input devices 106 and / or display unit 112. In some examples, images displayed by display unit 112 can be received by workstation 102 from one or more imaging devices arranged at a worksite. In other examples, the images displayed by display unit 112 can be generated by display unit 112 (or by a different connected device or system), such as for virtual representations of tools, the worksite, or for user interface components.

[0025] When using workstation 102, operator 108 can sit in a chair or other support in front of workstation 102, position his or her eyes in front of display unit 112, manipulate leader input devices 106, and rest his or her forearms on ergonomic support 110 as desired. In some embodiments, operator 108 can stand at the workstation or assume other poses, and display unit 112 and leader input devices 106 can be adjusted in position (height, depth, etc.) to accommodate operator 108.

[0026] In some embodiments, the one or more leader input devices 106 can be ungrounded (ungrounded leader input devices being not kinematically grounded, such as leader input devices held by the hands of operator 108 without additional physical support). Such ungrounded leader input devices can be used in conjunction with display unit 112. In some embodiments, operator 108 can use a display unit 112 positioned near the worksite, such that operator 108 manually operates instruments at the worksite, such as a medical instrument in a medical example, while viewing images displayed by display unit 112.

[0027] Computer-assisted system 100 also includes follower device 104, which can be commanded by workstation 102. In a medical example, follower device 104 can be located near an operating table (e.g., a table, bed, or other support) on which a patient can be positioned. In some medical examples, the worksite is provided on an operating table, e.g., on or in a patient, simulated patient, or model, etc. (not shown). The follower device 104 shown includes a plurality of repositionable structures 120, which are sometime referred to as manipulator arms. Each of the repositionable structures 120 is configured to couple to an instrument assembly 122. An instrument assembly 122 can include, for example, an instrument 126. In various embodiments, examples of instruments 126 include, without limitation, a sealing instrument, a cutting instrument, a sealing-and-cutting instrument, a suturing instrument (e.g., a suturing needle), a needle instrument (e.g., a biopsy needle), or a gripping or grasping instrument (e.g., clamps, jaws), and / or the like. As shown, each instrument assembly 122 is mounted to a distal portion of a respective repositionable structure 120. The distal portion of each repositionable structure 120 further includes a cannula mount 124 which is configured to have a cannula (not shown) mounted thereto. When a cannula ismounted to the cannula mount, a shaft of an instrument 126 passes through the cannula and into a worksite, such as a surgery site during a surgical procedure. A force transmission mechanism 130 of the instrument assembly 122 can be connected to an actuation interface assembly 128 of the repositionable structure 120 that includes drive and / or other mechanisms controllable from workstation 102 to transmit forces to the force transmission mechanism 130 to actuate the instrument 126.

[0028] In various embodiments, one or more of instruments 126 can include an imaging device for capturing images. The imaging device can include any combination of an endoscope, an optical camera, a hyperspectral camera, ultrasonic sensors, a monoscopic imager, a stereoscopic imager, and / or the like. The imaging device can capture images in any frequency spectrum, including visible light, infrared light, ultraviolet light, and / or the like. The imaging device can include an illumination source to light the region being imaged. The imaging device can be mounted on a straight (e.g., 0 degree) shaft, an angled (e.g., 20 degree, 30 degree, 35 degree, etc.) shaft, and / or the like. For example, one or more of instruments 126 can be an endoscope assembly that includes an imaging device, which can provide captured images of a portion of the worksite to be displayed via display unit 112.

[0029] In some embodiments, the repositionable structures 120 and / or instrument assemblies 122 can be controlled to move and articulate instruments 126 in response to manipulation of leader input devices 106 by operator 108, and in this way “follow” the leader input devices 106 through teleoperation. This enables the operator 108 to perform tasks at the worksite using the repositionable structures 120 and / or instrument assemblies 122. In some embodiments, a repositionable structure of a computer-assisted device can include a plurality of links that are rigid members and joints that are movable components that can be actuated to cause relative motion between adjacent links. For a surgical example, the operator 108 can direct the follower repositionable structures 120 to move instruments 126 to perform surgical procedures at internal surgical sites through minimally invasive apertures or natural orifices.

[0030] As shown, a control system 140 is provided external to workstation 102 and communicates with workstation 102. In other embodiments, control system 140 can be provided in workstation 102 or in follower device 104. During teleoperation, as operator 108 moves leader input device(s) 106, sensed spatial information including sensed position and / or orientation information is provided to control system 140 based on the movement of leader input devices 106. Control system 140 can determine or provide control signals to follower device 104 to control the movement of repositionable structures 120, instrument assemblies122, and / or instruments 126 based on the received information and operator input. In one embodiment, control system 140 supports one or more wired communication protocols, (e.g., Ethernet, USB, and / or the like) and / or one or more wireless communication protocols (e.g., Bluetooth, IrDA, HomeRF, IEEE 1102.11, DECT, Wireless Telemetry, and / or the like).

[0031] Control system 140 can be implemented on one or more computing systems. One or more computing systems can be used to control follower device 104. In addition, one or more computing systems can be used to control components of workstation 102, such as movement of a display unit 112.

[0032] As shown, control system 140 includes a processor system 150 and a memory 160 storing a control module 170. In some embodiments, control system 140 can include one or more processors, non-persistent storage (e.g., volatile memory, such as random access memory (RAM), cache memory), persistent storage (e.g., a hard disk, an optical drive such as a compact disk (CD) drive or digital versatile disk (DVD) drive, a flash memory, a floppy disk, a flexible disk, a magnetic tape, any other magnetic medium, any other optical medium, programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a FLASH-EPROM, any other memory chip or cartridge, punch cards, paper tape, any other physical medium with patterns of holes, etc.), a communication interface (e.g., Bluetooth interface, infrared interface, network interface, optical interface, etc.), and numerous other elements and functionalities. The non-persistent storage and persistent storage are examples of non-transitory, tangible machine readable media that can include executable code that, when run by one or more processors (e.g., processor system 150), can cause the one or more processors to perform one or more of the techniques disclosed herein, including the process of method 200 described below. In addition, functionality of control system 140 can be implemented in any technically feasible software and / or hardware in some embodiments.

[0033] Various embodiments disclose techniques for use with a computer-assisted system 100 configured to support an instrument 126 including an imaging device. For convenience, the instrument including the imaging device is also referred to as “the imaging instrument.” The imaging instrument is configured to be mounted to a first repositionable structure 120. The control system 140 included in the computer-assisted system 100 drives the first repositionable structure 120 and / or the imaging instrument to automatically position and / or orient (“pose”) the field of view (FOV) of the imaging device to provide images of another instrument 126 during at least part of an insertion of another instrument 126. This techniquecan support the installation, deployment, and insertion of another instrument 126 into a workspace and / or toward a worksite in the workspace.

[0034] In an endoscopic surgery example, an instrument 126 including an imaging device (the “imaging instrument”) is already inserted into the patient. Then, another instrument 126 (the “inserted instrument”) is to be deployed to a workspace in the interior anatomy of the patient through a cannula. The cannula is inserted through an opening leading to the interior anatomy, such as through an incision or natural orifice of the patient. The inserted instrument 126 can be any appropriate instrument, and can even include a second imaging device. Before inserting the inserted instrument 126 all the way through the cannula, computer-assisted system 100 can help reposition and / or reorient the imaging device to put the imaging device in a pose where the imaging device can capture images of a position associated with the insertion of the inserted instrument 126. Examples of such a position include the end of the cannula, a predicted position of a distal portion of the instrument as the instrument is inserted, etc. Example distal portions of the inserted instrument 126 include the tip of an end effector, a clevis (for a jaws instrument), a wrist (for a wristed instrument), a predetermined section on a distal part of a shaft of the instrument, etc.

[0035] This technique is helpful to enable medical personnel to observe at least a portion of interest of the inserted instrument 126 (such as the distal end of the inserted instrument 126), as the inserted instrument 126 is inserted into the patient. In the example where the imaging device is posed to be able to capture images of the end of the cannula, the imaging device can be used to visualize the inserted instrument 126 as the inserted instrument 126 enters the workspace as applicable, emerges from the end of the cannula) and into the workspace (e.g., an interior anatomy of a patient). The imaging device can be further used to track the inserted instrument 126 as the inserted instrument 126 is inserted to a worksite within the workspace. The system facilitating such a technique can make the process easier for human operators, reduce workflow disruption, reduce time spent for instrument insertion, and increase efficiency.

[0036] In many instances, the imaging device pose needs to be changed to be able to locate such a position associated with the insertion of the inserted instrument 126 within a field of view (“FOV”) of the imaging device. In a particular medical example, the inserted instrument 126 is inserted through an opening into the workspace to perform a task at the worksite, and the opening correlates with a position to be captured by the FOV in this technique. The pose of the imaging device that directs the FOV to include the opening is often different from the poseof the imaging device that directs the FOV to include the worksite. For particular workspaces, worksites, procedures, instrument types and numbers of instruments, and the like, these poses can be very different. The benefits of the computer-assisted system 100 facilitating the process is then quite apparent.

[0037] As an example workflow, the control system 140 receives an indication that a second instrument 126 that is mounted to a second repositionable structure 120 is ready to be inserted into the workspace. In a medical example, this second instrument 126 can be inserted through an opening, such as through a lumen in a port or cannula, or a natural orifice in a patient. In response to receiving the indication, the control system 140 determines a configuration of the joints of the first repositionable structure 120 and / or a first instrument 126 that includes an imaging device (the “imaging instrument”) that poses the field of view of the imaging device to include an intermediate position. In various instances, intermediate position is a position that a distal portion of the second instrument 126 would reach as the second instrument 126 is inserted into the workspace.

[0038] In some instances, the intermediate position is fixed relative a specified linkjoint, or feature of the repositionable structure, and the control system 140 determines the intermediate position based on kinematic data of the repositionable structure. In some instances, the intermediate position is not fixed to the repositionable structure, and the control system 140 determines the intermediate position by predicting one or more positions of the distal portion during insertion.

[0039] As a specific example, the second repositionable structure 120 can be designed such that a typical insertion of a second instrument 126 supported by the second repositionable structure 120 would cause a distal portion of the second instrument 126 to traverse through a remote center of motion (RCM) of the second repositionable structure 120. In this example, the control system 140 determines a configuration of the joint(s) of the first repositionable structure 120 and / or the imaging instrument that poses the field of view of the imaging device to include the remote center of motion of the second repositionable structure 120. As another example, the instrument 126 can be inserted through an opening to reach the workspace. In this example, if the system receives or is preprogrammed with data about the location of the opening, the system can be configured to determine a configuration of the joint(s) of the first repositionable structure 120 and / or the imaging instrument that poses the field of view to include the position of the opening. In determining this configuration, the control system 140 may also determine a physical or digital zoom level of the imaging device.

[0040] As shown by these examples, the intermediate position can be determined through preset data, determined dynamically, can be determined through a combination of preset and dynamic data, or the like. Further, the intermediate position can be a position disassociated with any particular physical feature or object in a workspace, such as a distance offset from a part of the repositionable structure from an end of a distal link or mounting feature of arepositionable structure), along a direction defined by the repositionable structure (e.g„ an insertion axis, a translational axis of a prismatic link of the repositionable structure, and / or the like), etc. Alternatively, the intermediate position can be associated with tangible features or objects such as a tip of a cannula or an opening allowing access to the workspace. As some specific examples, the intermediate position can be based on a stored location of an opening or based on image analysis, kinematics, etc. As another example, information about the intermediate position can be entered manually into the system. As a particular example, the intermediate position can be coincident with a tip of a cannula mounted to the second repositionable structure 120 during operation. In this example, the system can determine the intermediate position by using kinematic data and one or more kinematic models of the second repositionable structure 120, and a geometric model of the cannula, to determine the position of the tip of the cannula. The models can be complex or simple, depending on the physical structure. For example, in some instances, the model of the cannula can comprise just offset distances from a mounting location of the cannula to the second repositionable structure 120.

[0041] To determine the configuration of the first repositionable structure to include the intermediate position, the system can use one or more kinematic models of the first repositionable structure 120 and / or the imaging instrument. For example, the system can use such model(s) to determine a configuration of the joints of the first repositionable structure 120 and / or the imaging instrument, such that the configuration would pose the FOV in a manner to include the space at and around the intermediate position. In such a pose, the imaging device can then capture images including the intermediate position, and of the space around the intermediate position.

[0042] The control system 140 then commands the first repositionable structure 120 and / or the imaging instrument to a configuration which poses the FOV of the imaging device to include the space at and around the intermediate position. In some instances, as the second instrument 126 is inserted toward the intermediate position, the control system 140 can cause the configuration of the first repositionable structure 120 and the imaging instrument to be held at least until the second instrument 126 is detected to have reached or passed theintermediate position. The second instrument 126 supported by the second repositionable structure, which is an example of an “inserted instrument 126” described above, is also referred to as “tracked instrument” in the following discussion.

[0043] In some embodiments, the control system 140 determines the location of a distal portion of the tracked instrument 126 as the tracked instrument 126 is inserted. The control system 140 can use any applicable manner of location determination, such as by visual tool tracking, by using the one or more kinematic models, etc. As the distal portion of the tracked instrument 126 reaches the intermediate position (or is within a tolerance of the intermediate position), the imaging device is in a pose that can provide images of the space around the intermediate position. In some cases, the intermediate position is set such that images of the space around the intermediate position include part or all of the opening, the distal portion of the cannula, and / or the distal portion of the tracked instrument 126 as the distal portion emerges from the tracked instrument 126 exits the opening or cannula, and / or the like (along with the surrounding workspace).

[0044] In some embodiments, after the distal portion of the tracked instrument 126 is inserted past the intermediate position (or a tolerance past the intermediate position), the control system 140 keeps the first repositionable structure 120 and the imaging instrument in the same configuration. In some embodiments, after the distal portion of the tracked instrument 126 is inserted past the intermediate position (or a tolerance past the intermediate position), the control system 140 uses the one or more kinematic models to determine and command additional configurations of the first repositionable structure 120 and / or the imaging instrument that reposition and / or reorient the FOV of the imaging device so that the imaging device can continue to capture images including the distal portion of the tracked instrument 126 as the distal portion of the tracked instrument 126 is inserted. In this way, the imaging device can continue to capture images of the distal portion of the tracked instrument 126 as further insertion of the tracked instrument 126 occurs. In some instances, this further tracking only occurs for insertion and not retraction. In some instances, this further tracking occurs for both insertion and retraction. In some instances, the system determines and commands new configurations such that the FOV of the imaging device is reoriented and / or repositioned to keep the distal portion of the tracked instrument 126 within a central region of the FOV. This updating of configurations can be periodic on a regular time sequence, event driven (e.g., triggered by an amount of movement of the tracked instrument 126 in particular directions or overall), and the like. In some embodiments, this workflow is performed only at operatorcommand, or only for particular instruments 126 or repositionable structures 120. In some embodiments, this workflow is repeated for each instrument 126 that is not the imaging instrument, as such instrument 126 is newly inserted into the workspace or re-inserted into the workspace.

[0045] Each of the one or more processors of processor system 150 can be an integrated circuit for processing instructions. For example, the one or more processors can be one or more cores or micro-cores of a processor, a central processing unit (CPU), a microprocessor, a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a graphics processing unit (GPU), a tensor processing unit (TPU), and / or the like. Control system 140 can also include one or more input devices, such as a touchscreen, keyboard, mouse, microphone, touchpad, electronic pen, or any other type of input device.

[0046] A communication interface of control system 140 can include an integrated circuit for connecting the computing system to a network (not shown) (e.g„ a local area network (LAN), a wide area network (WAN) such as the Internet, mobile network, or any other type of network) and / or to another device, such as another computing system. Further, control system 140 can include one or more output devices, such as a display device (e.g., a liquid crystal display (LCD), a plasma display, touchscreen, organic LED display (OLED), projector, or other display device), a printer, a speaker, external storage, or any other output device. One or more of the output devices can be the same or different from the input device(s). Many different types of computing systems exist, and the aforementioned input and output device(s) can take other forms. In some embodiments, control system 140 can be connected to or be a part of a network. The network can include multiple nodes. Control system 140 can be implemented on one node or on a group of nodes. By way of example, control system 140 can be implemented on a node of a distributed system that is connected to other nodes. By way of another example, control system 140 can be implemented on a distributed computing system having multiple nodes, where different functions and / or components of control system 140 can be located on a different node within the distributed computing system. Further, one or more elements of the aforementioned control system 140 can be located at a remote location and connected to the other elements over a network.

[0047] Some embodiments can include one or more components of a teleoperated medical system such as a da Vinci® Surgical System, commercialized by Intuitive Surgical, Inc. of Sunnyvale, California, U.S.A. Embodiments on da Vinci® Surgical Systems are merelyexamples and are not to be considered as limiting the scope of the features disclosed herein. For example, different types of teleoperated systems having follower devices at worksites, as well as non-teleoperated systems, can make use of features described herein.

[0048] Figure 2 is a flow diagram of method steps for manipulating an imaging device when inserting an instrument in a computer-assisted system 100 in accordance with one or more embodiments. Although the method steps are described in conjunction with the computer-assisted system 100 of Figure 1 and the examples of Figures 3A-3F and 4A-4G, persons of ordinary skill in the art will understand that any system configured to perform the method steps, in any order, is within the scope of the present disclosure. One or more of the processes 202-216 of method 200 can be implemented, at least in part, in the form of executable code stored on non-transient, tangible, machine-readable media. This executable code, when executed by a processor system the processor system 150 in the controlsystem 140), can cause the processor system to perform one or more of the processes 202-216. In some embodiments, method 200 can be performed by a control system, such as the control system 140. In some embodiments, method 200 can be applied to one or more repositionable structures, such as repositionable structures 120, of a computer-assisted system 100 to direct a field of view of an imaging device to include an intermediate position for a distal portion of an instrument as the instrument is inserted into a workspace. In some embodiments, the computer-assisted system 100 continues to maintain the field of view to include an intermediate position for the distal portion until detecting an indication to stop tracking the insertion of the instrument. In some embodiments, in response to detecting an indication to stop tracking the insertion of the second instrument 126, the control system 140 commands a display device, such as display unit 112, to display a synthesized view of the distal portion of the second instrument 126 relative to the workspace. The synthesized view can depict the second instrument 126 in poses and positions determined from a model of the second instrument 126. In some instances, the synthesized view is based on a geometric model of the second instrument 126, modified with kinematic information from the instrument 126 or the repositionable structure 120, or image data from images of the second instrument 126 previously acquired by the imaging device.

[0049] Aspects of method 200 are described with reference to Figures 3-4. Figures 3A-3F illustrate facilitating viewing an instrument by an imaging device. Figures 4A-4G illustrate facilitating viewing an instrument by an imaging device as the instrument is inserted toward a worksite. However, it is understood that the examples of Figures 3A-3F and 4A-4G are not restrictive, and that other repositionable structures, instruments, behaviors, and / or the likedepicted in Figures 3 A-3F and 4A-4G may be different for other computer-assisted systems, different repositionable structures, different imaging devices, different instruments, different DOFs, different procedures, and / or the like. Further, although method 200 illustrates processes 202-216, implementations of method 200 may have fewer, more, or different processes than illustrated. For example, various embodiments may lack one or more of processes 202 to 216. As another example, various embodiments may lack one, some, or all of process 204, 208, 210, 212, or 216. As some examples, an embodiment may include processes 202, 206, and 210, and may or may not include the other processes shown in Figure 2; an embodiment may include processes 202, 206, 208, 210, 216, and may or may not include the other processes shown in Figure 2; an embodiment may include processes 202, 206, 212, 214, and may or may not include the other processes shown in Figure 2. As a further example, some embodiments may have alternatives to one or more of processes 202 to 216. As yet a further example, some embodiments may have additional processes not illustrated in Figure 2, such as any of the other processes discussed herein. For example, an embodiment that includes process 204, may also include a process (not shown in Figure 2), to move the first repositionable structure to the first configuration when exiting from tracking the insertion of a tracked instrument.

[0050] At a process 202, a control system, such as control system 140, detects an indication to track an insertion of a second instrument into a workspace. As shown in Figure 3 A, a computer-assisted system 100 includes four repositionable structures 310(0), 312(0), 314(0), and 316(0) that are configured to support respective cannulas 320(0), 322(0), 324(0), and 326(0) used to provide access to a workspace 360(0) for respective instruments. The repositionable structures 310(0), 312(0), 314(0), and 316(0) are example repositionable structures analogous to the repositionable structures 120 of Figure 1. The cannulas 320(0), 322(0), 324(0), and 326(0) are analogous to the cannulas that can be mounted to cannula mounts 124 of Figure 1. As also shown, a first instrument 330(0) including an imaging device (also called an “imaging instrument”) is mounted onto the first repositionable structure 310(0). A display device 350(0) is used to present images captured by the imaging device of the imaging instrument 330(0) in a field of view of the imaging device. Figure 3A further shows an operator 370(0) positioning a second instrument 340(0) to be supported and controlled by the second repositionable structure 312(0).

[0051] Figures 3 A-3F depict four repositionable structures 310(0), 312(0), 314(0), and316(0), all mechanically grounded to a common kinematic base. In some other examples, the repositionable structures do not share such a common kinematic base, and are mounted to two or separate carts, configured to be mounted to walls, tables, ceilings, floors independently ofeach other, etc. As appropriate, the control system may employ registration techniques, applied with appropriate geometric models and reference frame transformations, to locate the repositionable structures, and components such as cannulas and instruments supported by the repositionable structures, relative to each other. Techniques related to registration and frame transforms include those described in U.S. Pat. 9,259,289, filed January 27, 2012, and titled “Estimation of a position and orientation of a frame used in controlling movement of a tool,” U.S. Pat. 11,534,252, filed November 13, 2018, and titled “Master / slave registration and control for teleoperation,” and U.S. Pat. Publication 2023 / 0028689, filed January 1, 2021, and titled “System and method for inter-arm registration.”

[0052] As shown in Figure 3B, the operator 370 has coupled the second instrument 340(1) onto the second repositionable structure 312(1) of the computer-assisted system 100. The other three repositionable structures 310(1), 314(1), and 316(1) remain in the configurations shown in Figure 3A. The display device 350(1) displays an image captured by the imaging device in the imaging instrument 330(1). The computer-assisted system 100 is now ready to enter instrument insertion tracking mode in order to facilitate insertion for the second instrument 340(1).

[0053] The control system can employ various techniques to detect the indication to enter the instrument insertion tracking mode. In some examples, the control system detects the indication to enter the instrument insertion tracking mode by receiving a command from the operator 370. In some embodiments, the system receives a direct command to enter the instrument insertion tracking mode from the operator 370 pressing one or more buttons. These one or more buttons can be located on the imaging instrument, the repositionable structure to which the imaging instrument is mounted, the second instrument, and / or the second repositionable structure. Additionally or alternatively, the system receives a direct command to enter the instrument insertion tracking mode from the operator 370 selecting an option directly or indirectly linked to the instrument insertion tracking mode from a user interface located locally or remotely to the first or second repositionable structure. Additionally or alternatively, the operator 370 can generate the command by uttering a voice command, by making a gesture with a hand or other body part, and / or the like.

[0054] In some examples, the control system detects the indication to enter the instrument insertion tracking mode through context or indirect commands. In some embodiments, the indication to enter the instrument insertion tracking mode comprises the control system receiving a signal indicative of a mounting of the instrument to a repositionable structure, suchas the coupling of the second instrument 340 to the second repositionable structure 312. In some embodiments, the signal indicative of a mounting of the instrument to a repositionable structure is an indication to enter the instrument insertion tracking mode at each repositionable structure of an instrument other than the imaging instrument. In some embodiments, the signal indicative of a mounting of the instrument to a repositionable structure is not an indication to enter the instrument insertion tracking mode when an instrument has been mounted to a repositionable structure and a different mode is active (e.g., a clutch mode allowing manual articulation of the repositionable structure, a guided tool change mode described further below, etc.).

[0055] In some embodiments, the control system determines a signal received to be an indication to enter the instrument insertion tracking mode only during particular modes or stages of a procedure, such as for a setup mode or setup stage, for changing the instrument mounted to a repositionable structure, etc.

[0056] In some embodiments, the system further supports a guided tool change mode, in which the system facilitates the “changing” of an instrument previously mounted to a repositionable structure. This guided tool “change” mode does not require that the instrument actually be changed. For example, an instrument can be removed for cleaning and remounted, an instrument can be removed to reload the instrument with staples, clips, sutures, or other material and then remounted, and / or the like. Techniques related to guided tool change executed by a computer-assisted system are described in U.S. Patent No. 6,645,196 Bl, filed June 16, 2000, and entitled “GUIDED TOOL CHANGE,” which is incorporated herein by reference.

[0057] In some examples, the control system detects the indication to enter the instrument insertion tracking mode by detecting the mounting of the to-be-inserted instrument before the operator 108 begins using leader input devices 106 to cause follower device 104 to perform tasks at the worksite. In some examples where the system supports a guided tool change mode, the control system detects the indication to enter the instrument insertion tracking mode by detecting an instrument removal and mounting under conditions where the guided tool change mode is not entered, or is exited before the instrument has been inserted. Examples of where such may occur include: during or after a port clutching operation, a change in an orientation or position of a link of the repositionable structure beyond a threshold rotation or distance, an application of a force or torque to the repositionable structure or the instrument beyond a threshold force or torque, a timeout, etc. In some examples, the control system detects theindication to enter the instrument insertion tracking mode by detecting that the computer- assisted system 100 was previously in a guided tool change mode, and subsequently exited the guided tool change mode.

[0058] In some examples, the control system detects the indication to enter the instrument insertion tracking mode by receiving a signal indicative of an insertion movement of an instrument along an insertion axis of the instrument. In some examples, the control system detects the indication to enter the instrument insertion tracking mode by receiving a signal indicative of a mounting of a subsequent instrument to a repositionable structure.

[0059] At a process 204, in response to detecting the indication to track the insertion of the second instrument, the control system stores a first configuration of a first repositionable structure to which a first instrument including an imaging device is mounted. The control system can subsequently retrieve the stored first configuration in later steps if needed. In some examples, to store the first configuration, the control system can record the current joint positions of the joints of the first repositionable structure and / or the imaging instrument. In some examples, the first configuration is stored as a complete kinematic description of the joints of the first repositionable structure. In some examples, the first configuration poses the field of view of the imaging device in a pre-tracking pose.

[0060] At a process 206, the control system determines a second configuration of the first repositionable structure that poses the field of view of the imaging device to include an intermediate position for a distal portion of the second instrument as the second instrument is inserted through an opening and into the workspace. The intermediate position can be determined using any appropriate technique, including any of techniques described in conjunction with Figure 1. The control system can employ any of various techniques to determine such second configuration.

[0061] In some examples, the control system determines the second configuration by determining what is visualized in the field of view of the imaging device. In some examples, the control system determines the second configuration so as to encompass some particular feature(s) or object(s) (e.g„ at least a portion of the cannula, of the target anatomy, and / or of another feature or object) in the field of view of the imaging device. The target anatomy or other object can include the target tissue, one or more instruments, other region(s) of interest, and / or the like. In some examples, the particular feature(s) or object(s) change depending on context. For example, in some instances, when a second instrument is inserted, a distal portion of the second instrument determined as the particular feature, and the control systemdetermines the second configuration to encompass the distal portion of the second instrument in the field of view.

[0062] In various instances, when a subsequent instrument is inserted, or mounted to the computer-assisted system, after the second instrument, the particular feature may or may not change. In an example, the particular feature does not change (e.g., still the distal portion of the second instrument), and the control system does not determine an updated configuration of the first repositionable structure, and determines updated configurations to continue to encompass the distal portion of the second instrument within in the FOV. In an example, the particular feature does change (e.g., to be the distal portion of the subsequent instrument), and the control system determines updated configurations to encompass the distal portion of the subsequent instrument within in the FOV. In another example, the particular feature changes to be a composite or average feature (e.g., to be both the distal portion of the second instrument and the distal portion of the subsequent instrument, or a centroid between the distal portions of the second and subsequent instruments), and the control system determines updated configurations to encompass such particular feature within in the FOV.

[0063] In some examples, the control system determines the second configuration based on the pre-tracking pose of the field of view associated with the first configuration stored at process 202. For example, in some instances, the control system is configured to determine the second configuration to be as similar to the pre-tracking field of view as reasonable, while also being able to visualize the intermediate position during insertion of the inserted instrument. This can be done using any appropriate method. For example, this can be done by characterizing different parameters of the field of view in the second configuration as compared to the pre-tracking configuration. Example parameters include: amount of translational and rotational displacement, amount of zoom difference, how much of the field of view in the pre-tracking configuration is overlapped by the field of view in the second configuration, location of the intermediate position in the field of view in the second configuration, zoom level in the second configuration, etc. These parameters can be weighed, prioritized, captured in a cost function that optimized for determining the second configuration, or the like.

[0064] Other instances may use other criteria for determining the second configuration instead of, or in addition, to the above or each other. As another example, in some instances, the control system determines the second configuration to maintain a visualization of multiple instruments simultaneously in the workspace, such as an already-present instrument at theworksite while a second instrument is being inserted, second and third instruments being inserted simultaneously, etc. Similar approaches to evaluating the visualization of these multiple instruments can be used.

[0065] In some examples, the control system determines the second configuration by determining a position and / or an orientation of the field of view of the imaging device so that the field of view includes the space around the intermediate position. The control system can determine the position and / or orientation of the field of view of the imaging device through a model of the field of view, such as based on a position or orientation of imaging sensor(s) of the imaging device, a proxy fixed relative to the field of view such as an image capture portion (e.g., a lens) of the imaging device or an instrument comprising the imaging device, and / or the like. The position and / or orientation can further use kinematic information of a fixture or manipulator supporting the imaging instrument, and forward kinematic calculations to relate the position and / or orientation to a kinematic base of such fixture or manipulator. The control system can further determine the second configuration by aligning a direction of the field of view such that the direction of the field of view intersects the intermediate position.

[0066] In some examples, the control system determines the second configuration by using inverse kinematics of the first repositionable structure and / or the imaging instrument. Inverse kinematics can be used to determine joint commands for the first repositionable structure and / or the imaging instrument, such as to position or orient the field of view of the imaging device.

[0067] In some examples, the control system determines the second configuration so that the imaging device is positioned and / or oriented to provide a wider view of the workspace. In a particular example, the control system determines the second configuration to include retracting the imaging instrument. In another example, the control system determines the second configuration such that a base of the field of view is located at or near an end portion of a cannula, or at or near an opening, through which the imaging instrument is inserted. In yet another example, the control system determines the second configuration to position the imaging device at a predetermined distance from the intermediate position (e.g., a distance along a vector pointing from the intermediate position toward a central point in the worksite), in a direction defined based on an insertion axis of the tracked instrument, to include a predetermined pose of one or more links of the first repositionable structure, a combination of one or more of the foregoing, and / or the like.

[0068] In some instances, the control system can determine the second configuration to include a change of a zoom level of the imaging device to achieve a narrower or wider view of the workspace than the initial configuration. Adjusting the zoom level of the imaging device can help bring the region around the intermediate position into the field of view the imaging device. In some embodiments, the control system can determine the second configuration to include a change in the zoom level based on the distance between the imaging portion of the imaging device and the intermediate position. The zoom mechanism can include an optical zoom mechanism, a digital zoom mechanism, physical movement of the imaging device, and / or the like.

[0069] In some instances, the control system can determine the second configuration to include a pan of the imaging device to achieve a different view of the workspace. In some embodiments, the control system can determine the second configuration to include a panning of the field of view based on the distance between the imaging device and the intermediate position. The pan mechanism can include an optical pan mechanism, a digital pan mechanism, physical movement of the imaging device, a combination of one or more of the foregoing, and / or the like.

[0070] The control system can determine the second configuration so that the intermediate position is positioned in a central region of the field of view of the imaging device when the second configuration is adopted. In some embodiments, the control system can determine positions of the joints of the first repositionable structure to locate the intermediate position in a central region of the field of view. The control system can determine the configuration of the joints of the first repositionable structure based on kinematics, image processing, using kinematic or geometric models of the first repositionable structure and the imaging instrument, accessing pre-stored parameters about the imaging device or the system itself, using real-time data received regarding the environment or the system, and / or the like.

[0071] Additionally or alternatively, the control system can determine the second configuration so that the intermediate position is not specifically positioned in a central region of the field of view of the imaging device. In some embodiments, the control system can determine the second configuration without requiring that the intermediate position be located in the central region, or by requiring that the intermediate position is off-center within the field of view of the imaging device. In some examples, the control system can use an alternative to the intermediate position defining a pose for the field of view of the imaging device. Examplealternates include a position fixed relative to a remote center of motion of the first repositionable structure, a position fixed relative to a cannula, and / or the like.

[0072] In some examples, the control system does not determine updated configurations to update the pose of the FOV and track the tracked instrument as the tracked instrument moves. In some instances, the FOV does not translate or rotate as the tracked instrument is inserted past the intermediate position, even as the distal portion of the tracked instrument is inserted past the FOV and can no longer be visualized by the imaging device. In some instances, the control system has determined the second configuration such that the FOV includes both the intermediate position and the worksite, such that the imaging device can visualize the insertion of the second instrument to the worksite without further changes in FOV pose. In some instances, a zoom level of the imaging device is changed as the second instrument is inserted, such as to zoom out and maintain the distal portion of the second instrument in the FOV as the second instrument is inserted.

[0073] In some embodiments, the control system determines the second configuration to reduce the likelihood of collision. For example, the control system can determine the second configuration with a greater preference (e.g., by applying weights, priorities, costs) for lower joint speeds, joint positions that result in less displacement of the repositionable structure and / or the imaging instrument, that place the relevant portion(s) of the repositionable structure and the imaging system at least a predetermined distance from personnel, objects, other instruments, other repositionable structures of the computer-assisted system 100, etc. In some instances, the control system determines the second configuration with greater preference for one or more degrees of freedom, such as a roll degree of freedom. With imaging instruments where the field of view is angled relative to a shaft of the imaging instrument (e.g., a 30 degree angled endoscope, in a medical example), where a center of the field of view is not coincident with the shaft axis (e.g., a stereoscopic endoscope), and / or where the field of view is not infinitely rotationally symmetric, a pure roll of the imaging instrument about the shaft axis moves the field of view without substantive change to a volume occupied by the imaging instrument.

[0074] In some examples, the control system can determine the second configuration to limit an amount of motion of the field of view of the imaging device, an amount of motion of the imaging instrument, and / or an amount of motion of the first repositionable structure when moving from the first configuration to the second configuration. The control system can limit at least one degree of freedom by allowing the imaging instrument to move only in a set ofallowed degrees of freedom while limiting movement of the imaging instrument in other nonallowed degrees of freedom. The control system can limit movement of the imaging instrument by disallowing the imaging instrument from moving in the non-allowed degrees of freedom. Alternatively, the control system can limit movement of the imaging instrument by limiting an amount of motion of the imaging instrument in the non-allowed degrees of freedom. In some embodiments, the control system determines the second configuration such that the second configuration relative to the first configuration can be reached using limited motions of the first repositionable structure and / or the imaging instrument. In some such examples, the control system can limit the motion in one or more spatial degrees of freedom, and / or joint degrees of freedom. As one example, the control system can determine a second configuration that limits the motion to be along the insertion / retraction axis of the imaging instrument. The insertion / retraction axis can be a longitudinal axis associated with a shaft of the imaging instrument. In some examples, the control system can determine a second configuration that limits the motion to rotation about the roll axis of the imaging instrument. The roll axis can be a longitudinal axis associated with the shaft of the instrument.

[0075] In some examples, the control system determines the intermediate position relative to a reference position associated with the second instrument. For example, the reference position can be a position of the opening into the workspace through which the cannula and the second instrument are inserted, a position (e.g„ distal portion) associated with the cannula, a position of a remote center of motion associated with the second repositionable structure attached to the cannula, and / or the like. In certain examples, the intermediate position can coincide with the reference position associated with the second instrument, be offset by a predefined amount from the reference position associated with the second instrument, and / or the like.

[0076] In some embodiments, the control system determines the intermediate position using forward kinematics applied to kinematic information about the second repositionable structure (such as received through joint encoders, processing of images including the second repositionable structure, etc.). Some embodiments use other techniques instead of, or in addition to, techniques involving kinematics. In some such examples, the control system uses image processing on images captured by the imaging device to segment and identify components in the images, and locate parts of the components relative to the FOV. This image processing can be performed using any suitable technique, and be used by the control system to locate a certain reference position in the FOV of the imaging instrument. Examples ofreference positions that can be used include: a position of the opening into the workspace through which the second instrument is (or will be) inserted, a position of a feature of a cannula through which the second instrument is to be inserted (e.g., a marking detectable by the imaging device, a distal end of the cannula), etc. As an example, a marking visible to the human eye is made on the cannula, and coincides with a remote center of the second repositionable structure when the cannula is attached to the second repositionable structure. In this example, when the instrument is designed to pass through this remote center of motion during insertion, the intermediate position can be defined as this remote center, and the control system can include the intermediate position in the field of view of the imaging device by locating the remote center of motion in the field of view.

[0077] In some examples, the control system determines the intermediate position by employing image processing techniques to identify or predict the intermediate position. For example, the control system can store a series of one or more image frames captured by the imaging device over a duration of time. The control system can perform image analysis on each frame included in the stored image frames to determine the intermediate position based on a reference position identified in the FOV of the imaging device. This reference position can coincide with, or be offset from, the intermediate position. In one example, the reference position can be based on the position and / or orientation of the distal portion of the instrument being inserted. The control system can determine the change in position and / or orientation of the distal portion to determine the estimated speed and direction of travel of the distal portion. Based on the estimated speed and direction of travel of the distal portion, the control system can estimate the likely position and / or orientation of the distal portion when the imaging device captures the next image frame. The control system can set the intermediate position as the likely position and / or orientation of the distal portion.

[0078] At a process 208, in response to determining the second configuration, the control system commands each of one or more other repositionable structures to move toward an avoidance configuration for reducing the likelihood of collision between the first repositionable structure and one or more other repositionable structures. To move the one or more other repositionable structures away from the motion path of the first repositionable structure, the control system determines avoidance configurations of the other repositionable structures. The avoidance configurations are kinematic configurations of the other repositionable structures that place the other repositionable structures out of one or more avoidance regions. The other repositionable structures adopting the avoidance configurationshelp reduce the likelihood of physical interference with the first repositionable structure (and / or the imaging instrument) as the first repositionable structure moves to position or orient the imaging device during insertion tracking. For example, moving the other repositionable structures out of these avoidance regions while allowing the first repositionable structure to enter and move within the avoidance regions reduces potential physical interference as the first repositionable structure moves toward the second configuration to visualize the intermediate position, and / or to further configurations to position or orient the imaging device to track the second instrument as it is inserted. In some instances, the avoidance regions change with a change in the configuration of the first repositionable structure, in the configuration of the second repositionable structure, with an amount of insertion of the second instrument, or other contextual parameters associated with the computer-assisted system, instruments or cannulas or other components supported by the computer-assisted system, the type or stage or procedure being performed, changes in the worksite or workspace, etc. In some instances, the control system also stores pre-avoidance configurations of the other repositionable structure(s). Pre-avoidance configurations are the configurations of the other repositionable structures before moving those repositionable structures to their respective avoidance configurations.

[0079] In some examples, while further tracking the insertion of the second instrument to the workspace, the control system can determine that a currently commanded configuration of the first repositionable structure would cause a collision between the first repositionable structure and another repositionable structure, or between the imaging instrument and another instrument. In response to determining that the currently commanded configuration would cause the collision, or has caused collision, with another repositionable structure (and / or another instrument), the control system determines and commands an avoidance configuration of another repositionable structure (and / or another instrument) that would reduce or avoid the collision. In some instances, the control system can command another repositionable structure (and / or another instrument) to move toward the avoidance configuration.

[0080] In some embodiments, if an instrument had already been mounted to one of the other repositionable structures and inserted into the workspace (“existing instrument”), then the control system performs additional steps before moving this repositionable structure supporting the existing instrument. As one example, the control system saves the insertion position for the existing instrument and corresponding repositionable structure. In addition, the control system retracts the existing instrument into the cannula before moving thecorresponding repositionable structure to the avoidance configuration. These additional steps can help reduce the likelihood that the existing instrument will interact in an unintended with the workspace when the corresponding repositionable structure moves away from the motion path of the first repositionable structure. Further, these additional can include moving the existing instrument to the pre-tracking insertion position, such as at or after process 216, after tracking is terminated.

[0081] Figure 3C shows the repositionable structures 314(2) and 316(2) after the repositionable structures 314(2) and 316(2) have been moved to respective avoidance configurations. As shown, the repositionable structures 314(2) and 316(2) have been angled away from the repositionable structures 310(2) and 312(2), respectively, so as to reduce the likelihood of collisions with the repositionable structures 310(2) and 312(2).

[0082] At a process 210, in response to determining the second configuration, the control system commands the first repositionable structure to move from the first configuration toward the second configuration. The control system determines a trajectory of the first repositionable structure and / or the imaging instrument that moves the first repositionable structure from the first configuration to the second configuration and that poses the field of view of the imaging device to include the intermediate position. In so doing, the control system causes the first repositionable structure to move so as to move the FOV of the imaging device to include the intermediate position.

[0083] In this example, the control system causes the first repositionable structure and / or the imaging instrument to move by transmitting commands to actuators that control joints of the first repositionable structure and / or the imaging instrument. The actuators cause motion in the degrees of freedom provided by joint(s) of the first repositionable structure and / or the imaging instrument. The actuators can be disposed in or outside, and transmit motive force (e.g., linear force or rotary torque) to, the first repositionable structure and / or the imaging instrument. Examples of actuators include rotary motors, linear motors, solenoids, and / or the like.

[0084] Figure 3C shows the second repositionable structure 312(2) in the second configuration so that the FOV of the imaging device of imaging instrument 330(2) is positioned to allow the imaging device to capture images of the intermediate position for the second instrument 340(2). As shown in Figure 3C, the second repositionable structure 312(2) of the computer-assisted system 100 has moved from the configuration shown in Figure 3B to a different configuration. As a result, the distal portion of the second instrument 340(2) hascorrespondingly moved within the workspace 360(2). (In some embodiments, the intermediate position is unchanged as the distal portion is further inserted. In some embodiments, the control system updates the intermediate position, as discussed further below).Correspondingly, the distal portion 352(2) of the second instrument 340(2) is in the field of view of the imaging device of imaging instrument 330(2), as shown by the image displayed by display device 350(2).

[0085] Figure 4A shows a cross-sectional view of a workspace 460(0) and a worksite 462(0) within the workspace 460(0). A first cannula 420(0) is supported by a first repositionable structure 410(0). In some examples, an imaging device 432(0) is attached to a first instrument 430(0) (also, an “imaging instrument”). In some examples, an imaging device 432(0) is included in, and is part of, the first instrument 430(0). The imaging device 432(0) and a distal portion of the shaft of the first instrument 430(0) are inserted into the workspace 460(0) via the first cannula 420(0). The FOV of the imaging device 432(0) is directed so that the imaging device 432(0) is able to capture images of the worksite 462(0). A second cannula 422(0) is supported by a second repositionable structure 412(0). In this example, each of the first and second cannulas 420(0), 422(0) has a black band that, when mounted to the first and second repositionable structures 410(0), 412(0), coincides with a remote center of the first and second repositionable structures 410(0), 412(0). As shown, the second repositionable structure 412(0) is ready to receive an instrument. For clarity of illustration, Figure 4A shows only part of the first and second repositionable structures 410(0), 412(0).

[0086] Figure 4B illustrates mounting of a second instrument 440(1) to the second repositionable structure 412(1). The second instrument 440(1) includes an end effector 442(1) in the form of a gripping device. The end effector 442(1) and a distal portion of the shaft of the second instrument 440(1) are inserted into the workspace 460(1) via the second cannula 422(1). The instrument insertion tracking mode is not yet enabled, and the imaging device 432(1) and the first instrument 430(1) are physically configured such that the field of view of the imaging device 432(1) remains directed in the same way as in Figure 4A. The first instrument 430(1) includes a shaft and an imaging device 432(1), where the imaging device may or may not articulate in relation to the shaft. Therefore, the field of view of the imaging device 432(1) can be changed by re-configuring the first instrument 430(1), which may include re-configuring the shaft and / or articulating the imaging device 432(1).

[0087] Figure 4C illustrates the movement of the imaging device 432(2) after the instrument insertion tracking mode is enabled, and the control system has caused the firstrepositionable structure to adopt a configuration for enabling the field of view of the imaging device to visualize an intermediate position. In the example shown, the intermediate position is located at the distal portion (e.g., a distal end) of the cannula 422(2), and is offset from the marking on the cannula 422(2). In Figure 4C, the first repositionable structure 410(2), the first instrument 430(2), and the imaging device 432(2) are moved to direct the FOV of the imaging device 432(2) so that the FOV includes the distal portion of the second cannula 422(2). The end effector 442(2) and distal portion of the second instrument 440(2) are still inserted into the second cannula 422(2) but have not yet entered the workspace 460(2). The second instrument 440(2) is also called the “inserted instrument” and the “tracked instrument.”

[0088] In the example shown in Figure 4C, the control system has caused the first instrument 430(2) to articulate the imaging device relative to a shaft of the first instrument 430(2) in posing the FOV. In various other examples shown in Figures 4A-4G, the imaging instrument is articulated from a previous configuration. However, in other examples, the control system does not cause the imaging instrument to articulate the imaging device relative to a shaft of the imaging instrument; this may be because the control system has determined configurations that do not include articulating the imaging device relative to the shaft, or because the imaging instrument lacks joint(s) that enable the field of view to move relative to the shaft. The technique described in Figures 2-4 can be applied with imaging instruments with or without the ability to articulate the imaging device relative to the shaft. In various instances, the configurations determined by the controller for the first repositionable structure differ depending on the amount and direction that the imaging instrument can articulate the imaging device.

[0089] Further, the examples shown in Figures 4A-4G generally illustrate tracking of the end effector of the tracked instrument. In various other examples, the tracking may be of a different part of the tracked instrument, such as an instrument joint proximal to an end effector, or some other distal portion of the instrument that is not the end effector.

[0090] Figure 4D illustrates no further tracking by the FOV of the imaging device 432(3) as the end effector 442(3) (and the distal portion of the second instrument 440(3)) have been inserted past the distal end of the second cannula 422(3) and into the workspace 460(3). The end effector 442(3) has inserted past the distal end of the second cannula 422(3), towards the worksite 462(3). The first repositionable structure 410(3), the first instrument 430(3), and the imaging device 432(3) have not moved from the configurations shown in Figure 4C. In the instance shown in Figure 4D, the FOV of the imaging device 432(3) still includes the endeffector 442(3) and the distal portion of the second instrument 440(3). In another example, the control system causes further tracking by the FOV of the imaging device 432(3) as the end effector 442(3) is further inserted, and the first repositionable structure 410(3), the first instrument 430(3), and the imaging device 432(3) to change from the configurations shown in Figure 4C. The further tracking may be to keep the end effector 442(3) in a central region of the FOV of the imaging device 432(3).

[0091] In some examples, the control system also generates haptic feedback by changing the resistance on the tracked instrument as the instrument is inserted further into the workspace. The resistance can be based on an amount of insertion of the tracked instrument. For example, greater resistance for larger amounts of insertion can lead to slower insertion. (As one example, the control system commands actuators of the second instrument or the second repositionable structure to provide resistance as a function of insertion depth, with higher resistance correlating to greater insertion depth). In some instances, adjusting resistance can reduce insertion speeds on deeper instruments, and can serve to improve synchronized visualization of multiple instruments.

[0092] In some examples, the control system can determine the second configuration by employing various techniques to determine shorter motion paths for the first repositionable structure, the imaging device, and / or the imaging instrument. In some instances, shorter motion paths provide lower energy expenditure, faster response, more efficient movement, and / or better user experience. The control system can determine the shortest path of the imaging device, or imaging instrument, in the workspace. Additionally or alternatively, the control system can determine as the shortest path the smallest amount of movement of one or more joints of the repositionable structure and / or one or more joints of the imaging instrument within their respective joint spaces.

[0093] In some examples, the control system can determine the second configuration while preferring (e.g„ by applying weights, priorities, optimizing cost functions, limiting degrees of freedom or amounts of motion, etc.) motion paths that include joint rotation over motion paths that include joint translation. The control system can use weighted objectives to reduce the change in the spatial volume occupied by the imaging instrument as the imaging device is moved to adjust the FOV. This can help reduce the potential interference to motion of the imaging instrument due to other objects, materials, anatomy, and / or the like in the workspace. For example, in some cases, the control system can prefer rolling the imaging instrument about a shaft of the imaging instrument. In many embodiments, rolling the imaging instrumentabout a shaft of the imaging instrument causes little or no change to the volume occupied by the imaging instrument.

[0094] In some examples, the control system can determine motion paths that combine multiple motion components (e.g., translate and rotate contemporaneously). In some examples, the control system can determine motion paths that use separate sequences of moves. In a particular example, when commanding the joints of the first repositionable structure and / or the imaging instrument towards the second configuration, the control system commands the joints to translate and / or rotate the field of view sequentially. As one example, the control system can determine motion paths that isolate Cartesian movements of the field of view (such as by separating translation movements from rotation movements), by moving each individual joint in sequence, and / or the like. In some examples, the control system can determine motion paths that do not use separate sequences of moves, and instead comprise combined, simultaneous motion paths of all of the degrees of freedom to be moved.

[0095] In some examples, the control system can determine motion paths that reduce energy consumption, such as by determining motion paths that minimize joint motion, motion paths that consider the mass and moment of inertia of the repositionable structures and instruments, and / or the like. In some examples, the control system can determine motion paths improve user experience, such as selecting motion paths with cinematic camera motion.

[0096] In some examples, the computer-assisted system 100 further includes a human interface system configured to provide feedback detectable by a human. In such examples, the control system can determine that the joints of the first repositionable structure and / or the joints of the imaging instrument cannot achieve the second configuration. In response to determine that the joints of the first repositionable structure and / or the joints of the imaging instrument cannot achieve the second configuration, the control system can cause the human interface system to provide a human-detectable indication. The human-detectable indication can be one or more visual indications, audio indications, haptic indications, and / or the like.

[0097] At a process 212, the control system commands the first repositionable structure to track the distal portion of the second instrument as the second instrument is further inserted into the workspace. In some embodiments, the control system determines a current position of the distal portion after the distal portion is inserted past the intermediate position. The control system determines an updated configuration of the first plurality of joints that poses the field of view based on the current position. In response to determining the updated configuration, the control system commands the first repositionable structure to move the first plurality ofjoints toward the updated configuration. In some examples, in order to determine the updated configuration of the first plurality of joints that poses the field of view based on the current position, the control system determines an updated intermediate position based on the current position. The control system determines the updated configuration to pose the field of view to include the updated intermediate position.

[0098] In some examples, after the distal portion of the tracked instrument has been inserted past the intermediate position, the control system commands the first repositionable structure and / or the imaging instrument to redirect the field of view, and keep the distal portion of the tracked instrument within at least a portion of the field of view. In some instances, the control system causes motion of the first repositionable structure and / or the imaging instrument to keep a direction of the field of view pointed toward the distal portion or other portion of the tracked instrument.

[0099] In the example shown in Figure 3D, the repositionable structures 314(3) and 316(3) of the computer-assisted system 100 remain in their respective avoidance configurations as the operator 370(3) moves the instrument 340(3) mounted to the second repositionable structure 312(3) and cause the instrument 340(3) to be inserted into the workspace. The first repositionable structure 310(3) continues to move so as to maintain the distal portion of the second instrument 340(3) within the field of view of the imaging device included in imaging instrument 330(3). Correspondingly, the imaging device can provide real-time images including the distal portion 352(3) of the second instrument 340(3) as the second instrument 340(3) is further inserted into the workspace, as shown by the image displayed by display device 350(3).

[0100] Figure 4E shows a first repositionable structure 410(4) supporting a first cannula 420(4) and a first instrument 430(4) with an imaging device 432(4) as well as a second repositionable structure 412(4) supporting a second cannula 422(4) and a second instrument 440(4) with an end effector 442(4). The end effector 442(4) and a distal portion of the second instrument 440(4) have been inserted past the distal end of the second cannula 422(4) and towards the worksite 462(4). The imaging device 432(4) and the first instrument 430(4) are inserted into the workspace 460(4). The first repositionable structure 410(4), the first instrument 430(4), and the imaging device 432(4) have moved from previous configurations to the configurations shown in Figure 4E, such that the FOV of the imaging device 432(4) is able to capture images of the end effector 442(4) and the distal portion of the second instrument 440(4). An example of previous configurations are configurations similar to that shown inFigure 4D for first repositionable structure 410(3), first instrument 430(3), and imaging device 432(3). As the end effector 442(4) is further inserted toward the worksite 462(4), the control system causes the first repositionable structure 410(4), the first instrument 430(4), and the imaging device 432(4) to move to further configurations in order to pose the FOV such that the end effector 442(4) is maintained in the FOV.

[0101] Figure 4F shows the end effector 442(5) as the end effector 442(5) is further inserted beyond the distal end of the second cannula 422(5) and has reached the worksite 462(5). The control system has caused the first instrument 430(5) to articulate the imaging device 432(5) to track the end effector 442(5) such that the FOV of the imaging device 432(5) is directed to be able to capture images of the end effector 442(5). In the example shown, the FOV is also directed such that the FOV includes the distal portion of the second instrument 440(5) and the worksite 462(5). In the example shown in Figure 4F, the control system has determined that the first repositionable structure can maintain the configuration from Figure 4E in tracking. In other examples, the control system causes the first repositionable structure 410 to change in configuration, and the first instrument 430 not to change in configuration, to track the end effector 442. In yet another example, the control system causes both the first repositionable structure 410 and the first instrument 430 to change in configuration to track the end effector 42. In yet other examples, a distal portion of the second instrument 440, other than the end effector, is tracked.

[0102] In some embodiments, during the further tracking, the control system performs steps to determine the position of the distal portion of the tracked instrument as the instrument is further inserted past the intermediate position. In a first step, the control system determines the current position of the distal portion of the tracked instrument relative to the position associated with the imaging device. The control system determines this current position by employing techniques similar to the techniques described in conjunction with process 206. In a second step, the control system determines the command to the first repositionable structure and / or the imaging instrument by employing techniques similar to the techniques described in conjunction with process 210. The control system employs these techniques to maintain the distal portion of the tracked instrument within the field of view of the imaging device, such as within a central region or a peripheral region of the field of view of the imaging device. In some instances, the control system maintains the distal portion of the tracked instrument within the field of view of the imaging device by causing adjustment of a zoom level of the imaging device. In some examples, the control system adjusts the zoom level by maintaining a desirable instrument-to-field-of-view size ratio. The mechanism to adjust the zoom level caninclude an optical zoom mechanism, a digital zoom mechanism, physical movement of the imaging device, and / or the like. In some instances, the control system maintains the distal portion of the tracked instrument within the field of view of the imaging device by causing panning of the imaging device. The mechanism to adjust the pan can include an optical pan mechanism, a digital pan mechanism, physical movement of the imaging device, and / or the like.

[0103] At a process 214, the control system detects an indication to stop tracking the insertion of the tracked instrument to the workspace and / or exit the instrument insertion tracking mode. The control system can employ various techniques to detect the indication to stop tracking the insertion of the tracked instrument to the workspace.

[0104] In some examples, the control system detects the indication to stop tracking the insertion of the instrument by detecting an indication to track an insertion of an instrument into the workspace, where the new instrument is different from the current instrument being tracked. In such examples, the control system stops tracking the insertion of the first instrument and begins tracking the insertion of the new instrument.

[0105] In some examples, the control system detects the indication to stop tracking the insertion of the tracked instrument by receiving a command from the operator 370 to exit the instrument insertion tracking mode. The operator 370 can generate the command by pressing one or more buttons, selecting an option on a user interface, uttering a voice command, by making a hand and / or body gesture, by entering a command via a user interface, and / or the like.

[0106] As shown in Figure 3E, in some embodiments, the operator 370(4) pushes a button associated with the imaging instrument 330(4) while the FOV of the imaging device visualizes the distal portion 352(4) of the second instrument 340(4). In response, the control system stops tracking the insertion of the second instrument 340(4) into the workspace 360(4).

[0107] In some examples, the control system detects the indication to stop tracking the insertion of the instrument by determining that the insertion of the tracked instrument by the operator 370 has stopped, or that the speed of insertion is below a threshold speed. In so doing, the control system can apply a time-out duration for a predetermined period of time. If the insertion of the tracked instrument has stopped and / or the speed of insertion is below the threshold speed for at least the predetermined period of time, then the control system determines that tracking the insertion of the instrument should stop.

[0108] In some examples, the control system detects the indication to stop tracking the insertion of the second instrument by detecting that a force applied to the first repositionable structure, on the imaging instrument, or another repositionable structure or instrument, has exceeded a threshold level. Stopping tracking in response to such force can reduce the chances of damage to the repositionable instrument, damage to the instrument, unintended motion of the instrument within the workspace, and / or the like.

[0109] In some examples, the control system detects the indication to stop tracking the insertion of the tracked instrument by detecting that the operator 370 is moving (or commanding motion of) the tracked instrument in a direction other than along the insertion axis, or in a direction deviating from some other expected insertion direction. In some examples, the control system can detect the indication to stop tracking the insertion of the instrument by detecting that the operator 370 is retracting the tracked instrument or commanding the tracked instrument to retract, rolling the tracked instrument about an insertion axis of the tracked instrument, moving the tracked instrument laterally relative to the insertion axis, operating an end effector of the tracked instrument (such as opening a jaw device, closing a jaw device, firing a staple, etc.), and / or the like. In some examples, such non-insertion-axis motion may indicate aberrant input. In some examples, such non-insertion-axis motion may indicate that insertion has been completed.

[0110] In some examples, the control system detects the indication to stop tracking the insertion of the instrument by detecting that the distal portion of instrument has been inserted by at least a threshold distance. In some embodiments, the control system can detect insertion of the instrument past the field of view of the imaging device, past a defined region within the field of view of the imaging device, past a threshold distance from the end of the cannula, past a threshold distance from the end of the opening into the workspace, and / or the like.[OHl] In some examples, the control system detects the indication to stop tracking the insertion of the instrument by detecting that the tracked instrument has arrived at the worksite within the workspace. The control system can detect such arrival by determining that a distal portion of the tracked instrument is within a tolerance of a target location, in a worksite space, within a region defined based on already-inserted instruments or previously inserted instruments, and the like. In some instances, the arrival of the tracked instrument at the worksite indicates that insertion of the tracked instrument is complete, and that the tracked instrument is in position to begin a procedure.

[0112] In some examples, the control system detects the indication to stop tracking the insertion of the tracked instrument by detecting that a pose of the field of view of the imaging device is within some tolerance of a target pose. Examples of such target poses include a pose that visualizes all deployed instruments within the field of view, a pose that visualizes a defined worksite where a procedure is to be performed, a pose that visualizes the field of view prior to tracking insertion of the instrument, and / or the like.

[0113] In some examples, the control system detects the indication to stop tracking the insertion of the tracked instrument by detecting an actual or pending collision of the imaging instrument with another object in the workspace. Similarly, the control system can detect the indication to stop tracking the insertion of the tracked instrument by detecting collision of an instrument other than the imaging instrument with another object in the workspace. The collision can be detected and / or measured by image processing techniques, 3D models of the workspace from a prior CT scan, MRI, etc., vision-based depth mapping techniques, and / or the like.

[0114] In some examples, the control system detects the indication to stop tracking the insertion of the tracked instrument by detecting a distance change beyond a safety threshold between the tracked instrument and an object in the workspace, such as a portion of anatomy, an object in the workspace environment, another instrument, and / or the like. For example, the safety threshold corresponds to the minimum distance to be maintained by the system to prevent a collision between the instrument and the object.

[0115] In some examples, the control system detects the indication to stop tracking the insertion of the tracked instrument by determining that a current pose of the field of view of the imaging device is within a tolerance of the pre-tracking pose stored at process 204.

[0116] At a process 216, the control system commands the one or more other repositionable structures to move toward their respective pre-avoidance configurations (e.g„ the respective pre-avoidance configurations stored during process 208) after the tracking of the second instrument has stopped. As shown in Figure 3F, the repositionable structures 314(5) and 316(5) of the computer-assisted system 100 have moved to their respective pre-avoidance configurations. The first repositionable structure 310(5) and the imaging instrument 330(5) maintain their configurations, such that the distal portion of the instrument 340(5) is maintained within the field of view of the imaging device.

[0117] The computer-assisted system 100 is now ready to begin a procedure. Alternatively, the computer-assisted system 100 is now ready to command the first repositionable structure 310(5) and / or imaging instrument 330(5) to begin tracking the insertion into the workspace 360(5) of a new instrument that is mounted, or is soon to be mounted, onto repositionable structure 314(5) or repositionable structure 316(5). In some embodiments, when an instrument had already been mounted to one of the other repositionable structures and that instrument was retracted during process 208, the control system restores the instrument to the insertion position stored during process 208.

[0118] In some embodiments, the method 200 then returns to process 202 and repeats processes 202-216 when tracking of additional instrument insertions is indicated. As an example in that regard, Figure 4G illustrates an additional instrument 444(6) with an additional end effector 446(6) being inserted into the workspace 460(6). The additional end effector 446(6) is in the form of a scalpel, and is entering the workspace 460(6) via a cannula 424(6). The additional instrument 444(6) is supported by an additional repositionable structure 414(6). The first instrument 430(6) is retracted within the workspace 460(6) toward the first cannula 420(6) such that the FOV of the imaging device 432(6) has a wider view of the workspace 460(6). The FOV of the imaging device 432(6) has been changed in pose to include the distal portion of the second instrument 440(6) and the distal portion of the additional instrument 444(6). In the particular example shown in Figure 4G, the FOV is directed so that the imaging device 432(6) is able to capture images of the end effector 442(6), the distal portion of the second instrument 440(6), the worksite 462(6), the additional end effector 446(6), and the distal portion of the additional instrument 444(6).

[0119] As discussed above and further emphasized here, Figures 2, 3A-3F, and 4A-4G are merely examples which should not unduly limit the scope of the claims. Many variations, alternatives, and modifications are possible. In some embodiments, the control system causes generation of haptic feedback to help guide the operator 370 when performing the processes of method 200. In some examples, the control system can cause the computer-assisted system 100 to enter the instrument insertion tracking mode at the time of mounting an instrument to the second repositionable structure. Additionally or alternatively, the control system can cause the computer-assisted system 100 to enter the instrument insertion tracking mode at a time other than the time of mounting an instrument to the second repositionable structure. In that regard, the control system can cause the computer-assisted system 100 to enter the instrument insertion tracking mode for a particular repositionable structure without an instrument firstbeing mounted to the particular repositionable structure. The computer-assisted system 100 can enter the instrument insertion tracking mode for the particular repositionable structure by using any of the techniques described herein in conjunction with process 202.

[0120] In some embodiments, the control system restores, at or after process 216, the first repositionable structure to the pre-tracking configuration. This helps to return the field of view of the imaging device to the pre-tracking field of view.

[0121] In some embodiments, the control system can maintain one or more additional or alternative positions in the field of view of the imaging device (additional or alternative in being other than the intermediate position). Examples of such additional or alternative positions include any relevant location that can be located relative to: the intermediate position, a feature of the tracked instrument, a feature of the repositionable structure supporting the tracked instrument, and / or a position of a part of the tracked instrument. In a first example, an alternative position can be an opening into the workspace, a distal portion of a cannula, a distal portion of any other instrument introduced into the workspace, a part of the worksite, a target object, and / or the like. In a second example, an additional or alternative position can be a fixed distance (such as 15 mm, 20 mm, 30 mm, and / or the like) beyond the intermediate position and toward the worksite. In a third example, an additional or alternative position can be any other relevant location relative to the intermediate position and / or an instrument.

[0122] In some embodiments, when the instrument insertion tracking mode for the current instrument is stopped (such as during process 214 because a new instrument is mounted to the second repositionable structure, or to another repositionable structure), then the control system omits process 216 and returns to process 206 to track the insertion of the new instrument. In some embodiments, when this occurs, the repositionable structures, other than the repositionable structure to which the new instrument is mounted, remain in their respective avoidance configurations or are moved to new respective avoidance configurations. This can be done by repeating process 208 to account for the intermediate position for the new instrument. These repositionable structures can remain in these avoidance configurations as the new instrument is mounted to a repositionable structure and inserted into the workspace (or until the instrument insertion tracking has been stopped).

[0123] In some embodiments, the control system is unable to pose the field of view to locate the intermediate position, or alternative position, in a central region of the field of view.In some such cases, the control system determines an expected location in the field of view of the imaging device. This expected location is a location where the intermediate position would be if the imaging device and / or the first repositionable structure are in the desired configuration. The control system then determines whether the intermediate position, or alternative position is within a tolerance range of the expected location in the field of view of the imaging device. This can be done while tracking the insertion of a tracked instrument and with the first repositionable structure and / or the imaging instrument in the configuration. If the intermediate position is not within a tolerance of the expected location in the field of view, then the control system performs at least one precautionary action. In various examples, that at least one precautionary action can includes one or more of generating a warning indication to provide to the operator 370, halting motion of the first repositionable structure, the second repositionable structure, and / or other repositionable structure(s) of the system, halting motion of the first instrument, the second instrument, and / or other instrum ent(s) supported by the system, commanding a retraction of the first instrument, the second instrument, and / or other instrum ent(s) supported by the system, etc., inhibiting the further insertion of the second instrument, commanding the first fist repositionable structure to move the field of view of the imaging device toward a target pose (e.g., a pre-tracking field of view of the imaging device), inhibiting further movement of the joints of the imaging instrument and / or the joints of the first repositionable structure, exiting the instrument insertion tracking mode for the second instrument, or the like. In some examples, if the instrument being inserted is not in view of the field of view of the imaging device, then the at least one precautionary action includes locking out instrument movement and / or commands to the repositionable structure(s) in order to prohibit further insertion of the instrument.

[0124] In some embodiments, the control system does not store previous positions of the distal portions of the instruments. In some embodiments, the control system stores previous positions of the distal portions of the instruments. After an instrument insertion is completed, the control system stores the position of the distal portion of that instrument. After that instrument is retracted into the cannula, the corresponding repositionable structure supporting that instrument is moved toward an avoidance configuration (as described in conjunction with process 208). Then, the control system can cause a virtual depiction of the distal portion of the retracted instrument to be displayed in conjunction with the image captured by the imaging device. This augmented display can aid the operator 370 in guiding the remaining instruments into the workspace.

[0125] In some embodiments, the control system tracks and assists in the insertion of multiple instruments. In such examples, the control system determines which instrument to track at any given time and determines how to continue insertion assistance from a current instrument to a different instrument. If the operator 370 generates a command to enter tracking for multiple instruments, then the control system selects an instrument for insertion tracking. The control system can select the instrument that was installed earliest (similar to a “first-in- first-ouf ’ approach), the instrument that was most recently installed (similar to a “last-in-first- ouf ’ approach), the instrument that was installed immediately after the imaging instrument was installed or inserted, and / or the like. After exiting the instrument insertion tracking mode for a current instrument, the control system selects a next instrument for insertion tracking. If the control system initially selects the instrument that was installed earliest for insertion tracking, then the control system can subsequently select the instrument that was next installed for insertion tracking, and so on. Similarly, the control system initially selects the instrument that was most recently installed for insertion tracking, then the control system can subsequently select the next most recently installed for insertion tracking, followed by the third to last instrument, and so on.

[0126] In some embodiments, the control system maintains the intermediate position within the field of view of the imaging device by causing the imaging instrument to articulate. In such examples, the control system generates commands for redirecting the field of view by commanding changes to the configuration of joints of the imaging instrument. In some embodiments, the control system generates commands for redirecting the field of view by preferring changes to the configuration of joints of the imaging instrument over changes to the configuration of joints of the first repositionable structure.

[0127] Some examples of control units, such as the control system 140 of Figure 1 can include non-transient, tangible, machine-readable media that include executable code that when executed by a processor system (e.g., the processor system 150 of Figure 1) can cause the processor system to perform the processes of method 200. Some common forms of machine-readable media that can include the processes of method 200 are, for example, floppy disk, flexible disk, hard disk, magnetic tape, any other magnetic medium, CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, RAM, PROM, EPROM, FLASH-EPROM, any other memory chip or cartridge, and / or any other medium from which a processor or computer is adapted to read.

[0128] Although illustrative embodiments have been shown and described, a wide range of modification, change and substitution is contemplated in the foregoing disclosure and in some instances, some features of the embodiments may be employed without a corresponding use of other features. One of ordinary skill in the art would recognize many variations, alternatives, and modifications. Thus, the scope of the invention should be limited only by the following claims, and it is appropriate that the claims be construed broadly and, in a manner, consistent with the scope of the embodiments disclosed herein.

Claims

WHAT IS CLAIMED IS:

1. A computer-assisted system comprising: a first repositionable structure configured to support a first instrument, the first instrument comprising an imaging device with a field of view, wherein the first repositionable structure supporting the first instrument comprises a first plurality of links coupled by a first plurality of joints; a second repositionable structure configured to support a second instrument; and a control system comprising one or more processors, the control system communicatively coupled to the first repositionable structure and the second repositionable structure; wherein the control system is configured to, while the first repositionable structure is supporting the first instrument: in response to detecting an indication to track an insertion of the second instrument into a workspace, determine a configuration of the first plurality of joints that poses the field of view to include an intermediate position for a distal portion of the second instrument as the second instrument is inserted into the workspace, and in response to determining the configuration, command the first plurality of joints toward the configuration.

2. The computer-assisted system of claim 1, wherein the second repositionable structure is further configured to support a cannula configured to be inserted through an opening leading to the workspace, and wherein the second instrument is configured to be inserted into the workspace by being inserted through the cannula.

3. The computer-assisted system of claim 1, wherein to determine the configuration, the control system is configured to: predict the intermediate position.

4. The computer-assisted system of claim 3, wherein to predict the intermediate position, the control system is configured to determine a reference position selected from the group consisting of: a position of an opening into the workspace; a position associated with a cannula inserted into the opening;a position associated with a cannula supported by the second repositionable structure; and a position of a remote center of motion for the second repositionable structure.

5. The computer-assisted system of claim 4, wherein to determine the configuration, the control system is configured to: determine the configuration to locate the reference position in a central region of the field of view.

6. The computer-assisted system of claim 1, wherein to detect the indication, the control system is configured to: receive a signal indicative of a mounting of the second instrument to the second repositionable structure; or receive a signal indicative of an insertion movement of the second instrument along an insertion axis of the second instrument.

7. The computer-assisted system of claim 1, wherein the computer-assisted system further comprises a third repositionable structure configured to support a third instrument, and wherein the control system is further configured to: detect the indication by receiving a signal indicative of a mounting of the third instrument to the third repositionable structure.

8. The computer-assisted system of claim 1, wherein, the control system is further configured to: detect the indication by determining that the second instrument was mounted to the second repositionable structure prior to the computer-assisted system receiving a command from an operator to teleoperate the first instrument or the second instrument.

9. The computer-assisted system of claim 1, wherein the control system is configured to: detect the indication by: determining that the second instrument was mounted to the second repositionable structure without the computer-assisted system entering a guided tool change mode; or determining that the computer-assisted system did not complete a guided tool change process for the second instrument.

10. The computer-assisted system of claim 1, wherein the first plurality of joints had a pretracking configuration before the control system commands the first plurality of joints toward the configuration, and wherein the control system is further configured to: in response to detecting an indication to stop tracking the second instrument, command the first plurality of joints to move toward the pre-tracking configuration.

11. The computer-assisted system of claim 1, wherein the control system is further configured to: stop tracking the second instrument in response to detecting an indication to track an insertion of a third instrument into the workspace.

12. The computer-assisted system of claim 1, wherein the control system is further configured to stop tracking the second instrument in response to: determining that an insertion movement of the second instrument has stopped for a predetermined period of time; or determining that a speed of the insertion movement of the second instrument is below a threshold speed; or determining that the second instrument is being moved or commanded to move in a degree of freedom other than an insertion direction; or determining that the second instrument is being retracted.

13. The computer-assisted system of claim 1, wherein the control system is further configured to stop tracking the second instrument in response to: determining that a distal portion of the second instrument is inserted past the field of view with the first plurality of joints in the configuration; or determining that a distal portion of the second instrument is inserted past a threshold distance relative to a reference; or determining that the second instrument is at a worksite within the workspace.

14. The computer-assisted system of claim 1, wherein the control system is further configured to stop tracking the second instrument in response to: determining that a force above a threshold level is applied to the first repositionable structure.

15. The computer-assisted system of any of claims 1 to 14, wherein the control system is further configured to: in response to the second instrument being inserted past the intermediate position, determine one or more further configurations of the first plurality of joints that repositions or reorients the field of view to keep the distal portion of the second instrument within the field of view; and command the first plurality of joints toward the one or more further configurations.

16. The computer-assisted system of claim 15, wherein to determine the one or more further configurations, the control system is configured to: determine the one or more further configurations to keep the distal portion of the second instrument in a central region of the field of view; or determine the one or more further configurations to also keep the intermediate position within the field of view.

17. The computer-assisted system of claim 15, wherein the field of view was at a pretracking pose before the first plurality of joints was moved toward the configuration, and wherein the control system is further configured to stop tracking the second instrument in response to: determining that a current pose of the field of view is within a tolerance of the pretracking pose.

18. The computer-assisted system of any of claims 1 to 14, wherein to determine the configuration, the control system is configured to: determine the configuration such that the field of view captures an image with a wider view of the workspace than a pre-tracking field of view.

19. The computer-assisted system of any of claims 1 to 14, wherein to determine the configuration, the control system is configured to: determine a zoom level for the field of view; determine a digital zoom for the field of view; or determine a digital pan for the field of view.

20. The computer-assisted system of any of claims 1 to 14, wherein the control system is further configured to track the insertion of the second instrument and, while tracking the insertion of the second instrument: determine a current position of the distal portion after the distal portion is inserted past the intermediate position; determine an updated configuration of the first plurality of joints that poses the field of view based on the current position; and in response to determining the updated configuration, command the first repositionable structure to move the first plurality of joints toward the updated configuration.

21. The computer-assisted system of claim 20, wherein to determine an updated configuration of the first plurality of joints that poses the field of view based on the current position, the control system is configured to: determine the updated configuration to pose the field of view to include the current position.

22. The computer-assisted system of claim 20, wherein to determine an updated configuration of the first plurality of joints that poses the field of view based on the current position, the control system is configured to: determine an updated intermediate position based on the current position; and determine the updated configuration to pose the field of view to include the updated intermediate position.

23. The computer-assisted system of claim 20, wherein to determine an updated configuration of the first plurality of joints that poses the field of view based on the current position, the control system is configured to: determine an updated intermediate position based on a velocity of the distal portion; and determine the updated configuration to pose the field of view to include the updated intermediate position.

24. The computer-assisted system of claim 20, wherein the control system is further configured to, while tracking the insertion of the second instrument:determine whether the distal portion of the second instrument is within the field of view; and in response to determining that the distal portion of the second instrument is not within the field of view, inhibit further movement of the second instrument.

25. The computer-assisted system of any of claims 1 to 14, wherein the computer-assisted system further comprises a third repositionable structure configured to support a third instrument, and wherein the control system is further configured to: store a pre-avoidance configuration of the third repositionable structure; command the third repositionable structure toward an avoidance configuration, wherein the third repositionable structure in the avoidance configuration avoids impeding a movement of the first plurality of joints toward the configuration; and in response to an exit from tracking the insertion of the second instrument, commanding the third repositionable structure toward the pre-avoidance configuration.

26. The computer-assisted system of any of claims 1 to 14, wherein the computer-assisted system further comprises a third repositionable structure configured to support a third instrument, and wherein the control system is further configured to: track the insertion of the second instrument; and automatically retract the third instrument while tracking the insertion of the second instrument.

27. The computer-assisted system of any of claims 1 to 14, wherein the computer-assisted system further comprises a third repositionable structure configured to support a third instrument, and wherein the control system is further configured to: track the insertion of the second instrument; while tracking the insertion of the second instrument to the workspace, determine that a currently commanded configuration of the first repositionable structure would cause a collision between the first repositionable structure and the third repositionable structure, or between the first instrument and the third instrument;in response to determining that the currently commanded configuration would cause the collision, determine an avoidance configuration of the third repositionable structure that would avoid the collision; and in response to determining the avoidance configuration, command the third repositionable structure toward the avoidance configuration.

28. The computer-assisted system of claim 27, wherein the control system is further configured to: in response to detecting an indication to stop tracking the insertion of the second instrument, command the third repositionable structure to move toward a preavoidance configuration of the third repositionable structure, the pre-avoidance configuration being a configuration of the third repositionable structure prior to moving toward the avoidance configuration.

29. The computer-assisted system of claim 27, wherein the control system is further configured to: in response to detecting an indication to stop tracking the insertion of the second instrument along with an indication to track another instrument supported by the computer-assisted system, not command the third repositionable structure to move toward a pre-avoidance configuration of the third repositionable structure, the pre-avoidance configuration being a configuration of the third repositionable structure prior to moving toward the avoidance configuration.

30. The computer-assisted system of any of claims 1 to 14, wherein: the computer-assisted system further comprises a display device; and the control system is further configured to: in response to detecting an indication to stop tracking the insertion of the second instrument, command the display device to display a synthesized view of the distal portion of the second instrument relative to the workspace.

31. The computer-assisted system of any of claims 1 to 14, wherein the control system is further configured to:determine an expected location in the field of view, the expected location coinciding with the intermediate position if the first plurality of joints is in the configuration; determine, with the first plurality of joints in the configuration, whether the intermediate position is within a tolerance of the expected location in the field of view; and in response to determining that the intermediate position is not within a tolerance of the expected location in the field of view, cause at least one precautionary action selected from the group consisting of providing a warning indication, commanding a retraction of the second instrument, inhibiting further insertion of the second instrument, commanding the first repositionable structure to move the first plurality of joints toward a pre-tracking configuration, the pre-tracking configuration being a configuration of the first plurality of joints before the first plurality of joints moved toward the configuration, inhibiting further movement of the first plurality of joints, and exiting the tracking of the insertion of the second instrument.

32. The computer-assisted system of any of claims 1 to 14, wherein to determine the configuration, the control system is configured to: limit at least one degree of freedom associated with the first plurality of joints.

33. The computer-assisted system of claim 32, wherein to limit at least one degree of freedom, the control system is configured to: allow the first instrument to move only in a set of allowed degrees of freedom; or disallow the first instrument from moving in the at least one degree of freedom; or limit an amount of motion of the first instrument in the at least one degree of freedom.

34. The computer-assisted system of claim 33, wherein the set of allowed degrees of freedom includes: a rotation about a longitudinal axis of the first instrument, and a translation along the longitudinal axis.

35. The computer-assisted system of any of claims 1 to 14, wherein to determine the configuration, the control system is configured to: minimize a translation of the first instrument or the field of view; or minimize a rotation of the first instrument or the field of view; or minimize a movement of at least one joint of the first plurality of joints.

36. The computer-assisted system of any of claims 1 to 14, wherein: the computer-assisted system further comprises a third repositionable structure configured to support a third instrument; and to determine the configuration, the control system is configured to: in response to the third instrument being in the workspace, determine the configuration to further keep at least a portion of the third instrument within the field of view, or to further keep an area of interest within the field of view.

37. The computer-assisted system of any of claims 1 to 14, wherein: the configuration of the first plurality of joints poses the field of view in a first pose; and to command the first plurality of joints toward the configuration, the control system is configured to: command the first plurality of joints to move the field of view to the first pose while minimizing a motion of the field of view in the workspace, or command the first plurality of joints to move the field of view to the first pose while minimizing a motion of at least one joint of the first plurality of joints in joint space.

38. The computer-assisted system of any of claims 1 to 14, wherein to command the first plurality of joints toward the configuration, the control system is configured to: command the first plurality of joints to translate and rotate the field of view sequentially.

39. The computer-assisted system of any of claims 1 to 14, wherein: the computer-assisted system further comprises a human interface system configured to provide feedback detectable by a human; and the control system is further configured to:in response to a determination that the first plurality of joints cannot achieve the configuration, cause the human interface system to provide a human-detectable indication.

40. The computer-assisted system of any of claims 1 to 14, wherein the computer-assisted system is a surgical system.

41. A method of commanding a first repositionable structure of a computer-assisted system, the first repositionable structure comprising a first plurality of joints, the first repositionable structure configured to support a first instrument, the first instrument comprising an imaging device with a field of view, wherein the method comprises: determining, by a control system and in response to detecting an indication to track an insertion of a second instrument into a workspace, a configuration of the first plurality of joints that poses the field of view to include an intermediate position for a distal portion of the second instrument as the second instrument is inserted into the workspace, the second instrument supported by a second repositionable structure of the computer-assisted system, and commanding, by the control system and in response to determining the configuration, the first plurality of joints toward the configuration.

42. The method of claim 41, wherein determining the configuration comprises: predicting the intermediate position.

43. The method of claim 42, wherein predicting the intermediate position comprises determining a reference position selected from the group consisting of: a position of an opening into the workspace; a position associated with a cannula inserted into the opening; a position associated with a cannula supported by the second repositionable structure; and a position of a remote center of motion for the second repositionable structure.

44. The method of claim 41, wherein detecting the indication comprises: receiving a signal indicative of a mounting of the second instrument to the second repositionable structure; orreceiving a signal indicative of an insertion movement of the second instrument along an insertion axis of the second instrument; or receiving a signal indicative of a mounting of a third instrument to a third repositionable structure configured to support the third instrument.

45. The method of claim 41, wherein detecting the indication comprises: determining that the second instrument was mounted to the second repositionable structure prior to the control system receiving a command from an operator to teleoperate the first instrument or the second instrument; or determining that the second instrument was mounted to the second repositionable structure without the computer-assisted system entering a guided tool change mode; or determining that the computer-assisted system did not complete a guided tool change process for the second instrument.

46. The method of claim 41, wherein the first plurality of joints had a pre-tracking configuration before the first plurality of joints moved toward the configuration, the method further comprising: commanding, by the control system and in response to detecting an indication to stop tracking the second instrument, the first plurality of joints to move toward the pre-tracking configuration.

47. The method of claim 41, further comprising: the control system commanding a stop to tracking of the second instrument in response to detecting an indication to track an insertion of a third instrument into the workspace.

48. The method of claim 41, further comprising: the control system commanding a stop to tracking of the second instrument in response to: determining that an insertion movement of the second instrument has stopped for a predetermined period of time; or determining that a speed of the insertion movement of the second instrument is below a threshold speed; or determining that the second instrument is being moved or commanded to move in a degree of freedom other than an insertion direction; ordetermining that the second instrument is being retracted; or determining that a distal portion of the second instrument is inserted past the field of view with the first plurality of joints in the configuration; or determining that a distal portion of the second instrument is inserted past a threshold distance relative to a reference; or determining that the second instrument is at a worksite within the workspace. determining that a force above a threshold level is applied to the first repositionable structure.

49. The method of any of claims 41 to 48, further comprising: determining, by the control system and in response to the second instrument being inserted past the intermediate position, one or more further configurations of the first plurality of joints that repositions or reorients the field of view to keep the distal portion of the second instrument within the field of view; and commanding, by the control system, the first plurality of joints toward the one or more further configurations.

50. The method of claim 49, wherein the field of view was at a pre-tracking pose before the first plurality of joints was moved toward the configuration, the method further comprising: the control system commanding a stop to tracking of the second instrument in response to: determining that a current pose of the field of view is within a tolerance of the pre-tracking pose.

51. The method of any of claims 41 to 48, wherein determining the configuration comprises: determining the configuration such that the field of view captures an image with a wider view of the workspace than a pre-tracking field of view.

52. The method of any of claims 41 to 48, further comprising, while tracking the insertion of the second instrument: determining, by the control system, a current position of the distal portion after the distal portion is inserted past the intermediate position;determining, by the control system, an updated configuration of the first plurality of joints that poses the field of view based on the current position; and commanding, by the control system and in response to determining the updated configuration, the first repositionable structure to move the first plurality of joints toward the updated configuration.

53. The method of any of claims 41 to 48, further comprising: storing, by the control system, a pre-avoidance configuration of a third repositionable structure of the computer-assisted system, the third repositionable structure configured to support a third instrument; commanding the third repositionable structure toward an avoidance configuration, wherein the third repositionable structure in the avoidance configuration avoids impeding a movement of the first plurality of joints toward the configuration; and commanding, by the control system and in response to an exit from tracking the insertion of the second instrument, the third repositionable structure toward the pre-avoidance configuration.

54. The method of any of claims 41 to 48, further comprising: tracking the insertion of the second instrument; and automatically retracting, by the control system, a third instrument supported by a third repositionable structure while tracking the insertion of the second instrument.

55. The method of any of claims 41 to 48, further comprising: tracking the insertion of the second instrument; and while tracking the insertion of the second instrument to the workspace, determining, by the control system, that a currently commanded configuration of the first repositionable structure would cause a collision between the first repositionable structure and a third repositionable structure configured to support a third instrument, or between the first instrument and the third instrument; in response to determining that the currently commanded configuration would cause the collision, determining, by the control system, an avoidance configuration of the third repositionable structure that would avoid the collision;in response to determining the avoidance configuration, commanding, by the control system, the third repositionable structure toward the avoidance configuration; and in response to detecting an indication to stop tracking the insertion of the second instrument, commanding, by the control system, the third repositionable structure to move toward a pre-avoidance configuration of the third repositionable structure, the pre-avoidance configuration being a configuration of the third repositionable structure prior to moving toward the avoidance configuration.

56. The method of any of claims 41 to 48, further comprising: determining, by the control system, an expected location in the field of view, the expected location coinciding with the intermediate position if the first plurality of joints is in the configuration; determining, by the control system and with the first plurality of joints in the configuration, whether the intermediate position is within a tolerance of the expected location in the field of view; and in response to determining that the intermediate position is not within a tolerance of the expected location in the field of view, causing, by the control system, at least one precautionary action selected from the group consisting of providing a warning indication, commanding a retraction of the second instrument, inhibiting further insertion of the second instrument, commanding the first repositionable structure to move the first plurality of joints toward a pre-tracking configuration, the pre-tracking configuration being a configuration of the first plurality of joints before the first plurality of joints moved toward the configuration, inhibiting further movement of the first plurality of joints, and exiting the tracking of the insertion of the second instrument.

57. The method of any of claims 41 to 48, wherein determining the configuration comprises: limiting at least one degree of freedom associated with the first plurality of joints.

58. The method of any of claims 41 to 48, wherein determining the configuration comprises: minimizing a translation of the first instrument or the field of view; or minimizing a rotation of the first instrument or the field of view; or minimizing a movement of at least one joint of the first plurality of joints.

59. The method of any of claims 41 to 48, wherein determining the configuration comprises: in response to a third instrument supported by a third repositionable structure being in the workspace, determining the configuration to further keep at least a portion of the third instrument within the field of view, or to further keep an area of interest within the field of view.

60. One or more non-transitory machine-readable media comprising a plurality of machine-readable instructions which when executed by a control system associated with a computer-assisted system are adapted to cause the control system to perform the method of any one of claims 41-59.

Images