Increasing mobility of computer-aided systems while maintaining partially constrained field of view

Abstract

A system includes proximal and distal structures and a processor system. The processor system determines a first commanded motion of the proximal structure; determining a second commanded motion for the distal structure or the joint (s) of the imaging device, where the field of view of the imaging device is maintained relative to the workspace when the first commanded motion and the second commanded motion are performed in combination; and in response to determining that driving the joint (s) in accordance with a second command motion will cause the distal structure or imaging device to violate the constraint: determining for the joint (s) Alternative commanded motion of the joint to maintain a defined geometric relationship between a first geometric feature fixed relative to the imaging device and a second geometric feature fixed relative to the workspace, drives the proximal structure in accordance with the first commanded motion, and drives the joint (s) in accordance with the alternative commanded motion.

Description

[0001] Related Applications

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 413,041, filed Oct. 4, 2022, titled “Increasing Mobility of Computer-assisted Systems while Maintaining a Partially Constrained Field of View”, which is incorporated herein by reference. Technical Field

[0003] The present disclosure generally relates to computer-assisted systems, and more particularly to increasing the mobility of computer-assisted systems while maintaining a partially constrained field of view. Background Art

[0004] Some computer-assisted systems include one or more devices that are repositioned to perform various procedures. The computer-assisted system can be automatic, semi-automatic, remotely operated, etc. In an example of a remotely operated system, a human operator manipulates one or more leader input controls to command the movement of one or more follower devices located in a workspace. In some examples, the remotely operated system is configured to support a device that includes an imaging device (such as a camera) that enables the operator to observe the workspace. In some cases, the field of view of the imaging device is directed so that the operator can see one or more other devices when the operator commands the movement of the device(s).

[0005] In some examples, a proximal repositionable structure is proximal to one or more distal repositionable structures. In an example of multiple distal repositionable structures, a first distal repositionable structure is configured to support a first device that includes an imaging device, and a second distal repositionable structure is configured to support a second device. In operation, the movement of the proximal repositionable structure can be used to produce the movement of the first and / or second distal repositionable structures. Thus, in operation, the movement of the proximal repositionable structure can be used to produce a movement that moves or assists the movement of the imaging device supported by the first distal repositionable structure or one or more second devices supported by the second distal repositionable structure. In the case where the proximal repositionable structure moves, the movement of the proximal repositionable structure can also move the first distal repositionable structure that supports the imaging device, and thus the movement of the proximal repositionable structure can also move the field of view of the imaging device. To maintain the orientation, position, or both the orientation and position of the field of view of the imaging device, the computer-assisted system can command the movement of one or more joints of the imaging device and / or the first distal repositionable structure while commanding the movement of the proximal repositionable structure.

[0006] However, in a case where one or more joints of the imaging device cannot fully compensate for movement of the proximally relocatable structure, the computer-assisted system can no longer maintain the field of view when moving the proximally relocatable structure. Accordingly, an operator command movement of one or more second instruments causes movement of the field of view of the imaging device. Such movement of the field of view can be unexpected and / or disturbing to an operator of the computer-assisted system, and so on.

[0007] Accordingly, improved techniques for controlling movement of a relocatable structure in the presence of fewer uncommanded movements of the field of view are desired. SUMMARY OF THE INVENTION

[0008] In accordance with some embodiments, a computer-assisted system and method implemented therein includes a computer-assisted system including: a proximally relocatable structure; a first distally relocatable structure physically coupled to the proximally relocatable structure; and a processor system communicatively coupled to the proximally relocatable structure and the first distally relocatable structure. The processor system is configured to determine a first commanded movement of the proximally relocatable structure. The processor system is further configured to determine a second commanded movement of one or more joints selected from the group consisting of the first distally relocatable structure and joints of an imaging device supported by the first distally relocatable structure, wherein when the second commanded movement is executed in combination with the first commanded movement, the field of view of the imaging device is maintained relative to a workspace. The processor system is further configured to determine whether driving the one or more joints according to the second commanded movement will cause the first distally relocatable structure or the imaging device to violate a first constraint. The processor system is further configured to, in response to determining that driving the one or more joints according to the second commanded movement will cause the first distally relocatable structure or the imaging device to violate the first constraint: determine an alternative second commanded movement for the one or more joints to maintain a defined geometric relationship between a first geometric feature fixed relative to the imaging device and a second geometric feature fixed relative to the workspace, drive the proximally relocatable structure according to the first commanded movement, and drive the one or more joints according to the alternative second commanded movement.

[0009] Consistent with some embodiments, a method includes determining, by a processor system, a first commanded motion of a proximal relocatable structure. The method further includes determining, by the processor system, a second commanded motion of one or more joints selected from the group consisting of a first distal relocatable structure and joints of an imaging device supported by the first distal relocatable structure, wherein a field of view of the imaging device is maintained relative to a workspace when the second commanded motion is executed in combination with the first commanded motion. The method further includes determining, by the processor system, whether driving the one or more joints according to the second commanded motion will cause the first distal relocatable structure or the imaging device to violate a first constraint. The method further includes, in response to determining that driving the one or more joints according to the second commanded motion will cause the first distal relocatable structure or the imaging device to violate the first constraint: determining, by the processor system, an alternative second commanded motion for the one or more joints to maintain a defined geometric relationship between a first geometric feature fixed relative to the imaging device and a second geometric feature fixed relative to the workspace, driving, by the processor system, the proximal relocatable structure according to the first commanded motion, and driving, by the processor system, the one or more joints according to the alternative second commanded motion.

[0010] Consistent with some embodiments, one or more non-transitory machine-readable media include a plurality of machine-readable instructions that, when executed by a processor system, are adapted to cause the processor system to perform any of the methods described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a schematic diagram of a computer-aided system according to one or more embodiments.

[0012] Figure 2 is a schematic diagram of a computer-aided system according to one or more embodiments.

[0013] Figure 3 is a flowchart of method steps for controlling movement of a constrained relocatable structure according to one or more embodiments.

[0014] Figures 4A - 4C illustrates movement of one or more joints of a distal relocatable structure when maintaining a fixed field of view according to one or more embodiments.

[0015] Figures 5A - 5D illustrates movement of one or more joints of a distal relocatable structure when maintaining a partially constrained field of view according to one or more embodiments.

[0016] Figures 6A - 6DIllustrated is the movement of one or more joints of a distally repositionable structure when maintaining a partially constrained field of view at infinity, in accordance with one or more embodiments.

[0017] Figures 7A - 7D Illustrated is the movement of one or more joints of a distally repositionable structure when maintaining a floating field of view, in accordance with one or more embodiments.

[0018] In the drawings, elements having the same name have the same or similar functions. Detailed Description

[0019] In this specification, specific details are set forth that describe some embodiments consistent with the present disclosure. Numerous specific details are set forth in order to provide a thorough understanding of the embodiments. However, it will be apparent to those skilled in the art that some embodiments may be practiced without some or all of these specific details. The specific embodiments disclosed herein are intended to be illustrative and not restrictive. Although not specifically described herein, those skilled in the art will recognize other elements within the scope and spirit of the present disclosure. Additionally, to avoid unnecessary repetition, unless otherwise specifically described or if one or more features would render an embodiment inoperative, one or more features shown and described in connection with one embodiment may be incorporated into other embodiments.

[0020] Furthermore, the terminology used in this specification is not intended to limit the invention. For example, spatially relative terms - such as "below", "beneath", "lower", "above", "upper", "proximal", "distal", etc. - may be used to describe the relationship of one element or feature to another element or feature, as illustrated in the figures. These spatially relative terms are intended to encompass different positions (i.e., orientations) and orientations (i.e., rotational placements) of the elements or their operations, in addition to the positions and orientations shown in the figures. For example, if the contents of one figure are flipped, an element described as "below" or "beneath" other elements or features will be "above" or "over" other elements or features. The device may be otherwise oriented, and the spatially relative descriptors used herein are to be interpreted accordingly. Similarly, descriptions of movement along and around various axes include various particular element positions and orientations. Additionally, unless the context clearly dictates otherwise, the singular forms "a / an" and "the" are also intended to include the plural forms. Also, the terms "comprises", "comprising", "includes", etc. specify the presence of the stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups. Components described as being coupled may be directly coupled electrically or mechanically, or indirectly coupled via one or more intermediate components.

[0021] Whenever feasible, elements described in detail with reference to one embodiment, implementation, or module may also be included in other embodiments, implementations, or modules in which they are not specifically shown or described. For example, if an element is described in detail with reference to one embodiment and not described with reference to a second embodiment, that element may still be claimed as being included in the second embodiment. Thus, to avoid unnecessary repetition in the following description, unless otherwise specifically described, one or more elements shown and described in connection with one embodiment, implementation, or application may be incorporated into other embodiments, implementations, or aspects, unless one or more of the elements would render the embodiment or implementation inoperative, or unless two or more of the elements provide conflicting functions.

[0022] In some instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

[0023] The present disclosure describes various devices, elements, and portions of computer-aided systems and elements in terms of their state in three-dimensional space. As used herein, the term "position" refers to the location of an element or portion of an element (e.g., three translational degrees of freedom in three-dimensional space (such as along Cartesian x, y, and z coordinates)). As used herein, the term "orientation" refers to the rotational placement of an element or portion of an element (e.g., three rotational degrees of freedom in three-dimensional space (such as represented by angular axes, rotation matrices, quaternions, etc. around roll, pitch, and yaw axes)). As used herein, and for devices having a kinematic series (such as a repositionable structure having a plurality of links coupled by one or more joints), the term "proximal" refers to the direction toward the base of the kinematic series, and "distal" refers to the direction along the kinematic series away from the base.

[0024] As used herein, the term "pose" refers to the multi-degree-of-freedom (DOF) spatial position and orientation of a coordinate system of interest attached to a rigid body. Generally, a pose includes pose variables for each DOF in the pose. For example, a full 6-DOF pose of a rigid body in three-dimensional space will include six pose variables corresponding to three position DOFs (e.g., x, y, and z) and three orientation degrees of freedom (e.g., roll, pitch, and yaw). A 3-DOF position-only pose includes only the pose variables for the three position DOFs. Similarly, a 3-DOF orientation-only pose includes only the pose variables for the three rotational DOFs. Additionally, the velocity of a pose captures the change of the pose over time (e.g., the first derivative of the pose). For a full 6-DOF pose of a rigid body in three-dimensional space, the velocity will include three translational velocities and three rotational velocities. A pose with a different number of DOFs will have a corresponding number of velocity translations and / or rotational velocities.

[0025] Aspects of the present disclosure are described with reference to a computer-assisted system, which may include devices that are remotely operated, externally manipulated, automated, semi-automated, etc. Additionally, aspects of the present disclosure are described in accordance with embodiments using a remote-operated surgical system (such as the da surgical system) commercialized by Intuitive Surgical, Inc. of Sunnyvale, California. However, one of ordinary skill in the art will understand that the inventive aspects disclosed herein may be embodied and implemented in various ways (including remotely operated and non-remotely operated) and in medical and non-medical embodiments and implementations (e.g., medical and non-medical devices, apparatuses, or systems). The da surgical system embodiments are merely exemplary and should not be considered as limiting the scope of the inventive aspects disclosed herein. For example, the techniques described with reference to surgical instruments and surgical methods may be used in other situations. Thus, the instruments, systems, and methods described herein may be used for humans, animals, parts of human or animal anatomies, industrial systems, general-purpose robots, or remote operating systems. As another example, the instruments, systems, and methods described herein may be used for non-medical purposes, including industrial uses, general-purpose robot uses, sensing or manipulating non-tissue workpieces, cosmetic improvements, imaging of human or animal anatomies, collecting data from human or animal anatomies, setting up or disassembling systems, training medical or non-medical personnel, etc. Additional example applications include procedures for tissue removal from human or animal anatomies (with or without return to the human or animal anatomy) and procedures for human or animal cadavers. Additionally, these techniques may also be used for medical treatment or diagnostic procedures that include or do not include surgical aspects.

[0026] Figure 1 is a schematic diagram of a computer-assisted system 100 according to one or more embodiments. AsFigure 1 As shown, computer - assisted system 100 includes a manipulation assembly 110 having one or more relocatable structures 120. In Figure 1 the example, the (one or more) relocatable structures are shown as manipulator arms including multiple linkages coupled by one or more joints. Each of the one or more relocatable structures 120 supports one or more instruments 130. In some examples, the manipulation assembly 110 includes a computer - assisted surgical assembly. Examples of medical devices include surgical instruments for interacting with tissue, imaging, sensing devices, etc. In some examples, the instrument 130 can include an end - effector capable of but not limited to performing, grasping, retracting, cauterizing, ablating, suturing, cutting, stapling, fusing, sealing, etc. and / or combinations thereof.

[0027] In a teleoperation example, the manipulation assembly 110 can also be communicatively coupled to a user input system (not shown) via a wired or wireless connection. The user input system includes one or more input controls, also referred to herein as input controls, for operating the manipulation assembly 110, one or more relocatable structures 120, and / or the instrument 130. In some examples, the one or more input controls can include a kinematic series of linkages and one or more joints, one or more actuators for driving portions of the (one or more) input controls, robotic manipulators, levers, pedals, switches, buttons, knobs, triggers, etc. In some examples, the one or more input controls include a leader device (also referred to as a “master” device in the industry), and the manipulation assembly 110 and / or one or more relocatable structures 120 (supporting or not supporting the instrument 130) include a follower device (also referred to as a “slave” device in the industry). The operator can use the one or more input controls to command the movement of the manipulation assembly 110, such as by commanding the movement of one or more relocatable structures 120 and / or the instrument 130 in a leader - follower configuration. The leader - follower configuration is a type of teleoperation configuration and is sometimes referred to as a master - slave configuration in the industry.

[0028] In an example of supporting external manipulation, the input controls can be located at the relocatable structure. As a specific example, the input controls can include joint sensors that detect joint deflections, and the computer - assisted system is configured to process certain joint deflections into commands to move the joints.

[0029] Figure 1The manipulation assembly 110 is coupled to the control unit 140 via an interface. The interface can be wired and / or wireless and can include one or more cables, optical fibers, connectors, and / or buses and can also include one or more networks having one or more network switching and / or routing devices. The operation of the control unit 140 is controlled by the processor system 150. The processor system 150 can include one or more central processing units, multi-core processors, microprocessors, microcontrollers, digital signal processors, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), graphics processing units (GPUs), tensor processing units (TPUs), etc. in the control unit 140. The control unit 140 can be implemented as a stand-alone subsystem and / or board added to a computing device or as a virtual machine. In some embodiments, the control unit 140 can be included as part of the user input system and / or the manipulation assembly 110, and / or operate separately from and in cooperation with the user input system and / or the manipulation assembly 110.

[0030] As an example, the manipulation assembly 110, the user input system, and / or the control unit 140 can correspond to the patient-side cart, the surgeon console, and the processing unit and associated software of the da Vinci Surgical System commercialized by Intuitive Surgical, Inc. of Sunnyvale, California. In some embodiments, components having other configurations, such as fewer or more repositionable structures, different user input systems or input controls, different repositionable structure hardware, etc., can include the computer-assisted system 100.

[0031] The memory 160 can be used to store software executed by the control unit 140 and / or one or more data structures used during the operation of the control unit 140. The memory 160 can include one or more types of machine-readable media. Some common forms of machine-readable media can include floppy disks, flexible disks, hard disks, magnetic tapes, any other magnetic media, CD-ROMs, any other optical media, punch cards, paper tapes, any other physical media with hole patterns, RAMs, PROMs, EPROMs, FLASH-EPROMs, any other memory chips or cartridge tapes, and / or any other media readable by a processor or computer.

[0032] As Figure 1As shown in the example, the memory 160 includes a control module 170 that can be used to support autonomous, semi-autonomous, and / or remote operation control of the manipulation assembly 110. The control module 170 includes one or more application programming interfaces (APIs) that are used to receive position, motion, force, torque, and / or other sensor information from the manipulation assembly 110, the repositionable structure 120, and / or the instrument 130, to share position, motion, force, torque, and / or collision avoidance information with other control units regarding other devices, and / or to plan and / or assist in planning the motion of the manipulation assembly 110 and / or the instrument 130 (such as the motion of the repositionable structure 120). In some examples, the control module 170 also supports autonomous, semi-autonomous, and / or remote operation control of the manipulation assembly 110 and / or the instrument 130 during the execution of various tasks. And although the control module 170 is depicted as a software application, the control module 170 can optionally be implemented using hardware, software, and / or a combination of hardware and software.

[0033] In some medical embodiments, the computer-aided system 100 can be found in a clinic, a diagnostic facility, an operating room, an interventional suite, or other medical environments. Although the computer-aided system 100 is shown including one manipulation assembly 110 having two repositionable structures 120, each repositionable structure supporting a corresponding instrument 130, those of ordinary skill in the art will understand that the computer-aided system 100 can include any number of manipulation assemblies, each manipulation assembly can include one or more repositionable structures, and each repositionable structure can support one or more instruments, and all of these elements can be similar or different in design from those specifically described in these figures. In some examples, each of the manipulation assemblies can include fewer or more repositionable structures and / or support fewer or more instruments compared to those specifically depicted in these figures.

[0034] Figure 2 is a schematic diagram of a computer-aided system 200 according to one or more embodiments. In Figure 2 the example, the computer-aided system 200 includes a repositionable structure shown as a manipulation assembly 210 and a user input system 250. In a remote operation scenario, the operator 298 uses the user input system 250 to operate the manipulation assembly 210, such as in a leader-follower configuration. In the leader-follower configuration for the Figure 1 example, the components of the user input system 250 (e.g., input controls) are the leader, and a part of the manipulation assembly 210 (e.g., a manipulation arm or other repositionable structure) is the follower.

[0035] The manipulation assembly 210 can be used to introduce a set of instruments into a work site through a single port 230 inserted into a hole (e.g., using the cannula shown). In a medical scenario, the work site can be on or within a patient's body, and the hole can be a minimally invasive incision or a natural body orifice. The port 230 can be free-floating, held in place by a fixing device separate from the manipulation assembly 210, or held by a linkage 222 or other parts of the manipulation assembly 210. The linkage 222 can be coupled to additional joints and linkages 214, 220 of the manipulation assembly 210, and these additional joints and linkages 214, 220 can be disposed on a base 212. The linkage 222 can also include a manipulator support linkage 224 in the proximal direction 262 of the port 230. A set of manipulators 226 in the proximal direction 262 of the port 230 can be coupled to the manipulator support linkage 224. A relocatable structure movable to follow commands from a user input system 250 can include one or more of any of the following: the linkage 222, the additional joints and linkages 214, 220, the base 212, the manipulator support linkage 224, and / or any additional linkages or joints coupled to the foregoing joints or linkages. Each manipulator 226 can include a carriage (or other instrument coupling linkage) configured to be coupled to an instrument, and each manipulator 226 can include one or more joints and / or linkages that can be driven to move the carriage. For example, the manipulator 226 can include a prismatic joint that, when driven, linearly moves the carriage and any instrument(s) coupled to the carriage. This linear motion can be along (parallel to) an insertion axis extending distally 264 to and through the port 230.

[0036] The additional joints and additional linkages 214, 220 can be used to position the port 230 at a hole or another location. Figure 2 A prismatic joint for vertical adjustment (as indicated by arrow “A”) and a set of rotary joints for horizontal adjustment (as indicated by arrows “B” and “C”) are shown, which can be used to translate the position of the port 230. The linkage 222 is used to deflect, pitch, and roll-rotate the port 230 (and the instrument(s) then disposed within the port) about a remote center of motion (RCM) positioned adjacent the port 230 as indicated by arrows D, E, and F respectively, without translating the RCM.

[0037] Actuation of the degrees of freedom provided by one or more couplings (not shown) of one or more instruments can be provided by an actuator disposed within the instrument, or the power thereof (e.g., linear force or rotational torque) is transmitted to the one or more instruments. Examples of actuators include rotary motors, linear motors, solenoids, etc. The actuator can drive the manipulation assembly 210 and / or transmission elements within the instrument to control the degrees of freedom of the one or more instruments. For example, the actuator can drive a rotary disk of a manipulator coupled to a drive element (e.g., rotary disk, linear slide rail) of the one or more instruments, wherein driving the drive element of the instrument will drive the transmission elements within the instrument, and the transmission elements are coupled to move one or more couplings of the instrument, or to actuate some other function of the instrument, such as the degrees of freedom of an end effector. Thus, the degrees of freedom of the one or more instruments can be controlled by an actuator that drives the one or more instruments in accordance with a control signal. The control signal can be determined to cause instrument movement or other actuation automatically determined by the system, as indicated by movement of an input control or other manipulation or any other control signal command. Additionally, appropriately positioned sensors, such as encoders, potentiometers, etc., can be provided to enable measurement of an indication of the coupling position, or other data that can be used to derive the coupling position, such as coupling speed. The actuator and sensors can be disposed within the one or more manipulators 226, or transmit signals to or receive signals from the one or more manipulators 226. Techniques for manipulating multiple instruments in a computer-aided system are more fully described in Patent Cooperation Treaty Patent Application No. PCT / US2021 / 047374, filed on August 24, 2021, and titled "METHOD AND SYSTEM FOR COORDINATED MULTIPLE-TOOL MOVEMENT USING A DRIVABLE ASSEMBLY", which is incorporated herein by reference.

[0038] Although Figure 2 a particular configuration of the manipulation assembly 210 is shown, those skilled in the art will understand that embodiments of the present disclosure can be used with a manipulation assembly or other repositionable structure of any design. In some examples, the manipulation assembly can have any number and any type of degrees of freedom, can be configured to be coupled or not coupled to an access port, and can optionally use ports other than cannulas, such as catheters, etc. In some examples, the manipulation assembly 210 can also include an arrangement of links and couplings that do not provide a remote center of motion.

[0039] In Figure 2 the example shown, the user input system 250 includes one or more input controls 252 configured to be operated by an operator 298. InFigure 2 In the example shown, one or more input controls 252 are contacted and manipulated by the hands of the operator 298, one input control 252 per hand. Examples of such hand input devices include any type of device that can be manually operated by a human user, such as a joystick, a trackball, a cluster of buttons, and / or other types of haptic devices that typically have multiple degrees of freedom. Position, force, and / or haptic feedback devices (not shown) can be used to provide position, force, and / or haptic feedback from the instrument to the hands of the operator 298 via the input controls 252.

[0040] The input controls 252 are supported by the user input system 250 and are shown as being mechanically grounded, and in other embodiments may be mechanically ungrounded. In some embodiments, an ergonomic support 256 may be provided; for example, Figure 2 an ergonomic support 256 including a forearm rest is shown, and the operator 298 may rest his or her forearm on the forearm rest when manipulating the input controls 252. In some examples, the operator 298 may control the manipulation assembly 210 by using the input controls 252 and perform tasks at a work location near the manipulation assembly 210 during a procedure.

[0041] A display unit 254 is included in the user input system 250. The display unit 254 may display images for the operator 298 to view. The display unit 254 may provide a view of the work location where the manipulation assembly 210 interacts therewith. The view may include a stereoscopic image or a three-dimensional image to provide depth perception of the work location and the instrument(s) of the manipulation assembly 210 in the work location. The display unit 254 may be movable in various degrees of freedom to accommodate the viewing position of the operator 298 and / or to provide control functions. In the case where the display unit (e.g., the display unit 254) is also used to provide control functions, such as commanding the manipulation assembly 210, the display unit further includes input controls (e.g., another input control 252).

[0042] When using the user input system 250, the operator 298 may sit on a chair or other support, position his or her eyes to view the images displayed by the display unit 254, grasp and manipulate the input controls 252, and rest his or her forearm on the ergonomic support 256 as needed. In some embodiments, the operator 298 may stand at a workstation or assume other postures, and the display unit 254 and the input controls 252 may be structurally different, adjustable in position (height, depth, etc.), etc.

[0043] In some examples, the relocatable structure includes a base manipulator and a plurality of instrument manipulators coupled to the base manipulator. In some examples, the relocatable structure includes a single instrument manipulator without a series coupling of manipulators. In some examples, the relocatable structure includes a single instrument manipulator coupled to a single base manipulator. In some examples, a computer-assisted system can include a movable base mounted on a cart or to an operating table, and one or more manipulators mounted to the movable base.

[0044] In many of the embodiments described in this application, the relocatable structure includes one or more proximal relocatable structures and one or more distal relocatable structures. In some examples, one or more proximal relocatable structures can include a linkage 222, additional joints and / or linkages 214, 220, a manipulator support linkage 224, and / or any one or more of any additional linkages and / or joints coupled to the foregoing joints or linkages. One or more distal relocatable structures can include a manipulator 226, a carriage (or other instrument coupling linkage) configured to couple to an instrument, and / or one or more of one or more joints and / or linkages that can be actuated to move the carriage. The operator 298 views the workspace via an imaging device that is coupled to one of the one or more distal relocatable structures in the form of the manipulator 226 and an associated carriage, joint, and / or linkage. The imaging device has a field of view that can be displayed on the display unit 254. The operator 298 contacts and manipulates one or more input controls 252 to generate command motions to move the relocatable structure. In response, one or more corresponding distal relocatable structures move so as to move the instrument according to the command motions.

[0045] During the procedure, the workspace may or may not move relative to the larger environment that contains the workspace. In various cases, the workspace can be fixed in position and / or orientation relative to the larger environment that contains the workspace for the duration of the procedure. For example, the workpiece that contains or otherwise defines the workspace can be rigid and can be located in the same position and orientation on a platform; furthermore, the platform can not move within the physical environment that contains the platform during the procedure. In various cases, the workspace can translate and / or rotate relative to the larger environment that contains the workspace during the procedure. For example, the workpiece that contains or otherwise defines the workspace can be compliant and physically reconfigured at one or more times during the procedure. As another example, the workpiece that contains or otherwise defines the workspace can move autonomously or be externally manipulated to move relative to the physical environment that contains the platform during the procedure. In a medical example, the patient on whom the procedure is being performed can change shape or orientation due to breathing or other involuntary movements, can be moved by medical personnel, or can be moved by the movement of the examination table on which the patient is resting.

[0046] In some cases, the corresponding distal repositionable structures may be subject to certain constraints such that these distal repositionable structures cannot move in accordance with commanded motion. As described herein, a constraint can be a physical constraint, such as a mechanical limitation on the position of the repositionable structure and / or an instrument supported by the repositionable structure. In some cases, such mechanical limitation is imposed by the physical design or construction of the repositionable structure and / or the instrument. In some cases, such mechanical limitation is imposed by physical objects in the operating environment of the repositionable structure and / or the instrument. Example physical objects include people, equipment, walls, and floors, among others. A constraint can be a motion constraint, where the repositionable structure and / or the instrument is limited in one or more motion parameters, such as position, velocity, speed, and / or acceleration. In some cases, such mechanical limitation is imposed by the physical design or construction of the repositionable structure and / or the instrument (such as by any actuator or transmission component of these objects). In some cases, such mechanical limitation is imposed by other operating considerations, such as reducing the likelihood of collision or damage, reducing power consumption, reducing vibration, increasing motion precision, and the like.

[0047] In cases where the distal repositionable structures cannot move in accordance with commanded motion, instead of or in addition to these distal repositionable structures, one or more proximal repositionable structures can be moved such that the combined motion of the (one or more) proximal repositionable structures and the (one or more) distal repositionable structures moves the instrument in accordance with commanded motion. The movement of the (one or more) proximal repositionable structures can also cause the movement of the (one or more) distal repositionable structures coupled to the imaging device. Accordingly, the field of view of the imaging device also moves, which can disorient the operator 298 or be non-intuitive to the operator 298. Accordingly, the (one or more) distal repositionable structures coupled to the imaging device move in a manner to maintain the field of view of the imaging device.

[0048] In some cases, one or more joints of the (one or more) distally repositionable structures coupled to the imaging device can be commanded to maintain the field of view of the imaging device, but the one or more joints of the one or more distally repositionable structures are restricted in movement as needed in order to implement the commanded movement and also maintain the field of view. The restriction can be a range of motion (ROM) limitation due to physical limitations of the repositionable structure, obstacles, etc. Additionally or alternatively, the commanded movement will cause the distal portion of the imaging device to enter a restricted area in the workspace. The distal portion of the imaging device can include the distal end of the imaging device, such as the imaging end of the imaging device. For example, the restriction will limit the distal portion of the imaging device from penetrating an entrance to the workspace that is not deeper than a specified distance. The commanded movement will cause the distal portion of the imaging device to penetrate an entrance deeper than the specified distance. In such cases, the control module determines the position of the distal portion of the imaging device based on the known geometry of the joints and the repositionable structure and / or the support instrument. In some cases, such a restriction is employed to limit the imaging device from penetrating the workspace beyond a certain depth, to limit the imaging device from getting close enough to an object in the workspace to overheat the object, etc. In other examples, the movement of the repositionable structure is restricted in order to avoid collisions with nearby objects, such as components, devices, and / or personnel, to avoid avoidance zones, etc.

[0049] Generally, the field of view is the extent of the observable world that the imaging device can effectively detect and display. The field of view of the imaging device varies based on the position and / or orientation of the imaging device, the type and size of the lens of the imaging device, the zoom setting of the lens, the focal length of the lens, etc. In such a case where the one or more joints cannot maintain the field of view due to the restriction, further movement of the proximally repositionable structure in a manner that requires the imaging device to move past the restriction can no longer be compensated for by the one or more joints. This further movement of the proximally repositionable structure will result in a change in the position (i.e., orientation and / or position) of the imaging device or the field of view of the imaging device. Thus, even though the proximally repositionable structure has an additional range of motion that produces movement of one or more second instruments, the proximally repositionable structure is prevented from moving due to the restriction associated with the field of view of the imaging device.

[0050] A potential way to respond to the inability to maintain the field of view due to constraints is to disallow the movement of a proximally repositionable structure that requires one or more joints to move past the constraint and thus maintain a fixed field of view regardless of whether the one or more joints are constrained. Accordingly, the system does not allow the operator to command further movement of one or more second instruments that involves movement of the proximally repositionable structure that requires movement of one or more joints that would cause a violation of the constraint. This approach may limit the achievable positions, orientations, or movements of the second instrument; may require the operator to adjust the physical configuration of the proximally or distally repositionable structure to allow the second instrument to perform the command; or may prevent the operator from completing a desired movement or task with the second instrument.

[0051] Another possible way to respond to the inability to maintain the field of view is to continue to compensate for movement of the proximally repositionable structure to some extent. For example, some embodiments are configured to compensate for movement of the proximally repositionable structure as much as possible. With this possible approach, the field of view of the imaging device or imaging devices changes as the proximally repositionable structure moves the imaging device in a manner that one or more joints are no longer able to compensate for due to the constraint. This results in a floating field of view because the constraint causes the field of view to change as movement of the proximally repositionable structure that cannot be compensated for by one or more joints continues to occur. This approach may be beneficial in some cases and may cause less desirable effects in other cases.

[0052] For example, in some cases, such resultant changes in the field of view relative to the workspace and / or world frame may reduce efficiency, increase the time required to execute a procedure, misalign the field of view or otherwise mislead, be disorienting or non-intuitive to a viewer of an image captured by an imaging device (e.g., an operator or another person when viewing such an image), and so on. Additionally, in some examples, such a change in the field of view causes an object on which the operator is performing a procedure, an instrument that the operator is controlling, or an object that the operator is observing to move out of the operator's visual attention area within the field of view, or out of the field of view entirely. If the object and / or instrument moves out of the field of view (or out of a preferred position within the field of view), the operator may have to interrupt the procedure until the object and / or instrument is again within the field of view (or in the preferred position), or perform additional tasks to bring the object and / or instrument back into the field of view (or to the preferred position). Additionally, such resultant changes in the field of view may reduce the efficiency with which the operator must re-orient to the changed field of view, re-position the imaging device, re-position the instrument, and so on. As a specific example, in some cases, a 90-degree clockwise rotation of the field of view may cause the operator to mistake a movement to the right for a movement downward, and a 180-degree rotation of the field of view may cause the operator to mistake a movement for the opposite direction. Additionally, such a change in the field of view may be directed in a manner that is less effective for visualizing the object to be manipulated, the instrument being commanded, the work site, or the operating environment. In some cases, such a change in the field of view may also confuse an operator who is accustomed to visualizing the workspace (such as an anatomical structure) in a particular orientation.

[0053] Generally, the workspace can be any area within the observable world that is accessible by a computer-assisted system. The workspace can include a workbench or examination bench, a chamber, a device, a portion of the anatomical structure of a medical patient, and so on. In cases where the viewer is an operator who commands the movement of the system, such a change in the field of view may cause the operator's commands to the system to slow down (and the procedure time to increase), reduce the perceived responsiveness of the system to the operator's commands, increase the tolerances and clearances required to execute the procedure, and so on. In some cases, an unexpected change in the field of view of an imaging device can give the viewer the impression that objects viewable by the imaging device (e.g., other objects in the workspace, such as a manual instrument or patient tissue in a medical example) are moving even though they are stationary relative to the workspace, are moving differently relative to the workspace than they actually are, and so on. In some cases resulting from the movement of the imaging device, the imaging device may collide with an object (such as patient tissue in a medical example), resulting in an unintentional contact with these objects, pressure on these objects, and / or damage to these objects.

[0054] Some embodiments of the present disclosure include techniques for enabling limited movement of an imaging device and / or the field of view of the imaging device when one or more joints of a computer-assisted system are no longer able to compensate for movement of a proximally repositionable structure due to constraints. In doing so, the computer-assisted system maintains a partially constrained field of view, allowing for limited deviation of the field of view, as further described below, while also allowing the repositionable structure to continue to move, thereby allowing for a greater range of motion of one or more second instruments. In some examples, maintaining a partially constrained field of view enables an operator to complete a procedure in less time or with fewer operator inputs, as compared to other methods.

[0055] When maintaining a partially constrained field of view, the computer-assisted system maintains a defined geometric relationship between a first geometric feature that is fixed relative to the imaging device and a second geometric feature that is fixed relative to the workspace. In some examples, the first geometric feature is the optical axis of the imaging device, the second geometric feature is fixed relative to the workspace (e.g., at a fixed position in the workspace, where the position may move with translation or rotation of the workspace), and the defined geometric relationship is concurrency of the two (i.e., the optical axis must pass through the point). When the proximally repositionable structure and the distally repositionable structure move, the computer-assisted system maintains the concurrency relationship. This results in a change in the position of the imaging device, the viewing direction, or both. Maintaining a partially constrained field of view has fewer limitations than maintaining a fixed field of view, while reducing the amount of disruption to the field of view that can be caused by maintaining a floating field of view. In some examples, when the second geometric feature is a point at a greater distance (effectively at infinity) from the imaging device, maintaining a partially constrained field of view allows movement of the repositionable structure as long as the orientation of the imaging device (i.e., the viewing direction of the imaging device) remains fixed. Additional examples of maintaining a partially constrained field of view are discussed in further detail below.

[0056] In some examples, the geometric features may include simple geometric elements, such as line segments (or lines including such line segments), points, and / or the like. The line segments may be straight or curved. Alternatively or additionally, the geometric features may include more complex elements, such as two-dimensional (2D) features, such as parabolas, hyperbolas, ellipses, circles, polygons, piecewise linear or nonlinear curves, planes, etc. Further alternatively or additionally, the geometric features may include three-dimensional (3D) features, such as cones, polygons, spheres, three-dimensional spatial regions with linear or nonlinear edges or surfaces, etc. The geometric features may also vary with changes in the operating environment, the operating mode of the system, the stage of the procedure being performed by the system, user preferences, etc.

[0057] In some examples, the defined geometric relationships can include simple geometric relationships such as concurrency. At the same time, at least some portions of the first geometric feature coincide in position with at least some portions of the second geometric feature. For example, a point that coincides with another point, intersects a line, lies in a plane, or intersects a region in space are examples of points that coexist with another geometric feature. Other examples of concurrency include a line segment (or multiple lines) that is collinear with and overlaps another line, intersects a plane or lies in a plane, intersects a region in space or lies within a region in space; a two-dimensional feature (such as a polygon, conic section, other two-dimensional shape, a plane, etc.) that intersects or overlaps another geometric feature. Additional examples include the intersection or overlap of other one-dimensional, two-dimensional, or three-dimensional geometric features. The defined geometric relationships can also include parallelism, where the first geometric feature is parallel to the second geometric feature. Specific examples of parallelism include parallel lines, a line parallel to a two-dimensional region or a plane, etc. In some cases, a defined geometric relationship is obtained when a portion of the first and second geometric features has such a defined geometric relationship. In some cases, a defined geometric relationship is achieved only when all of the first geometric features and the second geometric features have such a defined geometric relationship. There are many other defined geometric relationships, including orthogonality, minimum, maximum, or target separation distances, etc. In addition, the defined geometric relationships can include multiple such relationships. More generally, the defined geometric relationships can include a constraint on the relative velocity between the first geometric feature and the second geometric feature.

[0058] Figure 3 is a flowchart of method steps for controlling the movement of a repositionable structure subject to constraints according to one or more embodiments. Although the method steps are described in connection with Figures 1 - 2 the system and the examples of 4A - 7D, those of ordinary skill in the art will understand that any system configured to perform the method steps in any order is within the scope of the present disclosure. One or more of the processes 302 - 324 of method 300 can be implemented at least in part in the form of executable code stored on a non-transitory tangible machine-readable medium. When executed by a processor system (e.g., the processor system 150 in the control unit 140), the executable code can cause the processor system to perform one or more of the processes 302 - 324. In some embodiments, method 300 can be performed by a module such as control module 170. In some embodiments, method 300 can be applied to one or more proximal repositionable structures and / or one or more distal repositionable structures of a computer-aided system to maintain a partially constrained field of view of an imaging device. When a fixed field of view can no longer be maintained due to constraints associated with one or more distal repositionable structures coupled to the imaging device, the computer-aided system maintains a partially constrained field of view.

[0059] Refer to Figures 4A - 4CAspects of method 300 are described, Figures 4A - 4C illustrates the movement of one or more joints of a distally repositionable structure when maintaining a fixed field of view, according to one or more embodiments. Additionally, reference is made to Figures 5A - 5D Aspects of method 300 are described, Figures 5A - 5D shows the movement of one or more joints of a distally repositionable structure when maintaining a partially constrained field of view, according to one or more embodiments. Additionally, reference is made to Figures 6A - 6D Aspects of method 300 are described, Figures 6A - 6D shows the movement of one or more joints of a distally repositionable structure when maintaining a partially constrained field of view at infinity, according to one or more embodiments. Additionally, reference is made to Figures 7A - 7D Aspects of method 300 are described, Figures 7A - 7D shows the movement of one or more joints of a distally repositionable structure when maintaining a floating field of view, according to one or more embodiments. However, it should be understood that Figures 4A - 4C 、 Figures 5A - 5D 、 Figures 6A - 6D and Figures 7A - 7D The examples of Figures 4A - 4C 、 Figures 5A - 5D 、 Figures 6A - 6D and Figures 7A - 7D are not restrictive, and other values, shapes, behaviors, etc. depicted in

[0060]

[0061] At process 302, a control module (such as control module 170) receives a command to move a proximally repositionable structure with a commanded motion. In some examples, the commanded motion of the proximally repositionable structure is determined to move an instrument supported by the distally repositionable structure. The control module 170 can receive the command via any technically feasible technique, such as by detecting an input from one or more input controls 252 in response to manipulation by the operator 298, receiving an input from a semi-autonomous or autonomous software application executed by a processor system (e.g., processor system 150 in control unit 140), etc. The control module 170 can operate in a remote operation mode, a semi-autonomous or autonomous mode. The control module 170 can operate in a single mode during a procedure, or can switch between multiple modes during a procedure.In the remote operation mode, the control module 170 receives commands from the operator 298 via one or more input controls, such as one or more input controls 252. For example, the input controls 252 are touched and manipulated by the hands of the operator 298, such as one input control 252 per hand. In the semi-autonomous mode, the control module 170 receives commands from a software application executed by a processor system and from the operator 298 via one or more input controls 252. In some examples, the control module 170 receives commands from the operator 298 during certain steps of a program and receives commands from the software application during certain other steps of the program. Additionally or alternatively, the control module 170 receives commands from the software application, where the operator 298 may override the software application and generate commands via one or more input controls 252. During autonomous operation, the control module 170 generally receives commands from a software application program throughout the program.

[0062] At process 304, the control module determines a first commanded motion of the proximal relocatable structure to move the first distal relocatable structure and / or an instrument supported by the first distal relocatable structure. The proximal relocatable structure is proximal to a plurality of distal relocatable structures such that movement of the proximal relocatable structure causes movement of the plurality of distal relocatable structures including the first distal relocatable structure.

[0063] More specifically, the first distal relocatable structure may be subject to certain constraints such that the first distal relocatable structure cannot move according to the commanded motion. In such a case, instead of or in addition to the first distal relocatable structure, the proximal relocatable structure is moved such that the combined movement of the proximal relocatable structure and the first distal relocatable structure moves the instrument according to the commanded motion, as described in connection with Figure 2 described.

[0064] When determining the first commanded motion of the (one or more) proximal relocatable structures, the control module performs calculations using a kinematic model and / or a Jacobian matrix for each of the proximal relocatable structure, the distal relocatable structure, and / or the instrument. The control module uses these calculations to generate various numerical parameters for the joints, including joint position, velocity, and / or acceleration detected by joint sensors. These numerical parameters are then used to drive one or more joints of the proximal relocatable structure, the distal relocatable structure, and / or the instrument based on the first commanded motion.

[0065] In some examples, a second distally repositionable structure of a plurality of distally repositionable structures is configured to support an imaging device. The imaging device may include one-dimensional (1D), 2D, 3D, or higher dimensional imaging capabilities. Examples of medical imaging devices include endoscopes and ultrasound probes. Additional examples of imaging devices include optical cameras, cameras that image in the visible spectrum, infrared spectrum, ultraviolet spectrum, RF spectrum (e.g., gamma probes), hyperspectral spectrum, etc. In operation, movement of the proximally repositionable structure can be used to generate movement or assist in the movement of one or more second instruments supported by the second distally repositionable structure.

[0066] At process 306, the control module determines a second commanded movement to move the second distally repositionable structure so as to maintain a fixed field of view of the imaging device. The movement of the proximally repositionable structure described in process 304 can also move the second distally repositionable structure that supports the imaging device, and thus the movement of the proximally repositionable structure can also move the imaging device. Accordingly, the field of view of the imaging device also moves, which may disorient the operator 298 or be non-intuitive to the operator 298. Accordingly, the second distally repositionable structure coupled to the imaging device is moved in a manner so as to maintain the field of view of the imaging device. To maintain the orientation, position, or both the orientation and position of the imaging device or the field of view of the imaging device, the control module commands the movement of one or more joints of the imaging device and / or the supporting second distally repositionable structure while commanding the movement of the proximally repositionable structure. When determining the second commanded movement of the second distally repositionable structure, the control module performs calculations using the kinematic models and / or Jacobian matrices for each of the distally repositionable structures, the distally repositionable structures, and / or the instruments. The control module uses these calculations to generate various numerical parameters of the joints, including joint positions, velocities, and / or accelerations detected by joint sensors. These numerical parameters are used to drive one or more joints of the proximally repositionable structure, the distally repositionable structure, and / or the instrument based on the second commanded movement.

[0067] Figures 4A - 4C Illustrated is the movement of one or more joints of the distally repositionable structure 400 when maintaining a fixed field of view in accordance with one or more embodiments. In some examples, the distally repositionable structure 400 corresponds to the second distally repositionable structure of process 306. As Figure 4AAs shown, the imaging device axis 402 is coupled to the joint 404 of the distal repositionable structure 400. The imaging device axis 402 is coupled to the joint 404, and the joint 404 is arranged such that the distal portion of the imaging device is in the target position 410 and has the position and orientation of the target orientation 412. As shown, the target orientation 412 passes through the center of the first object 420. Correspondingly, the image 430 of the first object 420 is located at the center of the field of view 440. Similarly, the target orientation 412 passes through the center of the second object 422. Correspondingly, the image 432 of the second object 422 is also at the center of the field of view 440. As Figure 4B shown, the imaging device axis 402 moves due to the movement of the proximal repositionable structure (not shown). In some examples, the movement of the proximal repositionable structure has moved or assisted the movement of the first distal repositionable structure of the support instrument. In response, the joint 404 has moved to maintain the target position 410 and the target orientation 412 of the imaging device. Correspondingly, the image 430 of the first object 420 and the image 432 of the second object 422 remain fixed relative to the field of view 442. As Figure 4C shown, the imaging device axis 402 moves further due to additional movement of the proximal repositionable structure. In response, the joint 404 has moved further to maintain the target position 410 and the target orientation 412 of the imaging device. Correspondingly, the image 430 of the first object 420 and the image 432 of the second object 422 remain fixed relative to the field of view 444. The movement of the joint 404 that maintains a fixed field of view corresponds to the second commanded movement of the second distal repositionable structure and / or the imaging device determined during process 306.

[0068] At process 308, the control module determines whether the second commanded movement will cause the distal repositionable structure 400 to violate a constraint. In some examples, the second distal repositionable structure is constrained from moving as needed to implement the commanded movement. In particular, due to physical limitations, one or more joints of the distal repositionable structure 400 can be constrained by range-of-motion limitations and / or restrictions. Additionally or alternatively, the second commanded movement will cause the distal portion of the imaging device to enter a restricted area in the workspace. For example, the constraint will limit the distal portion of the imaging device from penetrating an entrance to the workspace deeper than a specified distance. The second commanded movement will cause the distal portion of the imaging device to penetrate an entrance deeper than the specified distance. In such cases, the control module can determine the position of the distal portion of the imaging device based on the known geometry of the joints and the repositionable structure and / or the support instrument. In other examples, the movement of the distal repositionable structure 400 can be constrained to avoid collisions with nearby objects (such as components, devices, and / or personnel), to avoid avoidance areas, etc.

[0069] If the control module determines that the second commanded motion will violate the constraints of the distal repositionable structure 400, the method proceeds to procedure 314. If the control module determines that the second commanded motion will not violate the constraints of the distal repositionable structure 400, method 300 proceeds to procedure 310, where the control module drives the proximal repositionable structure based on the first commanded motion. During procedure 310, driving the proximal repositionable structure includes driving the joints of the proximal repositionable structure using the corresponding numerical parameters determined during procedure 304. In some examples, the numerical parameters are provided as set points to one or more control systems or controls for controlling the joints of the proximal repositionable structure.

[0070] At procedure 312, the control module drives the distal repositionable structure 400 based on the second commanded motion. During procedure 312, driving the second distal repositionable structure can involve driving the respective joints of the second distal repositionable structure using the corresponding numerical parameters determined during procedure 306. In some examples, the numerical parameters are provided as set points to one or more control systems or controls for controlling the joints of the second distal repositionable structure. Then, method 300 returns to procedure 302 above to process other commanded motions of the instrument.

[0071] Returning to procedure 308, if the control module determines that the second commanded motion will cause the distal repositionable structure 400 to violate a constraint, method 300 proceeds to procedure 314, where the control module determines an alternative second commanded motion to maintain a defined geometric relationship between a first geometric feature fixed relative to the imaging device and within the field of view and a second geometric feature fixed relative to the workspace. The defined geometric relationship between the geometric features is determined such that at least one degree of freedom of the imaging device is restricted (e.g., the degrees of freedom of the distal portion of the imaging device or the field of view of the imaging device). In some examples, the defined geometric relationship between the geometric features is determined such that fewer than all six degrees of freedom of the distal portion of the imaging device or the field of view are restricted. For example, in some cases, one, two, or three translational degrees of freedom are unrestricted, and / or one, two, or three rotational degrees of freedom are unrestricted (but at least one degree of freedom is restricted). Thus, the defined geometric relationship between the geometric features is determined such that the imaging device is not permitted to move completely without any restrictions.

[0072] In some embodiments, the first geometric feature is the optical axis of the imaging device, and the second geometric feature is a fixed position in the workspace. The control module applies a defined geometric relationship to maintain the concurrency of the geometric features. In this example, the defined concurrent geometric relationship means that the optical axis (first geometric feature) passes through the fixed position (second geometric feature) in the workspace. Thus, when the field of view of the imaging device moves, an object that is positioned and held at a fixed position will appear stationary in the image captured by the imaging device while the system maintains a partially constrained field of view. Additionally, an object that is positioned and held at a location closer to the distal portion of the imaging device than the fixed position will appear to drift in a direction opposite to the direction of movement of the field of view of the imaging device in the image captured by the imaging device. An object that is positioned and held at a location farther from the distal portion of the imaging device than the fixed position will appear to drift in the same direction as the direction of movement of the field of view of the imaging device in the image captured by the imaging device.

[0073] During an example process, the control module determines a first geometric feature that is fixed relative to the imaging device (e.g., a position that is fixed relative to a portion of the imaging device, relative to the field of view of the imaging device, etc., even if such a portion or field of view moves). Examples of such first geometric features include those described above and also include: the optical axis of the imaging device, a point on the optical axis of the imaging device, a line formed by connecting the distal portion of the imaging device to the distal portion of an instrument (determined at the start of the partially constrained field of view state), etc. Additionally or alternatively, the first geometric feature can include a coordinate system that is fixed relative to the distal portion of the imaging device. For example, a coordinate system such as a triple of x, y, and z axes can be used as the first geometric feature that is fixed relative to the imaging device. In some examples, the origin of the coordinate system is located at the distal tip of the imaging device, and the z axis of the coordinate system coincides with the optical axis. The coordinate system is fixed relative to the distal portion of the imaging device and maintains a constant orientation relative to the workspace.

[0074] During an example process, the control module registers a second geometric feature that is fixed relative to the workspace via one or more of the techniques described herein. Examples of such second geometric features include those described above and also include: a fixed position in the workspace, a fixed line in the workspace, the entire workspace, etc. In some examples, the control module is configured to determine the second geometric feature based on the position of the distal portion of the instrument. The control module determines the position up to the start of the partially constrained field of view technique and / or the position prior to driving a repositionable structure or one or more joints when using the partially constrained field of view technique. In some cases, the partially constrained field of view technique is associated with a mode of the control module. In some cases, the partially constrained field of view technique is not associated with a particular mode and is used when the system cannot move the field of view exactly as commanded.

[0075] In some examples, the control module is configured to determine a second geometric feature based on the (one or more) position(s) of one or more distal portions of a plurality of instruments supported by a relocatable structure. The control module determines the position of the relocatable structure or one or more joints before initiating the field-of-view technique with partial constraints and / or while using the field-of-view technique with partial constraints. In some examples, the control module determines the second geometric feature as a fixed position in the workspace and defines the point based on a line extending from the distal portion of the imaging device in a direction based on the orientation of the imaging device at a defined moment. The control module establishes the second geometric feature at a point at a specified distance from the distal portion of the imaging device along the line. In some examples, the specified distance is fixed at the midpoint of the focal length of the imaging device. In some cases, the midpoint represents an area in the workspace where a procedure may be performed by an operator. In some examples, the specified distance is based on a depth sensor located on the imaging device, which determines the depth of an anatomical structure or other object in the workspace. In some examples, the specified distance is based on the average position of a set of instruments supported by the (one or more) relocatable structures. In such examples, the specified distance is dynamic and changes as the position and / or orientation of the instruments change. In some examples, the specified distance is set based on the type of procedure being performed. In some examples, when the control module initiates a state of maintaining a field of view with partial constraints, the fixed position in the workspace is at a reference distance from the distal portion of the imaging device and lies on the optical axis of the imaging device.

[0076] In some embodiments, the control module determines a second geometric feature fixed relative to the workspace as a point based on the position of the distal portion of one or more instruments, where the one or more instruments include any one or more of a plurality of instruments supported by a distally repositionable structure. In a specific example, the second geometric feature includes a point fixed relative to the workspace. In some examples, the control module determines the fixed position based on a first instrument introduced into the workspace and / or any one or more subsequent instruments introduced into the workspace. In some examples, the control module determines the fixed position based on a plurality of instruments. In some examples, the control module determines the vertices of a polygon, where each vertex of the polygon represents the position of the distal portion of a different instrument. In some examples, the control module establishes the fixed position based on a representative position such as the average position corresponding to the vertices of the polygon, the centroid of the polygon, the center of the smallest disk encompassing all the vertices of the polygon, etc. In some examples, the control module restricts the selection of the instruments on which the fixed position is based based on the type of instrument and / or the geometry of the instrument. In some examples, the control module causes the fixed position to be based on the static position of the distal portion of the instrument. In such an example, the control module receives the static position of the instrument positioned by the operator 298 to specify the fixed position. In some examples, the control module causes the fixed position to be based on the dynamic movement of the distal portion of the instrument, such as the trajectory of the position of the distal portion of the instrument around a desired fixed position.

[0077] In some examples, the control module sets the fixed position based on the insertion depth of one or more instruments. In some examples, the control module establishes the fixed position based on the arithmetic mean of the depths of a plurality of instruments, the geometric mean of the depths of a plurality of instruments, the median depth of a plurality of instruments, the instrument with the longest depth, the instrument with the shortest depth, etc.

[0078] In some examples, the control module causes the fixed position to be based on the focal length of the imaging device. In some cases, the focal length is the depth of focus of the imaging device, and the fixed position can be the point at that depth along the optical axis of the imaging device. In some examples, the focal length of certain imaging devices is fixed, in which case the fixed position is at a fixed distance relative to the distal portion of the imaging device. In other examples, the depth of focus of certain imaging devices is adjustable by the operator. In such cases, the control module causes the fixed position to be based on the current setting of the depth of focus of the imaging device.

[0079] In some embodiments, the control module causes the fixed position to be based on the position of at least one visible feature within the field of view of the imaging device. The visible feature can include an object in the workspace, a fixture associated with one or more instruments, etc. In some examples, the visible feature can include a fiducial indicator that appears as an object in the field of view of the imaging device or a marker on a fixture. In some medical examples, the control module causes the fixed position to be based on the position of an object that includes a portion of an anatomical structure. In some examples, the control module causes the fixed position to be based on the position of a fixture that includes means for retracting, holding, and / or moving an object (such as a portion of the anatomical structure of a medical patient).

[0080] In some examples, the position (including depth) of an object, fixture, or other fiducial marker is determined via a 3D / stereoscopic imaging device. In some examples, the position (including depth) of an object, fixture, or other fiducial marker is determined via analysis of a plurality of images captured by the imaging device as the imaging device moves. In some examples, the second geometric feature is based on the type of imaging device. In some examples, the second geometric feature is based on the particular procedure being performed.

[0081] In some embodiments, the operator 298 specifies a first geometric feature fixed relative to the imaging device and a second geometric feature fixed relative to the workspace via a user interface. Additionally or alternatively, the operator 298 activates an input to register the first geometric feature fixed relative to the imaging device and the second geometric feature fixed relative to the workspace, such as by pressing a button, issuing a voice command, activating a control on the user interface, etc. After specifying the first geometric feature fixed relative to the imaging device and the second geometric feature fixed relative to the workspace, the control module generates a mapping, where the mapping is a defined geometric relationship between the first geometric feature and the second geometric feature. The control module can generate the defined geometric relationship at any time before and / or during the execution of method 300.

[0082] In some embodiments, the control module determines the fixed position based on a line fixed relative to the imaging device, such as when starting a partially constrained field of view, the line intersects a fixed line in the workspace at a defined moment. The control module can determine the fixed line in the workspace by determining the line between two fixed positions in the workspace, determining the line between geometric features within the workspace, etc.

[0083] Typically, the control module determines a defined geometric relationship between a first geometric feature fixed relative to the imaging device and a second geometric feature fixed relative to the workspace. This defined geometric relationship is maintained throughout the process of maintaining a partially anchored field of view. As discussed herein, the relationship can be concurrency, parallelism, constancy of relative orientation, constancy of angular velocity in a defined direction, etc. In some examples, the defined geometric relationship can be a composite relationship. For example, if the first geometric feature is a coordinate system attached to the distal end of the imaging device and the second feature is a fixed position in the workspace, the control module will maintain a) the angular velocity of the first geometric feature (coordinate system) about its z-axis is zero, and b) the z-axis of the first geometric feature (coordinate system) is concurrent with the second geometric feature (i.e., the fixed position in the workspace).

[0084] Figures 5A - 5D Illustrated is the movement of one or more joints of a distal repositionable structure when maintaining a partially constrained field of view according to one or more embodiments. In some examples, the distal repositionable structure 500 corresponds to the second distal repositionable structure of process 314. As Figure 5A shown, the imaging device axis 502 is coupled to the joint 504 of the distal repositionable structure 500. The imaging device axis 502 is coupled to the joint 504 which is arranged such that the distal portion of the imaging device is in the target position 510 and has the position and orientation of the target orientation 512. As shown, the target orientation 512 passes through the center of the first object 520. Correspondingly, the image 530 of the first object 520 is located at the center of the field of view 540. Similarly, the target orientation 512 passes through the center of the second object 522. Correspondingly, the image 532 of the second object 522 is also at the center of the field of view 540. The second object 522 is located at a reference distance 524 from the distal portion of the imaging device. As Figure 5B shown, the imaging device axis 502 moves due to the movement of a proximal repositionable structure (not shown). In some examples, the movement of the proximal repositionable structure has moved or assisted the movement of the first distal repositionable structure of the support instrument. In response, the joint 504 has moved in order to maintain the target position 510 and the target orientation 512 of the imaging device. Correspondingly, the image 530 of the first object 520 and the image 532 of the second object 522 remain fixed relative to the field of view 542.

[0085] As Figure 5CAs shown, the imaging device axis 502 is further moved due to additional movement of the proximal repositionable structure. In response, the joint 504 has been further moved, but cannot maintain the target position 510 and target orientation 512 of the imaging device. Instead, a partially constrained field of view 544 is maintained such that the image 532 of the second object remains fixed relative to the partially constrained field of view 544. As discussed in connection with process 314, the control module maintains a defined geometric relationship between a first geometric feature fixed relative to the imaging device and a second geometric feature fixed relative to the workspace. As shown, the second geometric feature is a point in the field of view 544 at a fixed reference distance 524 from the first geometric feature (such as the distal portion of the imaging device) and lies on the optical axis of the imaging device (shown as the target orientation 512). An object closer to the first geometric feature than the reference distance to the second geometric feature appears to drift in the opposite direction of the direction of movement of the imaging device. An object farther from the first geometric feature than the reference distance to the second geometric feature appears to drift in the same direction as the direction of movement of the imaging device. In particular, the first object 520 is closer to the distal portion of the imaging device relative to the reference distance 524. Correspondingly, when the joint 504 has been further moved to the left, the image 530 of the first object 520 appears to move to the right relative to the partially constrained field of view 544.

[0086] As Figure 5D As shown, the imaging device axis 502 is further moved due to additional movement of the proximal repositionable structure. In response, the joint 504 has been further moved, but still cannot maintain the target position 510 and target orientation 512 of the imaging device. Instead, a partially constrained field of view 546 is maintained such that the image 532 of the second object remains fixed relative to the partially constrained field of view 546. The first object 520 is closer to the distal portion of the imaging device relative to the reference distance 524. Correspondingly, the image 530 of the first object 520 appears to move farther to the right relative to the partially constrained field of view 546. The movement of the joint 504 that maintains the partially constrained field of view corresponds to an alternative second commanded movement of the second distal repositionable structure and / or the imaging device determined during process 314.

[0087] Returning to reference process 314, in some embodiments, where the first geometric feature includes an optical axis, the second geometric feature includes a fixed position in the workspace, and the defined geometric relationship includes concurrency, the control module operates as follows. The control module determines an alternative second commanded motion to maintain concurrency of the optical axis of the imaging device with the fixed position in the workspace, where the point is at an infinite distance or at a large enough distance to effectively be at infinity for the operating parameters of the imaging device or typical human visual acuity. To specify maintaining the partially constrained field of view at infinity, the control module sets a reference viewing direction based on input from the operator 298. In some examples, the reference viewing direction is selected prior to and / or at any time during the execution of method 300.

[0088] In some examples, the reference viewing direction is selected via one or more techniques. In some examples, when the control module begins maintaining the partially constrained field of view at infinity, the control module sets the reference field of view direction to the current field of view direction of the imaging device. In some examples, the operator selects an object in the field of view that is greater than a threshold distance from the distal portion of the imaging device. In response to the selection, the control module determines that the object in the field of view is greater than a threshold distance from the distal portion of the imaging device and begins maintaining the partially constrained field of view at infinity. The control module sets the reference viewing direction based on the angle between the distal portion of the imaging device and the selected object. In some examples, the operator manually selects to maintain the partially constrained field of view in infinity mode via the user interface. In some examples, the operator manually sets the reference viewing direction via a touch screen, stylus, joystick, trackball, button cluster, and / or other input controls.

[0089] Regardless of the technique used to establish the reference viewing direction, the operator specifies the reference viewing direction and / or activates an input to register the reference viewing direction via the user interface, such as by pressing a button, issuing a voice command, activating a control on the user interface, etc. The selected reference for maintaining the partially constrained field of view at infinity is at a reference distance at infinity or at a large distance greater than a threshold amount effectively at infinity. An object at a greater distance from the distal portion of the imaging device is effectively at infinity while maintaining the partially constrained field of view at infinity. An object not at infinity is in the foreground. All objects not at infinity appear to drift in the opposite direction of the direction of movement relative to the imaging device. An object located at infinity (or an object at a large distance greater than a threshold amount effectively at infinity relative to the distal portion of the imaging device) remains fixed within the field of view of the imaging device.

[0090] Figures 6A - 6DIllustrated is the movement of one or more joints of a distal repositionable structure when maintaining a partially constrained field of view at infinity, according to one or more embodiments. In some examples, the distal repositionable structure 600 corresponds to the second distal repositionable structure of process 314. As Figure 6A shown, the imaging device axis 602 is coupled to joint 604 of the distal repositionable structure 600. The imaging device axis 602 is coupled to joint 604, which is arranged such that the distal portion of the imaging device is in a target position 610 and has a target orientation 612 of position and orientation. As shown, the target orientation 612 passes through the center of the first object 620. Correspondingly, the image 630 of the first object 620 is located at the center of the field of view 640. Similarly, the target orientation 612 passes through the center of the second object 622. Correspondingly, the image 632 of the second object 622 is also at the center of the field of view 640. The target orientation 612 is set in a reference viewing direction relative to the distal portion of the imaging device. The second object 622 is located at a reference distance 624 from the distal portion of the imaging device. As Figure 6B shown, the imaging device axis 602 moves due to the movement of a proximal repositionable structure (not shown). In some examples, the movement of the proximal repositionable structure has moved or assisted in the movement of the first distal repositionable structure of the support instrument. In response, joint 604 has moved to maintain the target position 610 and target orientation 612 of the imaging device. Correspondingly, the image 630 of the first object 620 and the image 632 of the second object 622 remain fixed relative to the field of view 642.

[0091] As Figure 6C shown, the imaging device axis 602 moves further due to additional movement of the proximal repositionable structure. In response, joint 604 has moved further, but cannot maintain both the target position 610 and target orientation 612 of the imaging device. Instead, a partially constrained field of view at infinity 644 is maintained such that the distal portion of the imaging device moves from the target position 610. However, the orientation of the imaging device is maintained at the target orientation 612, which is the reference viewing direction. Both the first object 620 and the second object 622 are located at a distance less than infinity from the distal portion of the imaging device. Correspondingly, the image 630 of the first object 620 and the image 632 of the second object 622 appear to move to the right relative to the partially constrained field of view at infinity 644. Since the first object 620 is closer to the distal portion of the imaging device than the second object 622, the image 630 of the first object 620 appears to move further to the right relative to the image 632 of the second object 622.

[0092] As Figure 6DAs shown, the imaging device axis 602 is further moved due to additional movement of the proximal repositionable structure. In response, the joint 604 has been further moved, but still cannot maintain the target position 610 and target orientation 612 of the imaging device. Instead, a partially constrained field of view at infinity 646 is maintained, such that the distal portion of the imaging device is further moved from the target position 610. However, the orientation of the imaging device is maintained at the target orientation 612, which is the reference viewing direction. Both the first object 620 and the second object 622 are located at a distance less than infinity from the distal portion of the imaging device. Correspondingly, the image 630 of the first object 620 and the image 632 of the second object 622 appear to move further to the right relative to the partially constrained field of view at infinity 646. Since the first object 620 is closer to the distal portion of the imaging device than the second object 622, the image 630 of the first object 620 appears to move further to the right relative to the image 632 of the second object 622. Additionally, relative to Figure 6C the partially constrained field of view at infinity 644 of Figure 6D the divergence between the image 630 of the first object 620 and the image 632 of the second object 622 is greater in

[0093] the partially constrained field of view at infinity 646. The movement of the joint 604 that maintains the partially constrained field of view at infinity corresponds to an alternative second commanded movement of the second distal repositionable structure and / or the imaging device determined during process 314.

[0093] When maintaining a partially constrained field of view (including a partially constrained field of view at infinity), the control module can prohibit movement of the second distal repositionable structure in certain directions and / or degrees of freedom while enabling movement of the second distal repositionable structure in other directions and / or degrees of freedom. In some examples, as further movement of the proximal repositionable structure occurs, the imaging device can continue to move as long as the reference position can be maintained within a small tolerance of a fixed position in the workspace. In various examples, the small tolerance can include a position accuracy tolerance and an orientation accuracy tolerance. In some examples, the position accuracy tolerance can be in the range of 0.25, 0.5, 1, or 2 cm, and the orientation accuracy tolerance can be in the range of 5, 10, 20, or 30 degrees. For example, in the case of a partially constrained field of view at infinity, even if the control module nominally maintains the orientation as fixed, the control module allows an orientation drift up to the orientation accuracy tolerance. In some examples, the characteristic dimension of the cross-section of the instrument is from about 3 mm to 10 mm. Additionally, the instrument workspace of such an instrument can be a cylinder with a diameter of 5 cm to 20 cm or some other 3D region with a characteristic dimension larger than the characteristic dimension of the instrument. In an example where the instrument has a characteristic dimension of 5 mm, the instrument workspace is a cylinder with a diameter of 6 cm and the position accuracy tolerance is 1 cm, then the position accuracy is approximately twice the characteristic dimension of the instrument and approximately 17% of the diameter of the instrument workspace.

[0094] At process 316, the control module determines whether an alternative second commanded motion can be performed while maintaining a partially constrained field of view. When maintaining a partially constrained field of view, the control module determines whether an alternative second commanded motion can be performed and whether the partially constrained field of view can be maintained. In some examples, the control module determines that the partially constrained field of view can no longer be maintained due to constraints on one or more joints, physical limitations of the second distal repositionable structure, etc. In such cases, the control module determines that the alternative second commanded motion cannot be performed.

[0095] If the control module determines that an alternative second commanded motion can be performed while maintaining a partially constrained field of view, method 300 proceeds to process 318, where the control module drives the proximal repositionable structure based on the first commanded motion. During process 318, driving the proximal repositionable structure can involve driving the respective joints of the proximal repositionable structure using the corresponding numerical parameters determined during process 304. In some examples, the numerical parameters are provided as set points to one or more control systems or controls for controlling the joints of the proximal repositionable structure.

[0096] At process 320, the control module drives the second distal repositionable structure based on an alternative second commanded motion. During process 320, driving the second distal repositionable structure can involve using the corresponding numerical parameters determined during process 314 to drive the respective joints of the second distal repositionable structure. In some examples, the numerical parameters are provided as set points to one or more control systems or controls for controlling the joints of the second distal repositionable structure. The method 300 then returns to process 302 above to process other commanded motions of the instrument.

[0097] Returning to process 316, if the control module determines that the alternative second commanded motion cannot be performed while maintaining a partially constrained field of view, the method 300 proceeds to process 322, where the control module exits the partially constrained field of view and determines a remedial action in lieu of the alternative commanded motion.

[0098] In some examples, the remedial action includes initiating a mode for maintaining a fixed field of view, as described in connection with processes 304 - 306. In doing so, the control module does not permit movement of the proximal repositionable structure that would cause one or more joints to violate the constraints. The control module continues to disallow movement of the proximal repositionable structure until a command to move in a direction that will no longer result in a constraint is received.

[0099] In some examples, the remedial action includes initiating a mode for maintaining a floating field of view. In doing so, the control module permits the proximal repositionable structure to move without restriction while attempting to maintain as closely as possible the defined geometric relationship between the first geometric feature and the second geometric feature given the system constraints or other operating constraints. The control module continues to maintain the floating field of view until a command to move in a direction that will no longer result in a constraint is received.

[0100] Figures 7A - 7D Illustrated is movement of one or more joints of a distal repositionable structure when maintaining a floating field of view in accordance with one or more embodiments. In some examples, the distal repositionable structure 700 corresponds to the second distal repositionable structure of processes 306 and 308. As Figure 7A shown, the imaging device axis 702 is coupled to a joint 704 of the distal repositionable structure 700. The imaging device axis 702 is coupled to joint 704, which is positioned such that the distal portion of the imaging device is at a target position 710 and has a target orientation 712. As shown, the target orientation 712 passes through the center of a first object 720. Correspondingly, an image 730 of the first object 720 is located at the center of the field of view 740. Similarly, the target orientation 712 passes through the center of a second object 722. Correspondingly, an image 732 of the second object 722 is also at the center of the field of view 740. As Figure 7BAs shown, imaging device axis 702 moves due to movement of a proximal repositionable structure (not shown). In some examples, movement of the proximal repositionable structure has moved or assisted movement of a first distal repositionable structure of a support instrument. In response, joint 704 has moved to maintain target position 710 and target orientation 712 of the imaging device. Correspondingly, image 730 of first object 720 and image 732 of second object 722 remain fixed relative to field of view 742.

[0101] As Figure 7C shown, imaging device axis 702 moves further due to additional movement of the proximal repositionable structure. In response, joint 704 has moved further but cannot maintain target position 710 and target orientation 712 of the imaging device. Additionally, joint 704 cannot maintain a partially constrained field of view. Instead, a floating field of view 744 is maintained such that the distal portion of the imaging device moves from target position 710. Additionally, the orientation of the imaging device moves from target orientation 712. Correspondingly, image 730 of first object 720 appears to move to the right relative to floating field of view 744. Similarly, image 732 of second object 722 also appears to move to the right relative to floating field of view 744.

[0102] As Figure 7D shown, imaging device axis 702 moves further due to additional movement of the proximal repositionable structure. In response, joint 704 has moved further but cannot maintain target position 710 and target orientation 712 of the imaging device. Moreover, joint 704 cannot maintain a partially constrained field of view. Instead, a floating field of view 744 is maintained such that the distal portion of the imaging device moves away from target position 710. Additionally, the orientation of the imaging device moves further away from target orientation 712. Correspondingly, image 730 of first object 720 appears to move further to the right relative to floating field of view 744. Similarly, image 732 of second object 722 also appears to move further to the right relative to floating field of view 744. Because first object 720 is closer to the distal portion of the imaging device relative to second object 722, image 730 of first object 720 appears to move further to the right relative to image 732 of second object 722. Additionally, relative to Figure 7C the floating field of view 744, the divergence between image 730 of first object 720 and image 732 of second object 722 is greater in Figure 7D the floating field of view 746. Movement of joint 704 that maintains the floating field of view corresponds to a remedial action in lieu of moving a second distal repositionable structure and / or an alternative second commanded movement of the imaging device.

[0103] Returning to reference process 322, in some examples, the remedial action includes generating a request for the operator to select between maintaining a fixed field of view or a floating field of view when a defined geometric relationship between the first geometric feature and the second geometric feature can no longer be maintained due to a commanded motion. In some examples, when the defined geometric relationship can no longer be maintained at a fixed position relative to the workspace due to a commanded motion, the control module allows the operator to override the constraint and allows the commanded motion. At process 324, the control module performs the remedial action. Then, method 300 returns to process 302 described above to process other commanded motions of the instrument.

[0104] As discussed above and further emphasized here, Figures 3 - 7D are merely examples that should not unduly limit the scope of the claims. Many variations, alternatives, and modifications are possible. In some embodiments, the control module generates haptic feedback to assist in guiding the operator 298 during the execution of method 300. In some examples, when starting or exiting the state of maintaining a partially constrained field of view and / or maintaining a partially constrained field of view at infinity, the control module generates haptic feedback of short duration, such as a brief pulse. In some examples, when maintaining a partially constrained field of view, the control module generates haptic resistance proportional to the difference between: (1) the position and orientation of the distal portion of the imaging device when starting to maintain the partially constrained field of view; and (2) the current position and / or orientation of the distal portion of the imaging device. In some examples, when transitioning to maintaining a fixed field of view or a floating field of view, the control module generates haptic feedback consistent with maintaining a fixed field of view or a floating field of view, respectively.

[0105] In some embodiments, and as discussed above, the first geometric feature and / or the second geometric feature may not be a point and may be a line, a plane, etc. In some examples of geometric features, the control module determines two separate fixed positions in the workspace based on any one or more of the techniques described herein. For example, the control module may determine two fixed positions based on the positions of the distal portions of any two instruments introduced into the workspace. Then, the control module determines a fixed line in the workspace that connects the two fixed positions. The control module determines a line in the field of view, where a portion of the workspace along the fixed line in the workspace is visible in the field of view of the imaging device. The control module maps the 2D line in the field of view to the 3D fixed line in the workspace. In some examples, the control module may determine the geometric feature based on two lines in the workspace set at different depths from the imaging device, where the two lines would intersect if projected onto a plane orthogonal to the viewing direction of the imaging device.

[0106] Some examples of the control unit (such as Figure 1The control unit 140) may include a non-transitory tangible machine-readable medium that includes executable code that, when executed by a processor system (e.g., Figure 1 the processor system 150) of, causes the processor system to perform the processes of method 300. Some common forms of machine-readable media that may include the processes of method 300 may include floppy disks, flexible disks, hard disks, magnetic tapes, any other magnetic medium, CD-ROMs, any other optical medium, punch cards, paper tapes, any other physical medium with hole patterns, RAMs, PROMs, EPROMs, FLASH-EPROMs, any other memory chips or cartridge tapes and / or any other medium readable by a processor or computer.

[0107] Although the illustrative embodiments have been shown and described, various modifications, variations, and alternatives are encompassed within the foregoing disclosure, and in some instances, some features of the embodiments may be employed without corresponding use of other features. Those of ordinary skill in the art will recognize many variations, alternatives, and modifications. Accordingly, the scope of the present invention should be limited only by the appended claims, and the claims should be broadly construed in a manner consistent with the scope of the embodiments disclosed herein.

Claims

1. A computer-aided system, comprising: a proximal relocatable structure; a first distal relocatable structure physically coupled to the proximal relocatable structure; and a processor system communicatively coupled to the proximal relocatable structure and the first distal relocatable structure; wherein the processor system is configured to: determine a first commanded motion of the proximal relocatable structure; determine a second commanded motion of one or more joints selected from the group consisting of the first distal relocatable structure and joints of an imaging device supported by the first distal relocatable structure, wherein the field of view of the imaging device is maintained relative to a workspace when the second commanded motion is executed in combination with the first commanded motion; determine whether driving the one or more joints according to the second commanded motion will cause the first distal relocatable structure or the imaging device to violate a first constraint; and in response to determining that driving the one or more joints according to the second commanded motion will cause the first distal relocatable structure or the imaging device to violate the first constraint: determine an alternative second commanded motion for the one or more joints to maintain a defined geometric relationship between a first geometric feature fixed relative to the imaging device and a second geometric feature fixed relative to the workspace, drive the proximal relocatable structure according to the first commanded motion, and drive the one or more joints according to the alternative second commanded motion.

2. The computer-aided system according to claim 1, wherein executing the alternative second commanded motion in combination with the first commanded motion will keep the defined geometric relationship within a tolerance, wherein the tolerance includes a position accuracy tolerance or an orientation accuracy tolerance.

3. The computer-aided system according to claim 1, wherein the first constraint includes a physical constraint or a motion constraint, and wherein, to determine whether driving the one or more joints according to the second commanded motion will cause the first distal relocatable structure or the imaging device to violate the first constraint, the processor system is configured to: determine whether driving the one or more joints according to the second commanded motion will cause a first joint among the one or more joints to violate the physical constraint or the motion constraint.

4. The computer-aided system according to claim 1, wherein the first constraint includes at least a portion of an avoidance region, and wherein, to determine whether driving the one or more joints according to the second commanded motion will cause the first distal relocatable structure or the imaging device to violate the first constraint, the processor system is configured to: determine whether driving the one or more joints according to the second commanded motion will cause a distal portion of the imaging device to enter the avoidance region.

5. The computer-aided system according to claim 1, wherein: The computer - aided system further includes: a second distal re - positionable structure physically coupled to the proximal re - positionable structure and communicatively coupled to the processor system; and the first commanded motion of the proximal re - positionable structure is determined to move an instrument supported by the second distal re - positionable structure.

6. The computer - aided system according to claim 5, wherein: the computer - aided system further includes an input system configured to be manipulated by an operator; and the first commanded motion of the proximal re - positionable structure is determined based on an operator command received by the input system to move the instrument.

7. The computer - aided system according to claim 5 or 6, wherein the processor system is further configured to determine the second geometric feature based on the position of the distal portion of the instrument at a time prior to driving the proximal re - positionable structure in response to the first commanded motion.

8. The computer - aided system according to any one of claims 1 to 6, wherein the second geometric feature includes a reference position.

9. The computer - aided system according to any one of claims 1 to 6, wherein the second geometric feature is positioned along an axis defined with respect to the field of view of the imaging device.

10. The computer - aided system according to claim 9, wherein the axis is the optical axis of the imaging device, and wherein the second geometric feature is a point located along the optical axis at a specified distance distally from the distal portion of the imaging device, and wherein the specified distance is selected from the group consisting of: the mid - point of the focal length of the imaging device, the depth of an object in the workspace, the distance to the object in the workspace, and the average position of the parts of a plurality of instruments supported by the computer - aided system.

11. The computer - aided system according to any one of claims 1 to 6, wherein: the first geometric feature includes a first line segment fixed relative to the imaging device; the second geometric feature includes a point fixed relative to the workspace or a second line segment fixed relative to the workspace; and the defined geometric relationship includes the intersection of the first geometric feature and the second geometric feature.

12. The computer - aided system according to claim 11, wherein: the first line segment is straight; and the second line segment is straight.

13. The computer - aided system according to any one of claims 1 to 6, wherein the processor system is further configured to: for each instrument included in a plurality of instruments supported by the computer - aided system, determine the position of the distal portion of the instrument; and determine the second geometric feature based on the position of the distal portion of the instrument at a time prior to driving the proximal re - positionable structure in response to the first commanded motion.

14. The computer - aided system according to claim 13, wherein the processor system is further configured to: Determine the defined geometric relationship based on a line connecting a first position of a first distal portion of the first of the plurality of instruments and a second position of a second distal portion of the second of the plurality of instruments; or Determine the defined geometric relationship based on a representative position representative of the positions of the distal portions of the plurality of instruments.

15. The computer - aided system according to any one of claims 1 to 6, wherein the processor system is further configured to: position the second geometric feature at a specified distance from a distal portion of the imaging device.

16. The computer - aided system according to claim 15, wherein the processor system is further configured to determine the specified distance based on at least one parameter selected from the group consisting of: The focal length of the imaging device; The depth of an object in the workspace; The position of an instrument supported by the computer - aided system; The positions of a plurality of instruments supported by the computer - aided system; And The type of program being executed using the computer - aided system.

17. The computer - aided system according to any one of claims 1 to 6, wherein the processor system is further configured to: position the second geometric feature at an infinite distance from a distal portion of the imaging device.

18. The computer - aided system according to any one of claims 1 to 6, wherein the processor system is further configured to: determine the defined geometric relationship relative to the workspace based on the position of at least one visible feature in the field of view of the imaging device.

19. The computer - aided system according to any one of claims 1 to 6, wherein the processor system is further configured to: Determine a third commanded motion of the proximal repositionable structure; Determine a fourth commanded motion of the one or more joints, wherein when the fourth commanded motion is executed in combination with the third commanded motion, the field of view of the imaging device is maintained relative to the workspace; Determine whether driving the one or more joints according to the fourth commanded motion will cause the first distal repositionable structure or the imaging device to violate a second constraint; and In response to determining that driving the one or more joints according to the fourth commanded motion will cause the first distal repositionable structure or the imaging device to violate the second constraint: Drive the proximal repositionable structure according to the third commanded motion.

20. The computer - aided system according to claim 19, wherein the second constraint Comprises: Physical constraints of the proximal repositionable structure, the first distal repositionable structure, or the imaging device; Or Motion constraints of the proximal repositionable structure, the first distal repositionable structure, or the imaging device.

21. The computer - aided system according to any one of claims 1 to 6, wherein the processor system is further configured to: Determine a third commanded motion of the proximal repositionable structure; Determine a fourth commanded motion of the one or more joints, wherein when the fourth commanded motion is performed in combination with the third commanded motion, the field of view of the imaging device is maintained relative to the workspace; Determine whether driving the one or more joints according to the fourth commanded motion will cause the first distally repositionable structure or the imaging device to violate a second constraint; and In response to determining that driving the one or more joints according to the fourth commanded motion will cause the first distally repositionable structure or the imaging device to violate the second constraint: Determine an alternative fourth commanded motion of the one or more joints to maintain the defined geometric relationship between the first geometric feature and the second geometric feature, and In response to determining that driving the one or more joints according to the alternative fourth commanded motion will not cause the first distally repositionable structure or the imaging device to violate a third constraint, allow the proximally repositionable structure to be driven according to the third commanded motion; and In response to determining that driving one or more joints according to the alternative fourth commanded motion will cause the first distally repositionable structure or the imaging device to violate the third constraint, do not allow the proximally repositionable structure to be driven according to the third commanded motion.

22. The computer-aided system according to claim 21, wherein each of the second constraint and the third constraint comprises: a physical constraint of the proximally repositionable structure, the first distally repositionable structure, or the imaging device, or a motion constraint of the proximally repositionable structure, the first distally repositionable structure, or the imaging device.

23. The computer-aided system according to any one of claims 1 to 6, wherein the computer-aided system is a surgical system.

24. A method, comprising: determining, by a processor system, a first commanded motion of a proximally repositionable structure; determining, by the processor system, a second commanded motion of one or more joints selected from the group consisting of joints of a first distally repositionable structure and an imaging device, the first distally repositionable structure being physically coupled to the proximally repositionable structure, and the imaging device being supported by the first distally repositionable structure, wherein when the second commanded motion is performed in combination with the first commanded motion, the field of view of the imaging device is maintained relative to the workspace; determining, by the processor system, whether driving the one or more joints according to the second commanded motion will cause the first distally repositionable structure or the imaging device to violate a first constraint; and In response to determining that driving the one or more joints according to the second commanded motion will cause the first distally repositionable structure or the imaging device to violate the first constraint: The processor system determines an alternative second commanded motion for the one or more joints to maintain a defined geometric relationship between a first geometric feature fixed relative to the imaging device and a second geometric feature fixed relative to the workspace. The processor system drives the proximal repositionable structure in accordance with the first commanded motion, and The processor system drives the one or more joints in accordance with the alternative second commanded motion.

25. The method according to claim 24, wherein performing the alternative second commanded motion in combination with the first commanded motion will maintain the defined geometric relationship within a tolerance, wherein the tolerance includes a position accuracy tolerance or an orientation accuracy tolerance.

26. The method according to claim 24, wherein the first constraint includes a physical constraint or a motion constraint, and wherein determining whether driving the one or more joints in accordance with the second commanded motion will cause the first distal repositionable structure or the imaging device to violate the first constraint comprises: The processor system determines whether driving the one or more joints in accordance with the second commanded motion will cause a first joint among the one or more joints to violate the physical constraint or the motion constraint.

27. The method according to claim 24, wherein the first constraint includes at least a portion of an avoidance region, and wherein determining whether driving the one or more joints in accordance with the second commanded motion will cause the first distal repositionable structure or the imaging device to violate the first constraint comprises: The processor system determines whether driving the one or more joints in accordance with the second commanded motion will cause a distal portion of the imaging device to enter the avoidance region.

28. The method according to claim 24, wherein determining the first commanded motion of the proximal repositionable structure to move an instrument supported by a second distal repositionable structure physically coupled to the proximal repositionable structure and communicatively coupled to the processor system.

29. The method according to claim 28, wherein the first commanded motion of the proximal repositionable structure is determined based on an operator command received by an input system configured to be manipulated by an operator.

30. The method according to claim 28 or 29, further comprising the processor system determining the second geometric feature based on a position of a distal portion of the instrument supported by the second distal repositionable structure at a time prior to driving the proximal repositionable structure in response to the first commanded motion.

31. The method according to any one of claims 24 to 29, wherein the second geometric feature includes a reference position.

32. The method according to any one of claims 24 to 29, wherein the second geometric feature is positioned along an axis defined relative to the field of view of the imaging device.

33. The method according to claim 32, wherein the axis is the optical axis of the imaging device, and wherein the second geometric feature is a point located at a specified distance from the distal portion of the imaging device along the optical axis, and wherein the specified distance is selected from the group consisting of: the midpoint of the focal length of the imaging device, the depth of an object in the working space, the distance to the object in the working space, and the average position of the parts of a plurality of instruments supported by a computer-aided system including the processor system.

34. The method according to any one of claims 24 to 29, wherein: the first geometric feature includes a first line segment fixed relative to the imaging device; the second geometric feature includes a point fixed relative to the working space or a second line segment fixed relative to the working space; and the defined geometric relationship includes the intersection of the first geometric feature and the second geometric feature.

35. The method according to claim 34, wherein: the first line segment is straight; and the second line segment is straight.

36. The method according to any one of claims 24 to 29, further comprising: for each instrument included in a plurality of instruments supported by a computer-aided system including the processor system, determining, by the processor system, the position of the distal portion of the instrument; and determining, by the processor system, the second geometric feature based on the position of the distal portion of the instrument at a time prior to driving the proximal repositionable structure in response to the first command.

37. The method according to claim 36, further comprising: determining, by the processor system, the defined geometric relationship based on a line connecting a first position of a first distal portion of a first instrument among the plurality of instruments and a second position of a second distal portion of a second instrument among the plurality of instruments; or determining, by the processor system, the defined geometric relationship based on a representative position representing the positions of the distal portions of the plurality of instruments.

38. The method according to any one of claims 24 to 29, further comprising: positioning, by the processor system, the second geometric feature at a specified distance from the distal portion of the imaging device.

39. The method according to claim 38, further comprising determining, by the processor system, the specified distance based on at least one parameter selected from the group consisting of: the focal length of the imaging device; the depth of an object in the working space; the position of an instrument supported by a computer-aided system including the processor system; the positions of a plurality of instruments supported by the computer-aided system; and the type of program being executed using the computer-aided system.

40. The method according to any one of claims 24 to 29, further comprising: positioning, by the processor system, the second geometric feature at an infinite distance from the distal portion of the imaging device.

41. The method according to any one of claims 24 to 29, further comprising: The processor system determines the defined geometric relationship relative to the workspace based on the position of at least one visible feature in the field of view of the imaging device.

42. The method according to any one of claims 24 to 29, further comprising: determining, by the processor system, a third commanded movement of the proximal repositionable structure; determining, by the processor system, a fourth commanded movement of the one or more joints, wherein when the fourth commanded movement is executed in combination with the third commanded movement, the field of view of the imaging device is maintained relative to the workspace; determining, by the processor system, whether driving the one or more joints according to the fourth commanded movement will cause the first distal repositionable structure or the imaging device to violate a second constraint; and in response to determining that driving the one or more joints according to the fourth commanded movement will cause the first distal repositionable structure or the imaging device to violate the second constraint: driving, by the processor system, the proximal repositionable structure according to the third commanded movement.

43. The method according to claim 42, wherein the second constraint comprises: a physical constraint of the proximal repositionable structure, the first distal repositionable structure, or the imaging device; or a motion constraint of the proximal repositionable structure, the first distal repositionable structure, or the imaging device.

44. The method according to any one of claims 24 to 29, further comprising: determining, by the processor system, a third commanded movement of the proximal repositionable structure; determining, by the processor system, a fourth commanded movement of the one or more joints, wherein when the fourth commanded movement is executed in combination with the third commanded movement, the field of view of the imaging device is maintained relative to the workspace; determining, by the processor system, whether driving the one or more joints according to the fourth commanded movement will cause the first distal repositionable structure or the imaging device to violate a second constraint; and in response to determining that driving the one or more joints according to the fourth commanded movement will cause the first distal repositionable structure or the imaging device to violate the second constraint: determining, by the processor system, an alternative fourth commanded movement of the one or more joints to maintain the defined geometric relationship between the first geometric feature and the second geometric feature, and in response to determining that driving the one or more joints according to the alternative fourth commanded movement will not cause the first distal repositionable structure or the imaging device to violate a third constraint, allowing the proximal repositionable structure to be driven according to the third commanded movement; and in response to determining that driving the one or more joints according to the alternative fourth commanded movement will cause the first distal repositionable structure or the imaging device to violate the third constraint, not allowing the processor system to drive the proximal repositionable structure according to the third commanded movement.

45. The method according to claim 44, wherein each of the second constraint and the third constraint comprises: Physical constraints of the proximal repositionable structure, the first distal repositionable structure, or the imaging device, or Motion constraints of the proximal repositionable structure, the first distal repositionable structure, or the imaging device.

46. The method according to any one of claims 24 to 29, wherein the processor system is included in a computer-assisted system including a surgical system.

47. One or more non-transitory machine-readable media including a plurality of machine-readable instructions that, when executed by a processor system associated with a computer-assisted system, are adapted to cause the processor system to execute the method according to any one of claims 24 - 46.

Images