Aligning instrument supported by computer-aided system

Abstract

Computer-assisted systems and methods for performing instrument targeting therewith are provided. A computer-aided system includes a first repositionable structure configured to support a first instrument and a control system including one or more processors. The control system may be communicatively coupled to the first repositionable structure. To perform instrument targeting, a control system: (1) determines a target in a workspace for positioning a first instrument supported by a first repositionable structure; (2) defining a first target axis based on the target and the entry venue location; (3) determining a configuration of the first plurality of joints, the configuration aligning the first instrument with the first target axis such that the first instrument can be advanced along the first target axis towards the target; and (4) commanding the first plurality of joints based on the determined configuration.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority and benefit on the filing date of Provisional U.S. Patent Application No. 63 / 585,912, filed September 27, 2023, entitled “ALIGNING AN INSTRUMENT SUPPORTEDBY A COMPUTER-ASSISTED SYSTEM”. The entire contents of that provisional application are expressly incorporated herein by reference. Technical Field

[0003] This disclosure relates generally to computer-aided systems, and more specifically to the alignment of instruments supported by computer-aided systems. Background Technology

[0004] Some computer-aided systems include one or more instruments engaged in performing various procedures. Computer-aided systems can be automated, semi-automated, remotely operated, etc. In a remotely operated example, a human operator manipulates one or more guide input controls to command the movement of one or more follower devices located in the workspace. Example follower devices include instruments, repositionable structures configured to support instruments, etc. In some examples, the remote operating system is configured to support instruments such as catheters, electrocautery devices, cutting devices, grasping devices, sutures, etc. In some cases, the computer-aided system moves instruments within the workspace to perform tasks at the worksite, such as manipulating specific tissues within a patient's internal anatomy in a medical example.

[0005] Inserting and aligning instruments into a workspace to perform a task in the workplace can be complex or inefficient. For example, repositionable structures or instruments with more complex kinematic designs, larger amounts of material, or higher inertial components may be more difficult to manipulate or control. As another example, workspaces with smaller entry sites (e.g., minimally invasive applications) may require more precise alignment.

[0006] Therefore, improved technologies are desired for aligning instruments, for example, for advancing toward the workspace. Such technologies could allow for faster and more efficient instrument alignment. Summary of the Invention

[0007] The following is a simplified overview of the various examples described herein and is not intended to identify key or important elements or define the scope of the claims.

[0008] In one embodiment, a computer-aided system is provided. The computer-aided system may include: (i) a first repositionable structure configured to support a first instrument, wherein the first repositionable structure supporting the first instrument includes a first plurality of joints; and (ii) a control system including one or more processors, wherein the control system is communicatively coupled to the first repositionable structure. For performing instrument aiming, the control system may be configured to: (1) determine a target in a workspace for positioning the first instrument supported by the first repositionable structure; (2) define a first target axis based on the target and the location of entry; (3) determine a configuration of the first plurality of joints that aligns the first instrument with the first target axis, enabling the first instrument to be advanced along the first target axis; and (4) command the first plurality of joints based on the determined configuration.

[0009] In another embodiment, a method for performing instrument aiming using a computer-aided system is provided, the computer-aided system comprising: (i) a first repositionable structure configured to support a first instrument, wherein the first repositionable structure supporting the first instrument includes a first plurality of joints; and (ii) a control system including one or more processors communicatively coupled to the first repositionable structure. The method includes: (1) determining, via the control system, a target in a workspace for positioning the first instrument supported by the first repositionable structure; (2) defining a first target axis via the control system based on the target and the location of entry; (3) determining, via the control system, a configuration of the first plurality of joints aligning the first instrument with the first target axis such that the first instrument can be advanced along the first target axis; and (4) commanding the first plurality of joints via the control system based on the determined configuration.

[0010] In another embodiment, one or more non-transitory machine-readable media include a plurality of machine-readable instructions that, when executed by a processor system, are adapted to cause the processor system to perform any of the methods described herein.

[0011] It should be understood that both the foregoing general description and the following detailed description are illustrative and explanatory in nature and are intended to provide an understanding of this disclosure, without limiting its scope. In this regard, additional aspects, features, and advantages of this disclosure will be apparent to those skilled in the art from the following detailed description. Attached Figure Description

[0012] Figure 1 It is a diagram of a computer-aided system according to one or more implementation methods.

[0013] Figure 2 It is an example depiction of defining a target axis and / or target position using a computer-aided system according to one or more embodiments.

[0014] Figure 2 It is an example depiction of using a computer-aided system to define the target axis and / or the destination position according to one or more embodiments.

[0015] Figure 3 It is an example graphical user interface depicting a visual indication of a target axis defined by a computer-aided system according to one or more embodiments.

[0016] Figures 4A to 4D An example process is shown for aligning a follower device with a target axis using a computer-aided system according to one or more embodiments.

[0017] Figure 5 It is a flowchart of method steps for manipulating an imaging device when inserting an instrument into a computer-aided system, according to one or more embodiments.

[0018] Examples of this disclosure and its advantages will be best understood by referring to the following detailed description. It should be understood that similar reference numerals are used to identify one or more similar elements shown in the figures, wherein the illustrations in the figures are for illustrative purposes and not for limiting the scope of this disclosure. Detailed Implementation

[0019] In this specification, specific details are set forth in relation to some embodiments consistent with this disclosure. Numerous specific details are set forth to provide a thorough understanding of the embodiments. However, it will be apparent to those skilled in the art that some embodiments may be practiced without some or all of these specific details. The specific embodiments disclosed herein are intended to be illustrative and not restrictive. Although not specifically described herein, those skilled in the art will recognize other elements within the scope and spirit of this disclosure. Furthermore, to avoid unnecessary repetition, one or more features shown and described in association with one embodiment may be incorporated into other embodiments unless otherwise specifically described or if one or more features would render the embodiment inoperable.

[0020] Furthermore, the terminology used in this specification is not intended to limit the invention. For example, spatially relative terms—such as “below,” “under,” “lower,” “above,” “upper,” “near,” “far”, etc.—may be used to describe the relationship of one element or feature to another element or feature as shown in the figures. In addition to the positions and orientations shown in the figures, these spatially relative terms are intended to include different positions (i.e., locations) and orientations (i.e., rotational placements) of elements or their operations. For example, if one of the contents in the figures is flipped, an element described as being “below” or “under” other elements or features will be “above” or “on” other elements or features. Devices may be oriented in other ways, and the spatially relative descriptors used herein are interpreted accordingly. Similarly, descriptions of movement along and about various axes include various specific element positions and orientations. Additionally, unless the context otherwise indicates, the singular forms “a,” “an,” and “the” are also intended to include the plural forms. Additionally, the terms “comprising,” “containing,” “including,” etc., specify the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups. Components described as coupled may be directly electrically or mechanically coupled, or they may be indirectly coupled via one or more intermediate components.

[0021] An element described in detail with reference to one embodiment, implementation, system, or module may be included in other embodiments, implementations, systems, or modules where feasible, even if not specifically shown or described therein. For example, if an element is described in detail with reference to one embodiment and not with reference to a second embodiment, that element may still be required to be included in the second embodiment. Therefore, to avoid unnecessary repetition in the following description, one or more elements shown and described in association with one embodiment, implementation, or application may be incorporated into other embodiments, implementations, or aspects unless specifically described otherwise, unless one or more elements would render the embodiment or implementation inoperable, or unless two or more of the elements provide conflicting functionality.

[0022] In some cases, well-known methods, processes, components, and circuits are not described in detail to avoid unnecessarily obscuring aspects of the implementation.

[0023] This disclosure describes various devices, components, and parts of a computer-aided system, and elements, based on the state of the computer-aided system and its components in three-dimensional space. As used herein, the term "position" refers to the orientation of an element or part of an element (e.g., three translational degrees of freedom in three-dimensional space, such as along Cartesian x, y, and z coordinates). As used herein, the term "orientation" refers to the rotational placement of an element or part of an element (e.g., three rotational degrees of freedom in three-dimensional space, such as around roll, pitch, and yaw axes, expressed in terms of angular axes, rotation matrices, quaternions, etc.). As used herein, and for devices having a kinematic series, such as a device having a repositionable structure with multiple links coupled by one or more joints, the term "proximal" refers to a direction toward the base of the kinematic series, and "distal" refers to a direction away from the base along the kinematic series.

[0024] As used herein, the term “pose” refers to the position and orientation in a multi-degree-of-freedom (DOF) space attached to a coordinate system of interest of a rigid body. Typically, a pose includes pose variables for each of the DOFs in the pose. For example, a complete 6-DOF pose of a rigid body in 3D space would include six pose variables corresponding to three position DOFs (e.g., x, y, and z) and three orientation DOFs (e.g., roll, pitch, and yaw). A 3-DOF position-only pose would include only pose variables for the three position DOFs. Similarly, a 3-DOF orientation-only pose would include only pose variables for the three rotation DOFs. Furthermore, the velocity of the pose captures the change of the pose over time (e.g., the first derivative of the pose). For a complete 6-DOF pose of a rigid body in 3D space, the velocity would include three translational velocities and three rotational velocities. Pose with other numbers of DOFs would have corresponding numbers of translational and / or rotational velocities.

[0025] The aspects of this disclosure are described with reference to computer-aided systems, which may include remotely operated, externally manipulated, autonomous, semi-autonomous, and other devices. Furthermore, aspects of this disclosure are described according to implementations using remotely operated surgical systems, such as the da Vinci® Surgical System commercially available from Intuitive Surgical, Sunnyvale, California. However, those skilled in the art will understand that the inventive aspects disclosed herein can be embodied and implemented in various ways, including remotely operated and non-remotely operated, as well as medical and non-medical implementations. The implementations of the da Vinci® Surgical System are merely exemplary and should not be considered as limiting the scope of the inventive aspects disclosed herein. For example, the techniques described with reference to surgical instruments and methods can be used in other settings. Therefore, the instruments, systems, and methods described herein can be used with humans, animals, parts of human or animal anatomy, industrial systems, general robots, or remote operating systems. As further examples, the instruments, systems, and methods described herein can be used for non-medical purposes, including industrial applications, general robotic applications, sensing or manipulating non-tissue artifacts, cosmetic enhancements, imaging of human or animal anatomy, collecting data from human or animal anatomy, setting up or disassembling systems, training medical or non-medical personnel, etc. Additional example applications include procedures concerning tissue removed from human or animal anatomy (with or without return to the anatomy) and procedures concerning human or animal cadavers. Furthermore, these techniques can also be used in medical treatments or diagnostic procedures, with or without surgical aspects.

[0026] Figure 1 This is a simplified diagram of an example computer-aided system 100 according to various embodiments. In some examples, the computer-aided system 100 is a remote operating system. In medical examples, the computer-aided system 100 may be a remotely operated medical system, such as a surgical system. As shown, the computer-aided system 100 includes a follower device 104 that can be remotely operated by being controlled by one or more guide devices (also referred to as "guide input devices" when designed to accept external input), which are described in more detail below. A system that includes both guide devices and follower devices is called a guide-follower system and is sometimes also referred to as a master-slave system. Figure 1 The document also illustrates an input system that includes a workstation 102 (e.g., a console), and in various embodiments, the input system may be of any suitable form and may or may not include a workstation 102.

[0027] exist Figure 1In one example, workstation 102 includes one or more guide input devices 106 designed to be accessed and manipulated by operator 108. For example, workstation 102 may include one or more guide input devices 106 for use by operator 108's hand, head, or other body parts. In this example, the guide input devices 106 are supported by workstation 102 and may be mechanically grounded. In some embodiments, an ergonomic support 110 (e.g., a forearm support) may be provided on which operator 108 rests his or her forearm. In some examples, operator 108 may command follower device 104 using guide input devices 106 to perform tasks at a workplace within the workspace near follower device 104 during a procedure. In a medical example, the workplace may be a surgical workplace associated with a patient.

[0028] Workstation 102 also includes a display device 112. Display device 112 can be configured to display images for operator 108 to view. Display device 112 can move at various degrees of freedom (DOF) to adapt to the viewing position of operator 108 and / or provide control functionality. In embodiments where display device 112 provides control functionality, guide input device 106 may include display device 112. In examples of computer-aided system 100, the displayed images may depict a workplace where operator 108 is performing various tasks by manipulating guide input device 106 and / or display device 112. In some examples, the images displayed by display device 112 may be received by workstation 102 from one or more imaging devices arranged in the workplace. In other examples, the images displayed by display device 112 may be generated by display device 112 (or by different connected devices or systems), for example, for tools, virtual representations of the workplace, or for user interface components.

[0029] When using workstation 102, operator 108 can sit in a chair or other support in front of workstation 102, positioning his or her eyes in front of display device 112, manipulating guide input device 106, and resting his or her forearms on ergonomic support 110 as needed. In some embodiments, operator 108 can stand at workstation or adopt other postures, and display device 112 and guide input device 106 can be adjusted in position (height, depth, etc.) to accommodate operator 108's posture.

[0030] In some embodiments, one or more guide input devices 106 may be ungrounded (an ungrounded guide input device is not kinematically grounded, such as a guide input device 106 held by the operator 108's hand without additional physical support). Such ungrounded guide input devices 106 can be used in conjunction with a display device 112. In some embodiments, the display device 112 is positioned near the workplace so that the operator 108 can manually operate instruments in the workplace, such as medical instruments in a medical example, while viewing images displayed on the display device 112.

[0031] As shown, the computer-aided system 100 also includes a follower device 104 that can be commanded by workstation 102. In medical examples, the follower device 104 may be located near an operating table (e.g., a table, bed, or other support) on which a patient can be positioned. In some medical examples, the workspace is set on or in an operating table, such as on or in a patient, a simulated patient or model, a training dummy, etc. (not shown). As shown, the follower device 104 may include a plurality of repositionable structures 120 (sometimes referred to as “manipulator arms” in robotic embodiments). In some embodiments, the repositionable structures 120 may include joints that can be individually actuated as part of a kinematic series and a plurality of links as rigid members. Additionally, each of the repositionable structures 120 is configured to be coupled to instrument 122. Although Figure 1 A follower device 104 with four repositionable structures 120 is shown, but in other embodiments, the follower device 104 may include one, two, three, four, five, six, or additional or fewer repositionable structures 120.

[0032] The device 122 may include, for example, a working portion 126 and one or more structures for supporting and / or driving the working portion 126. Example working portions 126 include end effectors that physically contact or manipulate material, energy application elements that apply electrical, RF, ultrasonic, or other types of energy, sensors (e.g., temperature sensors, imaging devices, etc.) that detect characteristics of the workspace environment, etc. In various embodiments, examples of the device 122 include, but are not limited to, sealing devices, cutting devices, sealing and cutting devices, energy devices for applying energy, clamping devices (e.g., jigs, jaws), suture devices, imaging devices (e.g., imaging devices using optical, RF, or ultrasonic imaging modalities), sensing devices, flushing devices, suction devices, etc. Additionally, the device 122 may include a drive mechanism 128 that may be coupled to a drive assembly 130 of a corresponding repositionable structure 120. The drive assembly 130 may include a driver and / or other mechanisms controllable from workstation 102 that transmit force to the drive mechanism 128 to engage or otherwise actuate the device 122.

[0033] As shown, each device 122 can be mounted to a portion of the corresponding repositionable structure 120. Figure 1 In this diagram, the drive assembly 130 is physically coupled to the transmission mechanism 128. The distal portion of each repositionable structure 120 also includes a sleeve mount 124 to which a sleeve (not shown) is mounted. When the sleeve is mounted to the sleeve mount 124, the shaft of the instrument 122 passes through the sleeve and enters the workspace.

[0034] In various embodiments, one or more of the working portions 126 of the device 122 may include imaging means for capturing images. The imaging means may include any sensing technology capable of acquiring images. Example imaging devices include optical endoscopes, hyperspectral imaging devices, ultrasonic sensors, etc. Imaging devices may include single-field-of-view imagers, stereo imagers, etc. Radio frequency domain-based imaging means can capture images in any spectrum including visible light, infrared light, ultraviolet light, etc. Imaging means may include an illumination source to illuminate the area being imaged. In embodiments where one or more of the working portions 126 of the device 122 include imaging means, the device 122 may be configured to capture an image of a portion of the workspace for display via the display device 112.

[0035] In some embodiments, the repositionable structure 120 and / or instrument 122 can be controlled to move the working portion 126 in response to manipulation of the guide input device 106 by the operator 108. Thus, the repositionable structure 120 and / or instrument 122 can be said to "follow" the guide input device 106 via remote operation. This enables the operator 108 to perform tasks in the workplace using the repositionable structure 120 and / or instrument 122. For a surgical example, the operator 108 may guide the repositionable structure 120 of the follower device 104 to move the working portion 126 as part of a surgical procedure performed in an internal surgical setting accessed via one or more micro-incisions or natural orifices.

[0036] In some embodiments, the repositionable structure 120a of the computer-aided system 100 may be configured to support a working portion 126a, which includes an imaging device (also referred to herein as "imaging device 126a"). For convenience, the instrument 122 including the imaging device is also referred to herein as "imaging instrument". The control system 140 may be configured to command the repositionable structure 120a and / or the imaging instrument 122 including the imaging device 126a to automatically position and / or orient ("set pose") the field of view (FOV) of the imaging device 126a to provide images of the workspace and / or other instruments 122.

[0037] In the illustrated embodiment, the control system 140 is communicatively coupled to the workstation 102. In other embodiments, the control system 140 may be configured as a component of the workstation 102 and / or the follower device 104. During remote operation, when the operator 108 moves the guide input device 106, the control system 140 is configured to detect spatial and / or orientation movement data generated by one or more sensors of the guide input device 106 and provided to the control system 140. The control system 140 may interpret the spatial and / or orientation information to determine control signals and / or provide control signals to the follower device 104 to control the movement of the repositionable structure 120, the instrument 122, and / or the working part 126. In one embodiment, the control system 140 supports one or more wired communication protocols (e.g., Ethernet, USB, etc.) and / or one or more wireless communication protocols (e.g., Bluetooth, IrDA, HomeRF, IEEE 1102.11, DECT, wireless telemetry, etc.) for communication between the control system 140 and the workstation 104.

[0038] In some implementations, the control system 140 may be implemented at one or more computing systems. For example, one or more computing systems may be used to control the follower device 104. As another example, one or more computing systems may be used to control the movement of components of workstation 102, such as display device 112.

[0039] As shown, the control system 140 includes a processor system 150 and a memory 160. The memory 160 may store the control module 170. The processor system 150 may include one or more processors with different processing architectures for processing instructions. For example, the one or more processors may be one or more cores or microcores of a multi-core processor, a central processing unit (CPU), a microprocessor, a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a graphics processing unit (GPU), a tensor processing unit (TPU), etc.

[0040] In some embodiments, the processor system 150 includes a circuitry for supporting one or more communication interfaces (e.g., Bluetooth, infrared, network, optical, etc.). Additionally, the communication interface of the control system 140 may include an integrated circuit for connecting the control system 140 to a network (not shown) (e.g., a local area network (LAN), a wide area network (WAN) such as the Internet, a mobile network, or any other type of network) and / or another device such as workstation 102 and / or follower device 104.

[0041] Additionally, memory 160 may include non-persistent storage devices (e.g., volatile memory, such as random access memory (RAM), cache memory), persistent storage devices (e.g., hard disks, optical drives such as optical disc (CD) drives or digital versatile disc (DVD) drives, flash memory, floppy disks, flexible disks, magnetic tape, any other magnetic media, any other optical media, programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), FLASH-EPROM, and / or any other memory chip or cartridge). Non-persistent and persistent storage devices are examples of non-transitory tangible machine-readable media that can store executable code, which, when run by one or more processors (e.g., processor system 150), can cause one or more processors to perform one or more of the techniques and / or methods disclosed herein.

[0042] Additionally, the control system 140 may also include one or more input devices (e.g., a touchscreen, keyboard, mouse, microphone, touchpad, electronic pen, or any other type of input device) and / or output devices (e.g., a display device, speaker, external storage device, printer, or any other output device). In some embodiments, the control system 140 may be implemented on a specific node of a distributed computing system (e.g., a cloud computing system). As another example, different functions associated with the control system 140 may be implemented on different nodes of the distributed computing system. Furthermore, one or more components of the aforementioned control system 140 may be located in a remote location and connected to other components via a network.

[0043] In an endoscopic surgery example, an imaging instrument including imaging device 126a can be inserted into the patient before other instruments 122 (including a second instrument 122b having a second working portion 126b). Instrument 122b can include any suitable working portion 126b, and may even include a second imaging device. Thus, imaging device 126a can be manipulated for positioning to identify a target for defining an insertion axis (also referred to herein as the "target axis") along which other instruments 122 will move to reach the work site. Based on this position, control system 140 can automatically command the corresponding repositionable structure 120 to align the axis supporting the working portion 126 with its corresponding insertion axis.

[0044] In some examples, this technology enables the computer-assisted system 100 to automatically align the instrument 122 supported by the repositionable structure 120 before the procedure is performed, so as to insert its corresponding working portion 126 into the work area. By automatically aligning the instrument 122, the disclosed technology reduces the likelihood of instruments (or the repositionable structure supporting the instrument) colliding with each other and / or with avoidance areas (e.g., the subject's anatomy in a medical example). Thus, when the operator 108 is ready to perform the procedure, the operator 108 can simply command the follower device 104 to insert the instrument 122 along the insertion axis, or another operator (e.g., an operator who can physically contact the instrument 122 or the follower device 104) can simply push the instrument 122 along the insertion axis. Systems that facilitate such technology can make process setup easier for human operators, reduce workflow interruptions, reduce the time spent on instrument alignment, and improve the overall efficiency of the medical procedure. Such technology can also improve the efficiency of operating computer-assisted systems or instruments, simplify user control of computer-assisted systems or instruments, and improve the accuracy of instrument alignment. Furthermore, although a surgical example is shown, the disclosed technology provides an improvement to the computer-aided system 100 in the non-surgical aspect of the procedure and can be used to improve computer-aided systems applied in non-medical settings.

[0045] As an example workflow, the control system 140 may receive an indication that a second instrument 122b, mounted to the second repositionable structure 120b, will be inserted into the workspace. In a medical example, the second instrument 122b may be inserted via an access site, such as through an incision or natural orifice in the patient's body. Accessories may be inserted into the access site and provide an opening for instrument insertion. For example, the instrument may be inserted via an access site through a port or cannula already inserted into the access site.

[0046] In response to receiving an instruction, the control system 140 can operate in an alignment mode, in which the control system 140 determines a target (e.g., a position and / or line identified by the operator and / or the system) for determining a target axis, and the second instrument 122b will be inserted into the workspace along the target axis. Aligning the second instrument 122b to the target axis may include aligning the insertion axis of the second instrument 122b with the target axis.

[0047] In some embodiments where a target is derived from operator identification (e.g., operator-identified location), workstation 102 and / or display device 112 may include a user interface that enables operator 108 to indicate a location associated with imaging device 122a, including imaging device 126a. For example, workstation 102 and / or input device 106 may include buttons, switches, toggle keys, or other user-interactive elements that operator 108 can use to indicate that the operator-identified location will be determined. In response to detected input, control system 140 may analyze kinematic data associated with imaging device 122a and / or image data generated by its imaging device 126a to determine the operator-identified location. In some embodiments, control system 140 utilizes the kinematic information associated with imaging device 122a to position the operator-identified location at the tip of imaging device 122a or at the origin of the field of view (FOV) of imaging device 126a included in imaging device 122a. In other embodiments, the control system 140 sets the operator-identified position to be offset relative to the imaging instrument 122a along a defined direction. In some examples, the defined direction is along the longitudinal axis of the imaging instrument 122a in a direction toward the workplace, or along the observation direction of the imaging instrument 122a. In some examples, the offset is a predetermined offset. In some other examples, the offset is a dynamically determined offset such that the operator-identified position coincides with a physical structure (e.g., an object such as a portion of a patient's anatomy) along the defined direction. Some examples of predetermined offsets include the focal length of the imaging device 126a, a typical offset distance for the current procedure, etc.

[0048] The observation direction of an imaging device is the direction around which the field of view (FOV) is centered. For example, the observation direction of an endoscope can be described as a central axis extending from the origin of the endoscope's FOV toward the center of the FOV. For a single-view endoscope with an FOV having a truncated cone or frustum shape, the observation direction extends from the origin of the FOV toward the center of the base of the cone or frustum. For a stereo endoscope, the observation direction can be defined for each of the two imaging devices of the endoscope, or for a combination or average of the observation directions or FOVs of the two imaging devices (e.g., the average of the two observation directions of the two imaging devices; the central axis of the union of the FOVs of the two imaging devices; the intersection of the FOVs (if the FOVs intersect) etc.). As another example, the observation direction of mode A ultrasound is the direction outward from the ultrasound sensor along a single dimension of the mode A ultrasound image, and the observation direction of mode B ultrasound is a central axis extending from the base of the ultrasound image toward the center of the range of the ultrasound image in two-dimensional space.

[0049] In other embodiments, the operator can more precisely indicate the operator-identified location. As another example, the display device 112 may include a graphical user interface (GUI) depicting image data generated by an imaging instrument including the imaging device 126a. In this example, the operator 108 can indicate a specific location by interacting with the GUI (e.g., by moving a cursor object to the target's location, by touching the display device 112, and / or other known means of interacting with the GUI). As another example, the GUI may enable the operator 108 to input the location, size, and / or shape of the target's volume into one or more parameter input fields. It should be understood that although the term "operator-identified location" is used herein, in some embodiments, the operator may alternatively identify the line via the GUI (e.g., by identifying multiple locations of the line defined by the control system 140, by manipulating a virtual line indicator, etc.). Therefore, the term "operator-identified location" also refers to embodiments in which the operator identifies the line.

[0050] In embodiments where the control system 140 defines the location of a target, the control system 140 may use pre-configured data indicating the workplace. For a medical example, a user may mark preoperative image data of the workplace with an indication of the desired location of the instrument 122 to be positioned. In this example, the user may mark a target lesion (or other anatomical feature of interest) in the preoperative image data. Therefore, in this example, the control system 140 may be configured to process image data generated by an imaging instrument 122a, including an imaging device 126a, to detect when an anatomical feature is included in the field of view (FOV). Upon detection of the presence of an anatomical feature, the control system 140 may automatically define the location associated with the anatomical feature, from which the target is derived.

[0051] Regardless of whether the location is operator-identified and / or system-defined, the control system 140 can then define a target axis for aligning the instrument 122b. More specifically, the control system 140 can define a target axis that coincides with a straight line defined by the location identified by the operator and / or system-defined and the entry point associated with the instrument 122b.

[0052] Additionally, in some embodiments, the control system 140 may define a target position along a target axis, at which the instrument 122b will be inserted to perform the process. For example, the control system 140 may define a volume centered on a position identified by the operator and / or defined by the system. In some embodiments, the volume has a spherical and / or ellipsoidal shape. The control system 140 may then define the point on the surface of the target volume closest to the entry point positioning as the target position. The control system 140 may then define the target axis to coincide with a line intersecting the target position and the entry point positioning.

[0053] In various embodiments, a point on the surface of the target volume is the point on the surface closest to the entry point of the instrument 122b via which it is inserted. Therefore, in most cases, the target location problem is constrained to have only one solution. However, in some scenarios, this solution results in the instrument 122b passing through a restricted area (e.g., in a medical setting, a portion of a patient's anatomy and / or its vicinity) of the target insertion axis upon insertion; results in a collision between the repositionable structure 120 and / or the instrument 122; or the solution simply lies outside the range of motion of the instrument 122 when supported by the repositionable structure 120. Therefore, in these scenarios, the target location problem can be redefined to allow a solution at any point on the surface of the target volume. Thus, the control system 140 can introduce additional constraints such that the set of new potential solutions can be reduced to a single solution. For example, additional constraints may include avoidance of restricted areas and / or collisions. In some embodiments, additional sensor data (e.g., line image recognition from the imaging instrument 122a) is used to determine the constraints. In these embodiments, the control system 140 can redefine the target axis of the instrument 122 based on the solution to the determined alternative target location problem.

[0054] While the foregoing describes how control system 140 determines the target axis and intended position of instrument 122b, control system 140 can perform similar techniques to determine the target axis and intended position of any instrument 122 supported by a corresponding repositionable structure 120 that is ready for aiming (e.g., ready to receive a target axis for alignment). To determine that an instrument 122 supported by repositionable structure 120 is ready for aiming, control system 140 may verify one or more of the following: (i) repositionable structure 120 has an instrument 122 mounted thereto; (ii) repositionable structure 120 is physically coupled to a sleeve supported by sleeve mount 124; (iii) instrument 122 is not inserted beyond a threshold distance from the uninserted position or remote center; (iv) the distal portion or working portion of instrument 122 is not inserted beyond a threshold distance from the uninserted position, remote center, entry point positioning, or a characteristic of the sleeve; and / or (v) the distal portion or working portion of instrument 122 is not inserted beyond the tip of the sleeve corresponding to the entry point positioning. The location of the distal portion or working portion or some other portion of the device 122 can be determined by any suitable technique, such as by identifying and locating known features of the device 122 through image analysis to position and orient the device, and / or by detecting kinematic information about the repositionable structure 120 and the device 122 with sensors and applying positive kinematics, etc.

[0055] The control system 140 can store data associated with the target axis and / or the intended position in memory. For example, the data may include position data and / or orientation data defined relative to a coordinate system associated with the kinematic model of the follower device 104. Therefore, the control system can then access the stored data, for example, when determining the configuration of the joints of the repositionable structure 120 and / or when presenting a GUI.

[0056] In this example, the control system 140 can determine the configuration of the joints of the repositionable structure 120 supporting the instrument 122 and / or the instrument 122, such that the respective instrument 122 is aligned for insertion along its respective target axis. To determine this configuration, the control system 140 can analyze one or more kinematic models of the instrument 122 and / or the repositionable structure 120 supporting the instrument 122. For example, the control system 140 can analyze the kinematic models to determine the configuration of the joints of the instrument 122 and / or the repositionable structure 120 supporting the instrument 122, such that the instrument 122 is aligned with the respective target axis. In such a pose, the instrument 122 can be positioned and inserted through its respective entry point to reach its respective target position.

[0057] The control system 140 can then command the apparatus 122 and / or the repositionable structure 120 based on the determined configuration. Techniques for commanding the apparatus 122 and / or the repositionable structure 120 to pivot relative to the entry site (sometimes referred to as "software centering") are described in U.S. Patent No. 8,823,308, the entire disclosure of which is incorporated herein by reference. In some embodiments, the control system 140 commands the apparatus 122 and / or the repositionable structure 120 toward the determined configuration in parallel. In other embodiments, the control system 140 commands the apparatus 122 and / or the repositionable structure 120 toward the determined configuration sequentially. For example, if the configuration requires the control system 140 to rotate multiple apparatuses 122 and / or the repositionable structure 120 in the same direction, the current pose of the first repositionable structure 120 (typically the outermost repositionable structure along the axis of rotation) may inhibit the ability of the second apparatus 122 and / or the repositionable structure 120 to reach the determined configuration without collision. Therefore, in this example, the control system 140 can command the first instrument 122 and / or the repositionable structure 120 to face the configuration, and after the first instrument 122 and / or the repositionable structure 120 reaches the pose associated with the configuration, command the second instrument 122 and / or the repositionable structure 120 to face the configuration.

[0058] In embodiments where the imaging device 122a is used to indicate a position identified by the operator for aiming, the field of view (FOV) of the imaging device 126a is not typically oriented in a manner optimal for the execution of the process. For example, the operator 108 may adjust the digital zoom level and / or pose of the imaging device 122a in a specific manner to indicate a position identified by the operator. Therefore, the control system 140 may also command the imaging device 122a and / or the repositionable structure 120a to adjust the pose (e.g., by retracting the imaging device 122a along the insertion axis) and / or the zoom level (e.g., by digital zoom-out) of the imaging device 126a so that the FOV of the imaging device 126a provides the operator with a better workspace view. In various embodiments, the control system 140 may command the repositionable structure 120a to adjust the imaging device 122a before aligning the repositionable structure 120 with the target axis, simultaneously with commanding other repositionable structures 120, or after other repositionable structures 120 have reached a determined configuration.

[0059] In some embodiments, the control system 140 may be configured to advance the instrument 122 along its respective target axis to its respective target position, such that the working part 126 is located near the workplace. In other embodiments, the operator 108 interfaces with the workstation 102 and / or the input device 106 to advance the instrument 122 along its respective target axis to its respective target position. In operator-advanced scenarios, the control system 140 may provide feedback to the operator 108 when the instrument 122 reaches the target position. For example, the control system 140 may provide tactile feedback by changing the resistance on the instrument 122 as it approaches the target position, and / or by locking the pose of the instrument 122 and / or the repositionable structure 120 supporting the instrument 122 when it reaches the target position. It should be understood that improved collision avoidance and a more efficient aiming and alignment process do not require the insertion of the instrument 122 to achieve the described benefits.

[0060] Figure 2 The image depicts the definition of the alignment device 122 (e.g., Figure 1 Example Figure 200 shows an instrument 122 (for entering a target axis 236 of a workspace 242). In some embodiments, the technique used to define the target axis 236 is... Figure 1 The control system 140 is executed.

[0061] Operator (e.g., Figure 1 The operator 108) can control the supporting imaging device 222a (e.g., Figure 1 The repositionable structure of the imaging device 122a (e.g., Figure 1 The pose of the repositionable structure 120a) allows the imaging device 226a (e.g., Figure 1 The field of view (FOV) of the imaging device 126a includes a position 232 for aiming at other instruments 222. As described above, in some embodiments, the control system can analyze image data generated by the imaging device 226a to detect the presence of anatomical features 244, thereby automatically defining the position 232. In other embodiments, the operator can interact with an operator interface (e.g., physical buttons or GUI elements) to identify the position 232 to the control system.

[0062] After the control system automatically defines position 232 and / or the operator identifies position 232, the control system can define a target axis 236 for the repositionable structure ready for aiming. In the depicted scenario, there are two repositionable structures ready for aiming. Each of the repositionable structures can be associated with a corresponding entry point 230. That is, an imaging device 222a can be inserted through entry point 230a, a second device 222b can be inserted through entry point 230b, and a third device 222c can be inserted through entry point 230c. To define the target axis 236b for the second device 222b, the control system can define the axis connecting position 232 to entry point 230b. Similarly, to define the target axis 236c for the third device 222c, the control system can define the axis connecting position 232 to entry point 230c. The control system can then determine the configuration of the repositionable structures such that the axis of device 222b is aligned with the target axis 236b, and the axis of device 222c is aligned with the target axis 236c.

[0063] Additionally, in some embodiments, the control system may define a target position 238 along the target axis 236, at which the instrument 222 will be positioned prior to the process. Therefore, the control system may define a target volume 234 centered at the target position 232. It should be understood that, although Figure 2 A spherical target volume 234 is depicted, but in other embodiments, the target volume 234 may have an ellipsoidal shape, a polygonal shape, a shape with a convex or concave surface, a shape with a convex surface facing the access area, or any other suitable shape. For example, in an alternative scenario where the target volume 234 would include portions of the patient's anatomy within its volume if a spherical and / or ellipsoidal target volume is defined, the control system may alternatively truncate the target volume to ensure that the target volume does not include the patient's anatomy (or, in some embodiments, a threshold distance close to the patient's anatomy).

[0064] In any case, the control system can define the target position 238 as the intersection between the target axis 236 and the surface of the target volume 234 (i.e., the point on the target volume 234 closest to the corresponding entry point 230). Therefore, the control system can define the target position 238b of the instrument 226b as the intersection of the target axis 236b and the surface of the target volume 234, and define the target position 238c of the instrument 226c as the intersection of the target axis 236c and the surface of the target volume 234.

[0065] It should be understood that in the depicted scenario, the control system uses the same target volume 234 for both instruments 222b and 222c. However, in other embodiments, the target volume 234 may be different for instruments 222b and 222c. For example, the operator may preferably position different types of instruments at different distances from position 232. As an example, an instrument with a larger working portion may be positioned further away from position 232 to provide additional space for other instruments to operate within workspace 242. As another example, an instrument used later in the process may be positioned further away from position 232 to maximize the space within workspace 242 for instruments used earlier in the process. Thus, in these embodiments, the control system may generate multiple target volumes 234 with different radii and / or shapes based on the specific instrument associated with the defined target position 238. Therefore, the control system is able to maximize the workspace associated with workspace 242, thereby providing improved working conditions for the execution process.

[0066] Additionally, in some embodiments, the control system can be configured to support a guided tool change mode in which the control system facilitates the “replacement” of an instrument previously mounted to the repositionable structure. This guided tool “replacement” mode does not require actual instrument replacement. For example, the instrument can be removed for cleaning and reinstalled; the instrument can be removed to be reloaded with staples, clips, sutures, or other materials and then reinstalled; and so on. Techniques related to guided tool changes performed by a computer-aided system are described in U.S. Patent No. 6,645,196 B1, filed June 16, 2000, entitled “GUIDED TOOL CHANGE,” which is incorporated herein by reference. Therefore, in embodiments implementing a guided tool change mode, when the control system detects that an instrument (new or previous) has been mounted to the repositionable structure, the control system can analyze the indications of the mounted instrument to determine whether target position 238 should be updated. If the indication indicates that a different type of instrument is being installed into the repositionable structure, the control system can use the target volume 234 of the new type of instrument to redefine the target position 238 to ensure that the new tool is reinserted to the appropriate depth.

[0067] Figure 3 Example GUI 300 is shown being provided to the operator (e.g., Figure 1 Operator 108) to facilitate the use of computer-aided systems (e.g., Figure 1 The process of a computer-aided system 100. In some embodiments, the GUI 300 is transmitted via a display device associated with the workstation (e.g., a computer-aided system 100). Figure 1The workstation 102's display device 112) displays the information. It should be understood that the control system (e.g., Figure 1 The control system 140), workstations, and / or combinations thereof can generate the GUI 300 and / or the data displayed therefrom.

[0068] As shown, GUI 300 enables a user to visualize image data 325 generated by imaging instruments (e.g., imaging instruments 122a, 222a) and / or other imaging devices. In some scenarios, the follower device supporting the imaging instruments may include multiple imaging instruments mounted to a repositionable structure. Therefore, GUI 300 may include indications 320 identifying the repositionable structure and / or imaging instruments associated with the displayed image data 325.

[0069] In some implementations, the GUI includes a visual indicator 336 representing the target axis of the instrument. For example, the control system can access kinematic data to determine the pose of the imaging instrument, thereby identifying a portion of the coordinate space within the FOV of the imaging instrument. If the stored data associated with the target axis is included within the aforementioned portion of the coordinate space, the control system can generate a visual indicator 336b representing the target axis in the image data 325. Thus, the operator can visually see the insertion path of the instrument.

[0070] In some implementations, this allows the operator to confirm that the automatically defined target axis is appropriate (e.g., does not collide with patient anatomy, aligns with operator preferences, etc.) before the control system commands the follower device to orient itself towards a configuration that aligns the instrument axis with the target axis. If the operator is satisfied with the target axis, they can interface with GUI element 346 to accept and confirm the target axis and command the control system to orient the follower device towards the configuration. Although Figure 3 GUI element 346 is depicted as a button, but in other implementations any suitable GUI element may be used. In some implementations, one or more hardware user interface elements (e.g., physical buttons, dials, foot pedals, etc.) may be used instead of, in addition to, or concurrent with GUI element 346 to command the follower device toward the configuration.

[0071] Figures 4A to 4D A computer-aided system 400 is shown (e.g., Figure 1 The computer-aided system 100) will follow the device 404 (e.g., Figure 1 An example process for aligning the follower device 104 with a target axis defined according to the technique described elsewhere herein. It should be understood that... Figures 4A to 4D The examples are not restrictive, and Figures 4A to 4DOther repositionable structures, instruments, behaviors, etc., described in the text may differ from other computer-aided systems, different repositionable structures, different imaging devices, different instruments, different DOFs, different processes, etc.

[0072] Figures 4A to 4D Four repositionable structures 420 are depicted (e.g., Figure 1 The repositionable structures 120 are all mechanically grounded to a common kinematic base. In some other examples, the repositionable structures 420 do not share such a common kinematic base and are mounted to two or more separate trolleys, which are configured to be mounted independently to a wall, table, ceiling, floor, etc. Where appropriate, a control system (e.g., Figure 1 The control system 140 may employ registration techniques (which are applied in conjunction with appropriate geometric model and reference frame transformations) to position the repositionable structure 420 and, for example, the apparatus 422 supported by the repositionable structure 420 relative to each other. Figure 1 The components of the instrument 122). Techniques related to registration and reference frame transformation include those described in U.S. Patent 9,259,289, filed January 27, 2012, entitled "Estimation of a position and orientation of a frame used in controlling movement of a tool"; U.S. Patent 11,534,252, filed November 13, 2018, entitled "Master / slave registration and control for teleoperation"; and U.S. Patent Publication 2023 / 0028689, filed January 1, 2021, entitled "System and method for inter-arm registration".

[0073] from Figure 4A Initially, the repositionable structure 420 exhibits an initial and / or default pose. Therefore, the instrument axis of instrument 422 is not aligned with its corresponding target axis. In the illustrated example, instrument 422b of the repositionable structure 420b is equipped with imaging instruments (e.g., imaging instruments 122a, 222a), and instruments 422a, 422c, 422d of the repositionable structures 420a, 420c, 420d are equipped with corresponding instruments and have been verified in some embodiments to be ready for aiming.

[0074] Therefore, the control system may have previously determined the configuration for aligning the instrument axis of the instrument 422 with its corresponding target axis. For example, the control system can determine the configuration using the inverse kinematics of the repositionable structure 420 and / or the instrument 422. More specifically, inverse kinematics can be applied to determine joint commands for the repositionable structure 420 and / or the instrument 422, for example, to align the instrument axis with its corresponding target axis. In addition to using inverse kinematics, the control system can also apply image processing, geometric modeling, pre-stored parameters, real-time data about the environment or system, etc., to determine appropriate joint commands.

[0075] As described herein, in some embodiments, the control system may sequentially command the instrument 422 and / or the repositionable structure 420 toward a determined configuration (e.g., by issuing joint commands associated with the respective repositionable structure 420). To reduce the likelihood of collisions, the control system may command instruments and / or repositionable structures (e.g., repositionable structures 420a, 420d) further away from the center of the cluster of repositionable structures 420 before commanding instruments and / or repositionable structures (e.g., repositionable structure 420c) closer to the center of the cluster.

[0076] Go to Figure 4B The control system has issued joint commands to move instrument 422a and repositionable structure 420a toward the determined configuration. Therefore, the instrument axis of instrument 422a is now aligned with the target axis of instrument 422a. The other three repositionable structures 420b, 420c, and 420d remain in position. Figure 4A In the configuration shown.

[0077] Go to Figure 4C The control system has issued joint commands to move instrument 422d and repositionable structure 420d toward the determined configuration. Therefore, the instrument axis of instrument 422d is now also aligned with the target axis of instrument 422a. The other three repositionable structures 420a, 420b, and 420c remain in position. Figure 4B In the configuration shown.

[0078] Go to Figure 4D The control system has issued joint commands to move the instrument 422c and the repositionable structure 420c toward the determined configuration. Therefore, the instrument axis of the instrument 422c is now aligned with the target axis of the instrument 422c. Consequently, each of the instruments 422a, 422c, 422d and the repositionable structures 420a, 420c, 420d now exhibits a pose consistent with the determined configuration.

[0079] It should be understood that, in the illustrated scenario, the imaging device of instrument 422b is used to indicate the location for operator identification and / or automatic definition of the target axis (e.g., Figure 2 (Position 232). Therefore, the control system also issues commands to change the pose of instrument 422b and / or repositionable structure 420b, so that the FOV of the imaging instrument mounted thereon provides a wider workspace field of view, allowing the operator to see the end of the sleeve positioned near the workspace. More specifically, in the illustrated scenario, the control system issues commands to retract the instrument axis along the insertion axis of instrument 422b. In other embodiments, the control system may alternatively change the zoom level of the imaging instrument to achieve a wider workspace field of view than previously configured. The zoom mechanism may include an optical zoom mechanism, a digital zoom mechanism, physical movement of the imaging device, etc.

[0080] Figure 5 This is a flowchart of an example method 500 for performing instrument aiming in a computer-aided system 100 according to one or more embodiments. Method 500 may be performed by one or more processors (e.g., Figure 1 The processor system 150 of the control system 140 executes instructions stored in one or more computer-readable media (e.g., non-volatile memory).

[0081] As described herein, the control system may be communicatively coupled to a first repositionable structure (e.g., repositionable structures 120, 220, 420) configured to support a first instrument (e.g., instrument 122, 222). The first repositionable structure may include a first plurality of joints.

[0082] When the control system determines a target in the workspace (e.g., workspaces 242, 342) for positioning the first instrument supported by the first repositionable structure (e.g., ... Figure 2 When the position is 232), method 500 may begin at block 502. For example, in some embodiments, computer assistance may also include a second repositionable structure (e.g., repositionable structure 120a, 420b) configured to support a second instrument (e.g., instrument 122a, 222a), wherein the second instrument includes an imaging device. In these embodiments, the control system may use kinematic information associated with the imaging device to determine the target.

[0083] As an example, a control system can determine a target by detecting operator interaction with an input device. The input device can be a physical input device (e.g., with...). Figure 1 The input device 106 is associated with buttons or toggle keys) or a virtual input device (e.g., on a display device). Figure 1(Graphical user interface elements presented on the display device 112). In this example, the control system can determine the position of the second instrument in response to detecting interaction with an operator on the input device (e.g., by detecting interaction with an image displayed on the display device). In some embodiments, the position is the position of a portion of the second instrument (e.g., the tip of the second instrument or the origin of the field of view (FOV) of the imaging device). In other embodiments, the system positions the target at a longitudinal distance along the viewing direction of the second instrument (e.g., "in front" of the second instrument, such as at...). Figure 2 (as depicted in the example). In yet other embodiments, the location may be based on anatomical features within the FOV of the imaging device (e.g., Figure 2 (244) Anatomical features. In any of these embodiments, the location of the target can be determined based on the position of the second instrument.

[0084] As another example, the control system can automatically define a target based on analysis of image data generated by the imaging device. For example, the control system can detect anatomical features within the field of view (FOV) of the imaging device and define the target as being located at a position associated with the anatomical feature (e.g., on the surface of the anatomical feature, at a specific sub-feature of the anatomical feature (e.g., a lesion), at an offset position relative to the anatomical feature, etc.). In some embodiments, the control system can automatically define multiple potential targets from which targets can be selected (e.g., by detecting operator interaction with an input or display device). Thus, according to this example, the target can be a location identified by the operator or automatically defined.

[0085] At box 504, the control system locates the target and the entry point based on the location (e.g., Figure 2 The entry point 230b) is used to define the first target axis (e.g., Figure 2 (Target axis 236b). For example, the control system can define the first target axis based on the line connecting the target to the entry site.

[0086] In some implementations, the computer-aided system includes a display device (e.g., Figure 1 The display device 112). In these embodiments, the display device can be configured to display an image based on image data generated by the imaging device (e.g., Figure 3 Image data 325). In these embodiments, the control system may render the image to include a representation of the first target axis within the image (e.g., Figure 3 (One of the visual indicators 336).

[0087] At box 506, the control system can determine the configuration of the first plurality of joints, which aligns the first instrument with the first target axis, enabling the first instrument to be advanced along the first target axis. As described herein, the control system can be configured to determine the configuration of the first plurality of joints using kinematic and / or geometric models of the plurality of joints, imaging data, pre-configured parameters, and / or sensed real-time data.

[0088] At box 508, the control system can then command the first plurality of joints based on the determined configuration. In some embodiments, the control system can implement collision avoidance techniques when commanding the first plurality of joints. For example, the control system can determine that the current position of the first repositionable structure prevents the second repositionable structure from aligning the second instrument supported by the second repositionable structure with the second target axis, and command the first plurality of joints toward the configuration before commanding the plurality of joints of the second repositionable structure toward the configuration. This example is illustrated by... Figures 4A to 4D The scenario depicted reflects that the control system commands multiple joints of the repositionable structure 420d before commanding multiple joints of the repositionable structure 420c.

[0089] As described herein, in embodiments of the imaging apparatus used for target identification, the field of view (FOV) may not provide a sufficiently wide workspace field of view. Therefore, in some embodiments, the control system may receive an instruction that a first plurality of joints have reached a configuration; and in response to receiving such an instruction, command to expand the field of view of the imaging apparatus by causing at least one action selected from the group consisting of: commanding a second plurality of joints to retract a second instrument; commanding the second plurality of joints to move the imaging apparatus further away from the workspace; reducing the magnification level of the imaging sensor of the imaging apparatus; and reducing the digital magnification of the image captured by the imaging apparatus.

[0090] In some implementations, the control system may perform one or more verifications before executing method 500. For example, the control system may perform a first verification to verify that the first instrument meets insertion conditions, which include at least one criterion selected from the group consisting of: the first instrument is not inserted beyond a threshold distance from an uninserted position, and the distal portion of the first instrument is not inserted beyond the tip of the cannula corresponding to the entry site positioning; perform a second verification to verify that the first repositionable structure is physically coupled to the cannula; and / or perform a third verification to verify that the first instrument is mounted to the first repositionable structure. In this example, the control system may perform instrument aiming only in response to the first instrument and / or the first repositionable structure meeting the verifications. Additionally, in some implementations where the operator controls the second instrument to determine a target, the control system may wait until it receives an indication that the positioning of the second instrument has been completed before performing instrument aiming.

[0091] In some embodiments, the control system may additionally determine a target position along a first target axis prior to the execution process, the target position indicating the insertion depth of the first instrument. Thus, in these embodiments, the control system may identify a position on a surface defined at least partially by an operator or automatically defined location. For example, the control system may define a spherical shape centered on a target and identify a position on the surface of the spherical shape based on entry site positioning. In some embodiments, the control system determines the size and / or shape of the surface based on the instrument type associated with the first instrument (e.g., the typical working distance of the first instrument during the current process).

[0092] In many scenarios, the control system defines the first candidate for a target location on a surface as the location on the surface closest to the entry point. However, in some scenarios, the control system may determine that setting the target location as the first candidate would result in the first instrument being commanded to cross a restricted area or move beyond its movement limits when inserted into the workspace. Therefore, in these scenarios, the control system may identify a second candidate for a location on the surface such that setting the target location as the second candidate would result in the first instrument not being commanded to cross a restricted area or move beyond its movement limits when inserted into the workspace. As another example, a computer-aided system may include a third repositionable structure configured to support a third instrument. Therefore, the control system may identify a second location on the surface based on detecting a potential collision between first target axes selected from at least one of the following parameters: (i) the physical configuration of the third repositionable structure; (ii) the insertion axis of the third instrument; or (iii) the target axis of the third instrument.

[0093] In some embodiments, the control system may also be configured to control the first repositionable structure to advance the first instrument toward the target along a first target axis. For example, the control system may be configured to advance the first instrument until it reaches the target position. In other embodiments, the control system may be configured to provide feedback to an operator who is manually and / or semi-automatically advancing the first instrument to indicate when the first instrument has reached the target position.

[0094] One or more components of the examples discussed in this disclosure (e.g., control system 140) can be implemented in software for execution on one or more processors of a computer system. The software may include code that, when executed by one or more processors, configures one or more processors to perform the various functions discussed herein. The code may be stored in a non-transitory computer-readable storage medium (e.g., memory, magnetic storage device, optical storage device, solid-state storage device, etc.). The computer-readable storage medium may be a computer-readable storage device, such as an electronic circuit, semiconductor device, semiconductor memory device, read-only memory (ROM), flash memory, erasable programmable read-only memory (EPROM), floppy disk, CD-ROM, optical disk, hard disk, or other storage device. The code may be downloaded via a computer network such as the Internet, intranet, etc., for storage on the computer-readable storage medium. The code can be executed by any of a variety of centralized or distributed data processing architectures. The programming instructions of the code may be implemented as multiple separate programs or subroutines, or they may be integrated into multiple other aspects of the system described herein. Components of the computing system discussed herein may be connected using wired and / or wireless connections. In some examples, wireless connectivity can use wireless communication protocols such as Bluetooth, Near Field Communication (NFC), Infrared Data Association (IrDA), HomeRF, IEEE 502.11, Digital Enhanced Cordless Telecommunications (DECT), and Wireless Medical Telemetry Service (WMTS).

[0095] Various general-purpose computer systems can be used to perform one or more of the processes, methods, or functions described herein. Alternatively or additionally, various special-purpose computer systems can be used to perform one or more of the processes, methods, or functions described herein. Furthermore, various programming languages ​​can be used to implement one or more of the processes, methods, or functions described herein.

[0096] While certain examples and instances have been described above and shown in the accompanying drawings, it should be understood that such examples and instances are merely illustrative and are not limited to the specific constructions and arrangements shown and described, as those skilled in the art will recognize various other alternatives, modifications and equivalents.

Claims

1. A computer-aided system, comprising: A first repositionable structure, configured to support a first instrument, wherein the first repositionable structure supporting the first instrument includes a first plurality of joints; and A control system, comprising one or more processors, communicatively coupled to the first repositionable structure; The control system is configured to perform instrument aiming by: Identify a target in the workspace for positioning the first instrument supported by the first repositionable structure; The axis of the first target is defined based on the target and the location of the entry point; Determine the configuration of the first plurality of joints, the configuration aligning the first instrument with the first target axis such that the first instrument can be advanced along the first target axis; and Command the first plurality of joints based on the determined configuration.

2. The computer-aided system according to claim 1, wherein: The computer-aided system further includes: a second repositionable structure configured to support a second instrument, wherein the second instrument includes an imaging device; and Determining the target includes using kinematic information associated with the imaging device.

3. The computer-aided system according to claim 1, wherein: The computer-aided system further includes: a second repositionable structure configured to support a second instrument, wherein the second instrument includes an imaging device; and Determining the target includes: Detection and input device interaction with the operator; In response to the operator's interaction, the position of the second instrument is determined; and The target is determined based on the location of the second instrument.

4. The computer-assisted system of claim 3, wherein, Determining the position of the second instrument includes: Determine the position of the tip of the second instrument or the position of the origin of the field of view (FOV) of the imaging device.

5. The computer-assisted system of claim 3, wherein, Determining the target based on the location of the second instrument includes: Position the target at a longitudinal distance along the observation direction of the second instrument.

6. The computer-aided system according to claim 1, wherein: The computer-aided system further includes: a second repositionable structure configured to support a second instrument, wherein the second instrument includes an imaging device; and Determining the target includes analyzing image data generated by the imaging device to automatically determine the target or to automatically determine a plurality of potential targets from which the target can be selected.

7. The computer-assisted system of claim 6, wherein, Determining the target or the plurality of potential targets includes: Anatomical features are identified based on the image data.

8. The computer-aided system according to claim 1, wherein: The computer-aided system further includes: a display device configured to display an image based on image data generated by an imaging device; and Determining the target includes: Detecting operator interaction with the image displayed on the display device; and The target is determined based on the operator's interaction.

9. The computer-aided system according to claim 1, wherein, Defining the axis of the first target based on the target and the location of the entry point includes: The surface or volume is defined based on the stated objective; Identify the target location on the surface or in the volume; and The first target axis is defined to intersect with the target location and the entry site location.

10. The computer-aided system according to claim 9, wherein, The shape of the surface or the volume, or the size of the surface or the volume, is determined based on the type of instrument associated with the first instrument.

11. The computer-aided system according to claim 9, wherein, Identifying the target location includes: The first candidate is identified as the location on the surface that is closest to the location of the entry point; Setting the target position as the first candidate will result in the following determination: when the first instrument is inserted into the workspace, the first instrument is commanded to pass through the restricted area or move beyond the movement limit of the first instrument; In response to making the determination, a second candidate is identified on the surface, wherein setting the target position as the second candidate will such that: when the first instrument is inserted into the workspace, the first instrument is not commanded to pass through the restricted area or move outside the range of motion; and Set the target location as the second candidate.

12. The computer-aided system according to claim 1, wherein, Defining the axis of the first target based on the target and the location of the entry point includes: Defined as a spherical shape centered on the target; The target location on the surface of the spherical shape is identified based on the location of the entry point; and The first target axis is defined to intersect with the target location and the entry site location.

13. The computer-aided system according to claim 12, wherein, Identifying the target location includes: The target location is defined as the location on the surface that is closest to the location of the entry point.

14. The computer-aided system according to any one of claims 1 to 13, wherein: The computer-aided system further includes: a second repositionable structure configured to support a second instrument, wherein the second instrument includes an imaging device; and The control system is configured to perform instrument aiming in response to receiving an indication that the positioning of the second instrument has been completed.

15. The computer-aided system according to any one of claims 1 to 13, wherein: The computer-aided system further includes: a second repositionable structure configured to support a second instrument, wherein the second instrument includes an imaging device, and wherein the second repositionable structure supporting the second instrument includes a second plurality of joints; and The control system is also configured to perform instrument aiming by: Receive an indication that the first plurality of joints have reached the determined configuration; and In response to receiving the instruction, the field of view of the imaging device is expanded by selecting at least one action from the group consisting of: commanding the second plurality of joints to retract the second instrument; commanding the second plurality of joints to move the imaging device further away from the workspace; reducing the magnification level of the imaging sensor of the imaging device; and reducing the digital magnification of the image captured by the imaging device.

16. The computer-aided system according to any one of claims 1 to 13, further comprising: A second repositionable structure is configured to support a second instrument, wherein the second instrument includes an imaging device; and The display device is configured to display an image based on image data captured by the imaging device, wherein the control system is further configured to render the image to include a representation of the first target axis.

17. The computer-aided system according to any one of claims 1 to 13, further comprising: A third repositionable structure is configured to support a third instrument. The first target axis is defined by: defining the first target axis based on at least one parameter selected from the group consisting of: (i) the physical configuration of the third repositionable structure; (ii) the insertion axis of the third instrument; and (iii) the target axis of the third instrument.

18. The computer-aided system according to any one of claims 1 to 13, wherein, The control system is also configured to: A first verification is performed to verify that the first instrument meets insertion conditions, the insertion conditions including at least one criterion selected from the group consisting of: the first instrument is not inserted beyond a threshold distance from an uninserted position, and the distal portion of the first instrument is not inserted beyond the tip of the cannula corresponding to the entry site positioning; and The instrument aiming is performed based on the first verification.

19. The computer-aided system according to any one of claims 1 to 13, wherein, The control system is also configured to: Perform a second verification to verify the physical coupling between the first repositionable structure and the sleeve; and The instrument aiming is performed based on the second verification.

20. The computer-aided system according to any one of claims 1 to 13, wherein, The control system is also configured to: Perform a third verification to verify that the first device is mounted to the first repositionable structure; and The instrument aiming is performed based on the third verification.

21. The computer-aided system according to any one of claims 1 to 13, wherein, The first plurality of joints are commanded to include: Determining the current configuration of the first repositionable structure inhibits the second repositionable structure from aligning the second instrument supported by the second repositionable structure with the second target axis; and Before commanding the second instrument to align with the second target axis, command the first plurality of joints to align with the configuration.

22. The computer-aided system according to any one of claims 1 to 13, wherein, The control system is configured to also perform the instrument aiming by: Control the first repositionable structure to advance the first instrument along the first target axis.

23. A method of performing instrument aiming using a computer-aided system, the computer-aided system comprising: (i) a first repositionable structure configured to support a first instrument, wherein the first repositionable structure supporting the first instrument includes a first plurality of joints; and (ii) a control system including one or more processors communicatively coupled to the first repositionable structure, wherein the method includes: The control system determines the target in the workspace for positioning the first instrument supported by the first repositionable structure; The control system defines the axis of the first target based on the target and the location of entry. The configuration of the first plurality of joints is determined via the control system, the configuration aligning the first instrument with the first target axis so that the first instrument can be advanced along the first target axis; and The control system commands the first plurality of joints based on the determined configuration.

24. The method according to claim 23, wherein, The computer-aided system further includes a second repositionable structure configured to support a second instrument, wherein the second instrument includes an imaging device, and determining the target includes: Use the kinematic information associated with the imaging device.

25. The method according to claim 23, wherein, The computer-aided system further includes a second repositionable structure configured to support a second instrument, wherein the second instrument includes an imaging device, and determining the target includes: The control system detects and interacts with the operator via the input device; In response to the operator's interaction, the position of the second instrument is determined; and The target is determined by the control system based on the position of the second instrument.

26. The method of claim 25, wherein, Determining the position of the second instrument includes: The position of the tip of the second instrument or the position of the origin of the field of view (FOV) of the imaging device is determined via the control system.

27. The method according to claim 25, wherein, Determining the target based on the location of the second instrument includes: The target is positioned at a longitudinal distance along the observation direction of the second instrument via the control system.

28. The method according to claim 23, wherein, The computer-aided system further includes a second repositionable structure configured to support a second instrument, wherein the second instrument includes an imaging device, and determining the target includes: The control system analyzes the image data generated by the imaging device to automatically determine the target or to automatically determine multiple potential targets from which the target can be selected.

29. The method according to claim 28, wherein, Determining the target or the plurality of potential targets includes: Anatomical features are identified based on the image data via the control system.

30. The method according to claim 23, wherein, The computer-aided system further includes: a display device configured to display an image based on image data generated by an imaging device, and determining the target includes: The control system detects operator interaction with the image displayed on the display device; and The target is determined by the control system based on the operator's interaction.

31. The method according to claim 23, wherein, Defining the axis of the first target based on the target and the location of the entry point includes: The control system defines the surface or volume based on the target; The target location on the surface or in the volume is identified via the control system; and The control system defines the first target axis to intersect with the target position and the entry location.

32. The method according to claim 31, wherein, The shape of the surface or the volume, or the size of the surface or the volume, is determined based on the type of instrument associated with the first instrument.

33. The method according to claim 31, wherein, Identifying the target location includes: The control system identifies the first candidate as the location on the surface that is closest to the location of the entry point; Setting the target position as the first candidate via the control system will result in the following determination: when the first instrument is inserted into the workspace, the first instrument is commanded to pass through the restricted area or move beyond the movement limit of the first instrument; In response to making the determination, a second candidate is identified on the surface, wherein setting the target position as the second candidate will such that: when the first instrument is inserted into the workspace, the first instrument is not commanded to pass through the restricted area or move outside the range of motion; and The target position is set as the second candidate via the control system.

34. The method according to claim 23, wherein, Defining the axis of the first target based on the target and the location of the entry point includes: The control system defines a spherical shape centered on the target; The control system identifies the target position on the surface of the spherical shape based on the location of the entry point; and The control system defines the first target axis to intersect with the target position and the entry location.

35. The method according to claim 34, wherein, Identifying the target location includes: The control system defines the target location as the position on the surface that is closest to the location of the entry point.

36. The method according to any one of claims 23 to 35, wherein, The computer-aided system further includes a second repositionable structure configured to support a second instrument, wherein the second instrument includes an imaging device, and the method includes: The instrument aiming is performed via the control system in response to receiving an indication that the positioning of the second instrument has been completed.

37. The method according to any one of claims 23 to 35, wherein, The computer-aided system further includes a second repositionable structure configured to support a second instrument, wherein the second instrument includes an imaging device, and wherein the second repositionable structure supporting the second instrument includes a second plurality of joints; and the method further includes: The control system receives instructions from the first plurality of joints to reach the determined configuration; and In response to receiving the instruction, the field of view of the imaging device is expanded by selecting at least one action from the group consisting of: commanding the second plurality of joints to retract the second instrument; commanding the second plurality of joints to move the imaging device further away from the workspace; reducing the magnification level of the imaging sensor of the imaging device; and reducing the digital magnification of the image captured by the imaging device.

38. The method according to any one of claims 23 to 35, wherein, The computer-aided system further includes: (i) a second repositionable structure configured to support a second instrument, wherein the second instrument includes an imaging device; and (ii) a display device configured to display an image based on image data captured by the imaging device, wherein the method further includes: The image is rendered via the control system to include a representation of the first target axis.

39. The method according to any one of claims 23 to 35, wherein, The computer-aided system further includes a third repositionable structure configured to support a third instrument and defining the first target axis as follows: The first target axis is defined by the control system based on at least one parameter selected from the group consisting of: (i) the physical configuration of the third repositionable structure; (ii) the insertion axis of the third instrument; and (iii) the target axis of the third instrument.

40. The method according to any one of claims 23 to 35, further comprising: The control system performs a first verification to verify that the first instrument meets the insertion conditions, the insertion conditions including at least one criterion selected from the group consisting of: the first instrument is not inserted beyond a threshold distance from the non-insertion position, and the distal portion of the first instrument is not inserted beyond the tip of the cannula corresponding to the entry site positioning; as well as The instrument aiming is performed via the control system based on the first verification.

41. The method according to any one of claims 23 to 35, further comprising: A second verification is performed via the control system to verify the physical coupling between the first repositionable structure and the sleeve. as well as The instrument aiming is performed via the control system based on the second verification.

42. The method according to any one of claims 23 to 35, further comprising: A third verification is performed via the control system to verify that the first instrument is installed in the first repositionable structure; as well as The instrument aiming is performed via the control system based on the third verification.

43. The method according to any one of claims 23 to 35, wherein, The first plurality of joints are commanded to include: The control system determines that the current configuration of the first repositionable structure inhibits the second repositionable structure from aligning the second instrument supported by the second repositionable structure with the second target axis; and The first plurality of joints are commanded to align with the second target axis via the control system.

44. The method according to any one of claims 23 to 35, further comprising: The first repositionable structure is controlled by the control system to advance the first instrument along the first target axis.

45. One or more non-transitory machine-readable media comprising a plurality of machine-readable instructions, which, when executed by a control system associated with a computer-aided system, are adapted to cause the control system to perform the method according to claims 23 to 44.

Images