System and method for view recovery
By recording and detecting kinematics and imaging information of the imaging device and determining its desired position and orientation, the problem of view recovery of the imaging device at different positions is solved, and image quality and operation efficiency are improved.
Patent Information
- Application Number
- CN202411037552.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2019-04-03
- Filing Date
- 2020-04-02
- Publication Date
- 2024-11-19
AI Technical Summary
When the imaging device moves between different repositionable arms and/or workspace ports, the usability of the image is reduced and the program execution is delayed, and a method is needed to restore the view of the imaging device to maintain the quality of the image and the efficiency of operation of the operation. .
The kinematic information and imaging information are recorded and detected by the computer-aided device and the controller, the desired position and orientation of the imaging device are determined and based on this, the imaging device is moved to restore the view after arm exchange and port jumps.
Effectively reduce the difference between the image forming device view after arm swap and port jump and the previous view, ensure that the operator can quickly identify and use the instrument, and improve the image quality and operation efficiency of the imaging device at different locations.
Smart Images

Figure CN118975854A_ABST
Abstract
Description
[0001] This application is a divisional application of Chinese patent application 2020800217197 (PCT / US2020 / 026469) filed on April 2, 2020 and entitled “System and Method for View Recovery”.
[0002] Related Applications
[0003] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 828,825, filed on April 3, 2019, which is incorporated herein by reference. Technical Field
[0004] The present disclosure relates generally to controlling a device having a repositionable arm for controlling an imaging device, and more particularly to restoring a view of an imaging device as the imaging device is moved between the repositionable arm and / or a workspace port. Background Art
[0005] More and more devices are being replaced by autonomous and semi-autonomous electronic devices. This is especially true in industrial, entertainment, educational, and other settings. As a medical example, today's hospitals find a large number of autonomous and semi-autonomous electronic devices in operating rooms, interventional rooms, intensive care units, emergency rooms, and / or similar locations. For example, glass and mercury thermometers are being replaced by electronic thermometers, IV drip lines now include electronic monitors and flow regulators, and traditional handheld surgical and other medical instruments are being replaced by computer-assisted medical devices.
[0006] These electronic devices provide advantages and challenges to the personnel who operate them. Many of these electronic devices may be able to autonomously or semi-autonomously move one or more repositionable arms and / or instruments. For example, in order to facilitate the flexible use of the electronic device, the electronic device can be configured to have an imaging device that can move between (e.g., mounted to) different repositionable arms and / or between different ports used to access the workspace. However, when the imaging device moves between different repositionable arms and / or different workspace ports, the image captured by the imaging device may change. In some embodiments, this may reduce the usability of the image from the imaging device and / or cause a delay in further executing a program that utilizes the image from the imaging device.
[0007] Therefore, improved methods and systems for restoring the view of an imaging device after the imaging device has been moved between different repositionable arms and / or different workspace ports are desired. Summary of the invention
[0008] In accordance with some embodiments, a computer-assisted device includes an imaging device and a controller coupled to the imaging device. The controller is configured to record kinematic information, imaging information, or both kinematic information and imaging information of the computer-assisted device prior to movement of the imaging device from a first repositionable arm to a second repositionable arm or from a first workspace port to a second workspace port; detect movement of the imaging device from the first repositionable arm to the second repositionable arm or from the first workspace port to the second workspace port; in response to the detection, determine a desired position and orientation of the imaging device based on the recorded kinematic information, the recorded imaging information, or both the recorded kinematic information and the recorded imaging information; and move the imaging device based on the desired position and orientation.
[0009] Consistent with some embodiments, the method includes recording, using a controller, kinematic information, imaging information, or both kinematic information and imaging information of a computer-assisted device prior to movement of the imaging device from a first repositionable arm to a second repositionable arm or from a first workspace port to a second workspace port; detecting, using the controller, movement of the imaging device from the first repositionable arm to the second repositionable arm or from the first workspace port to the second workspace port; in response to the detection, determining, using the controller, a desired position and orientation of the imaging device based on the recorded kinematic information, the recorded imaging information, or both the recorded kinematic information and the recorded imaging information; and moving the imaging device using the controller based on the desired position and orientation.
[0010] Consistent with some embodiments, a non-transitory machine-readable medium including a plurality of machine-readable instructions, when the instructions are executed by one or more processors associated with a computer-assisted device, is adapted to cause the one or more processors to perform any of the methods described herein.
[0011] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory in nature, and are intended to provide an understanding of the present disclosure without limiting the scope of the present disclosure. In this regard, additional aspects, features and advantages of the present disclosure will be apparent to those skilled in the art from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] Figure 1 is a simplified diagram of a computer-assisted system according to some embodiments.
[0013] Figure 2 is a simplified diagram of an imaging device introduced into a workspace according to some embodiments.
[0014] Figure 3 is a simplified diagram of a view restoration method according to some embodiments.
[0015] Figure 4Ais a simplified diagram of a view of an imaging device prior to arm swapping and / or port hopping in accordance with some embodiments.
[0016] Figure 4B is a simplified diagram of a view of an imaging device after an arm swap and / or port hopping and before view restoration according to some embodiments.
[0017] Figure 4C is a simplified diagram of a view of an imaging device after an arm swap and / or port hopping and after view restoration in accordance with some embodiments.
[0018] Figure 5 is a simplified diagram of a method of determining a view of an imaging device according to some embodiments.
[0019] Figure 6 is a simplified diagram of another method of determining a view of an imaging device in accordance with some embodiments.
[0020] Figure 7 is a simplified diagram of a method of moving an imaging device to obtain a desired view in accordance with some embodiments.
[0021] Figure 8 is a simplified diagram of a method of determining an aspect of interest from a view according to some embodiments.
[0022] Fig. 9 is a simplified diagram of a method of view restoration according to some embodiments.
[0023] In the drawings, elements with the same name have the same or similar functions. DETAILED DESCRIPTION
[0024] The present specification and drawings illustrating aspects, embodiments, implementations or modules of the invention should not be considered limiting - the claims define the protected invention. Various mechanical, compositional, structural, electrical and operational changes may be made without departing from the spirit and scope of the present specification and claims. In some cases, well-known circuits, structures or techniques are not shown or described in detail in order not to obscure the present invention. The same numbers in two or more figures represent the same or similar elements.
[0025] In this specification, the specific details of describing some embodiments consistent with the present disclosure are set forth. Many specific details are set forth to provide a thorough understanding of the embodiments. However, it is apparent to those skilled in the art that some embodiments may be practiced without some or all of these specific details. The specific embodiments disclosed herein are intended to illustrate rather than to limit. Those skilled in the art may appreciate that other elements within the scope and spirit of the present disclosure, although not specifically described herein. In addition, to avoid unnecessary repetition, one or more features shown and described in association with one embodiment may be incorporated into other embodiments, unless specifically described otherwise or when one or more features will render the embodiment non-functional.
[0026] In addition, the terms of this specification are not intended to limit the present invention. For example, spatial relative terms, such as "beneath", "below", "lower", "above", "upper", "proximal", "distal" and similar terms, can be used to describe the relationship between an element or feature and another element or feature, as shown in the figure. In addition to the position and orientation shown in the figure, these spatial relative terms are intended to cover the different positions (i.e., positioning) and orientations (i.e., rotational placement) of the elements or their operations. For example, if the content of one of the figures is flipped, the elements described as "below" or "beneath" other elements or features will be "above" or "over" other elements or features. Therefore, the exemplary term "below" can cover the position and orientation above and below. The device can be oriented in other ways (rotated 90 degrees or in other orientations), and the spatial relative descriptors used herein are interpreted accordingly. Likewise, descriptions of movement along and around various axes include various specific element positions and orientations. Furthermore, unless the context indicates otherwise, the singular forms "a," "an," and "the" are intended to include the plural forms as well. Also, the terms "comprises," "comprising," "includes," and similar terms list the presence of illustrated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups. Components described as being coupled may be directly coupled electronically or mechanically, or they may be indirectly coupled via one or more intermediate components.
[0027] The elements described in detail with reference to an embodiment, implementation or module can be included in other embodiments, implementations or modules in any practical situation, wherein they are not specifically shown or described. For example, if the elements are described in detail with reference to an embodiment and without reference to the second embodiment, the element can still be claimed to be included in the second embodiment. Therefore, in order to avoid unnecessary repetition in the following description, one or more elements shown and described in association with an embodiment, implementation or application can be incorporated in other embodiments, implementations or aspects, unless specifically described in addition, unless one or more elements will make an embodiment or implementation non-functional, or unless two or more elements provide conflicting functions.
[0028] In some instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.
[0029] The present disclosure describes various devices, elements, and portions of devices and elements in terms of their states in three-dimensional space. As used herein, the term "position" refers to the translational placement of an element or portion of an element in three-dimensional space (e.g., three translational degrees of freedom along Cartesian x-, y-, and z-coordinates). As used herein, the term "orientation" refers to the rotational placement of an element or portion of an element (three rotational degrees of freedom - e.g., roll, pitch, and yaw). As used herein, and for devices with repositionable arms, the term "proximal" refers to toward the base of the device and "distal" refers to away from the base.
[0030] Aspects of the present invention are primarily based on the use of a computer-assisted medical system such as the da However, those skilled in the art will appreciate that the inventive aspects disclosed herein may be embodied and implemented in a variety of ways, including robotic and non-robotic (if applicable) embodiments and implementations. The embodiments of any surgical system of the surgical system are merely exemplary and should not be considered as limiting the scope of the invention disclosed herein. For example, any reference to surgical instruments and surgical methods is non-restrictive, because the instruments, systems and methods described herein can be used for animals, human corpses, animal corpses, parts of human or animal anatomical structures, tissue removed from human or animal anatomical structures (with or without returning to human or animal anatomical structures), non-surgical diagnosis, industrial systems and general robots or remote operation systems. As a further example, the instruments, systems and methods described herein can be used for non-medical purposes, including industrial uses, general robot uses, sensing or manipulating non-tissue artifacts, cosmetic improvements, human or animal anatomical structure imaging, collecting data from human or animal anatomical structures, setting or removing systems, training medical or non-medical personnel and / or similar purposes. Additional example applications include procedures for tissue removed from human or animal anatomical structures (without returning to human or animal anatomical structures) and procedures for human or animal corpses. In addition, these technologies can also be used for medical treatment or diagnostic procedures including or not including surgical aspects.
[0031] Figure 1 is a simplified diagram of a computer-assisted system 100 according to some embodiments. Figure 1 As shown, the computer-assisted system 100 includes a computer-assisted device 110 having one or more repositionable arms 120. Each of the one or more repositionable arms 120 can support one or more instruments 125. In some examples, the computer-assisted device 110 can be consistent with a computer-assisted medical device such as a computer-assisted non-invasive diagnostic device and / or a computer-assisted surgical device. The one or more instruments 125 can include instruments, imaging devices, and / or the like. In some medical examples, the instruments can include medical instruments such as clips, graspers, retractors, cautery instruments, suction instruments, suturing devices, and / or the like. In some medical examples, the imaging devices can include endoscopes, cameras, ultrasound devices, fluoroscope devices, and / or the like.
[0032] The computer aided device 110 is coupled to the control unit 130 via an interface. The interface may include one or more cables, connectors and / or buses, and may also include one or more networks having one or more network switching devices and / or routing devices. The control unit 130 includes a processor 140 coupled to a memory 150. The operation of the control unit 130 is controlled by the processor 140. And although the control unit 130 is shown as having only one processor 140, it should be understood that the processor 140 may represent one or more central processing units, multi-core processors, microprocessors, microcontrollers, digital signal processors, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), graphics processing units (GPUs), and / or the like in the control unit 130. The control unit 130 may be implemented as a standalone subsystem and / or added to a board of a computing device or as a virtual machine.
[0033] The memory 150 may be used to store software executed by the control unit 130 and / or one or more data structures used during operation of the control unit 130. The memory 150 may include one or more types of machine-readable media. Some common forms of machine-readable media may include a floppy disk, a diskette, a hard disk, a magnetic tape, any other magnetic medium, a CD-ROM (Compact Disc Read Only Drive), any other optical medium, punch cards, paper tape, any other physical medium with a pattern of holes, a RAM (Random Access Memory), a PROM (Programmable Read Only Memory), an EPROM (Erasable Programmable Read Only Memory), a FLASH-EPROM (Flash Programmable Read Only Memory), any other memory chip or cartridge, and / or any other medium from which a processor or computer is suitable for reading.
[0034] In some embodiments, the computer-assisted system 100 may be found in a medical environment such as an operating room and / or an interventional room. And although the computer-assisted system 100 includes only one computer-assisted device 110 with two repositionable arms 120, a person of ordinary skill will understand that the computer-assisted system 100 may include any number of computer-assisted devices with repositionable arms and / or instruments, with designs similar to and / or different from the computer-assisted device 110. And although the computer-assisted device 110 is shown as a stand-alone unit (e.g., with a base on the ground), the computer-assisted device 110 may be wall-mounted, ceiling-mounted, mounted to a table and / or workspace, and / or the like. In some examples, each of the computer-assisted devices may include fewer or more repositionable arms 120 and / or instruments 125.
[0035] The imaging control module 160 may support autonomous and / or semi-autonomous control of the computer-assisted device 110. The imaging control module 160 may additionally include one or more application programming interfaces (APIs) for receiving position, motion, and / or other sensor information from the computer-assisted device 110, exchanging position, motion, and / or collision avoidance information about other devices with other control units, and / or planning and / or assisting in planning the motion of the computer-assisted device 110, the repositionable arm 120, the instrument 125, and / or the like. In addition, the imaging control module 160 may provide commands to one or more actuators to control the position and / or orientation of the repositionable arm 120, the instrument 125, and / or the like. And although the imaging control module 160 is depicted as a software module, the imaging control module 160 may be implemented using hardware, software, and / or a combination of hardware and software.
[0036] One of the tasks of the imaging control module 160 is to help support movement of an imaging device from a first one of the repositionable arms 120 to a second one of the repositionable arms 120, which is sometimes referred to as an arm swap. In some examples, the imaging device can be one of the instruments 125. Another task of the imaging control module 160 is to help support movement of the imaging device between different ports for accessing the workspace (with or without an arm swap), which is sometimes referred to as port hopping. In some examples, these tasks of the imaging control module 160 include determining relevant aspects of the view of the imaging device before the arm swap and / or port hopping, and then moving the imaging device after the arm swap and / or port hopping to minimize differences between the view of the imaging device after the arm swap and / or port hopping and the view of the imaging device before the arm swap and / or port hopping.
[0037] Figure 2 FIG. 2 is a simplified diagram of an imaging device 200 introduced into a workspace according to some embodiments. In some examples, the imaging device 200 may be one of the one or more instruments 125. Figure 2 As shown, the imaging device includes an elongated shaft 210 extending from a proximal end and a distal end, at which proximal end it is mounted to a repositionable arm (e.g., repositionable arm 120), and the imaging device is at the distal end. In some examples, the imaging device 220 is oriented so that the viewing direction of the image obtained by it is oriented to point to the distal end of the longitudinal axis 225 of the imaging device 220. In some examples, the imaging device 220 can correspond to a camera, an endoscope, a stereo endoscope, and / or the like. And although the imaging device 200 is shown as having a straight axis 210, an imaging device with an angled axis is also possible. In some examples, the angled axis can include a bend so that the viewing direction of the imaging device 220 and the axis 225 are angled relative to the longitudinal axis 215 of the shaft 210. In some examples, the angle of the bend can be between 0 degrees and 45 degrees.
[0038] According to some embodiments, the imaging device 200 may optionally include an articulated wrist 230, which may be used to dynamically adjust the angle between the viewing direction of the imaging device 220 and the longitudinal axis 215 of the shaft 210. In some examples, the articulated wrist 230 may allow for articulation of the directional view relative to the longitudinal axis 215 of the shaft 210 in one or more degrees of freedom. In some examples, the one or more degrees of freedom may include pitch and / or yaw degrees of freedom.
[0039] According to some embodiments, the imaging device 200 can be introduced into a workspace (e.g., the internal anatomical structure of a patient in a medical example) by inserting the imaging device 200 through the port 240. In some examples, the port 240 can be a cannula, a trocar, and / or the like. In some examples, the port 240 is typically located at the periphery of the workspace. In some examples, the periphery can correspond to the boundary of a pressure vessel, an isolation chamber, and / or the like. In some medical examples, the periphery can correspond to the dermis of an anatomical structure. In some examples, in order to avoid applying pressure on the imaging device 200, the port 240, and / or the periphery, the repositionable arm to which the imaging device 200 is mounted can have limited movement, so that the shaft 210 only rotates around the remote motion center 250 and / or the shaft 210 is inserted and / or retracted along the longitudinal axis 215. In some examples, the remote motion center 250 can be centered at a point where the port 240 allows the shaft 210 to pass through the periphery. In some examples, the rotation around the remote motion center 250 can include one or more rotational degrees of freedom. In some examples, the one or more rotational degrees of freedom may be one or more of a pitch rotation about remote center of motion 250 , a yaw rotation about remote center of motion 250 , and / or a roll rotation about remote center of motion 250 and longitudinal axis 215 .
[0040] Figure 3 is a simplified diagram of a view restoration method according to some embodiments. One or more of the processes 310-340 of the method 300 may be implemented at least in part in the form of executable code stored on a non-transitory, tangible, machine-readable medium, which executable code, when executed by one or more processors (e.g., processor 140 in control unit 130), may cause the one or more processors to perform one or more of the processes 310-340. In some embodiments, portions of the method 300 may be performed by a module such as an imaging control module 160. In some embodiments, the method 300 may be used to restore a view of an imaging device (e.g., imaging device 200) after the imaging device is moved between repositionable arms (e.g., repositionable arms 120) and / or between workspace ports / port jumps (e.g., between ports 240). Reference Figure 4A and Figure 4B Aspects of method 300 are described by way of example.
[0041] Figure 4A is a simplified diagram of a view of an imaging device prior to arm swapping and / or port hopping according to some embodiments. Figure 4A As shown, the view of the imaging device (roughly corresponding to the dashed rectangle) shows that the view includes three instruments. Figure 4A Three representative instruments are shown in FIG. 1 , but other instruments with other positions, orientations, and / or configurations are possible. Furthermore, even if three instruments are Figure 4A Although depicted as fully visible in the illustration, some or all of each instrument may also be obscured by each other, objects in the workspace (e.g., anatomical structures), and / or the like.
[0042] The instrument 410 extends into the view from the lower left. As shown, the instrument 410 includes an articulated wrist and an end effector having a gripper. The position of the instrument 410 and its end effector can be described using the position of a coordinate system 420, and the orientation of the end effector can be described using the orientation of the x-axis, y-axis, and z-axis of the coordinate system 420. In some examples, the position and orientation of the coordinate system 420 can be mapped by determining a relative transformation of the coordinate system 420 relative to the imaging device using one or more kinematic models of the instrument 410 and the repositionable arm to which the instrument 410 is mounted, as shown in Equation 1, where FK corresponds to the forward kinematics of the device indicated.
[0043]
[0044] The apparatus 430 extends into the view from the left side. As shown, the apparatus 430 includes an articulated wrist and an end effector. The position of the apparatus 430 and its end effector can be described using the position of the coordinate system 440, and the orientation of the end effector can be described using the orientation of the x-axis, y-axis, and z-axis of the coordinate system 440. In some examples, the position and orientation of the coordinate system 440 can be mapped to a world coordinate system using the inverse of one or more kinematic models of the apparatus 430 and the repositionable arm to which the apparatus 430 is mounted. In some examples, the position and orientation of the coordinate system 440 can be additionally mapped to the observation coordinate system of the imaging device using one or more kinematic models of the imaging device and the repositionable arm to which the imaging device is mounted.
[0045] The apparatus 450 extends into the view from the lower right. As shown, the apparatus 450 includes an articulated wrist and an end effector having a gripper. The position of the apparatus 450 and its end effector can be described using the position of a coordinate system 460, and the orientation of the end effector can be described using the orientation of the x-axis, y-axis, and z-axis of the coordinate system 460. In some examples, the position and orientation of the coordinate system 460 can be mapped to a world coordinate system using the inverse of one or more kinematic models of the apparatus 450 and the repositionable arm to which the apparatus 450 is mounted. In some examples, the position and orientation of the coordinate system 460 can be additionally mapped to the observation coordinate system of the imaging device using one or more kinematic models of the imaging device and the repositionable arm to which the imaging device is mounted.
[0046] Figure 4B is a simplified diagram of a view of an imaging device after an arm swap and / or port hopping and before view restoration according to some embodiments. Figure 4B , instrument 410 has been replaced by an imaging device (e.g., by swapping an imaging device arm to a repositionable arm to which instrument 410 is mounted and / or removing instrument 410 from the workspace and jumping the imaging device port to the port previously used by instrument 410). Figure 4B ed as fully visible in the workspace, but some or all of each instrument may be obscured by each other, objects in the workspace (e.g., anatomical structures), and / or the like.
[0047] Figure 4A and Figure 4B The comparison shows that arm swapping and / or port hopping before Figure 4A The view of the imaging device is significantly different after an arm swap and / or port jump Figure 4B The difference can be noted by observing that the coordinate system 442 of the instrument 430 after the arm exchange and / or port jump has a significantly different position and orientation than the coordinate system 440 of the instrument 430 before the arm exchange and / or port jump. Similarly, the coordinate system 462 of the instrument 450 after the arm exchange and / or port jump has a significantly different position and orientation than the coordinate system 460 of the instrument 450 before the arm exchange and / or port jump. In some cases, the difference in position and / or orientation may make it difficult for the operator to quickly discern Figure 4B Shown are views of the same instrument 430 and 450. Similar to coordinate systems 420, 440, and / or 460, coordinate systems 442 and / or 462 may be mapped to a world coordinate system and / or an imaging coordinate system of an imaging device using one or more kinematic models.
[0048] Figure 4Cis a simplified diagram of a view of an imaging device after an arm swap and / or port hopping and after view restoration according to some embodiments. Figure 4C As shown in the example of FIG. 4 , the imaging device has been moved to restore the view of instruments 430 and 450 and the surrounding workspace. Figure 4C ed as fully visible in the workspace, but some or all of each instrument may be obscured by each other, objects in the workspace (e.g., anatomical structures), and / or the like.
[0049] Figure 4A and Figure 4C The comparison shows that arm swapping and / or port hopping before Figure 4A The view of the imaging device is somewhat different after an arm swap and / or port jump Figure 4C of the imaging device, but Figure 4B This difference can be noted by observing that coordinate system 444 of instrument 430 after arm swap and / or port jump and view recovery is closer to the position and orientation of coordinate system 440 of instrument 430 before arm swap and / or port jump than coordinate system 442. Similarly, coordinate system 464 of instrument 450 after arm swap and / or port jump and view recovery is closer to the position and orientation of coordinate system 460 of instrument 450 before arm swap and / or port jump than coordinate system 462. Similar to coordinate systems 420, 440 and / or 460, one or more kinematic models can be used to map coordinate systems 444 and / or 446 to a world coordinate system and / or an imaging coordinate system of an imaging device.
[0050] However, because the imaging device has restored its view after the arm swap and / or port hopping, Figure 4A and Figure 4C The views are similar enough to assist an operator in being able to use instruments 430 and / or 450 following an arm swap and / or port.
[0051] Return to reference Figure 3 At process 310, kinematic and / or imaging information prior to an imaging device arm swap and / or port hop is recorded. The recorded kinematic and / or imaging information is used to determine a view of a workspace and / or one or more instruments in the workspace prior to the arm swap and / or port hop. The recorded information and corresponding views allow them to be used to determine a desired imaging device position and orientation after the arm swap and / or port hop. Figure 4A In an embodiment of the present invention, the recorded kinematic and / or imaging information may include information about the imaging coordinate system, coordinate systems 420, 440, and / or 460, kinematics used to determine these coordinate systems, and / or the like. Further examples of process 310 are described in more detail below.
[0052] At process 320, arm exchange and / or port jump of the imaging device are detected. In some examples, arm exchange can be automatically detected by observing that the imaging device is disassembled from the first repositionable arm and the imaging device is subsequently installed to the second repositionable arm. In some examples, one or more sensors, buttons, switches, electrical connections, magnetic detection parts and / or the like can be used to determine whether the imaging device is disassembled and / or installed to the corresponding repositionable arm. In some examples, port jump can be automatically detected by observing that the imaging device and / or the repositionable arm are disassembled from the first port and the imaging device and / or the repositionable arm are subsequently installed to the second port. In some examples, one or more sensors, buttons, switches, electrical connections, magnetic detection parts and / or the like can be used to determine whether the imaging device and / or the repositionable arm are disassembled and / or installed to the corresponding port. In some examples, arm exchange and / or port jump can be indicated by the operator by pressing a button, voice command, user interface command and / or the like. In some examples, port jump and arm exchange can occur together (e.g., the imaging device is moved between repositionable arms and between workspace ports). In some examples, arm swapping and / or port hopping can include replacing a first imaging device with a second imaging device.
[0053] At process 330, a desired position and orientation of the imaging device after the arm swap and / or port hop is determined. In some examples, the desired position and / or orientation is determined so that the view of the imaging device after the arm swap and / or port hop is as similar as possible to the view of the imaging device before the arm swap and / or port hop (as determined during process 310). Figure 4A and Figure 4C In an embodiment, a desired position and orientation of an imaging device may be determined to reduce and / or minimize differences between coordinate systems 440 and 470 relative to the imaging device and / or differences between coordinate systems 444 and 464 relative to the imaging device before and after an arm swap and / or port hopping and view recovery, and / or the like. An example of process 330 is described in more detail below.
[0054] At process 340, the imaging device is moved to a desired position and orientation. In some examples, before the movement is performed, the motion plan of the imaging device is determined, and then the imaging device is moved according to the motion plan. In some examples, the motion plan includes a movement that reduces the risk of collision between repositionable arms, between the imaging device and other instruments in the workspace, between the imaging device and objects in the workspace and / or no-fly zones, and / or the like. In some examples, after arm exchange and / or port jump, the movement of the imaging device can be performed using one or more actuators of the repositionable arm to which the imaging device is mounted. In some examples, one or more signals including one or more currents, one or more voltages, one or more pulse width modulation signals, and / or the like can be used to control the actuator. In some examples, one or more kinematic models, one or more Jacobian matrices, and / or the like of the repositionable arm and / or the imaging device can be used to plan the motion plan and / or for performing the movement of the repositionable arm and / or the imaging device. The example of process 340 is described in more detail below.
[0055] Figure 5 is a simplified diagram of a method 500 for determining a view of an imaging device according to some embodiments. One or more of the processes 510-540 of method 500 may be implemented at least in part in the form of executable code stored on a non-transitory, tangible, machine-readable medium, which executable code, when executed by one or more processors (e.g., processor 140 in control unit 130), may cause one or more processors to perform one or more of the processes 510-540. In some embodiments, a portion of method 500 may be performed by a module such as imaging control module 160. In some embodiments, method 500 may be performed as part of process 310. In some embodiments, method 500 determines a view of an imaging device based on information primarily associated with the imaging device. In some embodiments, method 500 may be used as a basis for a view of an imaging device when fewer than two devices are available.
[0056] At process 510, a viewing direction is determined based on the imaging device information. In some examples, the viewing direction may correspond to a direction between the imaging device and a center point of an image captured by the imaging device. In some examples, the viewing direction corresponds to a z-axis of an imaging device coordinate system of the imaging device. In some examples, the viewing direction may be determined based on an inverse of one or more kinematic models of the imaging device and / or the repositionable arm to which the imaging device is mounted. In some examples, the viewing direction may be determined in a world coordinate system, such as with reference to Figure 4A In some examples, the viewing direction may correspond to a distal direction along an imaging axis of the imaging device (eg, longitudinal axis 225 of imaging device 220).
[0057] At process 520, an upward viewing direction is determined based on the imaging device information. In some examples, the upward viewing direction corresponds to an upward direction in an image captured by the imaging device. In some examples, the upward viewing direction corresponds to the y-axis line of the imaging device coordinate system of the imaging device. In some examples, the upward viewing direction can be determined based on the inverse of one or more kinematic models of the imaging device and / or the repositionable arm to which the imaging device is mounted. In some examples, the upward viewing direction can be determined in a world coordinate system. In some examples, the upward viewing direction can correspond to a component in a direction opposite to gravity in a world coordinate system (e.g., the z-axis line of the world coordinate system) that is orthogonal to the viewing direction. In some examples, it may be preferred to use the direction of gravity to determine the upward viewing direction when the workspace is oriented so that at least the vertical direction corresponds to upward in the physical 3D world.
[0058] At process 530, the depth of the point of interest from the imaging device is determined. In some examples, the point of interest corresponds approximately to the center of the image captured by the imaging device. In some examples, the depth of the point of interest from the imaging device can be determined using a ranging sensor incorporated into the imaging device. In some examples, when the imaging device is stereoscopic, the depth of the point of interest from the imaging device can be determined based on the pixel similarity and / or disparity map between the left and right stereo images captured by the imaging device. In some examples, the depth of the point of interest from the imaging device corresponds to the distance (e.g., Euclidean distance) between the imaging device and the point of interest.
[0059] At process 540, the location of the center of view of the imaging device is determined. The center of view may be determined by projecting from the imaging device a distance corresponding to the depth of the point of interest from the imaging device (as determined during process 530) along the viewing direction (as determined during process 510). In some examples, the center of view may be determined in a world coordinate system.
[0060] Figure 6is a simplified diagram of another method 600 for determining a view of an imaging device according to some embodiments. One or more of the processes 610-660 of method 600 may be implemented at least in part in the form of executable code stored on a non-transitory, tangible, machine-readable medium, which executable code, when executed by one or more processors (e.g., processor 140 in control unit 130), may cause one or more processors to perform one or more of the processes 610-660. In some embodiments, portions of method 600 may be performed by a module such as imaging control module 160. In some embodiments, method 600 may be performed as part of process 310. In some embodiments, method 600 determines a view of an imaging device based on information associated primarily with multiple instruments within and / or near the field of view of the imaging device. In some examples, multiple imaging devices do not have to be visible in an image captured by the imaging device.
[0061] According to some embodiments, the multiple instruments to be considered by method 600 can be selected according to multiple strategies. In some examples, the multiple instruments can include all instruments in the workspace except the imaging device. In some examples, the multiple instruments can include other imaging devices. In some examples, the multiple instruments can include all instruments in the workspace except the imaging device. In some examples, the multiple instruments can include each instrument within the field of view (even if not visible) of the imaging device. In some examples, the instruments within the field of view can be determined based on one or more kinematic models of the multiple instruments and / or the repositionable arms to which they are mounted, the imaging device and / or the repositionable arms to which the imaging device is mounted, the width of the view of the imaging device, and / or the like. In some examples, the field of view can be limited to an area closer to the imaging device than the maximum depth of observation and / or an area farther away from the imaging device than the minimum depth of observation. In some examples, the multiple instruments can include only those instruments designated as active by the computer-assisted device and / or the operator. In some examples, the multiple instruments can include only those instruments designated by the operator as being related to view determination.
[0062] At process 610, a representative position of each of the plurality of instruments is determined. In some examples, the representative position of each of the plurality of instruments relative to the imaging device can be determined using one or more kinematic models of each of the instruments and / or a repositionable arm to which each of the plurality of instruments is mounted, such as by reference to Figure 4A-4C and described by Equation 1. In some examples, the representative position of the instrument can correspond to a representative point on the end effector of the instrument. In some examples, the representative position of the instrument can correspond to the origin of the coordinate system of the end effector of the instrument, such as at Figure 4A and Figure 4C In some examples, each of the representative positions may be represented by a bounding volume of the corresponding end effector. In some examples, the bounding volume may be a bounding sphere, a bounding rectangular shape, and / or a more complex shape that better approximates the shape of the corresponding end effector. In some examples, each of the representative positions may correspond to the center of mass of the corresponding bounding volume.
[0063] At process 620, a set of representative positions of the instrument is determined. In some examples, the set can include determining the centroid of the representative positions, the centroid of the bounding volume, and / or the like. In some examples, the set can include the union of the bounding volumes, a circumscribed sphere containing each of the representative positions and / or bounding volumes, and / or the like.
[0064] At flow 630, a view center is determined based on the set. In some examples, the view center can be a set of representative locations, a centroid of a bounding volume of the set, a centroid of a circumscribed sphere, and / or the like.
[0065] At flow 640, a viewing direction to the center of the view is determined. The viewing direction is determined based on a direction from the imaging device to the center of the view.
[0066] At process 650, a depth of the center of the view from the imaging device is determined. In some examples, the depth can be determined based on a distance (e.g., a Euclidean distance) from the imaging device to the center of the view. In some examples, the depth can be determined by determining an observation distance such that the view of the imaging device encompasses the set of bounding volumes determined during process 620 such that each of the representative locations and / or each of the bounding volumes is contained within the view of the imaging device.
[0067] At process 660 , an upward viewing direction is determined. In some examples, a process similar to process 520 may be used to determine the upward viewing direction.
[0068] According to some embodiments consistent with methods 300, 500, and / or 600, the viewing direction, the upward viewing direction, the depth of the point of interest / view center from the imaging device, and the location of the view center collectively determine the view of the imaging device. In some examples, when the view corresponds to the view of the imaging device before an arm swap and / or port hop, one goal would be to position and orient the imaging device after the arm swap and / or port hop so that the view of the imaging device after the arm swap and / or port hop is as close as possible to the view of the imaging device before the arm swap and / or port hop.
[0069] Figure 7is a simplified diagram of a method 700 of moving an imaging device to obtain a desired view according to some embodiments. One or more of the processes 710-760 of the method 700 may be implemented at least in part in the form of executable code stored on a non-transitory, tangible, machine-readable medium, which when executed by one or more processors (e.g., processor 140 in control unit 130) may cause the one or more processors to perform one or more of the processes 710-760. In some embodiments, portions of the method 700 may be performed by a module such as imaging control module 160. In some embodiments, the method 700 may be performed as part of processes 330 and / or 340. In some embodiments, the method 700 moves the imaging device to obtain a desired view based on features of the imaging device and / or degrees of freedom with which the imaging device may be manipulated.
[0070] At process 710, a desired position and orientation of the imaging device is determined based on the viewing direction, the upward viewing direction, and the depth from the imaging device to the point of interest / view center (e.g., as determined by process 310, method 500, and / or method 600). The desired position of the imaging device corresponds to a point determined by back-projecting from the point of interest / view center along the viewing direction a distance corresponding to the depth of the point of interest / view center from the imaging device. The desired orientation of the imaging device corresponds to aligning the imaging device with the viewing direction and rolling the imaging device so that an image captured by the imaging device is oriented to be consistent with the upward viewing direction.
[0071] In some embodiments, when it is not possible to position the imaging device at the desired position and / or orient the imaging device to be consistent with the desired orientation (e.g., due to a lack of sufficient degrees of freedom of the imaging device and / or the repositionable arm to which the imaging device is mounted), the goal of obtaining the desired position and desired orientation is balanced. In some examples, a trade-off between balancing the distance between the actual view center and the desired view center (e.g., the Euclidean distance) and balancing the distance between the actual upward viewing direction and the desired upward viewing direction (e.g., the angular difference between the upward viewing direction or the dot product of the direction vectors of the upward viewing direction) can be used to determine the desired position and orientation of the imaging device. In some examples, the trade-off can include minimizing the weighted sum of the distances. In some examples, the trade-off can further balance the visibility of each of the instruments, so that each instrument visible to the imaging device before the arm exchange and / or port jump is visible to the imaging device after the arm exchange and / or port jump and / or each instrument not visible to the imaging device before the arm exchange and / or port jump is not visible to the imaging device after the arm exchange and / or port jump.
[0072] At process 720, it is determined whether the imaging device has an articulated wrist. In some examples, it can be determined whether the imaging device has an articulated wrist based on the model and / or type of the imaging device. In some examples, the model and / or type of the imaging device can be determined when the imaging device is installed, such as by reading this information from a memory included in the imaging device. When the imaging device includes an articulated wrist, the imaging device is positioned and oriented starting at process 730. When the imaging device does not include an articulated wrist, the imaging device is positioned and oriented starting at process 750.
[0073] At process 730, the imaging device is positioned using the degrees of freedom of the repositionable arm on which the imaging device is mounted. In some examples, the degrees of freedom include one or more of a pitch degree of freedom (e.g., about a remote center of motion, such as remote center of motion 250), a yaw degree of freedom (e.g., about remote center of motion 250), an insertion degree of freedom (e.g., along an insertion axis, such as longitudinal axis 215), and / or the like.
[0074] At process 740, the imaging device is oriented using the roll degree of freedom of the articulated wrist and / or the repositionable arm. In some examples, the roll degree of freedom corresponds to a rotation of the imaging device about the insertion axis. In some examples, the articulated wrist can include one or more degrees of freedom, including a pitch degree of freedom and / or a yaw degree of freedom, e.g., consistent with the discussion of articulated wrist 230. In some examples, articulation or expected articulation of the articulated wrist can be compensated when the imaging device is positioned during process 730.
[0075] Once the imaging device is positioned and oriented, method 700 ends.
[0076] At process 750, the imaging device is positioned using the degrees of freedom of the repositionable arm on which the imaging device is mounted. In some examples, the degrees of freedom include one or more of the pitch degrees of freedom, yaw degrees of freedom, insertion degrees of freedom, and / or the like as described with reference to process 730.
[0077] At process 760, the imaging device is oriented using the degrees of freedom of the repositionable arm. In some examples, the degrees of freedom include one or more of a roll degree of freedom, a pitch degree of freedom, a yaw degree of freedom, and / or the like of the repositionable arm.
[0078] Once the imaging device is positioned and oriented, method 700 ends.
[0079] As discussed above and further emphasized herein, Figure 7This is merely an example and should not unduly limit the scope of the claims. One of ordinary skill in the art will recognize many variations, substitutions, and modifications. According to some embodiments, the order in which one or more of the execution processes 730-760 are performed may be different from Figure 7 In some examples, process 740 may be performed before process 730. In some examples, processes 730 and 740 may be performed simultaneously. In some examples, process 760 may be performed before process 750. In some examples, processes 750 and 760 may be performed simultaneously. In some examples, movement of the imaging device along the insertion degree of freedom of the repositionable arm may be performed last.
[0080] According to some embodiments, movement along the insertion degree of freedom may not be automatically performed. In some examples, the roll, pitch and / or yaw of the imaging device can be adjusted during processes 730-760, wherein the imaging device is fully or partially retracted along its insertion degree of freedom, and the insertion of the imaging device along the insertion degree of freedom is manually performed by an operator after the method 700 is completed. In some examples, manual insertion of the imaging device can be performed by placing the imaging device and / or the repositionable arm in a clutch mode that allows manual articulation along the insertion degree of freedom. In some examples, the insertion of the imaging device along the insertion degree of freedom can be performed via remote operator control of the operator.
[0081] According to some embodiments, when the imaging device includes an angled axis, process 710 may be adapted to consider two possible orientations of the mid-axis bend when selecting a desired position and orientation of the imaging device. The first orientation corresponds to the mid-axis bend being oriented in a first direction, and the second orientation corresponds to the mid-axis bend being oriented in a flipped second orientation, wherein the roll orientation of the axis of the imaging device is rotated 180 degrees. In some examples, the selection of the second orientation may also include inverting the image captured by the imaging device, and when the imaging device is stereoscopic, swapping the left and right images.
[0082] Figure 8is a simplified diagram of a method 800 for determining an aspect of interest from a view according to some embodiments. One or more of the processes 810-840 of method 800 may be implemented at least in part in the form of executable code stored on a non-transitory, tangible, machine-readable medium, which, when executed by one or more processors (e.g., processor 140 in control unit 130), may cause one or more processors to perform one or more of the processes 810-840. In some embodiments, a portion of method 800 may be performed by a module such as imaging control module 160. In some embodiments, method 800 may be performed as part of process 310. In some embodiments, method 800 determines a view of an imaging device based on information associated with the imaging device and one or more instruments within and / or near the field of view of the imaging device. In some examples, the instrument need not be visible in the image captured by the imaging device. In some embodiments, one or more of processes 830 and / or 840 are optional and may be omitted.
[0083] According to some embodiments, the selection of one or more instruments considered by method 800 may be selected according to any of the strategies described for the selection of the plurality of instruments considered by method 600 .
[0084] At process 810, the position of the distal portion of each instrument is determined. In some examples, the distal portion of each instrument can correspond to a representative point on the corresponding instrument (e.g., the distal tip of the end effector, the center of mass of the end effector, the pivot point between the jaws of the end effector, and / or the like). In some examples, the position of the distal portion of each end effector can be mapped to a world coordinate system using an inverse of one or more kinematic models of the corresponding instrument and the repositionable arm to which the corresponding instrument is mounted. In some examples, the position of the distal portion of each instrument can be additionally mapped to an observation coordinate system of the imaging device using one or more kinematic models of the imaging device and the repositionable arm to which the imaging device is mounted. Figure 4A and Figure 4B In the example, the position of the distal portion of instruments 410, 430 and / or 450 can correspond to the origin of coordinate system 420 of instrument 410, the origin of coordinate system 440 and / or 470 of instrument 430, and / or the origin of coordinate system 460 and / or 480 of instrument 450.
[0085] At process 820, an orientation axis of the distal portion of each instrument is determined. Each orientation axis of each of the one or more instruments may be determined using one or more kinematic models of each instrument and / or the repositionable arm to which each of the plurality of instruments is mounted, such as with reference to Figure 4A and Figure 4B Described. Figure 4A and Figure 4B In the examples, the orientation axes of the distal portions of instruments 410, 430 and / or 450 can correspond to the x-axis, y-axis and z-axis of coordinate system 420 of instrument 410, the x-axis, y-axis and z-axis of coordinate systems 440 and / or 470 of instrument 430, and / or the x-axis, y-axis and z-axis of coordinate systems 460 and / or 480 of instrument 450.
[0086] At optional process 830 , an upward viewing direction is determined. In some examples, the upward viewing direction can be determined using a process similar to processes 520 and / or 660 .
[0087] At optional process 840, the visibility, position and / or orientation of one or more points of interest are determined. In some examples, the one or more points of interest may correspond to additional representative points on the one or more instruments and / or one or more points of interest in the workspace. In some examples, each of the one or more points of interest may correspond to a fiducial marker and / or other elements detectable in an image captured by an imaging device and / or other elements detectable by other sensing pathways (e.g., radio frequency, magnetic and / or the like). In some examples, when a point of interest is associated with one of the one or more instruments and / or is located at a known position on one of the one or more instruments, one or more kinematic models of the instrument and / or the repositionable arm on which the instrument is mounted may be used to determine the position and / or orientation of the point of interest. In some examples, the visibility of each of the one or more points of interest may be used to help characterize the view as to which of the one or more points of interest are visible and which of the one or more points of interest are not visible.
[0088] Fig. 9 is a simplified diagram of a method 900 of view restoration according to some embodiments. One or more of the processes 910-980 of the method 900 may be implemented at least in part in the form of executable code stored on a non-transitory, tangible, machine-readable medium, which executable code, when executed by one or more processors (e.g., processor 140 in control unit 130), may cause the one or more processors to perform one or more of the processes 910-980. In some embodiments, a portion of the method 900 may be performed by a module such as imaging control module 160. In some embodiments, the method 900 may be performed as part of processes 330 and 340. In some embodiments, the method 900 uses a gradient descent approach to determine a desired position and orientation of an imaging device after an arm swap and / or port hop, and then moves the imaging device to the desired position and orientation.
[0089] At process 910, a representative position and orientation of the current simulated position and orientation of the imaging device is determined. In some examples, the representative position and orientation may include the position of the distal portion of each of the instruments in the imaging device coordinate system at the current simulated position and orientation of the imaging device, the orientation axis of the distal portion of each of the instruments, the position of each of the one or more points of interest, the orientation of each of the one or more points of interest, and / or the like. In some examples, the representative position and orientation may include the upward viewing direction of the imaging device when the imaging device is in the current simulated position and orientation. In some examples, the representative position and orientation may include which visible indications of one or more points of interest and / or indications within the field of view of the imaging device when the imaging device is in the current simulated position and orientation. In some examples, the representative position and orientation may correspond to any of the positions and orientations determined by method 800 and may be determined using a method consistent with method 800.
[0090] At process 920, the difference between the representative position and orientation and the same representative position and orientation before the arm exchange and / or port jump is determined. In some examples, the difference between two corresponding representative positions can be determined using any suitable distance metric (e.g., Euclidean distance). In some examples, the difference between two corresponding representative orientations (e.g., the directions of two corresponding orientation axes) can be determined based on the absolute angular difference between the orientations, the dot product of the two orientation axes, and / or the like. In some examples, the difference between the upward viewing vector before the arm exchange and / or port jump and the upward viewing direction when the imaging device is in the current simulated position and orientation can be determined based on the angular difference between the two upward viewing directions, the dot product of the unit axes corresponding to the two upward viewing directions, and / or the like. In some examples, the difference between the two visibility values can be zero when the visibilities are the same (both visible or both invisible), or the difference between the two visibility values can be one when the visibilities are different (one visible and the other invisible).
[0091] At flow 930, an objective function is applied to the differences determined during flow 920. In some examples, the objective function can include calculating a weighted sum of each of the differences. In some examples, each weight in the weighted sum can be selected based on which of the representative position and orientation is considered more important in finding two similar views. In some examples, the weight can be set to zero to ignore one of the representative position and / or orientation in the view comparison. In some examples, one or more other objective functions can be used.
[0092] At process 940, the effect of the incremental motion of each joint controlling the imaging device on the objective function is determined. For each joint in the imaging device (e.g., each joint in the articulated wrist of the imaging device) and each joint in the repositionable arm to which the imaging device is mounted, the incremental motion of the joint is simulated. As the incremental motion of each joint is simulated, the change in the incremental motion on the imaging device view is determined by the simulated motion. Once the incremental motion of the imaging device is simulated, the difference in the representative position and orientation due to the incremental motion relative to the imaging device is determined. The objective function is then applied to the difference (e.g., by repeating a process similar to processes 910-930).
[0093] At process 950, the gradient in joint space is determined by combining the effects of each of the incremental motions on the objective function, as determined during process 940. In some examples, numerical interpolation / extrapolation methods (e.g., use of mean differences) may be used to combine the effects of each of the incremental motions on the objective function to determine the gradient.
[0094] At process 960, movement of the imaging device along the negative gradient is simulated. In some examples, each of the joints controlling the imaging device is simulated to change its position in proportion to the coefficient of the gradient of the joint, but in the direction opposite to the gradient. In some examples, the amount of simulated movement can also be scaled by the coefficient. In some examples, movement along the negative gradient tends to reduce the cumulative difference between the view of the imaging device before the arm swap and / or port jump and the view of the imaging device with each successive movement through process 960.
[0095] At process 970, it is determined whether the objective function of the difference between the view of the imaging device before the arm swap and / or port jump and the imaging device view of the current simulated position and orientation of the imaging device has converged. In some examples, the objective function of the difference converges when the change in the objective function of the difference between consecutive simulated movements through process 960 is below a threshold. In some examples, the objective function of the difference converges when the magnitude of the gradient is below a threshold (e.g., the distance of the movement simulated by process 960 is below a threshold). When the objective function of the difference does not converge, processes 910-970 are repeated using the position and orientation of the imaging device after the simulated movement as the current simulated position and orientation of the imaging device. When the objective function of the difference converges, the view of the imaging device is restored using process 980.
[0096] At process 980, the imaging device is moved to a current simulated position and orientation of the imaging device. In some examples, process 980 can be substantially similar to process 340 and / or method 700, where the current simulated position and orientation of the imaging device is a desired position and orientation of the imaging device.
[0097] As discussed above and further emphasized herein, Fig. 9 This is merely an example and should not unduly limit the scope of the claims. One of ordinary skill in the art will recognize many variations, substitutions, and modifications. According to some embodiments, method 900 may be terminated when convergence of the objective function of the difference is not detected after a threshold number of iterations. In some examples, a counter that increments with each loop through process 910-970 may be used to determine the number of iterations. In some examples, when method 900 is terminated due to lack of convergence, an error indication or other indication may be made to an operator indicating that automatic view recovery cannot be completed. In some examples, when method 900 is terminated due to lack of convergence, view recovery may be performed manually by an operator.
[0098] According to some embodiments, method 900 may take into account other practical considerations. In some examples, the incremental motion considered by process 940 or included via gradients in processes 950 and / or 960 may be limited due to a range of motion limit of a joint controlling the imaging device to perform collision avoidance and / or similar measures. In some examples, the range of motion limit of the joint may be a physical range of motion limit and / or a software defined range of motion limit that is limited before the physical range of motion limit is reached.
[0099] Some examples of control units (e.g., control unit 130) may include non-transitory, tangible, machine-readable media that include executable code that, when executed by one or more processors (e.g., processor 140), may cause the one or more processors to perform the processes of methods 300, 500, 600, 700, 800, and / or 900. Some common forms of machine-readable media that may include the processes of methods 300, 500, 600, 700, 800, and / or 900 are, for example, floppy disks, diskettes, hard disks, magnetic tapes, any other magnetic media, CD-ROMs (Compact Disc Read Only Drives), any other optical media, punch cards, paper tapes, any other physical media with a pattern of holes, RAMs (Random Access Memory), PROMs (Programmable Read Only Memory), EPROMs (Erasable Programmable Read Only Memory), FLASH-EPROMs (Flash Programmable Read Only Memory), any other memory chip or cartridge, and / or any other media from which a processor or computer is suitable for reading.
[0100] Although illustrative embodiments have been shown and described, a wide range of modifications, variations, and substitutions are contemplated in the foregoing disclosure, and in some cases, some features of the embodiments may be employed without the corresponding use of other features. Those of ordinary skill in the art will recognize many variations, substitutions, and modifications. Accordingly, the scope of the present invention should be limited only by the following claims, and it is appropriate to interpret the claims broadly in a manner consistent with the scope of the embodiments disclosed herein.
Claims
1. A computer-aided device comprising: Imaging equipment; as well as A controller is coupled to the imaging device and configured to: recording kinematic information, imaging information, or both the kinematic information and the imaging information of the computer-assisted device before the imaging device is moved from a first repositionable arm to a second repositionable arm or from a first workspace port to a second workspace port; detecting said movement of said imaging device from said first repositionable arm to said second repositionable arm or from said first workspace port to said second workspace port; In response to the detecting, determining a desired position and orientation of the imaging device based on the recorded kinematic information, the recorded imaging information, or both the recorded kinematic information and the recorded imaging information; and The imaging device is moved based on the desired position and orientation.
2. A computer-assisted device according to claim 1, wherein the desired position and orientation minimizes the difference between a first view of the imaging device before the movement of the imaging device from the first repositionable arm or the first workspace port and a second view of the imaging device after the movement of the imaging device to the second repositionable arm or the second workspace port.
3. The computer-assisted device of claim 2, wherein the first view is determined based on one or more of: A viewing direction of the imaging device; an upward viewing direction of the imaging device, The center of view of the imaging device, the depth of the point of interest or the center of the view from the imaging device; the location of a distal portion of each of the one or more instruments; or One or more of visibility, location, or orientation of one or more points of interest.
4. The computer-assisted device of claim 1, wherein the imaging device is an endoscope.
5. The computer-assisted device of claim 1, wherein the controller is further configured to determine a viewing direction of the imaging device based on the recorded imaging information. 6 . The computer-assisted device of claim 1 , wherein the controller is further configured to determine an upward viewing direction of the imaging device based on recorded imaging information or a gravity direction.
7. The computer-assisted device of claim 1, wherein the controller is further configured to determine a depth between a point of interest and the imaging device.
8. The computer-assisted device of claim 1, wherein the controller is further configured to determine a center of view of the imaging device.
9. The computer-assisted device of claim 8, wherein to determine the center of view of the imaging device, the controller is configured to: determining a representative point or bounding volume for each of a plurality of instruments; and The view center is determined based on a set of each representative point or each bounding volume.
10. The computer-assisted apparatus of claim 9, wherein to determine the set, the controller is configured to determine a center of mass.
