Head movement control for observation systems

Abstract

A computer-assisted medical system includes a display unit configured to provide an image to an operator of the display unit; a headrest configured to receive mechanical input provided by the operator's head in mechanical contact with the headrest; a headrest sensor interfaced with the headrest and configured to provide a sensor signal based on the mechanical input; and a controller. The controller includes a computer processor and is configured to: process the sensor signal to obtain a drive input; drive a virtual mass via the drive input to obtain simulated virtual mass movement; and cause movement of the headrest, the movement of the headrest tracking the movement of the virtual mass.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of priority under 35 U.S. SC §119(e) to U.S. Provisional Patent Application Serial No. 62 / 891,229, filed on August 23, 2019, which is incorporated herein by reference in its entirety. Technical Field

[0003] The present invention generally provides improved robotic and / or medical (including surgical) devices, systems, and methods. Background Art

[0004] The system of robotic devices can be used to perform tasks at a workplace. An operator can observe images of the workplace obtained by an imaging device (e.g., an endoscope, an optical camera, or an ultrasound probe). The images can enable the operator to monitor and / or perform tasks using visual feedback from the workplace provided to the operator by a display unit.

[0005] The imaging device may be controllable to update the view of the workplace. The imaging device may be attached to a robotic manipulator, and the robotic manipulator may include two or more links coupled together by one or more joints. The joints may be moved to update the position and / or orientation of the imaging device in the workplace. The movement of the imaging device may be controlled by an operator, thereby enabling the operator to change the view of the workplace as needed or desired to perform a task.

[0006] Robotic systems equipped with display units that provide visual feedback to the operator include industrial and entertainment robotic systems. Robotic systems also include medical robotic systems used in operations such as diagnosis, non-surgical treatment, surgical treatment, etc. As a specific example, robotic systems include minimally invasive robotic telesurgery systems, in which surgeons can perform surgery on patients at the bedside or at a remote location. Telesurgery generally refers to surgical operations performed using a surgical system in which the surgeon uses some form of remote control, such as a servo mechanism, to manipulate the movement of surgical instruments rather than directly holding and moving the instruments with their hands. Robotic medical systems that can be used for telesurgery or other remote medical operations may include remotely controllable robotic manipulators. One or more robotic manipulators may be equipped with an imaging device to provide visual feedback to the operator via a display unit. The operator can remotely control the movement of one or more remotely controllable robotic manipulators. Summary of the Invention

[0007] Generally speaking, in one aspect, one or more embodiments relate to a computer-assisted medical system comprising: a display unit configured to provide an image to an operator of the display unit; a headrest configured to receive mechanical input provided by a head of an operator in mechanical contact with the headrest; a headrest sensor interfaced with the headrest and configured to provide a sensor signal based on the mechanical input; a controller comprising a computer processor configured to: process the sensor signal to obtain a drive input; drive a virtual mass via the drive input to obtain a simulated virtual mass movement; and cause movement of the headrest, the movement of the headrest tracking the virtual mass movement.

[0008] In general, in one aspect, one or more embodiments are directed to a method for operating a medical system. The method includes obtaining a sensor signal from a headrest sensor, wherein the headrest sensor interfaces with a headrest, the headrest being configured to receive a mechanical input provided by an operator's head, the head being in mechanical contact with the headrest, and wherein the sensor signal is based on the mechanical input; processing the sensor signal to obtain a drive input; driving a virtual mass via the drive input to obtain simulated virtual mass movement; and causing movement of the headrest, the movement of the headrest tracking the virtual mass movement.

[0009] Generally speaking, in one aspect, one or more embodiments relate to a non-transitory machine-readable medium comprising a plurality of machine-readable instructions for execution by one or more processors associated with a medical system, the plurality of machine-readable instructions causing the one or more processors to perform a method comprising: obtaining a sensor signal from a headrest sensor, wherein the headrest sensor interfaces with a headrest, the headrest being configured to receive a mechanical input provided by an operator's head, the head being in mechanical contact with the headrest, and wherein the sensor signal is based on the mechanical input; processing the sensor signal to obtain a drive input; driving a virtual mass via the drive input to obtain a simulated virtual mass movement; and causing movement of the headrest, the movement of the headrest tracking the virtual mass movement.

[0010] Other aspects of the invention will become apparent from the following description and appended claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 An example teleoperated surgical system is shown in accordance with one or more embodiments.

[0012] Figure 2 A display system is shown in accordance with one or more embodiments.

[0013] Figure 3 A control architecture for controlling movement of a display unit is shown in accordance with one or more embodiments.

[0014] Figure 4 A flow chart describing a method for head movement control of a display unit according to one or more embodiments is shown.

[0015] Figure 5A and Figure 5B An operator's head interacting with a head input device of a display unit is shown according to one or more embodiments.

[0016] Figure 6 A flow chart describing a method for sensing mechanical input from an operator's head is shown, according to one or more embodiments.

[0017] Figure 7 A flow chart describing a method of simulating movement using a virtual dynamics model driven by sensed mechanical input is shown in accordance with one or more embodiments.

[0018] Figure 8 A flow chart describing a method for causing movement of a headrest portion of a display unit to follow movement generated by a virtual dynamics model is shown, according to one or more embodiments. DETAILED DESCRIPTION

[0019] Now will be described in detail with reference to the accompanying drawings the specific embodiments of the present invention. For consistency, the same elements in various figures are represented by the same reference numerals.

[0020] In the following detailed description of embodiments of the present invention, numerous specific details are set forth to provide a more thorough understanding of the present invention. However, it will be apparent to one of ordinary skill in the art that the present invention may be practiced without these specific details. In other instances, well-known features have not been described in detail to avoid unnecessarily complicating the description.

[0021] Throughout the application, ordinal numbers (e.g., first, second, third, etc.) may be used as adjectives for elements (i.e., any noun in the application). The use of ordinal numbers does not imply or create any particular ordering of the elements, nor does it limit any element to being only a single element, unless explicitly disclosed, such as by the use of the terms "before," "after," "single," and other such terms. Rather, the use of ordinal numbers is to distinguish elements. As an example, a first element is different from a second element, and a first element may contain more than one element and be after (or before) a second element in the ordering of the elements.

[0022] Although some of the examples described herein relate to surgical procedures or tools, or medical procedures and medical tools, the disclosed technology is applicable to medical and non-medical procedures, as well as medical and non-medical tools. For example, the tools, systems, and methods described herein can be used for non-medical purposes, including industrial uses, general robotic uses, and sensing or manipulating non-tissue artifacts. Other example applications relate to cosmetic improvements, imaging of human or animal anatomical structures, collecting data from human or animal anatomical structures, assembling or dismantling systems, and training medical or non-medical personnel. Additional example applications include performing surgery on tissue removed from a human or animal anatomical structure (without returning it to the human or animal anatomical structure), and performing surgery on human or animal corpses. In addition, these technologies can also be used for medical treatment or diagnostic procedures that include or do not include surgical aspects.

[0023] Generally speaking, embodiments of the present disclosure can facilitate the use of robotic systems and improve workflows under various conditions. A robotic system may include an imaging device that enables an operator to observe the workspace. The position and / or orientation of the imaging device can be controlled by the operator. In one or more embodiments of the present disclosure, the operator controls the movement of the imaging device using head movements. The operator's head may be in contact with a headrest, and head movements may be recorded by a headrest sensor coupled to the headrest. Signals obtained from the headrest sensor can be used to update the position and / or orientation of the imaging device. If a fixed display unit displays images from the imaging device to the operator, the operator's head movement may cause misalignment between the operator's eyes and the display unit. This misalignment can reduce the visual information available to the operator, particularly when the display unit provides stereoscopic visual information. Therefore, in one or more embodiments, the operator's head movement causes compensatory movement of the display unit, causing the display unit to remain aligned with the operator's eyes. This compensatory movement can be performed while taking into account human anatomy, thereby ensuring ergonomic and effortless control while avoiding operator fatigue.

[0024] Reference is now made to the drawings, wherein like reference numerals refer to like parts throughout the several views. Figure 1 1 is a schematic diagram of an example teleoperation system (100) that can be used with one or more features disclosed herein and can be a surgical system. As shown, the teleoperation system (100) can include a master control subsystem (which can be in the form of a workstation (e.g., a console) (102)) and a slave device (104).

[0025] In this example, a master control workstation (102) includes one or more master control devices that are contacted and manipulated by an operator's hands, e.g., one master control device for each hand. The master control devices are supported by the workstation (102) and may be mechanically grounded. In some embodiments, an ergonomic support (110) (e.g., a forearm rest) may be provided on which the operator (108) may rest his or her forearm while grasping the master control device. In some examples, the operator (108) may perform tasks at a workstation near the slave device (104) during a medical procedure by controlling the slave device (104) using the master control device.

[0026] A display unit (112) is included in the workstation (102). The display unit (112) can display images for observation by the operator (108). The display unit (112) can move with various degrees of freedom to adapt to the operator's viewing position and / or provide control functions. In an example of a remote operation system (100), the displayed images can depict a workplace where the operator performs various tasks via control of a master control device. In some examples, the images displayed by the display unit (112) can be received by the workstation (102) from one or more image capture devices disposed at the remote workplace. In other examples, the images displayed by the display unit can be generated by the display unit (or by another connected device or system), such as for a virtual representation of a tool, a workplace, or for a user interface component.

[0027] When using the workstation (102), the operator (108) can sit in a chair or other support in front of the workstation (102), place his or her eyes in front of the display unit (112), grasp and manipulate the main control device, and lean his or her forearms on the ergonomic support (110) as needed. In some embodiments, the operator can stand at the workstation or assume other postures, and the position (height, depth, etc.) of the display unit (112) and main control device can be adjusted to suit.

[0028] The teleoperation system (100) may also include a slave device (104) that can be controlled by the master control workstation (102). In a medical example, the slave device (104) is located near an operating table (106) (e.g., a table, bed, or other support) on which a patient can be placed. A workspace (130) may be provided on or in the operating table (106), such as on or in a patient, simulated patient, or model (not shown). The illustrated teleoperation slave device (104) includes a plurality of manipulator arms (120), each configured to be coupled to an instrument assembly (122). The instrument assembly (122) may include, for example, an instrument (126) and an instrument holder (not shown) configured to hold the instrument (126).

[0029] In various embodiments, one or more of the instruments (126) may include an image capture device (e.g., a camera), such as a camera included in the endoscope assembly (124), which may provide a captured image of a portion of the workspace for display by the display unit (112) of the workstation (102) for output.

[0030] In some embodiments, the slave manipulator arm (120) and / or instrument assembly (122) can be controlled to move and articulate the instrument (126) in response to manipulation of the master control device by the operator (108) so that the operator (108) can perform tasks at the workspace (130). For a surgical example, the operator can direct a surgical procedure at an internal surgical site through a minimally invasive surgical port.

[0031] In some embodiments, a control system is provided in a master control workstation (102) or external to the workstation (102), and the control system communicates with the workstation (102). As the operator (108) moves the master control device(s), sensed spatial information and sensed orientation information are provided to the control system based on the movement of the master control device(s). The control system can determine or provide control signals to the slave device (104) to control the movement of the arm (120), the instrument assembly (122), and the instrument (126) based on the received information and user input. In one embodiment, the control system supports one or more wireless communication protocols, such as Bluetooth, IrDA, HomeRF, IEEE802.11, DECT, and wireless telemetry.

[0032] continue Figure 1 The control system may be implemented on one or more computing systems (not shown). The one or more computing systems may be used to control the slave device (104). Additionally, the one or more computing systems may be used to control components of the master control workstation (102), such as movement of the display unit (112), in response to the operator's head movement.

[0033] A computing system may include one or more computer processors, non-persistent storage (e.g., volatile memory such as random access memory (RAM), cache memory), persistent storage (e.g., a hard disk, an optical drive (e.g., a compact disc (CD) drive or a digital versatile disk (DVD) drive), flash memory, etc.), communication interfaces (e.g., a Bluetooth interface, an infrared interface, a network interface, an optical interface, etc.), and many other elements and functions.

[0034] A computer processor of a computing system may be an integrated circuit for processing instructions. For example, a computer processor may be one or more cores or microcores of a processor. A computing system may also include one or more input devices, such as a touch screen, keyboard, mouse, microphone, touchpad, electronic pen, or any other type of input device.

[0035] The communication interface of the computing system may include an integrated circuit for connecting the computing system to a network (not shown) (e.g., a local area network (LAN), a wide area network (WAN) (e.g., the Internet), a mobile network, or any other type of network) and / or another device (e.g., another computing system).

[0036] In addition, a computing system may include one or more output devices, such as a display device (e.g., a liquid crystal display (LCD), a plasma display, a touch screen, an organic LED display (OLED), a projector, or other display device), a printer, a speaker, an external storage device, or any other output device. The one or more output devices may be the same as or different from the input device(s). Many different types of computing systems exist, and the aforementioned input and output device(s) may take other forms.

[0037] The software instructions in the form of computer-readable program code for performing embodiments of the present disclosure may be stored, in whole or in part, temporarily or permanently, on a non-transitory computer-readable medium, such as a CD, DVD, storage device, disk, tape, flash memory, physical memory, or any other computer-readable storage medium. Specifically, the software instructions may correspond to a computer-readable program code that, when executed by a processor(s), is configured to perform one or more embodiments of the present invention.

[0038] continue Figure 1 , a computing system may be connected to a network or be part of a network. The network may include multiple nodes. Each node may correspond to a computing system, or a group of nodes. As an example, embodiments of the present disclosure may be implemented on a node connected to other nodes in a distributed system. As another example, embodiments of the present invention may be implemented on a distributed computing system having multiple nodes, wherein each part of the present disclosure may be located on a different node within the distributed computing system. In addition, one or more elements of the aforementioned computing system may be located at a remote location and connected to the other elements via a network.

[0039] In some embodiments, the display unit (112) can be operated by an operator in conjunction with operation of one or more ungrounded master control devices (ungrounded master control devices are not kinematically grounded, such as master control devices held by an operator's hand without additional physical support). In some embodiments, the operator can use the display unit (112) positioned near a workstation so that the operator can manually operate instruments at the workstation, such as laparoscopic instruments in surgical procedures, while viewing images displayed by the display unit (112).

[0040] Some embodiments may include one or more components of a remotely operated medical system, such as the da Surgical system. The implementation on the surgical system is only an example and is not to be considered as limiting the scope of the features disclosed herein.For example, different types of teleoperated systems and non-teleoperated systems with slave devices at the workplace can utilize the features described herein.

[0041] Figure 2 is a perspective view of an example display system (200) according to one or more embodiments. In one or more embodiments, the display system (200) is displayed in a master control workstation of a remote operating system (e.g., Figure 1 The system may be used in a master control workstation (102) of a remote operation system (100), or may be used in other systems or as a standalone system, for example, to allow an operator to observe a workplace or other physical location, a displayed virtual environment, etc.

[0042] The display system (200) includes a base support (202), an arm support (204), and a display unit (206). As described in more detail below, the display unit (206) is provided with multiple degrees of freedom of movement provided by a support linkage, the support linkage including the base support (202), the arm support (204) coupled to the base support (202), and a tilt member (224) (described below) coupled to the arm support (204), wherein the display unit (206) is coupled to the tilt member.

[0043] The base support (202) can be a vertical member that is mechanically grounded (e.g., directly or indirectly coupled to the ground). For example, the base support (202) can be mechanically coupled to a support structure (210) that is coupled to the ground to provide stability for the base support (202). The base support (202) includes a first base portion (212) and a second base portion (214) that are coupled such that the second base portion (214) can translate with linear degrees of freedom relative to the first base portion (212). Other embodiments may use different configurations.

[0044] The arm support (204) can be a horizontal member mechanically coupled to the base support (202). The arm support (204) includes a first arm portion (218) and a second arm portion (220). The second arm portion (220) is a distal portion of the arm support (204) coupled to the first arm portion (218) such that the second arm portion (220) can translate linearly relative to the first arm portion (218). Other embodiments may use different configurations.

[0045] In other embodiments, the arm support (204) may extend at various heights and / or configurations, such as below the operator's head or body, at the operator's head height, behind the operator and circling around the operator, etc.

[0046] continue Figure 2 The display unit (206) can be mechanically coupled to the arm support portion (204). The display unit (206) can move within two linear degrees of freedom provided by the linear translation of the second base portion (214) and the second arm portion (220).

[0047] In one or more embodiments, the display unit (206) includes a display device capable of displaying digital images, such as one or more display screens, projectors, or other display devices. The display unit (206) may include two viewing ports (223), wherein the display device is provided behind or included in the viewing ports. One or more display screens or other display devices may be located on the display unit (206) in place of the viewing ports (223).

[0048] In one or more embodiments, the display unit (206) displays an image of the surgical site captured by an imaging device such as an endoscope. The surgical site may alternatively be a virtual representation of the surgical site. The image may display a captured image or a virtual rendering of instruments (126) from the device (104), one or more of which are controlled by an operator via a master control device at the master control workstation (102). The image may also include information such as status information, alarms and warnings, notifications, etc. Such information may be displayed in conjunction with a view of the workspace or without a view of the workspace.

[0049] In one or more embodiments, the display unit (206) is rotationally coupled to the arm support (204) via a tilting member (224). In this example, the tilting member (224) is coupled at a first end to the second arm portion (220) of the arm support (204) via a rotational coupling configured to provide rotational movement of the tilting member (224) and the display unit (206) relative to the second arm portion (220) about a tilt axis (226). In one or more embodiments, the tilt axis (226) is located above a display device in the display unit (206). In one or more embodiments, the tilt axis (226) is located above the position of the operator's head when the operator operates the display unit (206).

[0050] continue Figure 2 Each of the various degrees of freedom discussed herein can be passive and require manual manipulation, or can be moved by one or more actuators, such as by one or more motors, solenoids, etc. For example, rotational movement of the tilt member (224) and display unit (206) about axis (226) can be driven by one or more actuators, such as a motor coupled to the tilt member at or near the tilt axis (226). The base support (202), arm support (204), and tilt member (224) can be considered to be a support link with the display unit (206) coupled at the distal end of the support link.

[0051] The display unit (206) can be rotatably coupled to the tilt member (224) and can rotate about a yaw axis (230). For example, this can be a lateral or left-right rotation from the perspective of an operator viewing an image of the display unit (206) via the viewing port (223). In this example, the display unit (206) is coupled to the tilt member via a rotation mechanism that can be a track mechanism. For example, in some embodiments, the track mechanism includes a curved track (228) that slidably engages a groove member (229) coupled to the tilt member (224), thereby allowing the display unit (206) to rotate about the yaw axis (230) by moving the curved track (228) through a groove of the groove member (229). In some embodiments, the curved track is coupled to the tilt member (224) and the groove member is coupled to the display unit (206). In some embodiments, the curved track (228) can be a curved cam follower that engages a cam roller.

[0052] The curvature (e.g., radius) of the curved track (228) and / or groove member provides a yaw axis (230) at a specific distance from the operator side of the display unit (206) and / or from the tilt axis (226). For example, this can be a specific horizontal distance parallel to the degree of freedom (222) of the second arm portion (220). For example, the yaw axis (230) can be set at a certain distance so that it approximately intersects a defined (e.g., virtual or software-defined) neck pivot axis corresponding to a pivot axis in the operator's neck. In some embodiments, the defined neck pivot axis can be used as a reference for movement of the display unit (206).

[0053] continue Figure 2 , the display system (200) can thus provide the display unit (206) with vertical linear degrees of freedom (216), horizontal linear degrees of freedom (222), rotational (tilt) degrees of freedom (227), and rotational yaw degrees of freedom (231). Coordinated movement of the components of the combined display system (200) in these degrees of freedom allows the display unit (206) to be positioned at various positions and orientations in its workspace. Movement of the display unit (206) in the tilt, horizontal, and vertical degrees of freedom allows the display unit (206) to remain close to the operator's head and eyes during movement of the operator's head, and / or maintain a physical connection between the operator's head (e.g., forehead) and the display unit (206).

[0054] For example, the display unit (206) is positionable (e.g., translatable and / or rotatable) in its workspace so that the operator's eyes are aligned with the viewing port of the display unit. Furthermore, the display unit (206) can be rotated in physical space about a defined eye pivot axis, which corresponds to, for example, an eye axis passing through the operator's two eyes to allow for a desired vertical (e.g., up and down) eye viewing angle and a desired yaw (e.g., left and right) viewing angle for the operator.

[0055] The degrees of freedom of the display system allow the display system (200) to provide pivotal movement of the display unit (206) in physical space about a pivot axis that can be located at different positions. For example, the system (200) can provide movement of the display unit (206) in physical space that corresponds to movement of the operator's head when operating the display system (200). The movement can include rotation about a defined neck pivot axis that roughly corresponds to a neck axis of the operator's head at the operator's neck. The rotation allows the display unit (206) to move in accordance with the operator's head that directs the movement of the display unit (206). In another example, the movement can include rotation about a defined forehead pivot axis that roughly corresponds to a forehead axis extending through the operator's head at the forehead when the display unit (206) is oriented about the yaw axis (230) in a center yaw rotation position as shown.

[0056] The display unit (206) may include an input device that allows an operator to provide input to manipulate the orientation and / or position of the display unit (206) in space, and / or to manipulate other functions or components of the display system (200) and / or a larger system (e.g., a teleoperation system).

[0057] The display unit (206) may include a head input device (242). In one or more embodiments, the head input device (242) is located on a surface of the display unit (206) that faces the operator's head during operation of the display unit (206).

[0058] The head input device (242) can be shaped to form a headrest portion that can contact the operator's head. More specifically, the head input device (242) can be located in an area above the viewing port (223) to contact the operator's forehead when the operator observes the image through the viewing port (223). The head input device (242) can include one or more head input sensors that sense the operator's head input received as a command to cause movement of the imaging device, thereby updating the view in the image presented to the operator. In addition, in one or more embodiments, the sensed head movement is used to move the display unit (206) to compensate for the head movement. Therefore, even when the operator performs head movement to control the view provided by the imaging device, the operator's head position can remain stationary relative to the viewing port (223). Therefore, it can be ensured that the operator's eyes are properly aligned with the viewing port.

[0059] In one or more embodiments, sensing the operator's head input includes sensing the presence or contact of the operator's head or a portion of the head (e.g., forehead) with the head input device (242). The one or more head input sensors may include any of a variety of types of sensors, such as a resistive sensor, a capacitive sensor, a force sensor, an optical sensor, etc.

[0060] continue Figure 2 The orientation and / or position of the display unit (206) can be changed by the display system (200) based on the operator's head input to the head input device (242). For example, the sensed operator input is provided to a control system that controls an actuator of the display system (200) to move the second base portion (214) with a linear degree of freedom (216), move the second arm portion (220) with a linear degree of freedom (222), move the tilt member (224) with a rotational degree of freedom (227), and / or move the display unit (206) with a rotational degree of freedom (231) so that the display unit (206) moves in accordance with (e.g., based on) the command of the sensed operator head input. The sensed operator head input can also be used to control the display system (200) and / or a larger system (e.g., Figure 1 Thus, in some embodiments, the operator can move his or her head to provide input to the input device to control the display unit (206) to be moved by the display system according to the movement of the head, thereby allowing the display unit to follow the movement of the operator's head and changes in viewing angle.

[0061] In some embodiments, images displayed by the display unit (206) and / or other controlled devices are changed and manipulated based on sensed motion of the display unit (206).

[0062] In some embodiments of the display system, the display unit (206) can be rotated about the yaw axis (230) in one or more of the degrees of freedom (231) and the other degrees of freedom (216), (222), and (227), and omitted from the display system (200). For example, the display unit (206) can be rotated about the yaw axis (230) (e.g., by actuator(s) and / or manually by an operator) and the display unit (206) can be manually positioned higher and / or lower (e.g., by actuator(s) and / or manually by an operator) (e.g., using the base support (202) or other mechanism), wherein the horizontal degree of freedom (222) and / or the tilt degree of freedom (227) are omitted.

[0063] Those skilled in the art will understand Figure 2Only examples of configurations of the display system (200) are shown. Alternative configurations that support movement of the display unit (206) based on operator input to the head input device (242) may be used without departing from the scope of this disclosure. Any linkage that supports the desired movement of the display unit (206) may be used in place of Figure 2 The configuration shown in . Figure 3 、 Figure 4 、 Figure 5A 、 Figure 5B 、 Figure 6 、 Figure 7 and Figure 8 A detailed description is provided of using signals captured at a head input device (242) to cause movement of a display unit (206).

[0064] Go to Figure 3 , shows a control architecture (300) for controlling the movement of a display unit according to one or more embodiments of the present disclosure. Figure 3 A high-level introduction to controlling movement of a display unit based on detected head movement is provided, but the flowchart discussed subsequently provides a description of the steps that may be performed. Figure 3 A key aspect of the control architecture is that the dynamics of moving the display unit are decoupled from the force inputs provided by the operator. More specifically, the force inputs made by the operator are used to simulate driving a virtual dynamics model. The display unit can then be driven to follow the virtual movements generated by the virtual dynamics model. Therefore, assuming that the movement of the display unit reflects the movement simulated by the virtual dynamics model, the dynamics of the display unit are transparent to the operator, while the dynamics experienced by the operator are instead governed by the virtual dynamics model. Therefore, the operator's experience of how head movement results in movement of the display unit (and updating of the view provided by the display unit) can be adjusted as needed by changing the characteristics of the virtual dynamics model.

[0065] exist Figure 3 In the control architecture of , the head position or changed head position (i.e., head movement) is considered as an input acting on the head input device. The interaction of the head with the head input device results in a force acting between the head and the head input device. The force is recorded by the head input sensor. Depending on the distance between the contact point between the head and the head input device and the position of the head input sensor, the force may result in different types of sensor signals. Specifically, as shown below Figure 4As shown, various forces and / or torques in various directions can be recorded for head movements in the horizontal and vertical planes. The head input sensor signals can be used to drive a virtual dynamics model. The virtual dynamics model can include a virtual mass that can be simulated and driven by the sensor signals, resulting in acceleration of the virtual mass. The resulting movement of the simulated mass can manipulate the movement of the display unit. Specifically, the velocity of the virtual mass can be used as the desired velocity input for an inverse kinematics model representing the linkage of the display unit. The output of the inverse kinematics model can be a joint velocity, which can be used to drive the actuators of the joints, thereby causing the display unit to move as specified by the simulated movement of the virtual mass. The movement of the display unit results in a change in the position of the head input device. In one or more embodiments, the control architecture is designed to compensate for the operator's head movement. Thus, the control architecture can cause the display unit to "follow" the head movement to reduce the offset between the contact point and the center of the head input device. The flowchart discussed below provides detailed information on possible implementations of the control architecture.

[0066] Go to the flowchart, Figure 4 A flow chart is shown according to one or more embodiments. Figure 4 The flowchart depicts a method for controlling movement of an observation system using head movements. The observation system may include the previously described imaging device and the previously described display unit of the user-controlled system. Head movements of an operator interacting with the display unit may cause movement of the imaging device. Furthermore, according to one or more embodiments, head movements cause movement of the display unit to maintain alignment between the display unit and the operator's head. Figure 4 One or more steps in the system may be performed by various components of the system, as previously described. Figure 1 and Figure 2 These figures depict a specific system configuration as an example. However, the methods described subsequently are not limited to a specific system configuration. Rather, these methods are applicable to any type of system that includes an imaging device that allows updating of a view provided by the imaging device paired with a movable display unit. Figure 4 The method can be used for head movement input in the horizontal plane and / or the vertical plane. Additional discussion of control in the horizontal and vertical planes is provided below.

[0067] Although the various steps in these flow charts are presented and described in sequence, one of ordinary skill will understand that some or all of the steps may be performed in a different order, may be combined or omitted, and may be performed in parallel. Additional steps may be further performed. In addition, the steps may be performed actively or passively. For example, according to one or more embodiments of the present invention, some steps may be performed using polling or may be interrupt driven. As an example, according to one or more embodiments of the present invention, the determining step may not require the processor to process the instruction unless an interrupt is received indicating that the condition exists. As another example, according to one or more embodiments of the present invention, the determining step may be performed by performing a test, such as checking a data value to test whether the value is consistent with a test condition. Therefore, the scope of the present disclosure should not be considered limited to Figure 4 The specific arrangement of the steps shown.

[0068] Go to Figure 4 Flowchart showing a series of steps which, when repeated, may form a loop in which movement of the operator's head is detected and causes compensatory movement of the display unit.

[0069] In step 400, according to one or more embodiments, a mechanical input provided by the operator's head is sensed. The sensing may be performed using the head input sensor described previously. The sensing may be performed in a vertical plane (e.g., Figure 5A shown) and / or in the horizontal plane (as Figure 5B In step 400, the head input sensor may be used to obtain various sensor signals representing the mechanical input by the operator's head. Figure 6 A detailed description of step 400 is provided.

[0070] In step 402, according to one or more embodiments, movement is simulated using a virtual dynamics model. The simulated movement can be driven by the sensed mechanical input obtained in step 400. In one or more embodiments, the virtual dynamics model establishes the dynamics of moving the display unit regardless of the mechanical configuration of the display unit. Therefore, when using head movement to control the viewing system, the virtual dynamics model can impose the desired dynamics to be experienced by the operator. Therefore, the desired dynamics can be set to meet a variety of different needs. For example, one operator may prefer a highly responsive system that requires minimal head movement as a control input, while another operator may prefer a less responsive system, for example, to achieve particularly high accuracy. These characteristics can be achieved by, for example, modulating the gain applied to the sensed mechanical input, adjusting the inertia of the virtual dynamics model, etc. Reference below Figure 7 These and other aspects of step 402 are discussed in detail.

[0071] In step 404, according to one or more embodiments, the display unit and the head input device of the display unit are caused to follow the movement of the virtual dynamic model. Therefore, the movement of the display unit reflects the movement simulated by the virtual dynamic model, thereby making the original dynamics of the display unit transparent to the operator. The execution of step 404 can drive the display unit in a direction that follows the movement of the operator's head, thereby reducing the existing offset between the contact point and the head input device. Figure 8 A detailed description of steps that may be used to drive a display unit is provided.

[0072] Go to Figure 5A and Figure 5B , illustrating the interaction of an operator's head with a headrest portion of a display unit according to one or more embodiments. Figure 5A provides a side view of the interaction, while Figure 5B A top view of the interaction is provided.

[0073] Go to Figure 5A , showing an operator's head (570) and torso (572). In interaction (500), the operator's head (570) contacts the headrest (510) at contact point (530). In one or more embodiments, a headrest sensor (520) interfaced with the headrest (510) is configured to sense forces and / or torques caused by the interaction of the operator's head (570) with the headrest (510). In one embodiment, the headrest sensor includes sensing modalities for forces applied in multiple directions and torques applied in multiple directions. Figure 5A A Cartesian coordinate system (548) is shown that establishes a reference system relative to the ground. When viewed in conjunction with the operator (head (570), torso (572)), the x-direction can be understood as representing the left / right displacement of the operator's head, the y-direction can be understood as representing the forward / backward displacement of the operator's head, and the z-direction can be understood as representing the upward / downward displacement of the operator's head. Rotation about the x-direction can be understood as pitch movement, while rotation about the z-direction can be understood as yaw movement of the operator's head. Sensing by the headrest sensor (520) can also be performed in a Cartesian reference system with up to six degrees of freedom (three translations and three rotations). In some embodiments, the headrest sensor can be a sensor that provides measurements in fewer translational and / or rotational degrees of freedom, such as forces only in the x-direction, y-direction, and z-direction, and torques only about the x-direction and z-direction. The degrees of freedom of the headrest sensor are not necessarily aligned with the reference system established by the coordinate system (548). Figure 5A In the example of , the headrest sensor (520) has a pivot offset about the x-direction. Figure 5A In the example of FIG. 1 , the headrest sensor obtains a force F corresponding approximately to the in / out movement.I / O (544), roughly corresponding to the upward / downward movement force F U / D (542) and torque τ X (540)). Conversions between different reference frames can be performed at any time using trigonometric operations. Specifically, in the ground-based Cartesian reference frame F y (538) and F z (536) can be based on F I / O (544) and F U / D The combination of (542) is calculated directly.

[0074] exist Figure 5A An offset (546) is shown in the diagram of FIG. Although an offset is shown between the contact point (530) and the headrest sensor (520), in a real scenario, there may be an offset between the instantaneous contact point (530) and the initial contact point (not shown). The offset may be the result of the operator's head movement, causing the contact point (530) to move away from the initial contact point. Because the method described later causes the display unit (206) to move to compensate for the offset (546), the offset may be temporary.

[0075] In one or more embodiments, the headrest portion (510) has a known geometry. Therefore, the offset (546) in the vertical yz plane can be directly estimated using the force / torque balance equation. Figure 5A In the example, F I / O (544), F U / D (542), τ X (540) may be used to determine an offset (546).

[0076] For illustration purposes, the offset between the contact point and the center of the headrest (or original contact point) is large, as shown in FIG. Figure 5A The actual offset encountered when performing the methods described subsequently may be much smaller because these methods drive compensating movements that reduce the offset.

[0077] Go to Figure 5B , originally displayed in Figure 5A The scene in is shown as a top view. Therefore, Figure 5A The various components introduced in Figure 5B In the interaction (500), the operator's head (570) contacts the headrest (510) at the contact point (530). Figure 5B In the example, the headrest sensor records the force F in the y direction. y (584), which roughly corresponds to the in / out displacement of the operator's head. The headrest sensor further records the force F in the x direction. x(582), which roughly corresponds to the left / right displacement of the operator's head. In addition, the headrest sensor records the torque τ about the z direction z (590), which roughly corresponds to the yaw motion of the operator's head. Figure 5B , the degrees of freedom of the headrest sensor are shown as aligned with the reference frame established by the coordinate system (548), but the sensing degrees of freedom of the headrest sensor may alternatively be in a reference frame having a rotational offset relative to the reference frame established by the coordinate system (548), for example, when including the headrest sensor (520) Figure 2 When the display unit (206) has a yaw offset, standard trigonometric operations can be used to convert between reference frames.

[0078] When installed Figure 2 A and Figure 2 When the display unit (206) of B is on, the headrest portion can be centered above the observation port (223). Figure 3 In the illustration of FIG, there is an offset between the contact point (330) and the headrest sensor (320). Although an offset is shown between the contact point (530) and the headrest sensor (520), in actual scenarios, there may be an offset between the instantaneous contact point (530) and the initial contact point (not shown). The initial contact point is not necessarily aligned with the headrest sensor (520). The offset may be the result of the operator's head movement. Because the method described later causes the display unit (206) to move to compensate for the offset, the offset may be temporary.

[0079] In one or more embodiments, the headrest (510) has a known curvature. Thus, whenever the operator's head (570) is not aligned with the center of the headrest at the initial contact point, the offset x in the x direction is offset (552) and the offset y in the y direction offset (554) may exist. Therefore, the force F (580) applied by the operator's head (570) results in the equilibrium equation x offset F y -y offset F x =τ z Among them F x (582), F y (584) represents the components of the force F (580) in the x and y directions, respectively, and τ z (590) is the opposite torque. The F provided by the headrest sensor (520) x (582), F y (584) and τ z (590), and x based on the known curvature of the headrest offset and y offsetKnowing the relationship between , the contact point in the horizontal xy plane can be determined (530).

[0080] Although the above description discusses Figure 5A and Figure 5B The offset of the headrest configuration shown in FIG can also be calculated for headrests with different geometries, different placement of head sensors, etc. without departing from the present disclosure. Those skilled in the art will understand that the calculation of the offset involves a force-torque balance equation that can be adapted based on the configuration used. Different configurations may require different trigonometric operations and / or force-torque balance equations.

[0081] Figure 5B Also shown is the calculated shear force F shear (586) and the calculated normal force F normal (588), which are later used to control the movement of the display unit (206). shear (586) is parallel to the surface of the headrest portion (510) at the contact point (530), and F normal (588) is perpendicular to the surface of the headrest portion (510) at the contact point (530). shear and F normal Calculations can be performed on surfaces of any shape.

[0082] For illustrative, non-limiting purposes, the offset between the contact point and the center of the headrest (or initial contact point) is large, such as Figure 5B The actual offset encountered when performing the methods described subsequently may be much smaller because these methods drive compensating movements that reduce the offset.

[0083] Go to Figure 6 A flowchart illustrating a method for obtaining a sensor signal reflecting an operator's head movement according to one or more embodiments is provided. The method can be used to record the operator's head movement within a substantially horizontal plane. The method can also be used to record the operator's head movement within a substantially vertical plane.

[0084] In step 600, the geometry of the headrest is obtained. The x / y plane (i.e., the horizontal plane, as previously described) Figure 5B The headrest portion may be curved in the y / z plane (i.e., in the vertical plane, as previously shown). Figure 5AThe headrest portion (shown in FIG) can be straight or substantially straight. The geometry of the headrest portion can be stored in the form of a headrest portion model, such as a polygonal headrest portion model. In one embodiment of the present disclosure, a polygonal headrest portion model smoothed using a cubic spline function is used. Without departing from the present disclosure, any other model that establishes a relationship between the x and y coordinates (in the horizontal plane) and the y and z coordinates (in the vertical plane) of the headrest portion can be used.

[0085] In step 602, a sensor signal is obtained from the headrest sensor. In one embodiment of the present disclosure, the obtained sensor signal at least includes a value indicating F x 、F y 、F z , τ x and τ z Other sensor signals may be obtained without departing from the present disclosure. Although the sensor signals may be initially obtained in a headrest sensor-specific reference frame, triangulation may be used to convert the sensor signals to any other reference frame based on the known current position and / or orientation of the headrest sensor.

[0086] The sensor signal may be obtained in any format (eg, as a digital or analog readout) and may be converted to a format capable of reading out a measured force / torque corresponding to the actual force / torque encountered by the headrest sensor.

[0087] In step 604, the location of the contact point between the head and the headrest is determined according to one or more embodiments. The contact point can be in a vertical plane (e.g. Figure 5A as shown) and / or horizontal plane (as Figure 5B As shown). Figure 5B In the example configuration shown, the contact points in the horizontal plane can be represented using the relationship x offset F y -y offset F x =τ z To determine. You can use x offset and y offset To describe the position of the contact point relative to the headrest sensor. Since the relationship between the x and y coordinates is known due to the previously obtained geometry of the headrest, the contact point can be described using x 、F y and τ z The obtained sensor signal is used to solve xo ffset and y offset In one embodiment of the present disclosure, x offset and y offsetDescribes the location of the contact point relative to the center of the headrest, which may or may not correspond to the location of the headrest sensor. Figure 5A In the example configuration shown, the contact point in the vertical plane can be determined in a similar manner. However, in the vertical plane, the contact point can be obtained relative to the initially recorded contact point (e.g., when the operator's head initially contacts the headrest), which does not necessarily correspond to the location of the headrest sensor. Therefore, perfect alignment between the operator's head and the center of the headrest or the headrest sensor is not necessary.

[0088] In step 606, according to one or more embodiments, F x and F y Decomposed into normal force F normal and shear force F shear The execution of step 606 is specific to the horizontal plane, such as Figure 5B As shown. Shear force and normal force can be ignored in the vertical plane, as shown in Figure 5A As shown. The decomposition can be performed based on the known geometry of the headrest at the contact point using trigonometric operations. Specifically, F is obtained at the contact point. shear The tangent direction of F is then determined shear and F normal Size. F x 、F y 、F normal and F shear Shown in Figure 5B In. F normal and F shear It can then be used to simulate the driving of the virtual dynamics model.

[0089] Although not in Figure 6 Although shown in the flowchart of , a sanity check can be performed on the obtained sensor signal. For example, a sensor signal can be accepted only if the corresponding force points in a physically meaningful direction. For example, if the direction is toward the operator, thereby suggesting a pulling force rather than a pushing force, the direction of the force may not be meaningful. Similarly, if F shear Relative to F normal If it is too large, the direction of the force may not make sense. Usually at least a minimum F is required. normal To allow a certain level of F shear , without letting the operator's head slide on the headrest. If physically implausible sensor signals are detected, they can be ignored. In addition or alternatively, the operator can receive a visual or audible warning.

[0090] In execution Figure 6 Following the method, the force and torque components used to simulate the actuation of the virtual dynamics model are available.

[0091] Go to Figure 7 , which shows a method for simulating the movement of a virtual dynamic model according to one or more embodiments. Figure 6 The sensed mechanical inputs obtained as described in the previous section are used to drive the virtual dynamics model.

[0092] In step 700, the vertical force is obtained. The vertical force can be Figure 5A F shown z . F z As previously mentioned, when the headrest sensor is not aligned with the ground-based Cartesian reference frame, F can be directly obtained from the force reported by the headrest sensor using trigonometric operations. z .

[0093] In step 702, the horizontal force is obtained. The horizontal force can be Figure 5A F shown in y . F y As previously mentioned, when the headrest sensor is not aligned with the ground-based Cartesian reference frame, F can be directly obtained from the force reported by the headrest sensor using trigonometric operations. y Alternatively, F y You can Figure 5B Obtained as shown.

[0094] In step 704, a pitch torque is obtained. The pitch torque can be obtained using the distance between the current contact point (in the same or similar manner as obtained in step 604) and the initial contact point. The distance can be multiplied by a pitch coefficient or gain. The initial contact point can be obtained at a point in time, for example, when the operator initially activates the control of the steerable display unit through head movement. The obtained pitch torque can be centered around the center of the headrest and can be oriented to drive the display unit and headrest in a direction that causes the current contact point to move closer to the initial contact point. As a result, the display unit and headrest can follow the operator's head movements, as discussed in further detail below.

[0095] In step 706, a yaw torque is obtained. Similar to the pitch torque, the yaw torque can be oriented to drive the display unit and the headrest in a direction that brings the current contact point closer to the center of the headrest. The obtained yaw torque can be about an axis passing through the center of the physical or virtual yaw joint, for example, Figure 2 The yaw torque may comprise an additive combination of two terms. The first term may be a contact point term, while the second term may be a shear force term.

[0096] The contact point term can be obtained using the distance between the current contact point (as obtained in step 604) and the center of the headrest (which may or may not coincide with the location of the headrest sensor). The distance can be multiplied by a yaw factor or gain. This factor can be adjustable to allow for adjustment of the effect of distance.

[0097] The contact point item may be particularly useful for detecting head movements including a "rolling" motion of the operator's head in the headrest, since rolling changes the contact point. More specifically, rolling head motion directly results in a shift in the contact point.

[0098] The shear force term can be obtained as follows: Figure 5B , can be achieved by using F normal F shear Derating is performed to obtain the shear term. Derating can scale F shear , the method is to normal Provides less horizontal force when raised, and normal Provides greater horizontal force when derated. Derating can be performed multiplicatively (e.g., by F shear With F normal The derating effect can be scaled by multiplying the inverse of the gain and using adjustable gain. Derating can allow the operator to more accurately control the movement of the viewing system. Specifically, when the operator leans heavily against the headrest, a high F normal , F shear May also increase, because when high F normal When the operator finely controls F shear In contrast, when the operator only lightly touches the headrest, the F shear Derating can further address potential stability issues in the control loop: higher F normal Effectively increases the rigidity of the entire closed-loop control system. shear The control loop gain is optimized for low F normal , that is, based on a lower rigidity mechanical system, when the control factor F normal When increasing the rigidity of the mechanical system, use F directly shear May cause instability, which may lead to oscillation. shear The derating ensures that the entire closed-loop control system remains within the stable region.

[0099] Derated F shear The radius can be multiplied by to obtain the shear term. The radius can be the distance between the contact point and the center of the physical or virtual yaw joint, e.g. Figure 2The shear force term may be particularly useful for detecting head movements that include head pivoting (yaw), since pivoting induces shear forces.

[0100] In step 708, the driving input is obtained by vectoring the horizontal force, vertical force, yaw torque, and pitch torque, so that the driving input provides a four-dimensional directional force-torque for simulating the driving virtual dynamics model.

[0101] In step 710, the drive input is adjusted for a bias force. The bias force can be perpendicular to the surface of the headrest, at the center of the headrest, and directed toward the operator. The bias force can be applied to maintain contact with the operator's head. In other words, the bias force causes the headrest to exert a force on the operator's head. The bias force can be selected to provide sufficient friction between the operator's head and the headrest to enable the operator to transmit force and / or torque to the headrest during head movements. Furthermore, when the operator retracts their head, the bias force causes the headrest to follow the operator's head. The bias force can be constant or progressive, increasing with insertion, requiring the operator's head to exert progressively greater force as the insertion progresses. Although the bias force can be in a direction perpendicular to the headrest, the horizontal and vertical forces of the drive input can be adjusted to introduce the bias force. The horizontal and vertical force adjustments can be calculated based on the current orientation of the headrest.

[0102] In step 712, the drive input is applied to the virtual dynamics model in simulation. In one or more embodiments, the virtual dynamics model includes a virtual mass. The virtual mass can be configured to adjust for the inertia of the simulated system. Velocity-related damping can act on the virtual mass. In addition, friction (e.g., Coulomb friction) can be included in the virtual dynamics. The force and torque represented by the drive input can act on the virtual mass, causing the virtual mass to accelerate. Therefore, at any point in time, based on the input driving the virtual mass, the virtual mass can move in virtual space.

[0103] In one or more embodiments, the virtual dynamics model has additional features. For example, the virtual mass may be constrained to a limited virtual space. The limitations of the virtual space may be derived from the physical workspace constraints of the imaging device and / or display unit. By constraining the virtual space to not exceed the physical workspace, it can be ensured that the physical workspace boundaries are not reached or exceeded when the display unit is driven to follow the virtual dynamics model (described below). Limitations on the virtual space can be introduced by reducing the speed of the virtual mass as it approaches the boundary. The speed reduction may be gradual until zero speed is reached at the boundary. The constraints of the virtual space can be static or dynamic. The static constraints of the virtual workspace can reflect the physical workspace constraints. The dynamic constraints of the virtual workspace can be set and adjusted based on spontaneously occurring events (such as predicted or actually detected collisions of the controlled imaging device).

[0104] In the above description, the virtual mass is primarily driven by the operator's head movement. Other factors may affect the simulated movement of the virtual mass. In one embodiment of the present disclosure, tactile events can be rendered by changing the dynamics of the virtual mass. These tactile events can implement virtual control elements, such as switches that can be implemented by simulating rigid or compliant surfaces, springs with configurable rigidity, etc. In step 708, tactile events can be introduced by modulation of the drive input. Tactile events can be introduced by additionally modulating the drive input. Because the movement of the display unit can reflect the movement of the simulated virtual mass, the operator can physically experience the tactile events.

[0105] Tactile events may be position-dependent. For example, a virtual switch may be implemented based on the current position of the display unit, allowing it to be placed, for example, at the boundaries of the physically available workspace. The current position of the display unit may be obtained using a position sensor (e.g., an encoder of an actuator driving a link of the display unit). Forward kinematics may be used to obtain the current position in a reference frame suitable for defining the location of the tactile event. Tactile events may be mapped to other features of the robotic system. For example, clicking on a virtual wall at the boundary of the available workspace may start recording a video, activate a specific tool, switch between different operating modes, etc.

[0106] Go to Figure 8 A flowchart illustrating a method for causing movement of a display unit and a headrest portion attached to the display unit according to one or more embodiments. Figure 8 The method may result in the headrest portion following the simulated movement of the virtual dynamic model. Alternatively, in one embodiment, the headrest portion is driven directly using the drive input. In this case, a virtual dynamic model may not be required.

[0107] In step 800, an inverse kinematics model of the connecting rod supporting the display unit and the headrest is run at the velocity of the virtual mass to generate joint movement commands for the actuators driving the joints of the connecting rod. Thus, the inverse kinematics model can be used to drive the joints of the connecting rod supporting the display unit and the headrest so that another point on the headrest or the display unit accurately follows the virtual mass. For example, the inverse kinematics model can be implemented using the inverse (or pseudo-inverse) Jacobian of the connecting rod supporting the display unit. While the output of the inverse kinematics model may be joint velocities, an integration can be performed to obtain the commanded joint positions.

[0108] In step 802, the joints of the link are driven using joint movement commands. The joint movement commands can be position or velocity commands, depending on the servo controller controlling the joint. Position-velocity-time (PVT) interpolation can be used for joint movement commands provided as positions.

[0109] Upon completion of step 802, the display unit can be moved in a direction based on the operator's head movement to achieve alignment between the display unit and the operator's head. Furthermore, in response to determining movement of the operator's head or the display unit, the imaging system can be caused to move and update the view provided to the operator. As a result, head movement enables the operator to control the view obtained from the imaging device, while compensating movement of the display unit ensures that the display unit remains aligned with the operator's head to provide a correct viewing configuration without misalignment between the operator's eyes and the display unit's viewport. Sometimes, the imaging device can be decoupled from the operator's head movement. A possible benefit of this decoupling is that, during this decoupling period, the view provided by the imaging device can remain stable even in the presence of small, naturally occurring shakes or other "jittery" head movements of the operator. Decoupling can be achieved by implementing an input deadband. The input deadband can be implemented by ignoring force / torque inputs below a set threshold as long as the operator stops moving. The deadband can be established around the location where the operator stops moving. Once the operator applies a force / torque input exceeding the threshold, the deadband can be deactivated.

[0110] Because the dynamics of the compensating movement according to one or more embodiments is governed by a virtual dynamics model that can be configured based on operator preferences, accuracy requirements, etc., the operator can receive a desired feedback experience regardless of the original dynamics of the display unit. As long as the inverse kinematics model accurately represents the display unit and the linkages supporting it, the display unit can accurately follow the simulated movement of the virtual mass, replacing the original dynamics with the desired dynamics.

[0111] While the present invention has been described with respect to a limited number of embodiments, those skilled in the art, having benefit of this disclosure, will appreciate that other embodiments may be designed which do not depart from the scope of the invention as disclosed herein. Accordingly, the scope of the present invention should be limited only by the claims appended hereto.

Claims

1. A computer-assisted robotic system comprising: a display unit configured to provide an image to an operator of the display unit, the display unit being supported by a link that enables movement of the display unit; a headrest portion provided on the display unit, the headrest portion configured to receive a mechanical input provided by a head of the operator in mechanical contact with the headrest portion; a headrest sensor interfacing with the headrest and configured to provide a sensor signal based on the mechanical input; A controller comprising a computer processor, the controller being configured to: processing the sensor signal to obtain a driving input; driving the virtual mass via the driving input to obtain simulated virtual mass movement; obtaining movement commands of joints of the link using an inverse kinematics model of the link based on the simulated virtual mass movement; and The movement of the headrest portion is caused by driving the joints of the links using the movement commands, the movement of the headrest portion tracking the movement of the virtual mass.

2. The computer-assisted robotic system according to claim 1, wherein the display unit comprises a viewing port configured to display the image, and The controller is configured to use the drive input to maintain alignment of the viewing port with the head during head movement that causes the mechanical input to the headrest.

3. The computer-assisted robotic system of claim 1 , wherein the headrest sensor senses at least one force or torque selected from the group consisting of: a plurality of forces caused by the mechanical input and a plurality of torques caused by the mechanical input.

4. The computer-assisted robotic system of claim 1 , wherein processing the sensor signals to obtain the actuation inputs comprises determining a point of contact between the head and the headrest.

5. The computer-assisted robotic system of claim 4, wherein determining the contact point comprises: A balance of forces and torques represented by the sensor signals and the geometry of the headrest are used.

6. The computer-assisted robotic system of claim 1, wherein processing the sensor signals to obtain the actuation inputs comprises determining shear forces at contact points between the head and the headrest.

7. The computer-assisted robotic system of claim 6, wherein processing the sensor signal further comprises de-rating the shear force based on the normal force at the contact point.

8. The computer-assisted robotic system of claim 1, wherein the drive inputs obtained from processing the sensor signals include horizontal force, vertical force, yaw torque, and pitch torque.

9. The computer-assisted robotic system of claim 1 , wherein the drive input obtained from processing the sensor signals reflects a determination of at least one head movement selected from the group consisting of: a roll head movement, a yaw head movement, a pitch head movement, and a horizontal head shift movement.

10. The computer-assisted robotic system according to any one of claims 1 to 9, wherein the controller is further configured to: The drive input is adjusted to include a biasing force.

11. The computer-assisted robotic system of claim 10, wherein the biasing force acts perpendicular to the surface of the headrest, at the center of the headrest, and is oriented toward the head.

12. The computer-assisted robotic system of claim 10, wherein the biasing force biases the horizontal force and the vertical force of the drive input.

13. The computer-assisted robotic system according to any one of claims 1 to 9, wherein driving the virtual mass comprises: Simulate friction or simulate damping.

14. The computer-assisted robotic system according to any one of claims 1 to 9, wherein the controller is further configured to render the haptic event by changing the actuation input or the dynamics of the virtual mass according to the haptic event.

15. The computer-assisted robotic system according to any one of claims 1 to 9, further comprising an imaging device, wherein the imaging device is configured to provide the image, and Wherein the controller is further configured to control movement of the imaging device based on the mechanical input.

16. A method for operating a robotic system, the robotic system comprising: a display unit configured to provide an image to an operator of the display unit, the display unit being supported by a link that enables movement of the display unit, a headrest portion provided on the display unit, the headrest portion being configured to receive a mechanical input provided by a head of the operator in mechanical contact with the headrest portion, a headrest sensor interfacing with the headrest and configured to provide a sensor signal based on the mechanical input; The method comprises: obtaining the sensor signal from the headrest sensor; processing the sensor signal to obtain a driving input; driving the virtual mass via the driving input to obtain simulated virtual mass movement; obtaining movement commands for joints of the link using an inverse kinematics model of the link based on the simulated virtual mass movement; and The movement of the headrest portion is caused by driving the joints of the links using the movement commands, the movement of the headrest portion tracking the movement of the virtual mass.

17. The method according to claim 16, further comprising: The drive input is used to maintain alignment of a viewing port of the display unit with the head during head movement that causes the mechanical input to the headrest.

18. The method of claim 16, wherein processing the sensor signal to obtain the drive input comprises determining a point of contact between the head and the headrest portion.

19. The method of claim 18, wherein determining the contact point comprises: A balance of forces and torques represented by the sensor signals and the geometry of the headrest are used.

20. The method of claim 16, wherein processing the sensor signal to obtain the drive input comprises determining a shear force at a contact point between the head and the headrest.

21. The method of claim 20, wherein processing the sensor signal further comprises de-rating the shear force based on the normal force at the contact point.

22. The method of claim 16, wherein the drive inputs obtained from processing the sensor signals include horizontal force, vertical force, yaw torque, and pitch torque.

23. The method according to any one of claims 16 to 22, further comprising: The drive input is adjusted to include a biasing force.

24. The method of claim 23, wherein the biasing force acts perpendicular to the surface of the headrest, at the center of the headrest, and is oriented toward the head.

25. The method of claim 23, wherein the biasing force biases a horizontal force and a vertical force of the drive input.

26. The method of any one of claims 16 to 22, wherein driving the virtual mass comprises: Simulate friction or simulate damping.

27. The method according to any one of claims 16 to 22, further comprising: The haptic event is rendered by varying the dynamics of the actuation input or the virtual mass according to the haptic event.

28. The method according to any one of claims 16 to 22, further comprising: Movement of the imaging device is controlled based on the mechanical input.

29. A non-transitory machine-readable medium comprising a plurality of machine-readable instructions executed by one or more processors associated with a medical system, The medical system includes: a display unit configured to provide an image to an operator of the display unit, the display unit being supported by a link that enables movement of the display unit, a headrest portion provided on the display unit, the headrest portion being configured to receive a mechanical input provided by a head of the operator in mechanical contact with the headrest portion, a headrest sensor interfacing with the headrest and configured to provide a sensor signal based on the mechanical input; The plurality of machine-readable instructions cause the one or more processors to perform a method comprising: obtaining the sensor signal from the headrest sensor, processing the sensor signal to obtain a driving input; driving the virtual mass via the driving input to obtain simulated virtual mass movement; obtaining movement commands for joints of the link using an inverse kinematics model of the link based on the simulated virtual mass movement; and The movement of the headrest portion is caused by driving the joints of the links using the movement commands, the movement of the headrest portion tracking the movement of the virtual mass.

30. The non-transitory machine-readable medium of claim 29, wherein the method further comprises: The drive input is used to maintain alignment of a viewing port of the display unit with the head during head movement that causes the mechanical input to the headrest.

31. The non-transitory machine-readable medium of claim 29, wherein processing the sensor signal to obtain the drive input comprises determining a point of contact between the head and the headrest.

32. The non-transitory machine-readable medium of claim 31 , wherein determining the contact point comprises: A balance of forces and torques represented by the sensor signals and the geometry of the headrest are used.

33. The non-transitory machine-readable medium of claim 29, wherein processing the sensor signal to obtain the drive input comprises determining a shear force at a contact point between the head and the headrest.

34. The non-transitory machine-readable medium of claim 33, wherein processing the sensor signal further comprises de-rating the shear force based on a normal force at the contact point.

35. The non-transitory machine-readable medium of claim 29, wherein the drive inputs obtained from processing the sensor signals include horizontal force, vertical force, yaw torque, and pitch torque.

36. The non-transitory machine-readable medium of any one of claims 29 to 35, wherein the method further comprises: The drive input is adjusted to include a biasing force.

37. The non-transitory machine-readable medium of claim 36, wherein the biasing force acts perpendicular to a surface of the headrest, at a center of the headrest, and is oriented toward the head.

38. The non-transitory machine-readable medium of claim 36, wherein the biasing force biases a horizontal force and a vertical force of the drive input.

39. The non-transitory machine-readable medium of any one of claims 29 to 35, wherein driving the virtual mass comprises: Simulate friction or simulate damping.

40. The non-transitory machine-readable medium of any one of claims 29 to 35, wherein the method further comprises: The haptic event is rendered by varying the dynamics of the actuation input or the virtual mass according to the haptic event.

41. The non-transitory machine-readable medium of any one of claims 29 to 35, wherein the method further comprises: Movement of the imaging device is controlled based on the mechanical input.

Images