System and method for haptic feedback of menu item selection in a remote control system

Abstract

The present invention relates to systems and methods for haptic feedback for selection of menu items in a teleoperated system. The haptic feedback method includes providing a teleoperated control system including a first mode of operation for operating a teleoperated instrument in response to movement of a control device in a first degree of freedom and a second mode of operation for controlling a graphical user interface. The method further includes entering the second mode of operation of the teleoperated control system. While in the second mode of operation, the method includes tracking movement of the control device in a second degree of freedom that is different from the first degree of freedom. While in the second mode of operation and in response to movement of the control device in the second degree of freedom, the method includes applying a first haptic force to the control device by the teleoperated control system.

Description

[0001] This application is a divisional application of application 202310560733X, filed on June 29, 2018, entitled "System and method for haptic feedback for menu item selection in a remote control system". Divisional application 202310560733X is a divisional application of Chinese patent application 2018800428788 (PCT / US2018 / 040475).

[0002] Cross-references to related applications This application claims the benefit of U.S. Provisional Application 62 / 529,038, filed July 6, 2017, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to systems and methods for controlling the operation of a remote operation control system, and more specifically to systems and methods for providing tactile feedback to a user when accessing a graphical user interface menu using a control device of the remote operation control system. Background Technology

[0004] Minimally invasive medical techniques aim to reduce the amount of tissue damaged during invasive medical procedures, thereby reducing patient recovery time, discomfort, and harmful side effects. Such techniques can be performed through natural openings in the patient's anatomy or through one or more surgical incisions. Through these natural openings or incisions, clinicians can insert medical instruments to reach target tissue locations. Minimally invasive medical instruments include devices such as therapeutic, diagnostic, and surgical instruments. They may also include imaging devices such as endoscopic instruments. Some minimally invasive medical instruments may be remotely controlled or computer-assisted. Various control devices can be used to control remotely operated or computer-assisted medical instruments. A graphical user interface (GUI) can be used to extend the functionality of a remote operating system without adding additional structural controls to the user's console. When accessing the GUI using a control device, the system and methods provide tactile feedback to the user, and this control device is also used to control instruments within the patient's body. Summary of the Invention

[0005] Embodiments of the present invention are summarized in the appended claims.

[0006] In one embodiment, the haptic feedback method includes providing a remote operation control system comprising a first operating mode for operating a remotely operated instrument in response to movement of a control device in a first degree of freedom, and a second operating mode for controlling a graphical user interface. The method further includes engaging the second operating mode of the remote operation control system. When in the second operating mode, the method includes tracking movement of the control device in a second degree of freedom different from the first degree of freedom. When in the second operating mode and in response to movement of the control device in the second degree of freedom, the method includes applying a first haptic force to the control device via the remote operation control system.

[0007] In another embodiment, the tactile feedback method includes entering an interface mode of a remote operation control system. In response to movement of a control device in the remote operation control system, the method further includes applying a first tactile force to the control device from a nominal position to urge the control device toward the nominal position. The method further includes determining that the control device has moved a first displacement distance with a first degree of freedom from the nominal position to an engaged position. The method further includes applying a second tactile force to the control device to provide a tactile sensation indicating engagement. The method further includes applying a third tactile force to the control device while in the engaged position to urge the control device back from the engaged position to the nominal position. Attached Figure Description

[0008] The various aspects of this disclosure are best understood by reading in conjunction with the accompanying drawings through the following detailed description. It should be emphasized that, in accordance with standard industry practice, the various features are not drawn to scale. In fact, the dimensions of the various features may be arbitrarily increased or decreased for clarity. Furthermore, reference numerals and / or letters may be repeated in various examples throughout this disclosure. This repetition is for simplicity and clarity and does not in itself indicate a relationship between the various embodiments and / or configurations discussed.

[0009] Figure 1A This is a schematic diagram of a remotely operated medical system according to an embodiment of the present disclosure.

[0010] Figure 1B This is a perspective view of a surgeon's console for remotely operating a medical system, according to many embodiments.

[0011] Figure 1C This is a perspective view of an electronic trolley for a remotely operated medical system according to many embodiments.

[0012] Figure 1D This is a perspective view of a patient-side trolley, an example of the principles described in this article.

[0013] Figure 2 The description includes a graphical user interface with a selector icon.

[0014] Figure 3A Indicates that it is in the nominal position Figure 2 The selector icon.

[0015] Figure 3B Instructions for rotating to the entry position Figure 2 The selector icon.

[0016] Figure 3C This describes the return from the entry position to the nominal position. Figure 2 The selector icon.

[0017] Figure 4 The selector icon is illustrated according to an optional embodiment.

[0018] Figure 5 This is a flowchart illustrating a method for providing tactile feedback via a remote operating system control device when using a graphical user interface menu.

[0019] Figure 6 This is a torque curve for providing tactile feedback according to an embodiment of the present disclosure.

[0020] Figure 7 This is a flowchart illustrating another method for providing haptic feedback via a remote operating system control device when using a graphical user interface menu.

[0021] Figure 8 This is a torque curve for providing tactile feedback according to another embodiment of the present disclosure.

[0022] Figure 9 This is a flowchart illustrating another method for providing haptic feedback via a remote operating system control device when using a graphical user interface menu. Detailed Implementation

[0023] To facilitate an understanding of the principles of this disclosure, reference will now be made to the embodiments illustrated in the accompanying drawings, and they will be described using specific language. However, it will be understood that this is not intended to limit the scope of the disclosure. Numerous specific details are set forth in the following detailed description of various aspects of the invention to provide a thorough understanding of the disclosed embodiments. However, it will be apparent to those skilled in the art that embodiments of the disclosure may be practiced without these specific details. In other instances, well-known methods, processes, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the embodiments of the invention.

[0024] As will commonly conceived by those skilled in the art to which this disclosure pertains, any changes and further modifications to the described apparatus, instruments, and methods, as well as any further application of the principles of this application, are foreseeable. In particular, it is fully contemplated that features, components, and / or steps described with respect to one embodiment may be combined with features, components, and / or steps described with respect to other embodiments of this disclosure. Furthermore, the dimensions provided herein are for specific examples, and it is contemplated that the concepts of this disclosure may be implemented using different sizes, dimensions, and / or ratios. To avoid unnecessary descriptive repetition, one or more components or actions described according to an illustrative embodiment may be used or omitted depending on the applicability of other illustrative embodiments. For brevity, numerous iterations of these combinations will not be described separately. For simplicity, in some cases, the same reference numerals are used throughout the drawings to refer to the same or similar parts.

[0025] The following embodiments will describe various instruments and instrument parts based on their states in three-dimensional space. As used herein, the term "position" refers to the location of an object or part of an object in three-dimensional space (e.g., the three translational degrees of freedom along the Cartesian X, Y, Z coordinates). As used herein, the term "orientation" refers to the rotational placement of an object or part of an object (three rotational degrees of freedom, e.g., roll, pitch, and yaw).

[0026] Refer to the attached diagram. Figure 1A Remotely operated medical systems, typically indicated by reference numeral 10, are used in medical procedures, including diagnostic, therapeutic, or surgical procedures. As will be described, this disclosed remotely operated medical system is under the remote operational control of a surgeon. In alternative embodiments, the remotely operated medical system may be under partial control of a computer programmed to execute a program or subroutine. In other alternative embodiments, a fully automated medical system under the complete control of a computer programmed to execute a program or subroutine may be used to execute the program or subroutine. Figure 1A As shown, the remote-operated medical system 10 typically includes a remote-operation component 12 mounted on or near an operating table O, with the patient P positioned on the operating table O. The remote-operation component 12 may be referred to as a patient-side trolley. A medical device system 14 and an endoscopic imaging system 15 are operatively coupled to the remote-operation component 12. An operator input system 16 allows the surgeon or other type of clinician S to view images of the surgical site or images representing the surgical site and to control the operation of the medical device system 14 and / or the endoscopic imaging system 15.

[0027] Operator input system 16 may be located on a surgeon's console, typically in the same room as the operating table O. However, it should be understood that the surgeon S may be located in a different room or in a completely different building from the patient P. Operator input system 16 typically includes one or more control devices for controlling medical device system 14. The one or more control devices may include one or more of any number of various input devices, such as handles, joysticks, trackballs, data gloves, trigger guns, manual controllers, voice recognition devices, touchscreens, limb movement or presence sensors, etc. In some embodiments, the one or more control devices are provided with the same degrees of freedom as the medical device of the remotely operated component to provide a remote presentation to the surgeon: the sensory effect of the one or more control devices integrated with the device gives the surgeon a strong awareness of directly controlling the device, as if at the surgical site. In other embodiments, the one or more control devices may have more or fewer degrees of freedom than the associated medical device and still provide a remote presentation to the surgeon. In some embodiments, the one or more control devices are manual input devices that move in six degrees of freedom and may also include actuable handles for actuating the device (e.g., for closing gripping jaws, applying a potential to electrodes, delivering therapeutic drugs, etc.).

[0028] While surgeon S views the surgical site via console 16, teleoperation component 12 supports and manipulates medical device system 14. Images of the surgical site can be obtained via endoscopic imaging system 15, such as a stereoscopic endoscope, which can be manipulated by teleoperation component 12 to position the endoscope 15. Electronic trolley 18 can be used to process images of the surgical site for subsequent display to surgeon S via the surgeon's console 16. The number of medical device systems 14 used at one time typically depends on factors such as diagnostic or surgical procedures and space constraints within the operating room. Teleoperation component 12 may include a kinematic structure of one or more non-servo-controlled links (e.g., one or more links that can be manually positioned and locked in place, often referred to as a set-up structure) and a remote manipulator. Teleoperation component 12 includes multiple motors that drive inputs on medical device system 14. These motors move in response to commands from a control system (e.g., control system 20). The motors include a drive system that, when coupled to medical device system 14, advances the medical device into a naturally or surgically created anatomical opening. Other motorized drive systems can move the distal end of a medical device in multiple degrees of freedom, including three linear motion angles (e.g., linear motion along the X, Y, and Z axes of a Cartesian reference system) and three rotational motion angles (e.g., rotation about the X, Y, and Z Cartesian coordinate axes). Additionally, electric motors can be used to actuate articulated end effectors of the device to grip tissue in the jaws of biopsy devices, etc. The electric motor can be controlled to generate command torque (or force under a linear motor).

[0029] The remote-operated medical system 10 also includes a control system 20. The control system 20 includes at least one memory and at least one processor (not shown), and typically multiple processors, for implementing control between the medical device system 14, the operator input system 16, and the electronic system 18. The control system 20 also includes programmable instructions (e.g., a computer-readable medium storing the instructions) to implement some or all of the methods described according to the aspects disclosed herein. Although in Figure 1A In the simplified schematic diagram, the control system 20 is shown as a single block, but the system may include two or more data processing circuits, wherein a portion of the processing is optionally executed on or near the remote operating component 12, another portion of the processing is executed at the operator input system 16, and so on. A wide variety of centralized or distributed data processing architectures can be employed. Similarly, programmed instructions may be implemented as multiple separate programs or subroutines, or they may be integrated into many other aspects of the remote operating system described herein. In one embodiment, the control system 20 supports wireless communication protocols such as Bluetooth, IrDA, HomeRF, IEEE 802.11, DECT, and wireless telemetry.

[0030] In some embodiments, the control system 20 may include one or more servo controllers that receive force and / or torque feedback from the medical device system 14. In response to the feedback, the servo controllers send signals to the operator input system 16. The servo controllers may also send signals to instruct the teleoperation component 12 to move the medical device system 14 and / or the endoscopic imaging system 15, which extends through an opening within the body to an internal surgical site within the patient. Any suitable conventional or dedicated servo controller may be used. The servo controller may be separate from or integrated with the teleoperation component 12. In some embodiments, the servo controller and the teleoperation component are provided as part of a teleoperation arm trolley positioned near the patient's body.

[0031] The remotely operated medical system 10 may also include optional operating and support systems (not shown), such as lighting systems, steering control systems, irrigation systems, and / or suction systems. In alternative embodiments, the remote operating system may include more than one remote operating component and / or more than one operator input system. The exact number of manipulator components will depend on factors such as the surgical procedure and space constraints within the operating room. Operator input systems may be juxtaposed or placed in separate locations. Multiple operator input systems allow more than one operator to control one or more manipulator components in various combinations.

[0032] Figure 1BThis is a perspective view of the surgeon's console 16. The surgeon's console 16 includes a left-eye display 32 and a right-eye display 34 for presenting the surgeon S with a coordinated stereoscopic view of the surgical site, enabling depth perception. The console 16 also includes one or more input controls 36, 37, which the surgeon uses to perform functions of the system 10. Input control 36 is a manual input control that allows the remote operation component 12 to manipulate one or more instruments or endoscopic imaging systems. Input control 36 can provide the same degrees of freedom as its associated instrument 14 to provide a remote presentation to the surgeon S, or input control 36 can be integrated with the instrument 14, giving the surgeon a strong awareness of direct control over the instrument 14. For this purpose, position, force, and tactile feedback sensors (not shown) can be used to transmit position, force, and tactile sensations from the instrument 14 back to the surgeon's hand via input control 36. Input control 37 allows the system 10 to switch between operating modes. Device 37 can be a pedal operated by the surgeon's foot, or it can be another type of hand or foot switch that functions as a clutch to disengage from a first operating mode and enter a second operating control mode. The operating modes of the remote-operated medical system 10 may include, for example, a surgical instrument control mode, a camera control mode, and a menu control mode. The surgical instrument control mode allows the surgeon to control the manipulation of the instrument 14 as described above. The camera control mode allows the surgeon to use the input control device 36 to manipulate the endoscopic imaging system 15 via the remote-operated component 12. The menu control mode allows the surgeon to use the input control device 36 to navigate the graphical user interface menu displayed to the surgeon via the left-eye display 32 and the right-eye display 34. For example, the control device 37 can be pressed to switch between the surgical instrument control mode and the menu mode.

[0033] Figure 1C This is a perspective view of the electronic trolley 18. The electronic trolley 18 may be coupled to the endoscope 15 and may include a processor to process the captured images for subsequent display to a surgeon, such as on a surgeon's console or on another suitable monitor located locally and / or remotely. For example, in the case of using a stereoscopic endoscope, the electronic trolley 18 may process the captured images to present the surgeon with a coordinated stereoscopic image of the surgical site. Such coordination may include alignment between relative images and may include adjusting the stereoscopic working distance of the stereoscopic endoscope. As another example, image processing may include compensating for imaging errors of the image capture device, such as optical aberrations, using previously determined camera calibration parameters. The electronic trolley 18 may also include components of a display monitor and a control system 20.

[0034] Figure 1DThis is a perspective view of one embodiment of a remote manipulation assembly 12, which may be referred to as a patient-side trolley. The patient-side trolley 12 shown provides manipulation of three surgical instruments 26 (e.g., instrument system 14) and an imaging device 28 (e.g., an endoscopic imaging system 15), such as a stereoscopic endoscope for capturing images of the procedure site. The imaging device transmits signals to the electronic trolley 18 via cable 56. Manipulation is provided by a remote manipulation mechanism with multiple connectors. The imaging device 28 and surgical instruments 26 can be positioned and manipulated through an incision in the patient, such that the kinematic remote center is held at the incision to minimize the incision size. Images of the surgical site may include images when the distal end of the surgical instrument 26 is positioned within the field of view of the imaging device 28.

[0035] The patient-side trolley 22 includes a driveable base 58. The driveable base 58 is connected to a telescopic column 57 that allows adjustment of the height of arms 54. Arms 54 may include rotary joints 55 that rotate and move up and down. Each arm 54 may be connected to a directional platform 53. The directional platform 53 may be able to rotate 360 ​​degrees. The patient-side trolley 22 may also include a telescopic horizontal cantilever 52 that moves the directional platform 53 in the horizontal direction.

[0036] In this example, each arm 54 is connected to a manipulator arm 51. The manipulator arm 51 can be directly connected to the medical device 26. The manipulator arm 51 can be remotely operated. In some examples, the arms 54 connected to the orientation platform are not remotely operable. Instead, these arms 54 are positioned as needed before the surgeon 18 begins operations with the remotely operated components.

[0037] Various configurations of rigid or flexible endoscopes can be included to provide endoscopic imaging systems (e.g., systems 15, 28). Rigid endoscopes include a rigid tube housing a relay lens system for transmitting images from the distal to the proximal end of the endoscope. Flexible endoscopes use one or more flexible optical fibers to transmit images. Endoscopes can be configured with different viewing angles, including a 0° angle for forward axial viewing or a viewing angle between 0° and 90° for forward tilt viewing. Digital image-based endoscopes feature a “cutting-edge chip” design where a distal digital sensor (such as one or more charge-coupled devices (CCDs) or complementary metal-oxide-semiconductor (CMOS) devices) stores image data. Endoscopic imaging systems can provide viewers with two-dimensional or three-dimensional images. Two-dimensional images provide limited depth perception. Three-dimensional stereoscopic endoscopic images provide viewers with more accurate depth perception. Stereoscopic endoscopic instruments use stereo cameras to capture stereoscopic images of the patient's anatomy.

[0038] To extend the functionality of the remote-operated medical system 10 without adding additional structural accessories such as pedals, physical switches, dials, and buttons to the console 16, a graphical user interface (GUI) menu can be used to provide additional options and actions for the operating system 10. When providing additional functionality to the system 10 using the GUI menu, providing haptic feedback to the operator at the console 16 is helpful, thereby facilitating interaction with the GUI menu elements and providing confirmation of executed commands. Other feedback mechanisms, such as auditory and visual feedback cues, can also provide confirmation of executed commands. Using haptic feedback when using GUI menus can provide a sense of immersion for the user.

[0039] Figure 2 The illustration describes the display 100 visible through the left-eye display 32 and right-eye display 34 of the surgeon's console 16. In this embodiment, the display includes a view of a surgical environment 102, which includes medical instruments 104, 106, and 108. In the "follow" operation mode of the surgical instrument control or remote operating system, the control device 36 can be manipulated to control the movement of instruments 104, 106, and 108 in finite or infinite degrees of freedom. For example, in the surgical instrument control mode, the control device can be operated to move the instruments with only three translational degrees of freedom in Cartesian coordinate space. When the graphical user interface mode of the remote operating system is invoked (e.g., by depressing the clutch pedal 37), the control device 36 is disengaged from instruments 104, 106, and 108 and can be used to select items from the graphical user interface menu. Figure 2 The description includes a graphical user interface menu with selector icon 110 and multiple menu items 112, 114, and 116. For example... Figure 2 As shown, the graphical user interface menu 113 can be overlaid on the image of the surgical environment 102, or alternatively, only the graphical user interface menu can be displayed without the surgical environment. In the graphical user interface mode where tools 104, 106, 108 are decoupled from control device 36, one or more of the control devices can be connected to position the selector icon 110 in a nominal or neutral position (e.g., ...). Figure 2 (as shown) and menu items 112, 114, and 116. In one embodiment, menu item 112 may be associated with an endoscope camera control mode in which the camera can be repositioned.

[0040] Figure 3A illustrate Figure 2The selector icon 110 is in its nominal position and has a clockwise direction of movement, which corresponds to the clockwise movement of the control device 36 around its rotation axis. When entering the graphical user interface mode, the nominal position of the control device 36 can be located in the space where the control device is positioned and oriented. Therefore, the nominal position of the control device 36 in space may be different each time the interface mode is entered.

[0041] The movement of the selector icon 110 can be coupled to a right-hand or left-hand control device. Different graphical user interface menus can be associated with each hand, allowing the right hand to select from a first menu and the left hand from a second menu. Optionally, the menu can be configured such that menu items are selected by counter-clockwise rotation. When the system is in other operating modes, such as surgical instrument control mode, the rotational movement of the control device 36 for moving the selector icon 110 can be a degree of freedom not used for tool operation. Thus, the user recognizes that the rolling degree of freedom about the axis of the control device is used for menu selection rather than tool operation. In other embodiments, the same degree of freedom used to control the selector icon can be used to control the movement of the tool in surgical instrument control mode. In other embodiments, the control device can move clockwise and counter-clockwise to select menu items. For example, clockwise rotation can point the selector icon to the menu item "Activate Camera Control Action," while counter-clockwise rotation can point the selector icon to the menu item "Activate Migration Action." In other embodiments, the control device can move about other rotational axes or can translate along an axis to produce selection motion. In other words, any Cartesian translation or rotation can produce selective motion.

[0042] Now for reference Figure 5 Flowchart 150 illustrates a method for providing tactile feedback via a control device of a remote operating system (e.g., system 10) when using a graphical user interface (GUI) menu (e.g., menu 113). Before initiating the process shown in flowchart 150, system 10 may be in a surgical instrument control mode or another operating mode of the system. In process 152, control system 20 determines whether input control device 37 has been actuated to initiate the GUI mode of system 10. This mode may be enabled, for example, by depressing the clutch pedal 37 of the surgeon's console to decouple from the surgical instrument control mode (including disengaging the surgical instruments from control device 36) and entering the GUI control mode, including coupling at least one of the control devices to a GUI menu selector icon. In process 154, menu 113 is displayed with or without an image of the surgical environment. For example, if menu 113 is associated with an instrument visible in the surgical environment, the menu may appear near the distal end of said instrument.

[0043] In process 156, system 10 receives and tracks a control signal from control device 36, which indicates that the control device is moving clockwise about the axis of the control device's lever. Therefore, selector icon 110 is as follows: Figure 3B As shown, from the nominal position N (see...) Figure 3A Move counterclockwise to menu item A.

[0044] In process 158, when the instrument control device moves a threshold displacement distance associated with the selected position corresponding to menu item A 114, a tactile force, such as tactile torque, is provided to the control device. The tactile force may be superimposed on the normal torque generated by the controller. The tactile torque provides the user with a tactile detent or feel indicating that the controller has selected menu item A. The normal torque may be a torque that, for example, is generated to reflect to the user the tracking error of the remotely operated component relative to the position / orientation of the control device.

[0045] In process 160, in response to user input (e.g., pressing a physical button on the control device), a command associated with menu item A is executed. For example, menu item A may activate camera control. Optionally, and particularly, if a single menu item exists, the command may be executed when the control device reaches the selected position. Optionally, selector icon 110 and control device 36 may remain in the selected position, awaiting further movement of the control device to move the selector icon clockwise to menu item B 116 or counterclockwise to the nominal position.

[0046] In process 162, optionally, after the command of menu item A is executed in process 160, another tactile torque can be provided to return control device 36 and selector icon 110 to the nominal position (see...). Figure 3CBy recentering the selector icon and control device 36 to the nominal position, a counter module can be used to count each time the selector icon and control device 36 return from the selected position to the nominal position. In this embodiment, the counter module increments its count each time the selector icon and control device 36 returns from the selected position to the nominal position. Each increment of the counter module can be associated with a menu item, allowing the user to switch between menu items by repeatedly moving the control device 36 back and forth. For example, a single click (e.g., a count of movement to the selected position and return to the nominal position) corresponds to a first menu item, and two clicks correspond to a second menu item. The incrementing menu items can be displayed to the user. Using the counter module requires the user to rotate the control device only a limited distance (e.g., to menu item A) to switch between several menu items, rather than requiring the user to rotate the control device away from the nominal position. This prevents the user from twisting their hand to a large angle and allows the user's hand to always be close to the nominal position, where they can quickly change modes and regain control of the surgical instruments coupled to the control device 36.

[0047] Figure 4 The optional graphical user interface menu 120 is described, in which the selector icon 122 moves along the translation direction. Other menu arrangements in which the controls and selector icons can move with other individual degrees of freedom are also possible.

[0048] In an alternative embodiment, clockwise or counterclockwise movement of the control device 36 can cause the selector icon 110 to move clockwise, or it can cause the selector icon 122 to move in a single translational direction. In other words, movement of the control device 36 in the left-right rolling direction will cause the same single forward movement of the selector icon. For each single action, the user can feel a tactile stop feature or a "click" sound. This embodiment can be adapted to make movement easy to perform in multiple positions of the control device 36. For example, when the control device 36 is pointed to the right, the user can more easily twist the tilt axis of the control device to the left rather than the right to make the selector icon move forward. By making the movement symmetrical (i.e., so that either control device can make the selector move forward), the user has more options for controlling the selector. This feature can be activated on both left and right control devices simultaneously. The software can detect which control device exceeds the δ1 threshold to determine which control device is activating the tactile stop feature.

[0049] Figure 6 Explanation of the tactile stop torque curve T according to one embodiment S The tactile stopping torque curve T S Superimposed on the controller torque curve T used to provide tactile feedback to the control device 36 CAbove, the applied electric motor torque simulates the rotation of a spring-loaded button. In this embodiment, the tactile stop torque curve T... S It produces the tactile sensation of a spring-loaded button. Figure 7 It is a description Figure 6 Flowchart 180 shows the torque curve. The torque curve is provided to one or more drive actuators in the control unit 36 ​​to provide force feedback felt by the surgeon S's hand. Controller torque curve T C A centering force or equilibrium is provided, which pushes the control device toward the nominal position. A greater torque is applied as the displacement δ (e.g., the rotation angle or rotation distance of the control device 36) increases, and as the displacement δ increases, the surgeon's hand on the control device feels an increase in resistance. The nominal position is located where the displacement δ equals zero. Superimposed on the controller torque curve T... C The tactile stopping torque curve T on the top S It has multiple regions.

[0050] refer to Figure 6 and Figure 7 In process 182, when the control device has displacement (relative to the nominal value) in the displacement region between zero and δ1, the torque curve T1 follows the controller torque curve T C No torque curve T is provided. C The additional resistance torque. In process 184, when the control device has displacement in the displacement region between δ1 and δ2, the torque curve T2 has a distance from the torque curve T. C The steep negative slope means that the resistance experienced by the surgeon decreases abruptly. In process 186, when the control device has displacement in the displacement region between δ2 and δ3, the torque curve T3 has a torque sign reversal, which provides a force that propels the control device away from the nominal position and toward the displacement distance δ2. The torque curve T3 provides a tactile sensation of a strong push toward the button "click" sound or entry point that occurs at the displacement distance δ3. In process 188, once the control device has reached the displacement distance δ3, the torque curve T4 provides a greater torque than the torque curve T at the same displacement distance. C Small centering (orientation) force. For example... Figure 6 As shown, the torque curve T C and T S A symmetrical curve representing the torque curves of the two "buttons" is provided. The symmetrical torque curves can provide the same tactile stop feel, but this can occur when the rotational movement of the control device is in opposite directions. In an alternative embodiment, the calculated torque curve can vary at different displacements associated with the position of different menu items, and the magnitude of the torque provided can increase or decrease depending on a variety of factors including user preference, the user's gripping action associated with the stop amplitude, or the expected time between consecutive actions.

[0051] Figure 8 Explanation of the superimposed torque curve T on the controller C The tactile stopping torque curve T on the top E It is used to provide tactile feedback to the control device 36 after reaching the entry or "click" displacement distance δ3. In this embodiment, the tactile stop torque curve T E Create a tactile feel for entering or "clicking" a spring-loaded button and a tactile feel for recentering the stop. Figure 9 It is a description Figure 8 The flowchart 190 shows the torque curve. Once the control device 36 has reached the displacement distance δ3, the torque curve T4 provides a torque curve T at the same displacement distance. C A small centering (towards nominal) force. Follow torque curve T4 until the control unit reaches the displacement distance δ5 upon returning to the nominal position. In process 192, when the control unit reaches the displacement distance δ5, torque curve T5 provides the increased torque to the controller torque curve T. C The level. When torque curve T5 is applied to the control device, the user feels a sudden increase in resistance compared to T4. This sudden increase in resistance may cause the user to instinctively relax or release their grip, causing the control device to recenter to the nominal position in process 194, where δ equals zero.

[0052] In one alternative embodiment, a torque superimposed on the normal control device torque simulates a translation switch that provides ON and OFF switch positions at a displacement distance along a line in space. In another alternative embodiment, a torque superimposed on the normal control device torque simulates a rotary switch that provides ON and OFF switch positions at an angular displacement distance around a spatial axis. This embodiment can be similar to the button embodiment described in detail above, except that a recentering force is not provided to move the control device toward a nominal position. Instead, the control device remains in the selected position (i.e., the "click state") until an action is performed, at which point the stop is deactivated or reset. In another alternative embodiment, a torque superimposed on the normal control device torque simulates a spring-loaded translation button moving along a line in space.

[0053] One or more elements in embodiments of the present invention may be implemented in software to execute on a processor of a computer system, such as a control processing system. When implemented in software, the elements of embodiments of the present invention are essentially code segments for performing necessary tasks. The program or code segment may be stored in a processor-readable storage medium or device, and may be downloaded via computer data signals embodied in the form of a carrier wave over a transmission medium or communication link. A processor-readable storage device may include any medium capable of storing information, including optical, semiconductor, and magnetic media. Examples of processor-readable storage devices include electronic circuits; semiconductor devices, semiconductor storage devices, read-only memory (ROM), flash memory, erasable programmable read-only memory (EPROM); floppy disks, CD-ROMs, optical disks, hard disks, or other storage devices. The code segment may be downloaded via a computer network such as the Internet, Intranet, etc.

[0054] It should be noted that the presented processes and displays are not inherently related to any particular computer or other device. Various general-purpose systems may be used with the programs based on the teachings herein, or it may prove convenient to construct more specialized devices to perform the described operations. The necessary structures for various such systems will appear as elements in the claims. Furthermore, embodiments of the invention are described without reference to any particular programming language. It should be understood that the teachings of the invention as described herein can be implemented using a variety of programming languages.

[0055] Although certain exemplary embodiments of the invention have been described and illustrated in the accompanying drawings, it should be understood that such embodiments are merely exemplary and not intended to limit the invention, and that the embodiments of the invention are not limited to the specific constructions and arrangements shown and described, as various other modifications can be made by those skilled in the art.

Claims

1. A haptic feedback method, comprising: A remote operation control system is provided, the remote operation control system comprising: a first operating mode for operating a remotely operated instrument in response to movement of a control device in a first degree of freedom; and a second operating mode for controlling a graphical user interface; Enter the second operating mode of the remote operation control system; When in the second operating mode, the movement of the control device is tracked in a second degree of freedom different from the first degree of freedom; and When in the second operating mode and in response to the movement of the control device in the second degree of freedom, a first tactile force is applied to the control device via a remote operation control system.

2. The tactile feedback method according to claim 1 further includes: It is determined that the control device has moved a first displacement distance from the nominal position to the entered position in the second degree of freedom, and A second tactile force is applied to the control device to provide a tactile sensation indicating the entry state of the second operating mode.

3. The tactile feedback method according to claim 2, wherein, The second tactile force provides a tactile sensation indicating the initial stage of the state of entry, and the method further includes: A third tactile force, opposite to the second tactile force, is applied to the control device to push the control device to the entry position. The third tactile force provides tactile sensation throughout the entire phase of the entry state, indicating the second operating mode.

4. The tactile feedback method according to claim 2 further includes: A third tactile force is applied to the control device to push the control device toward the nominal position.

5. The tactile feedback method according to claim 4 further includes: When the control device returns to the nominal position, the counter value is incremented.

6. The tactile feedback method according to claim 4 further includes: Determine the distance between the control device and the nominal position as a third displacement distance, and A fourth tactile force, greater than the third tactile force, is applied to the control device to push the control device toward the nominal position.

7. The tactile feedback method according to claim 4, wherein, The third tactile force is less than the first tactile force.

8. The tactile feedback method according to claim 1, wherein, The second degree of freedom is the rotational degree of freedom about the rotation axis of the control device.

9. The haptic feedback method according to claim 8, wherein, The first degree of freedom is a linear degree of freedom.

10. The tactile feedback method according to claim 2, wherein, The graphical user interface includes a selector icon, a nominal marker, and a selection marker, wherein the selector icon identifies the nominal marker when the control device is in the nominal position, and the selector icon identifies the selection marker when the control device is in the enter position.

Images