Medical robot guidance system having an integrated touch display, and operating method

Abstract

The invention relates to a medical robot guidance system (1) for a surgical procedure on a patient (P), comprising: a robot (2) with a movable robot arm (4) and a robot head (6) attached at the end of the robot arm (4), in particular comprising an end effector (8) on the robot head (6) or as a robot head (6); a control unit (10) customized to control and move at least the robot (2); at least one touch display (12) customized to visually output at least one operating menu (14), and to detect a touch-sensitive input as a user input and to transmit said user input to the control unit (10) in order to control the robot (2) in particular; wherein the at least one touch display (12) is rigidly fixed to the robot head (6) and moves along with the latter. The invention also relates to a robot operating method and a computer-readable storage medium according to the alternative independent claims.

Description

[0001] Medical robot guidance system with integrated touch display and operating procedures

[0002] Description

[0003] Technical area

[0004] The present disclosure relates to a medical, in particular surgical, robot guidance system for a medical procedure, in particular a surgical intervention, on a patient. The robot guidance system comprises a robot with a robot arm movably connected to a robot base and a robot head connected to the end of the robot arm. In particular, the robot head comprises an end effector, or an end effector itself forms the robot head. Furthermore, the robot guidance system comprises a (central) control unit (in particular with a processor and a memory unit) adapted to control and move at least the robot, in particular the robot arm and the robot head (and thus in particular the end effector).Furthermore, the robot guidance system comprises at least one touch display / touchscreen / touch-sensitive screen adapted to visually display at least one operating menu (for control) and to detect a touch-sensitive input as an operating input and send it to the control unit, in particular to control the robot. In addition, the disclosure relates to a (robot) operating method and a computer-readable storage medium according to the preambles of the independent claims.

[0005] Background of the Revelation

[0006] Surgical guidance systems are increasingly being used in surgical procedures, especially minimally invasive procedures. Due to technological advancements and the increasing specialization of various subsystems with corresponding integration, the number of functions of medical systems is increasing dramatically. The ongoing expansion of guidance systems to include this multitude of different functionalities is making it increasingly difficult to provide the user with a centralized control or operating modality that allows them to maintain an overview of the functions and maintain control.

[0007] According to the state of the art, touch displays are currently provided for the operation of a robotic guidance system, for example on a medical cart, a medical tower or on the base of a medical (surgical) microscope, via which a medical professional can make an (operating) input for a corresponding control by means of a (operating / operation) menu.

[0008] One problem, however, is that the touch display is not centrally located in the surgical area, but rather at a distance from it. This circumstance makes accessibility difficult, while also hampering hygiene requirements, as the touch display is not designed to be sterile. Due to the distance and the sterility requirements, the surgeon conducting the procedure is usually unable to operate the touch display themselves.

[0009] US 2005 / 0041282 A1, for example, discloses a surgical microscope as a robotic guidance system with a touch display rigidly attached to a base / carriage of the surgical microscope. Using the touch-sensitive display, different functions can be controlled via different areas of the display. Due to this configuration, however, the surgeon must give another medical professional an instruction to control a function, which this professional then executes. Summary of the Disclosure

[0010] It is therefore the object of the present disclosure to avoid or at least mitigate the disadvantages of the prior art and, in particular, to provide a medical robot guidance system and an operating method that provides a user, such as a medical professional, in particular a surgeon, with intuitive and clear operation with an intuitive user interface, allowing the user full control over the system or multiple systems used during a medical procedure such as a surgical intervention. In particular, the operation should centrally consolidate different operating modalities and preferably be provided in an area that is easily accessible to the surgeon.

[0011] The objects of the present disclosure are achieved with regard to a generic robot guidance system by the features of claim 1, with regard to a generic operating method by the features of claim 13, and with regard to a computer-readable storage medium by the features of claim 15.

[0012] A basic idea of ​​the present disclosure thus provides for an operating modality, such as a robot controller, in the area of ​​a robot head or an end effector. In contrast to the prior art, the touch display as an input and output unit is not provided at a separate, remote location, such as on a medical tower or the like, but directly on a movable part of the robot, namely on the terminal robot head and thus in the area of ​​the end effector.

[0013] In other words, the at least one touch display is rigidly attached / mounted / fixed to the robot head, in particular to the end effector, and moves with it. Instead of providing a static touch display on the robot base, for example on a carriage of a robot-assisted surgical microscope, according to the present disclosure, the touch display on the robot head is dynamically moved. Since the end effector is arranged on the robot head or the end effector itself forms the robot head, the operating modality in the form of the touch display is arranged directly in the area of ​​the end effector and is easy and safe for a surgeon to access. Using the dynamically moving touch display, the surgeon can then centrally control various functions of the operating modalities, so to speak, or is provided with the option of controlling at least one movement of the robot and thus of the robot head centrally in the area of ​​the end effector.In particular, the operator can control different functions via the touch display, for example by displaying an associated operating menu for the corresponding function.

[0014] In other words, the disclosure describes a touch display / touchscreen / touch-sensitive screen embedded in a robot-assisted guidance system / robot guidance system for medical interventions. The touch display is connected via data technology to a central control unit (as an execution unit for software) and enables flexible control of various functions, for example, in non-sterile and sterile environments, as well as visualization of information. The robot guidance system can, for example, include both visualization-based guidance (e.g., a surgical microscope) and guidance of instruments (e.g., a trocar).

[0015] In even further words, the present proposal is for an embedded touch display on or at a robotic end effector (on or as a robot head) that enables a user to flexibly provide information and offer various control options in both sterile and non-sterile environments (e.g., via an external monitor). The touch display is rigidly connected to the robot end effector and, in particular, is aligned so that the surgeon has a good viewing angle in the most common surgical positions (especially orientations) of the guidance system. The touch display can show both interactive content (e.g., the operating menu) and non-interactive content (e.g., an annotation). The visualized content can, for example, display the same content as a larger main monitor of the surgeon (not located on the robot arm) or as other control displays that are not attached to the control unit.Alternatively or additionally, independent content, which may be situation-dependent, can also be displayed via the touch display.

[0016] The term "position" refers to a geometric position in three-dimensional space, which is specified in particular using coordinates of a Cartesian coordinate system. In particular, the position can be specified by the three coordinates X, Y, and Z.

[0017] The term "orientation," in turn, indicates an orientation (e.g., position) in space. One could also say that orientation indicates an orientation with a direction or rotation specification in three-dimensional space. In particular, orientation can be specified using three angles.

[0018] The term "attitude" encompasses both position and orientation. In particular, attitude can be specified using six coordinates: three position coordinates X, Y, and Z, and three angular coordinates for orientation.

[0019] An operating input can, for example, be a control command associated with a selection in the operating menu, which is sent to the control unit so that it executes the corresponding function directly or indirectly via a control unit of a subsystem. For example, a central control unit can indirectly control a visualization system via a sub-control unit, or it can indirectly control a navigation system via a sub-control unit, for example, setting a waypoint.

[0020] Advantageous embodiments are claimed in the subclaims and are explained in particular below.

[0021] According to one embodiment, the robot guidance system can be designed in the form of a (surgical) operating microscope with a microscope head connected to the robot arm as the robot head, or in the form of a navigation system with a camera system, in particular with a laser system, connected to the robot arm. The microscope head can be actively controlled and moved, and a surgeon can make operating inputs directly on the microscope head via the touch display, for example with regard to zoom, alignment and / or illumination and / or movement / movement of the microscope head (change of position). In the case of a navigation system, for example, the view of the touch display and / or the view on the navigation monitor can be changed and adjusted in order to obtain a better overview or to be able to track instruments even better.

[0022] In particular, the robot arm of the robot can be configured such that the robot head is adjustable in both its position and its orientation, i.e., its attitude (or has six degrees of freedom / 6DOF). In particular, the robot arm can have at least a first and a second robot arm segment, which are connected to one another via a joint, and the robot head can be connected to the robot arm via a further joint, and the robot arm can be connected to the robot base via an additional joint. In particular, the joint of the first and second robot arm segments can have a rotational degree of freedom for rotation about an axis of rotation, wherein this axis of rotation can be arranged in a kinematic position of the robot arm, in particular in a horizontal direction (i.e., perpendicular to a top-bottom direction), in order to provide a type of boom (similar to an excavator arm).In particular, the robot can be designed in the form of an articulated arm robot.

[0023] In particular, the robot head is rigid and connected to the robot arm via a bearing or joint.

[0024] The robot arm can preferably have at least three robot arm segments, each of which is connected to one another via a joint. In particular, the robot (or the robot's control unit) can control the position and orientation of the robot head via multi-element actuated kinematics. The surgical microscope is preferably a digital microscope that creates and digitally provides a digital microscope image using a sensor, such as a CMOS sensor.

[0025] Preferably, the touch display can be arranged on a lateral side of the robot head (i.e., not on a rear side extending along a longitudinal axis of the robot head). In particular, the touch display can be arranged on the robot head such that a normal to the display surface is substantially perpendicular to a longitudinal axis of the robot head, such as a viewing axis of a surgical microscope.

[0026] In particular, in the case of a microscope head designed as a robot head, the touch display can be arranged laterally (i.e. not on the rear) in an area opposite the optical output. Alternatively or additionally, the touch display can be arranged on a side of a microscope head facing away from the optical output, i.e. virtually opposite (one front side is adapted for the optical output, in particular has a lens of an optical system, and the opposite front side has the touch display). It can also be said that the at least one touch display is arranged "on top" of the microscope (camera) head on a lateral and / or upper side, while the optical output is provided "below". An optional control button orAn actuating button or an input device in the form of a joystick or a 3D mouse can be arranged at an axial position between the optical output (bottom) and the touch display (top), as seen in the longitudinal axis direction of the robot head, in particular can be provided directly below the touch display. The robot arm can also preferably be connected to the robot head via a joint between the optical output (bottom) and the touch display (top), as seen in the longitudinal axis direction of the robot head. In particular, the touch display represents the "uppermost" or most terminal element or component, which is provided laterally and / or on the front side of the robot head. The touch display can preferably have a round outer contour or shape, in particular a circular outer contour or shape.

[0027] According to one embodiment, the robot head in the form of a microscope head can have a cylindrical / cylindrical base body, wherein one round end face forms the optical output for the digital microscope or the microscope camera and the opposite round end face has the circular touch display.

[0028] In particular, an additional input device, for example in the form of a joystick or a 3D mouse, can be provided as an extension of or at the same height opposite the connection of the robot arm. If the robot head is connected to the robot arm on one lateral side via a joint, in particular a rotary joint, the input device is provided on the opposite lateral side, extending from the connection in an extension of an axis perpendicular to an optical axis (one could also say perpendicular to a longitudinal axis of the robot head).

[0029] In particular, the robot guidance system can be adapted to display a menu structure for accessing various functions via the touch display. Examples of use cases for the touch display are listed below. Thus, the touch display can display not only a single operating menu or a single display, but also provide the operator with a multitude of operating menus for controlling various functions. In particular, it is possible to switch from a main menu structure to several submenu structures and back.

[0030] Preferably, the robot guidance system can be adapted to output a robot control menu as an operating menu via the touch display in order to control the robot via an operating input (a movement), in particular to control a robot movement in six degrees of freedom, for example in six translational directions (two opposite directions of a Cartesian coordinate system +X / -X, +Y / -Y, +Z / -Z) and / or six rotational directions (clockwise or counterclockwise rotations around the respective axis). In other words, the touch display can be adapted to display an (operating) menu structure for accessing the function, which is adapted to control robot movements, in particular, in six degrees of freedom.

[0031] In particular, the robot guidance system can be adapted to output a visualization control menu as an operating menu via the touch display in order to control / change settings of a visualization system, in particular a zoom, a focus, and / or a lighting intensity (as settings). In other words, the touch display can be adapted to display a menu structure for accessing the function: specifying settings of a visualization system (e.g., a surgical microscope), such as zoom, focus, and light intensity, via a touch bar / slider, in particular.

[0032] According to a further embodiment, the robot guidance system can be adapted to switch between at least two different control menus, in particular between at least one robot control menu and one visualization control menu, in order to control at least two different functions of the robot guidance system, in particular a movement of the robot as a first function and a visualization as a second function.

[0033] In particular, the robot guidance system can be adapted to output a visualization control menu as an operating menu via the touch display, which allows control of various light / imaging modes of the visualization system, in particular control of fluorescence for in particular ICG (indocyanine green), 5-ALA (5-aminolevulinic acid).

[0034] In particular, the robot guidance system can further be adapted to output a navigation control menu as an operating menu via the touch display, which allows saving of waypoints and / or robot configurations and / or navigation positions in relation to the patient (or a patient position) and in particular provides a so-called "Hold and Drive" function, in which a long press on one of the stored and displayed data causes the system to move to the stored points or predefined positions.

[0035] In particular, the robot guidance system can further be adapted to output a navigation control menu as an operating menu via the touch display, which allows or provides control of navigation processes, in particular point digitization and verification for patient registration and calibration / activation of tools.

[0036] In particular, the robot guidance system can further be adapted to output an instrument control menu as an operating menu via the touch display, which allows control of instrument guidance functions such as driving the robot or the instrument as an end effector on a target trajectory or provides switching on or off of an instrument function.

[0037] In particular, the robot guidance system can also be adapted to display a media control menu as an operating menu via the touch display, which allows control of the recording of images (e.g., as current snapshots) and / or video data from a visualization system. This allows the surgeon, for example, to create a recording at an initial point in time using an operator input and then display this recording on the touch display or an external monitor at a later point in time, for example, to perform a before-and-after comparison or to access information about the surgical site.

[0038] In particular, the robot guidance system can be adapted to output a media control menu as an operating menu via the touch display, which allows control of the playback and management of the recorded media data, wherein the played video can be shown, in particular, on other displays (in addition to or alternatively to the touch display). In particular, the robot guidance system can be adapted to output a navigation control menu as an operating menu via the touch display, which allows or provides control of settings for the information displayed on the visualization monitor, in particular allowing planned trajectories, operation targets, or other navigation information to be shown or hidden.

[0039] In particular, the robot guidance system may further be adapted to output a navigation control menu as an operating menu via the touch display, which provides control of switching between different monitor layouts of the main visualization monitor.

[0040] In particular, additional situation-dependent content based on information from the control unit / controller can be displayed to save the user time navigating through the operating menu. In particular, the following situation-dependent content (for example, depending on a robot configuration or the current status of an operation plan) can be displayed individually or in a selectable combination:

[0041] - Settings of a visualization system can be highlighted after the guidance system has been repositioned to enable quick adaptation of the visualization parameters to the new position;

[0042] - When movement of a navigated instrument is detected, the instrument guidance settings can be highlighted;

[0043] - Visualization of a current state / function of multi-purpose hardware buttons located in the vicinity of the display;

[0044] - Display of a safety stop button on the touch display to interrupt automated robot movements;

[0045] - Displaying notifications to the user (such as information, warnings and errors) both with and without requesting user feedback;

[0046] - Visualization of a distance to the target for instrument guidance with depth tracking / depth indication; - Rendering of an image of a visualization system for (rough) positioning of the system;

[0047] - In particular, a display mode of the touch display can be switched to a so-called "trackpad" mode, in which: o a mouse symbol is activated on the main visualization screen, in particular an operating room monitor, thus enabling the user to operate the visualization screen as a computer screen with a mouse / laptop trackpad (via the touch display); or o it enables the user to scroll through the navigation views / panes (segmentations) on the main visualization screen, in particular the operating room monitor.

[0048] In particular, the touch display attached to the robot head can have a sterile casing, which is preferably designed to be replaceable, in order to provide a sterile barrier against a sterile intervention point. For example, for operation in a sterile environment, the touch display can be covered with a transparent medical (surgical) drape, with connection points / coupling points for the surgical drape being provided.

[0049] According to a further embodiment, an inertial measurement unit (IMU) can be provided on or in the robot head, in particular provided on or in the touch display or fastened to the end effector, in order to detect a position and / or orientation of the touch display or the robot head, and the robot guidance system, in particular the control unit, can be adapted to adapt an orientation of a visual output of the touch display based on the detected position and / or orientation, in particular to adapt an output (for example of the operating menu) such that it is always displayed in a constant horizontal orientation.In other words, an inertial measurement unit (IMU) can optionally be connected to the touch display or the robot head or the end effector in order to be able to change the orientation of the visualized content in different positions and / or orientations, in particular extreme positions, of the guidance system. Alternatively or additionally, a position and / or orientation of the touch display or the robot head can be detected, preferably via robot kinematics or a time-current configuration of the robot with its robot arm segments and its robot head as an end effector or with a connected end effector. Furthermore, a position and / or orientation of the touch display or the robot head can preferably be detected via the navigation camera of the navigation system and corresponding data processing.In particular, the control unit can be adapted to adjust a display based on the detected position of the touch display so that it is optimally displayed relative to a predetermined position in space, namely the position at which a surgeon's head is located (for example, this position can be detected by a navigation camera). This can, in particular, comprise rotating the view of the touch display so that the view is preferably displayed approximately horizontally relative to the floor of an operating room, and thus relative to the surgeon, as well as stretching or compressing the view to provide the most neutral, natural view possible in the detected position of the surgeon (similar to arrows or labels on a roadway or road markings, which are adapted (stretched) for a driver so that the driver can recognize the information as clearly as possible).In particular, the control unit or the touch display can be adapted to output such a projection onto a virtual surface via the touch display, wherein the surface is perpendicular to a line of sight between the surgeon and the touch display, that the surgeon does not see a distorted representation (if the touch display is oblique to a line of sight), but such a representation similar to if he were looking perpendicularly at the touch display.

[0050] The touch display can preferably have a radio connection module to (independently) establish a wireless data connection to the control unit, in particular via WLAN or Bluetooth. According to an alternative embodiment, the touch display can also be connected to the control unit via a data cable. In other words, to control various functionalities of the guidance system for the situation-dependent presentation of content, the touch display can be connected to the control unit (as a computer system) via a touch display control unit, either by cable or wirelessly (with a radio connection module). In particular, the touch display itself has an independent sub-control unit as a computer system in order to independently form an independent control complement, which can be integrated into the overall system and can be coupled to the (central) control unit for data purposes.

[0051] Further preferably, the touch display can have a display diagonal or a display diameter of at least 4 cm and / or a maximum of 20 cm. In other words, the size of the touch display can be between 4 cm and 20 cm. In particular, the touch display can have a square, rectangular, or round form factor. This size and shape can be based, for example, on the number of functions and the size of the connected robot end effector. The size and shape facilitate the advantageous integration of the touch display into the end effector area.

[0052] According to one embodiment, the robot head can have at least one (physical) actuation button (hardware button), preferably three actuation buttons, and the touch display can be arranged directly adjacent to the button and adapted to display the current functional status ( / current functionality) of the button. This creates a multi-purpose button or a multi-function button that can record different operating inputs for different operating menus. This can also increase safety, in particular, if, for example, an input via the touch display fails or an input cannot be made accurately due to dirty surgical gloves.

[0053] In particular, the touch display has a color display to show color operating menus and / or information.

[0054] In a further embodiment, in addition to the touch display on the robot head, a touch display can be provided on the robot arm and / or on a robot base.

[0055] In particular, the robot guidance system can be designed as a robot guidance unit, with all components integrated into this unit or a single module. In particular, the robot guidance system can be designed to be mobile and can be independently deployed and moved within an operating room, for example, as a mobile surgical microscope with the touch display mounted on its microscope head (as the robot head and end effector).

[0056] The objects are achieved with regard to a medical robot operating method, in particular for a robot guidance system according to the present disclosure, by the steps of: outputting at least one visual operating menu via a touch display rigidly attached to a robot head of a robot; detecting a touch-sensitive input as operating input via the touch display; sending the operating input to a control unit adapted to control at least the robot; controlling a function, in particular a movement of a robot, by the control unit based on the operating input. As described above, this step can provide a flexible and central operating method by means of which the surgeon can perform various functions, for example, a movement of a robot or a setting of a visualization system.

[0057] Preferably, the (operating) method may further comprise the steps:

[0058] Detecting (a change in) the position and orientation of the robot head, in particular by means of an inertial measurement unit (IMU) or by means of detected robot kinematics or by means of a navigation camera of a navigation system; sending the detected position and orientation to the control unit; calculating an orientation of the visual operating menu adapted to the position and orientation; and outputting the adapted visual representation of the operating menu. In this way, the operator can be provided with the best possible or best-adapted visual output and, in particular, does not have to turn their head to read a label or information or reposition themselves in order to clearly recognize the information in the display, such as text, when viewed from an oblique view of the touch display.

[0059] With regard to a computer-readable storage medium, the objects are achieved in that it comprises instructions which, when executed by a computer, cause the computer to execute the method steps of the operating method according to the present disclosure. In particular, the control unit of the robot guidance system can comprise such a computer-readable storage medium.

[0060] Any disclosure related to the robot guidance system according to the present disclosure also applies to the operating method of the present disclosure, and vice versa.

[0061] Short description of the characters

[0062] The present invention will be explained in more detail below using preferred embodiments with reference to the accompanying figures. They show:

[0063] Fig. 1 is a schematic front view of a robot guidance system according to a preferred embodiment of the present disclosure;

[0064] Fig. 2 is a schematic view of a functional relationship between touch display, control unit and IMU of a robot guidance system according to another preferred embodiment;

[0065] Figs. 3a and 3b show an exemplary view of an operating menu in a table-like arrangement and in a circular arrangement;

[0066] Fig. 4 is a schematic view of an exemplary submenu of a visualization control menu for adjusting a light intensity;

[0067] Fig. 5 is a schematic view of an exemplary submenu of a media control menu for managing and playing back a recorded video; Fig. 6 is a schematic view of an operating menu with only a stop button as an interactive input;

[0068] Fig. 7 a schematic front view of a robot head or end effector with three buttons and a touch display;

[0069] Fig. 8 is a schematic front view of a robot guidance system according to another preferred embodiment of the present disclosure;

[0070] Figs 9 to 11 show various views of a robot guidance system with a robot head with a touch display, which is arranged on a side facing away from the optical output, at the top of the microscope head; and

[0071] Fig. 12 is a flowchart of an operating method according to a preferred embodiment.

[0072] The figures are merely schematic and are intended only to aid understanding of the invention. Identical elements are provided with the same reference numerals. The features of the various embodiments can be interchanged.

[0073] Detailed description of preferred embodiments

[0074] Fig. 1 shows a schematic front view of a medical robot guidance system 1 (hereinafter referred to as guidance system) for a surgical procedure on a patient P. The guidance system 1 has a robot 2 with a movable robot arm 4 and a robot head 6 connected to the end of the robot arm 4. In this embodiment, the guidance system 1 is designed in the form of a robot-guided surgical microscope. The microscope head is provided as the robot head on the robot arm 4 with several robot arm segments. Therefore, the robot head or microscope head as a whole can also be referred to as the end effector 8 of the surgical microscope. In order to control the robot 2 and its robot arm 4 and to adjust the position and orientation of the microscope head as the end effector 8, the guidance system 1 has a control unit 10.The control unit 10 can be designed as a central control unit, for example as a computer system that can process and control different functions, or it can have a subsystem of a robot control unit in order to control the robot accordingly.

[0075] Furthermore, the guidance system 1 has a touch display 12 as an input and output unit. This is adapted to visually display various operating menus 14 and displays for different functions, and to capture a touch-sensitive input as an operating input and send it to the control unit 10 in order to both control the robot 2 and make settings on the visualization system.

[0076] In contrast to the prior art, however, the touch display for control is not attached to a static base of the guidance system 1, but the touch display 12 is rigidly fixed directly to the robot head 6 or to the end effector 8 and moves with it.

[0077] In this way, the touch display 12 moves in the area of ​​the procedure, providing the surgeon with a central visualization of various operating menus 14 and allowing flexible control of the various functions of different subsystems via a single touch display. In this embodiment, the surgeon can select from at least one robot control menu and visualization control menu, or switch between the individual control menus as operating menus 14. Using the robot control menu, the surgeon can, for example, guide an instrument or move the robot head 6 so that the end effector 8 is in a better position for the procedure. In the present case, the microscope head, as the end effector, can be moved manually or automatically via the operating input of the touch display 12 into a position that allows a better view.Additional images can be output via an external (surgical) monitor 15, with media output being controllable via an additional media control menu on the touch display 12. For example, a current image from the microscope can be output via the surgical monitor 15, with brightness or color contrast being adjustable via a media control menu.

[0078] In the robot guidance system 1 according to the first embodiment, the operator is provided with at least three control menus: robot control menu, visualization control menu and media control menu, which he can operate centrally in order to control various functions flexibly and safely.

[0079] Fig. 2 shows a schematic functional view of an interaction between individual modules of a guidance system 1 according to a further, second preferred embodiment. Here, an inertial measurement unit (IMU) 16 is arranged directly on the touch display 12 in order to detect a movement (or a change in a movement via acceleration) and, based on an initial position, to determine a new position and orientation of the touch display 12 after a movement of the robot. The data from the IMU is sent directly to a central control unit 12. In addition, the central control unit 10 receives data from an end effector control unit 18 (as a sub-control unit). Furthermore, the central control unit 12 is in reciprocal data communication with a touch display control unit (as a sub-control unit). This central control unit 12 then calculates a visual output based on this data, which is then displayed on the touch display 12.

[0080] In particular, this allows a view most favorable for the surgeon to be calculated and displayed accordingly. Since, in contrast to the prior art, the touch display is no longer statically arranged but moves dynamically with the end effector 8, the orientation of the display changes relative to the surgeon. For example, if the robot arm 4 in Fig. 1 is pivoted upward by 90°, without correction, the visual view of the touch display 12 would also be rotated 90° counterclockwise relative to the surgeon.

[0081] However, to allow the surgeon to continue to read the visual display clearly, the control unit 12 calculates such an orientation—in the above example, a 90° clockwise rotation of the display relative to the touchscreen—so that the rotation and counter-rotation cancel each other out, allowing intuitive reading by the surgeon. In other words, when the robot 2 moves, the control unit calculates such an orientation of the display that appears as consistent as possible to the surgeon and, to a certain extent, enables a consistent relationship between the touchscreen display and the operator.

[0082] In one embodiment, for example, when changing from a front view of the touch display 12 (i.e., vertical view) to an oblique view, the visual output can also be displayed in a correspondingly distorted manner (similar to a road marking in the form of a label, which is also displayed lengthwise in order to offer the driver the best possible view).

[0083] Fig. 3a and Fig. 3b are exemplary views of an operating menu displayed by the touch display. Fig. 3a shows a table-like arrangement with rectangular, symbol-labeled (touch) buttons, and Fig. 3b shows a circular arrangement of symbol-labeled (touch) buttons. Using these, the user can, for example, select a submenu, which is illustrated in Figs. 4 and 5 and explained below.

[0084] Fig. 4 shows a submenu of a lighting control menu when the symbol-labeled light bulb is selected in the main menu. In this embodiment, the user can adjust the brightness of a white light, as well as the intensity of UV and IR radiation.

[0085] If the user selects the video icon in the main menu shown in Fig. 3a or 3b, the menu jumps to the video function submenu shown in Fig. 5, where video data management and playback functions can be selected. The video can be played back on an external surgical monitor (not shown here).

[0086] Fig. 6 shows a schematic view of a (digital) stop button on the touch display 12 in order to stop a movement of the robot 2 or the robot arm 4 during this movement (type of emergency button).

[0087] Fig. 7 shows a front view of an end effector of a robot guidance system 1 of another preferred embodiment. In this embodiment, three physical pushbuttons / switches / actuator buttons 22 are provided on the end effector 8, arranged in a straight line at equal distances from one another. Parallel to and above the linear arrangement of the three actuating buttons 22, a touch display 12 is attached to the end effector, which displays the function status above each corresponding actuating button 22. In this way, multiple functions can be assigned to the three actuating buttons 22 and visualized accordingly by the touch display.

[0088] For example, the control unit 12 can assign different functions to the actuation buttons for different steps in the operation plan and have them output via the touch display.

[0089] Fig. 8 shows a schematic front view of another embodiment of a guidance system 1. The robot 2 has a stationary robot base 24, to which the robot arm 4 is movably connected. The robot head 6 in the form of a (digital) microscope head is connected to the end of the robot arm 4 as an end effector, so that the robot arm 4 can adjust both the position and the orientation (i.e. the spatial location) of the microscope head in order to assume a suitable recording position and create a digital microscope image. The optical axis of the microscope head is shown as a (vertical) dashed line, which passes through the optical system (not shown) and, as an extension, a CMOS sensor for a digital recording.By adjusting both the position (three degrees of freedom) and the orientation (three degrees of freedom), six degrees of freedom / 6DOF (degree of freedom) can be set (within the robot kinematics).

[0090] In this embodiment, the robot arm 4 has a plurality of robot arm segments 26, each of which is connected to one another via a joint 28. The robot head 6 with the optical system is also connected to the robot arm 4 via a further joint 28, and the robot arm 4 is connected to the robot base 24 via a joint 28. In particular, the joint of the first and second robot arm segments 26 can have a rotational degree of freedom for rotation about an axis of rotation, wherein this axis of rotation can be arranged in a kinematic position of the robot arm 4 shown in Fig. 8, in particular in a horizontal direction (i.e., perpendicular to a top-bottom direction) in order to provide a type of boom.

[0091] The robot head 6 is rigid (or has a rigid robot head housing in which the optical system, the downstream CMOS sensor, and any other electronics are housed). Thus, the robot 2 (or the robot's control unit) can control the position and orientation of the robot head 6 via a multi-element actuated kinematics and adjust the optical axis accordingly.

[0092] In this embodiment, the touch display 12 is arranged on a lateral side of the robot head 6 (i.e., not on a rear side extending a longitudinal axis of the robot head). In particular, the touch display can be arranged on the robot head such that a normal to the display surface is substantially perpendicular to a longitudinal axis of the robot head, such as a viewing axis of a surgical microscope.

[0093] The touch display 12 is arranged such that, as seen in Fig. 8, it is at the top of the robot head 6. While, as seen in Fig. 8, a lower side forms the optical opening for the optical system of the digital microscope, the touch display is arranged in an upper region. One could also say that the optical output is provided in a first, lower region of the robot head 6, while the touch display is provided in a second, upper region of the robot head, with the first and second regions forming terminal regions facing away from one another. The touch display 12 is therefore arranged such that, during a normal operation, in which the surgical microscope or microscope head looks down onto the patient from above, the touch display arranged on the side is particularly clearly visible and operable for medical personnel. The operating buttons 22 are shown in Fig.8, it is arranged below the touch display 12, so that the touch display forms, in a sense, the uppermost element. Viewed along a longitudinal axis of the robot head 6, the following are provided in the axial position in the following order: touch display 12, control button, and optical output. The robot arm 4 is connected laterally to the robot head 6 via the joint 28, and the three dashed rotational indications are intended to represent that the robot head can adjust its orientation around three axes, or has three degrees of freedom of rotation.

[0094] Figs. 9 to 12 show a side view, a perspective top view, and a perspective isometric view of a medical robot guidance system 1 according to a further preferred embodiment. The robot 2 is designed in the form of a robot-guided surgical microscope, whose robot head 6 forms the end effector 8 as the microscope head, and the position and orientation of the microscope head can be adjusted via the robot arm 4. The optical output is provided on a lower side, as seen in Fig. 9, with the optical axis shown in dashed lines. The touch display 12 is arranged on the side opposite the optical output. In this embodiment, the touch display 12 is circular, with a center point of the circular touch display 12 lying essentially in extension of the optical axis, i.e., concentric with it to a certain extent.One can also say that the normal to the display is parallel to the optical axis, in particular on the optical axis (optical system and touch display are somewhat symmetrical to each other).

[0095] The touch display 12 forms a frontal, upper, flat surface and is offset from the rest of the microscope head (and even the robot arm 4) (i.e., forms a projection) to enable good operation. Viewed in the direction of the longitudinal axis of the robot head or the optical axis, from top to bottom, the touch display 12 is provided first, followed by an input device 30 in the form of a 3D mouse or a 3D space mouse, and then the optical output (with approximately the last lens of the optical system). The input device 30 is also arranged in extension to a longitudinal axis of a cylindrical arm with a pivot joint 28 to a robot arm segment 26 on an opposite side of the connection to the robot arm 4. A zero axis of the 3D mouse is concentric with an axis of the robot arm segment 26.

[0096] By means of additional operating buttons 22, a movement and / or a zoom and / or other functions can be controlled, depending on the function status.

[0097] The robot head 6 is connected to the robot arm 4 (or to the robot arm segment 26) by means of a rotary joint 28. An orthogonal line to the touch display 12 is also perpendicular to a rotation axis, whereby the orthogonal line to the touch display 12 is parallel to the optical axis.

[0098] Fig. 12 shows a flowchart of an operating method according to a preferred embodiment.

[0099] In a first step S1, an operating menu 14 is displayed via the touch display 12.

[0100] Specifically, in this embodiment of the operating method, the output of the operating menu is achieved through the following substeps. In a first substep S1.1, a position and orientation (i.e., the position) of the touch display 12 are detected (preferably also indirectly via the robot head 6), in particular by an inertial measurement unit (IMU) 16. Alternatively, the position can also be detected via a detected (mechanical) robot kinematics or by a tracking camera / navigation camera of a navigation system.

[0101] The position of the touch display 12 is then passed on to the control unit 12 in sub-step S1.2. This control unit then calculates an adjusted orientation for the output in sub-step S1.3 and then outputs this adjusted representation via the touch display 12 in sub-step S1.4.

[0102] In a subsequent second step S2, an input from the touch display 12 is then recorded.

[0103] In a third step S3, the recorded operating input is then sent to the control unit 12.

[0104] Finally, in step S4, a corresponding function, as displayed and selected on the touch display 12, is controlled.

[0105] In particular, this can be a control of the robot 2.

[0106] List of reference symbols

[0107] 1 Medical robot guidance system

[0108] 2 robots

[0109] 4 Robot arm

[0110] 6 Robot head

[0111] 8 End effector

[0112] 10 Control unit

[0113] 12 touch display

[0114] 14 Operating menu

[0115] 15 External monitor

[0116] 16 Inertial Measurement Unit (IMU)

[0117] 18 End effector control unit

[0118] 20 Stop button

[0119] 22 Actuation button

[0120] 24 Robot Base

[0121] 26 Robot arm segment / robot arm link

[0122] 28 joint

[0123] 30 input devices

[0124] P Patient

[0125] 51 Step Output of an operating menu through the touch display

[0126] 51 .1 Step Determine position and orientation by IMU

[0127] 51 .2 Step Send to control unit

[0128] 51 .3 Step Calculate adjusted alignment

[0129] 51 .4 Step Outputting the adjusted representation

[0130] 52 Step Capturing an Input from the Touch Display

[0131] 53 Step Send operating input to control unit

[0132] 54 Step Controlling a Function

Claims

Claims 1. A medical robot guidance system (1) for a surgical procedure on a patient (P), comprising: a robot (2) with a movable robot arm (4) and a robot head (6) connected to the end of the robot arm (4), in particular with an end effector (8) on the robot head (6) or as the robot head (6); a control unit (10) adapted to control and move at least the robot (2); at least one touch display (12) adapted to visually output at least one operating menu (14) and to detect a touch-sensitive input as an operating input and to send this to the control unit (10), in particular to control the robot (2); characterized in that the at least one touch display (12) is rigidly fixed to the robot head (6) and moves with it.

2. Medical robot guidance system (1) according to claim 1, characterized in that the robot guidance system (1) is designed in the form of a surgical microscope with a microscope head connected to the robot arm (4) as a robot head (6) or in the form of a navigation system with a camera system, in particular with a laser system, connected to the robot arm (4).

3. Medical robot guidance system (1) according to one of the preceding claims, characterized in that the robot guidance system (1) is adapted to output a robot control menu as an operating menu (14) via the touch display (12) in order to control the robot (2) via an operating input, in particular to control a robot movement in six degrees of freedom.

4. Medical robot guidance system (1) according to one of the preceding claims, characterized in that the robot guidance system (1) is designed is adapted to output a visualization control menu as an operating menu (14) via the touch display (12) in order to control settings of a visualization system, preferably a surgical microscope, in particular a zoom, a focus and / or an illumination intensity.

5. Medical robot guidance system (1) according to one of the preceding claims, characterized in that the robot guidance system (1) is adapted to switch between at least two different control menus in the touch display (12), preferably between the robot control menu and the visualization control menu, in order to control at least two different functions of the robot guidance system, in particular a movement of the robot (2) as a first function and a visualization as a second function.

6. Medical robot guidance system (1) according to one of the preceding claims, characterized in that the touch display (12) fixed to the robot head (6) has a sterile sheath, which is preferably designed to be replaceable in order to provide a sterile barrier against a sterile intervention site.

7. Medical robot guidance system (1) according to one of the preceding claims, characterized in that an inertial measuring unit is further provided on or in the robot head (6), in particular provided on or in the touch display (12) or fastened to the end effector (8) in order to detect a position and / or orientation of the touch display (12), and the robot guidance system (1), in particular the control unit (10), is adapted to adapt an orientation of a visual output of the touch display (12) based on the detected position and / or orientation, in particular to adapt an output such that it is always displayed in a constant horizontal orientation.

8. Medical robot guidance system (1) according to one of the preceding claims, characterized in that the touch display (12) has a radio connection module in order to establish a wireless data connection to the control unit, in particular via WLAN or Bluetooth.

9. Medical robot guidance system (1) according to one of the preceding claims, characterized in that the touch display (12) has a display diagonal or a display diameter of at least 4 cm and / or a maximum of 20 cm.

10. Medical robot guidance system (1) according to one of the preceding claims, characterized in that the robot head (6) has at least one actuation button (22), preferably three actuation buttons (22), and the touch display (12) is arranged directly adjacent to the actuation button (22) and is adapted to display the current functional status of the actuation button (22).

11. Medical robot guidance system (1) according to one of the preceding claims, characterized in that the robot guidance system (1) is designed in the form of a surgical microscope with a microscope head connected to the robot arm (4) as a robot head (6), wherein an optical output forms one side of the microscope head and opposite this side the touch display (12) is arranged on the front side, in particular an orthogonal to the touch display (12) is parallel to an optical axis.

12. Medical robot guidance system (1) according to one of the preceding claims, characterized in that the touch display (12) has a round shape, in particular a circular shape, and preferably when the robot (2) is designed as a robot-guided surgical microscope, a center point of the circular touch display (12) lies in the extension of an optical axis.

13. Medical robot operating method, in particular for a medical robot guidance system (1) according to one of the preceding claims, characterized by the steps: Outputting (S1) at least one visual operating menu (14) as a visual representation via a touch display (12) which is rigidly attached to a robot head (6) of a robot (2); Capturing (S2) a touch-sensitive input as operating input via the touch display (12); Sending (S3) the operating input to a control unit (10) adapted to control at least the robot (2); Controlling (S4), on the basis of the operating input, a function, in particular a movement of a robot (2), by the control unit (10).

14. Medical robot operating method according to claim 11, characterized in that the method further comprises the steps: Detecting (S1.1) a position and / or orientation of the robot head (6) by an inertial measuring unit; Sending (S1.2) the detected position and / or orientation to the control unit; Calculating (S1 .3) an orientation of the visual display, in particular of the visual operating menu (14), adapted to the position and / or orientation; Outputting (S1.4) the adapted visual representation, in particular the adapted visual representation of the operating menu (14).

15. A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the method steps of the robot operating method according to one of claims 11 or 12.

Images