Medical robotic guidance system with integrated touch display and method of operation

Abstract

The present disclosure relates to a medical robotic guidance system (1) for surgery on a patient (P), having a robot (2) with a movable robot arm (4) and a robot head (6) connected at the end to the robot arm (4), in particular with an end effector (8) on the robot head (6) or as a robot head (6); a control unit (10) adapted to at least control the robot (2) and to move the robot; at least one touch display (12) adapted to visually output at least one operating menu (14) and to detect a touch-sensitive input as an operating input and to send the touch-sensitive input to the control unit (10) in order to control, in particular, the robot (2); wherein the at least one touch display (12) is rigidly fixed to the robot head (6) and moves together with the robot head. The present disclosure also relates to a robot operation method according to the paralleling claims and a computer readable storage medium.

Description

Technical Field

[0001] The present disclosure relates to a medical robot guidance system for medical procedures, especially surgical operations, at a patient's location. The robot guidance system has a robot for this purpose, which has a robot arm movably connected to a robot base and a robot head connected to the robot arm at the end. In particular, the robot head has an end effector or the end effector itself constitutes the robot head. In addition, the robot guidance system has a (centralized) control unit (in particular with a processor and a storage unit), which is adapted to at least control and move the robot, in particular the robot arm and the robot head (and therefore in particular the end effector). In addition, the robot guidance system has at least one touch display / touch screen / touch-sensitive screen, which is adapted to visually output at least one operating menu (for control), and detects touch-sensitive input as an operating input and sends the input to the control unit, so as to control the robot in particular. The present disclosure also relates to a (robot) operating method and a computer-readable storage medium according to the preamble of the parallel claims. Background Art

[0002] Surgical guide systems are increasingly used in surgery, in particular minimally invasive surgery. Through further developments in technology and the continued specialization of different (sub)systems with corresponding combinations, the number of functions of medical systems is increasing significantly.

[0003] As navigation systems continue to expand with these multiple different functions, it becomes increasingly difficult to provide the user with a centralized operation or mode of operation that allows the user to maintain an overview and control over the functions.

[0004] In order to operate the robot guidance system, according to the prior art, a touch display is currently provided, for example on a medical cart, a medical tower or on the base of a medical (surgical) microscope, via which the medical professional can implement (operation) inputs for the corresponding control with the help of guidance through an (operation) menu.

[0005] However, the problem in this case is that the touch display is not arranged centrally in the operating area, but rather remotely therefrom. This makes accessibility difficult on the one hand and hygiene requirements difficult on the other hand, since the touch display is not arranged sterilely. Due to the distance on the one hand and the sterility requirements on the other hand, the surgeon leading the operation is usually unable to operate the touch display himself.

[0006] US2005 / 0041282 A1 discloses, for example, a surgical microscope as a robot guidance system with a touch display, which is rigidly fastened to a base / carriage of the surgical microscope. Different functions can be controlled via different areas of the display by means of the touch-sensitive display. However, due to this configuration, the surgeon must provide instructions to another medical professional for controlling the function, which the professional then controls. Summary of the invention

[0007] The object of the present disclosure is therefore to avoid or at least reduce the disadvantages of the prior art and in particular to provide a medical robot guidance system and an operating method which provide the user, such as a medical professional, in particular a surgeon, with an intuitive and clear operation with an intuitive operating interface which enables complete control of the system or systems used during a medical procedure, such as surgery. In particular, the operation should centrally bring together different operating modes and preferably be arranged in an area that is easily accessible to the surgeon.

[0008] The object of the disclosure is achieved according to the invention by the features of claim 1 with respect to a robot guidance system of this type, by the features of claim 13 with respect to an operating method of this type, and by the features of claim 15 with respect to a computer-readable storage medium.

[0009] The basic concept of the disclosure therefore provides that the operating mode, for example the control of the robot, is set in the area of ​​the robot head or the end effector. In contrast to the prior art, the touch display as an input and output unit is not arranged at a separate, remote location, for example on a medical tower or the like, but directly on the movable part of the robot, i.e. at the end of the robot head and therefore in the area of ​​the end effector.

[0010] In other words, at least one touch display is rigidly fastened / installed / fixed on the robot head, especially on the end effector and moves with it. Instead of setting a static touch display on the robot base, for example, on the vehicle of a robot-assisted surgical microscope, according to the present disclosure, the touch display moves dynamically on the robot head. Because the end effector is arranged on the robot head or the end effector itself constitutes the robot head, the operating mode in the form of a touch display is directly arranged in the area of ​​the end effector and can be easily and safely reached by the surgeon. Through the touch display that moves dynamically together, the surgeon can control the different functions of the operating mode to a certain extent, or provide the operator with the following possibility, that is, centrally control at least one movement of the robot in the area of ​​the end effector and thus control at least one movement of the robot head. In particular, the surgeon can control different functions through the touch display, for example, by displaying the operating menu belonging to the corresponding function.

[0011] In other words, the present disclosure describes a touch display / touch screen / touch-sensitive screen embedded in a robot-assisted guidance system / robotic guidance system for medical surgery. The touch display is connected to a central control unit (as a software implementation unit) in a data-technical manner and enables flexible control of different functions and visualization of information, for example in a non-sterile and sterile environment. The robotic guidance system can, for example, include not only a visualization-based guidance (e.g., a surgical microscope) but also a guidance for an instrument (e.g., a trocar).

[0012] In other words, the present invention proposes an embedded touch display (at or as a robot head) on or at a robot end effector, which is implemented to flexibly provide information and different control options to the user not only in a sterile environment but also in a non-sterile environment (e.g., through an external display). The touch display is rigidly connected to the robot end effector and is particularly oriented so that the surgeon has a good viewing angle in the most popular surgical positions (especially orientations) of the guidance system. The touch display can display not only interactive content (e.g., operating menus) but also non-interactive content (e.g., annotations). For example, the visualization content can display the same content as the surgeon's larger main monitor (not arranged on the robot arm) or display the same content as other control displays not installed on the controller. Alternatively or additionally, it is preferred that independent content that can depend on the situation can also be presented by the touch display.

[0013] The term "position" refers to a geometric position in three-dimensional space, which is described in particular with the aid of coordinates of a Cartesian coordinate system. In particular, the position can be described by three coordinates X, Y and Z.

[0014] The term "orientation" again describes the orientation in space (approximately at this position). It can also be said that the orientation is described by the orientation in a direction or rotation in three-dimensional space. In particular, the orientation can be described with the aid of three angles.

[0015] The term “position” includes both the position and the orientation. In particular, the position can be specified using six coordinates, three position coordinates X, Y and Z and three angular coordinates for the orientation.

[0016] The operating input may be, for example, a control command for selecting an operating menu, which is sent to the control unit so that the control unit directly or indirectly, for example, via the control unit of the subsystem, implements the corresponding function. For example, the central control unit may indirectly control the visualization system via the subcontrol unit of the visualization system, or the central control unit may indirectly control the navigation system via the subcontrol unit of the navigation system and, for example, may set a waypoint.

[0017] Advantageous embodiments are claimed in the dependent claims and are explained in particular below.

[0018] According to one embodiment, the robot guidance system can be implemented in the form of a (surgical) operating microscope having a microscope head connected to a robot arm as the robot head, or in the form of a navigation system having a camera system, in particular a laser system, connected to a robot arm. The microscope head can be actively controlled and moved, and the surgeon can make operating inputs directly on the microscope head via a touch display, for example, with regard to zoom, orientation and / or illumination and / or movement / displacement (position change) of the microscope head. In the case of a navigation system, for example, the view of the touch display and / or the view on the navigation monitor can be changed and adapted in order to obtain a better overview or also to better track the instrument.

[0019] In particular, the robot arm of the robot can be configured in such a way that the robot head is adjustable not only in its position but also in its orientation, i.e. in its position (or has six degrees of freedom / 6DOF (degrees of freedom)). In particular, the robot arm can have at least a first and a second robot arm segment, which are connected to each other via joints, and the robot head is connected to the robot arm via further joints, and the robot arm is connected to the robot base via additional joints. In particular, the joints of the first and second robot arm segments can have a rotational degree of freedom for rotation about an axis of rotation, wherein the axis of rotation can be arranged in particular in a horizontal direction (i.e. perpendicular to the up-down direction) in the kinematic posture of the robot arm to provide a type of cantilever (similar to an excavator arm). In particular, the robot can be constructed in the form of a bent arm robot.

[0020] In particular, the robot head is of rigid design and is connected to the robot arm via bearings or joints.

[0021] Preferably, the robot arm may have at least three robot arm segments, which are respectively connected to each other via joints. In particular, the robot (or a control unit of the robot) may control the position and orientation of the robot head via a multi-link actuated kinematic device.

[0022] Preferably, the surgical microscope is a digital microscope which is created with the aid of a sensor, for example a CMOS sensor, and which digitally provides digital microscope recordings.

[0023] The touch display can preferably be arranged on a lateral side of the robot head (that is, not on the back side of the extension of the longitudinal axis of the robot head). The touch display can especially be arranged on the robot head so that the normal of the display surface is substantially perpendicular to the longitudinal axis of the robot head, for example the observation axis of a surgical microscope.

[0024] In the case where the microscope head is implemented as a robot head, the touch display can be arranged in particular laterally (i.e. not on the back side) at the area opposite to the optical outlet. Alternatively or additionally, the touch display in the microscope head can be arranged on the side away from the optical output end, i.e. arranged almost oppositely (one end side is adapted to the optical output end, in particular, has a lens of the optical system, and the opposite end side has a touch display). It can also be said that at least one touch display is arranged on the microscope (camera) head on the lateral side and / or the upper side at the "upper part", and the optical output end is arranged at the "lower part". The optional operating button or operating key or input device in the form of a joystick or a 3D mouse can be arranged in an axial position between the optical output end (lower part) and the touch display (upper part) along the longitudinal axis direction of the robot head, in particular directly below the touch display. Preferably, the robot arm can also be connected to the robot head by a joint between the optical output end (lower part) and the touch display (upper part) along the longitudinal axis direction of the robot head. The touch display is in particular the “uppermost” or end-located element or component which is arranged laterally and / or at the end on the robot head.

[0025] Preferably, the touch display may have a round outer contour or shape, in particular a round outer contour or shape.

[0026] According to one embodiment, a robot head in the form of a microscope head can have a cylindrical / columnar base body, wherein the entire end side forms the optical output for a digital microscope or a microscope camera and the opposite entire end side has a rounded touch display.

[0027] In particular, an additional input device, for example in the form of a joystick or a 3D mouse, can be arranged opposite in the extension of the connection of the robot arm or at the same height. If the robot head is connected to the robot arm on a lateral side via a joint, in particular a swivel joint, the input device is arranged on the opposite lateral side starting from the connection in the extension of an axis perpendicular to the optical axis (or also perpendicular to the longitudinal axis of the robot head).

[0028] In particular, the robot guidance system can be adapted to display a menu structure for accessing different functions via a touch display. Exemplary use cases for the touch display are listed in particular below. Thus, not only a single operating menu or a single display section can be displayed on the touch display, but also a plurality of operating menus can be provided to the operator in order to control different functions. In particular, it is possible to switch from a top menu structure to a plurality of submenu structures and back again.

[0029] Preferably, the robot guidance system can be adapted to output a robot control menu as an operating menu via the touch display, so as to control the robot by operating input (movement), in particular to control the robot movement in six degrees of freedom, for example in six translation directions (two opposite directions of the Cartesian coordinate system +X / -X, +Y / -Y, +Z / -Z) and / or six rotation directions (rotation in clockwise or counterclockwise direction around the corresponding axis). In other words, the touch display can be adapted to display an (operating) menu structure for accessing functions, which menu structure is adapted to control the robot movement, in particular six degrees of freedom.

[0030] In particular, the robot guidance system can be adapted to output a visualization control menu as an operating menu via the touch display in order to control / change settings of the visualization system, in particular zoom, focus and / or lighting intensity (as settings). In other words, the touch display can be adapted to display a menu structure for accessing functions: by in particular touching a bar / slider, settings of the visualization system (e.g. a surgical microscope), such as zoom, focus and light intensity, are determined.

[0031] According to another embodiment, the robot guidance system can be adapted to switch between at least two different control menus, in particular between at least one robot control menu and a visualization control menu, so as to control at least two different functions of the robot guidance system, in particular movement of the robot as a first function and visualization as a second function.

[0032] In particular, the robot guidance system can be adapted to output a visualization control menu as an operating menu via a touch display, which visualization control menu allows control of different light modes / imaging modes of the visualization system, in particular control of fluorescence for, in particular, ICG (indocyanine green), 5-ALA (5-aminolevulinic acid).

[0033] Furthermore, the robot guidance system may in particular be adapted to output a navigation control menu as an operating menu via a touch display, which operating menu allows storing waypoints and / or robot configurations and / or navigation positions with respect to the patient (or patient position) and in particular provides a so-called "hold and drive" function, wherein driving to a stored point or a predefined position is achieved by long pressing one of the stored and displayed data.

[0034] Furthermore, the robot guidance system can in particular be adapted to output a navigation control menu as an operating menu via the touch display, which allows or provides control of the navigation process, in particular point digitization and point verification for patient registration and calibration / activation tools.

[0035] Furthermore, the robot guidance system can be adapted in particular to output an instrument control menu as an operating menu via a touch display, which instrument control menu allows controlling instrument guidance functions, such as moving a robot or instrument as an end effector on a target trajectory or providing switching on or off of instrument functions.

[0036] Furthermore, the robot guidance system can be adapted in particular to output a media control menu as an operating menu via the touch display, which allows controlling the image recording of the visualization system (e.g. as a current snapshot) and / or the recording of video data. Thus, the surgeon can, for example, create a recording at a first point in time by operating input and can output this recording at a later point in time on the touch display or an external monitor, for example, in order to make a before-and-after comparison or to retrieve information about the surgical site.

[0037] Furthermore, the robot guidance system can be adapted in particular to output a media control menu as an operating menu via a touch display, which media control menu allows control of the playback and management of the recorded media data, wherein the played video can in particular be shown on another display (in addition to or as an alternative to the touch display).

[0038] Furthermore, the robot guidance system may be adapted in particular to output a navigation control menu as an operating menu via a touch display, which operating menu allows or provides control of settings for information displayed on a visualization monitor, in particular displaying and hiding planned trajectories, surgical targets or other navigation information.

[0039] Furthermore, the robot guidance system may in particular be adapted to output a navigation control menu as an operating menu via the touch display, the operating menu providing control of switching between different monitor layouts of the main visualization monitor.

[0040] In particular, situation-related content based on information from the control unit / controller can be additionally displayed in order to save the user time in navigating through operating menus. In particular, the following situation-related content (e.g. related to the current state of the robot configuration or the surgical plan) can be displayed individually or in selectable combination with one another:

[0041] - The settings of the visualization system can be highlighted after the guidance system is repositioned, allowing the visualization parameters to be quickly adapted to the new position;

[0042] -When the movement of the navigation instrument is recognized, the settings for instrument guidance can be highlighted;

[0043] - Visualize the current state / current function of a multi-function hardware button in the surrounding environment of the display;

[0044] - Presentation of a safety stop button on the touch display for interrupting the automated robot movement;

[0045] - Display notifications (such as information, warnings, and errors) to the user both when and without user feedback;

[0046] - Utilize depth tracking / depth illustration to visualize distance to target for instrument guidance;

[0047] - Render an image of the visualization system to (roughly) locate the system;

[0048] The display mode of the touch display can in particular be switched into a so-called "track pad" mode, in which:

[0049] Activating a mouse symbol on a main visualization screen, in particular a surgical monitor, and thereby enabling the user to operate the visualization display screen as a computer screen with a mouse / laptop trackpad (by touching the display); or

[0050] o Enables the user to scroll through navigation views / slices (segments) on a main visualization screen, in particular a surgical monitor.

[0051] The touch display attached to the robot head can in particular have a sterile outer cover, which is preferably designed to be replaceable in order to provide a sterile barrier relative to the sterile surgical site. For example, in order to operate in a sterile environment, the touch display can be covered with a transparent medical (surgical) drape, wherein in particular connection points / coupling points for the surgical drape are provided.

[0052] According to another embodiment, an inertial measurement unit (IMU) can also be arranged at or in the robot head, especially on or in the touch display or fastened to the end effector, so as to detect the position and / or orientation of the touch display or the robot head, and the robot guidance system, especially the control unit, is adapted to adapt the orientation of the visual output of the touch display based on the detected position and / or orientation, especially to adapt the output (e.g., of the operating menu) so that the output is always shown with the same horizontal orientation. In other words, the inertial measurement unit (IMU) can be optionally connected to the touch display or the robot head or the end effector, so as to be able to change the orientation of the visualization content in different positions and / or orientations, especially in the extreme positions of the guidance system. Alternatively or additionally, the position and / or orientation of the touch display or the robot head can be detected preferably by the robot kinematics device or the time real-time configuration of the robot, which has its robot arm segments and its robot head as the end effector or has a connected end effector. In addition, preferably, the position and / or orientation of the touch display or the robot head can be detected by the navigation camera of the navigation system and the corresponding data technology processing. In particular, the control unit can be adapted to adapt the display part based on the detected orientation of the touch display so that the display part is optimally displayed relative to a predetermined position in space, i.e. the position of the surgeon's head (which position can be detected by a navigation camera, for example). This can include, in particular, a rotation of the view of the touch display so that the view is preferably displayed approximately horizontally relative to the floor of the operating room and therefore relative to the surgeon, as well as an extension or compression of the view in order to provide a view that is as neutral and natural as possible in the detected position of the surgeon (similar to arrows or markings on a lane or on a lane marking, which are adapted (stretched) for the driver so that the driver can recognize the information as well as possible). The control unit or the touch display can be adapted in particular to output such a projection on a virtual surface via the touch display, wherein the surface is perpendicular to the viewing axis between the surgeon and the touch display, so that the surgeon does not see a distorted presentation (when the touch display is tilted to the line of sight), but rather a presentation similar to that when the surgeon is viewing perpendicularly to the touch display.

[0053] The touch display can preferably have a radio connection module in order to (independently) establish a wireless data connection with the control unit, in particular via WLAN or Bluetooth. According to an alternative embodiment, the touch display can also be connected to the control unit via a data cable. In other words, in order to control different functions of the guidance system for presenting content according to the situation, the touch display is connected to the control unit (as a computer system) via a touch display controller via a cable or wirelessly (using a radio connection module). The touch display itself has, in particular, an independent sub-control unit as a computer system in order to autonomously form an independent control complement, which can be integrated into the entire system and can be coupled to the (centralized) control unit in terms of data technology.

[0054] Further preferably, the touch display can have a display diagonal or display diameter of at least 4 cm and / or a maximum of 20 cm. In other words, the size of the touch display can be between 4 cm and 20 cm. The touch display can especially have a square, rectangular or circular form factor. The size and shape can, for example, follow the number of functions and the size of the connected robot effector. The size and shape contribute to the advantageous integration of the touch display into the region of the end effector.

[0055] According to one embodiment, the robot head can have at least one (physical) operating key (hardware key), preferably three operating keys, and the touch display is arranged directly adjacent to the key and is adapted to display the current functional state / current operating mode of the key. This provides a universal key or multifunctional key, which can detect different operating inputs for different operating menus. This can also increase safety, in particular, if, for example, input via the touch display fails or input cannot be made accurately due to contaminated surgical gloves.

[0056] The touch display has, in particular, a color display in order to display operating menus and / or information in color.

[0057] In other embodiments, in addition to the touch display on the robot head, a touch display may also be provided on the robot arm and / or the robot base.

[0058] In particular, the robot guidance system can be designed as a robot guidance unit and all components can be integrated in this unit or a single module. In particular, the robot guidance system can be designed to be mobile and can be arranged and moved autonomously in the operating room, for example, as a mobile surgical microscope, on whose microscope head (as robot head and end effector) a touch display is arranged.

[0059] With regard to a medical robot operating method for a robot guidance system, in particular according to the present disclosure, the object is solved by the following steps: outputting at least one visual operating menu via a touch display, the touch display being rigidly fastened to a robot head of the robot; detecting a touch-sensitive input as an operating input via the touch display; sending the operating input to a control unit, the control unit being adapted to at least control the robot; controlling a function, in particular a movement of the robot, via the control unit based on the operating input. As described above, by means of this step a flexible and centralized operating method can be provided, with the aid of which the surgeon can perform different functions, such as a movement of the robot or a setting of a visualization system.

[0060] Preferably, the (operation) method may further comprise the steps:

[0061] The position and orientation (changes) of the robot head are detected, in particular by an inertial measurement unit (IMU) or by a detected robot kinematics or by a navigation camera of a navigation system; the detected position and orientation are transmitted to the control unit; an orientation of a visual operating menu adapted to the position and orientation is calculated; and an adapted visual presentation of the operating menu is output. In this way, the best possible or optimally adapted visual output can be provided to the surgeon, and the surgeon does not have to turn his head in order to read identifications or information or to reposition his head in order to clearly recognize the presented information, for example text, in an oblique top view on the touch display.

[0062] With regard to a computer-readable storage medium, the object is achieved in that the computer-readable storage medium comprises instructions which, when executed by a computer, cause the computer to carry out the method steps of the operating method according to the present disclosure. In particular, a control unit of a robot guidance system can comprise such a computer-readable storage medium.

[0063] Any disclosure related to the robot guidance system according to the present invention is also applicable to the operating method of the present disclosure, and vice versa. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The present invention will be explained in more detail below with reference to the accompanying drawings using preferred embodiments.

[0065] Figure 1 A schematic front view showing a robot guidance system according to a preferred embodiment of the present disclosure;

[0066] Figure 2 A schematic diagram showing the functional relationship of a touch display, a control unit and an IMU of a robot guidance system according to another preferred embodiment;

[0067] Figure 3a and Figure 3b Exemplary views of the operating menu are shown in a table arrangement and a circular arrangement;

[0068] Figure 4 a schematic view showing an example submenu of a visual control menu for setting light intensity;

[0069] Figure 5 A schematic diagram showing example submenus of a media control menu for managing and functionally reproducing recorded video;

[0070] Figure 6 A schematic view showing an operating menu having only one stop button as interactive input;

[0071] Figure 7 A schematic front view of a robot head or end effector is shown with three buttons and a touch display;

[0072] Figure 8 A schematic front view showing a robot guidance system according to another preferred embodiment of the present disclosure;

[0073] Figures 9 to 11 Different views of a robot guidance system having a robot head with a touch display which is arranged above the microscope head on the side facing away from the optical exit; and

[0074] Fig.12 is a flow chart of an operating method according to a preferred embodiment.

[0075] The accompanying drawings are merely schematic in nature and are only useful for understanding the invention. Identical elements are provided with identical reference numerals. The features of the various embodiments may be substituted for one another. DETAILED DESCRIPTION

[0076] Figure 1 A schematic front view of a medical robot guidance system 1 (hereinafter referred to only as guidance system) for surgical operations on a patient P is shown. The guidance system 1 has a robot 2, which has a movable robot arm 4 and a robot head 6 connected to the robot arm 4 at the end. In this embodiment, the guidance system 1 is designed in the form of a surgical microscope guided by a robot. A microscope head is arranged on the robot arm 4 having a plurality of robot arm segments as the robot head. Therefore, the robot head or microscope head can also be generally referred to as the end effector 8 of the surgical microscope.

[0077] In order to control the robot 2 and its robot arm 4 and to set the microlens as end effector 8 to a position and orientation, the guidance system 1 has a control unit 10. The control unit 10 can be designed as a centralized control unit, for example as a computer system that can process and control different functions, or the control unit can have a subsystem of the robot control unit in order to control the robot accordingly.

[0078] Furthermore, the guidance system 1 has a touch display 12 as an input unit and output unit. The touch display is adapted to visually output different operating menus 14 and displays for different functions and to detect touch-sensitive inputs as operating inputs and to transmit them to the control unit 10 in order not only to control the robot 2 but also to make settings on the visualization system.

[0079] However, unlike the prior art, the touch display used for control is not fastened to a static base of the guide system 1, but the touch display 12 is rigidly fixed directly to the robot head 6 or the end effector 8 and moves therewith.

[0080] In this way, the touch display 12 moves in the operating area and the surgeon has a centralized visualization of different operating menus 14 and can flexibly control different functions of different subsystems via a single touch display. In this embodiment, the surgeon can select from at least one robot control menu and a visualization control menu or switch between individual control menus as operating menus 14. Via the robot control menu, the surgeon can, for example, cause instrument guidance or move the robot head 6 in such a way that the end effector 8 is in a better position for surgery. Here, the microscope head as an end effector can be moved manually or automatically via operating inputs of the touch display 12 into such a position that allows better observation.

[0081] Further recordings can be output via an external (surgical) monitor 15, wherein the media output can be controlled via further media control menus via the touch display 12. For example, real-time recordings of a microscope can be output via the surgical monitor 15, wherein the brightness or color contrast can be set via the media control menu.

[0082] Therefore, in the robot guidance system 1 according to the first embodiment, at least three control menus are divided for the surgeon: a robot control menu, a visualization control menu, and a media control menu are provided, and the surgeon can centrally operate these menus to flexibly and safely control various functions.

[0083] Figure 2A schematic functional view of the interaction between the various modules of the guidance system 1 according to another second preferred embodiment is shown. Here, an inertial measurement unit (IMU) 16 is arranged directly on the touch display 12 to detect movement (or movement with acceleration changes) and determine the new position and orientation of the touch display 12 after the robot moves based on the initial orientation. The data of the IMU is sent directly to the centralized control unit 12. In addition, the centralized control unit 10 receives data from the end effector control unit 18 (as a sub-control unit). In addition, the centralized control unit 12 is data-connected to the touch display control unit (as a sub-control unit). The centralized control unit 12 then calculates a visual output based on these data, and then outputs the visual output on the touch display 12.

[0084] In particular, the most favorable view for the surgeon can be calculated and output accordingly in this way. Since, unlike the prior art, the touch display is no longer arranged statically, but moves dynamically with the end effector 8, the orientation of the display relative to the surgeon changes depending on the orientation of the display. Figure 1 When the robot arm 4 is pivoted upward by 90°, the visual view of the touch display 12 will also be rotated by 90° in the counterclockwise direction relative to the operator without correction.

[0085] However, in order to enable the surgeon to continue to read the visual display well, the control unit 12 calculates such an orientation that, in the above example, the display is rotated 90° clockwise relative to the touch display, so that the rotation and the counter-rotation cancel each other out and allow intuitive reading by the surgeon. In other words, the control unit calculates such an orientation of the display during the movement of the robot 2 that it appears to remain as constant as possible for the surgeon and achieves to a certain extent that the display of the touch display remains in the same relationship relative to the surgeon.

[0086] In one embodiment, for example when changing from a front view (i.e., a vertical view) to an oblique view of the touch display 12, the visual output can also be presented with a corresponding distortion (similar to a road marking in the form of an inscription, which is also presented extending longitudinally in order to provide the driver with the best possible view).

[0087] Figure 3a and Figure 3b is an exemplary view of an operation menu presented by a touch display, wherein Figure 3a A tabular assembly with rectangular, symbolic (touch) buttons / knobs is shown in FIG. Figure 3b A circular assembly of symbolized (touch) buttons / knobs is shown in FIG. By means of these, the user can, for example, select a submenu, which in Figure 4 and Figure 5and are described below.

[0088] Figure 4 The submenu of the lighting control menu when the symbolized incandescent bulb is selected in the main menu is shown. In this embodiment, the user can set the brightness of the white light and the intensity of the ultraviolet radiation and infrared radiation.

[0089] When the user Figure 3a or 3b. When the video symbol is selected in the main menu, the menu jumps to Figure 5 In the submenu of the video function shown in , the management and reproduction functions of the video data can be selected in this submenu. Here, the video can be played accordingly on an external surgical monitor (not shown here).

[0090] Figure 6 A (digital) stop button is schematically shown on the touch display 12 for stopping the movement of the robot 2 or the robot arm 4 during the movement (type of emergency button).

[0091] Figure 7 A front view of an end effector of a robot guidance system 1 of a further preferred embodiment is shown. In this embodiment, three equidistant physical keys / buttons / operating buttons 22 are arranged in a straight line on the end effector 8. Parallel to and above the straight line arrangement of the three operating buttons 22, a touch display 12 is fastened to the end effector, which in each case presents a functional state above the corresponding operating button 22. In this way, the three operating buttons 22 can be assigned a plurality of functions and visualized accordingly via the touch display.

[0092] For example, the control unit 12 can assign different functions to the operating buttons at different steps in the surgical plan and output them via the touch display.

[0093] Figure 8 A further embodiment of the guidance system 1 is shown in a schematic front view. The robot 2 has a fixed robot base 24, to which a robot arm 4 is movably connected. At the end, a robot head 6 in the form of a (digital) microscope lens is connected as an end effector 8, so that the robot arm 4 can not only set the position and orientation (i.e. the spatial orientation) of the microscope lens in order to be in the occupied shooting position and create a digital microscope shooting. Here, the optical axis of the microscope lens is presented as a (vertical) dotted line, which passes through the optical system (not shown) and in the extension through the CMOS sensor for digital shooting. Therefore, by setting not only the position (three degrees of freedom) but also the orientation (three degrees of freedom), six degrees of freedom / 6DOF (degrees of freedom) can be set (in the robot kinematics device).

[0094] In this embodiment, the robot arm 4 has a plurality of robot arm segments 26, which are each connected to one another via joints 28. The robot head 6 with the optical system is also connected to the robot arm 4 via a further joint 28, and the robot arm 4 is connected to the robot base 24 via the joint 28. In particular, the joints of the first and second robot arm segments 26 can have a rotational degree of freedom for rotation about an axis of rotation, wherein the axis of rotation can be located at the center of the robot arm 4. Figure 8 In the kinematic posture presented in the embodiment, it is arranged in particular in the horizontal direction (ie perpendicular to the up-down direction) so as to provide a type of cantilever.

[0095] The robot head 6 is of rigid construction (or has a rigid robot head housing in which the optical system, the CMOS sensor connected downstream and, for example, further electronics are accommodated). The robot 2 (or a control unit of the robot) can therefore control the position and orientation of the robot head 6 via a multi-actuated kinematic arrangement and set the optical axis accordingly.

[0096] In this embodiment, the touch display 12 is arranged on a lateral side of the robot head 6 (i.e. not on the back side in extension of the longitudinal axis of the robot head). The touch display can be arranged on the robot head in particular such that the normal of the display surface is substantially perpendicular to the longitudinal axis of the robot head, for example the observation axis of a surgical microscope.

[0097] The touch display 12 is arranged in this case so that it Figure 8 The robot head 6 is located above the robot head 6. Figure 8 When observed in the middle, the lower side forms an optical opening for the optical system of the digital microscope, and the touch display is arranged in the upper area. It can also be said that the optical output end is arranged in a first lower area of ​​the robot head 6, and the touch display is arranged in a second upper area of ​​the robot head, wherein the first and second areas form areas that are located at the ends and are separated from each other. The touch display 12 is therefore arranged in such a way that, in normal surgery, the surgical microscope or microscope head observes the patient from top to bottom to create the film and the touch display arranged on the side can be seen and operated particularly well by the medical professional. The operating buttons 22 are Figure 8 The robot arm 4 is arranged below the touch display 12, so that the touch display forms the uppermost element. Thus, viewed along the longitudinal axis of the robot head 6, the following axial positions are arranged in succession in this order: touch display 12 - operating button - optical output. The robot arm 4 is connected laterally to the robot head 6 via a joint 28, and the three dashed lines of the rotational illustration are intended to indicate that the robot head can be oriented about three axes, or has three degrees of rotational freedom.

[0098] Figures 9 to 12A medical robot guidance system 1 according to a further preferred embodiment is shown in a side view, a perspective top view and a perspective isometric view. The robot 2 is designed in the form of a robot-guided surgical microscope, whose robot head 6 forms an end effector 8 as a microscope lens and whose position and orientation can be set via a robot arm 4. Fig. 9 An optical output is provided on the lower side as seen in the figure, wherein the optical axis is shown in dashed lines. The touch display 12 is arranged on the side opposite the optical output. In this embodiment, the touch display 12 is designed to be round, wherein the center point of the rounded touch display 12 is substantially in the extension of the optical axis, i.e., concentric therewith to a certain extent. It can also be said that the normal of the display is parallel to the optical axis, in particular on the optical axis (the optical system and the touch display are symmetrical to each other to a certain extent).

[0099] The touch display 12 is designed as an end-side, upper, flat surface and is offset (i.e., protruded) relative to the rest of the microscope head (even the robot arm 4) to enable good operation. Viewed in the direction of the longitudinal axis or optical axis of the robot head, from top to bottom, the touch display 12 is first arranged, then the input device 30 in the form of a 3D mouse or 3D space mouse and then the optical output (for example, with the last lens of the optical system). The input device 30 is also arranged on the opposite side of the connection to the robot arm 4 in the extension of the longitudinal axis of the cylindrical arm, which has a rotary joint 28 relative to the robot arm segment 26. The zero axis of the 3D mouse is concentric with the axis of the robot arm segment 26.

[0100] By means of further operating buttons 22 , in particular movements and / or zooming and / or further functions can also be controlled, depending on the functional state.

[0101] The robot head 6 is connected to the robot arm 4 (or to the robot arm segment 26) by means of a revolute joint as joint 28. The orthogonal line on the touch display 12 is also perpendicular to the axis of rotation, wherein here the orthogonal line on the touch display 12 is parallel to the optical axis.

[0102] Fig.12 The operating method according to a preferred embodiment is shown in a flow chart.

[0103] In a first step S1 , an operating menu 14 is output via the touch display 12 .

[0104] Specifically, in this embodiment of the operating method, the output of the operating menu is solved by the following sub-steps. In a first sub-step S1.1, the position and orientation (i.e., position) of the touch display 12 is detected (preferably also indirectly via the robot head 6), in particular via the inertial measurement unit (IMU) 16. Alternatively, the position can also be detected by a detected (mechanical) robot kinematic device or by a tracking camera / navigation camera of a navigation system.

[0105] Thereafter, the position of the touch display 12 is forwarded to the control unit 12 in a sub-step S1 . 2 .

[0106] The control unit then calculates an adapted orientation for output in a substep S1 . 3 and then outputs the adapted presentation via the touch display 12 in a substep S1 . 4 .

[0107] In a subsequent second step S2 , an input to touch display 12 is then detected.

[0108] The detected operation input is then sent to the control unit 12 in a third step S3 .

[0109] Finally, in step S4 the corresponding function is controlled, as displayed and selected on the touch display 12 . This can be, in particular, the control of the robot 2 .

[0110] Reference numerals list

[0111] 1Medical Robot Guidance System

[0112] 2. Robot

[0113] 4 Robotic arms

[0114] 6 Robot Head

[0115] 8 End Effectors

[0116] 10 Control unit

[0117] 12 Touch Display

[0118] 14 Operation Menu

[0119] 15 External monitor

[0120] 16 Inertial Measurement Unit (IMU)

[0121] 18 End Effector Control Unit

[0122] 20 Stop button

[0123] 22 operation buttons

[0124] 24 Robot Base

[0125] 26 robot arm segments / robot arm links

[0126] 28 joints

[0127] 30 Input Devices

[0128] Patient P

[0129] S1 Steps for outputting the operation menu via the touch display

[0130] S1.1 Steps to detect position and orientation using IMU

[0131] S1.2 Steps sent to the control unit

[0132] S1.3 Steps for calculating the adapted orientation

[0133] S1.4 Output Adaptation Presentation Steps

[0134] S2 Steps for detecting input of touch display

[0135] S3 Steps for sending operation input to the control unit

[0136] S4 controls the steps of the function.

Claims

1. A medical robot guidance system (1) for surgical operations on a patient (P), comprising: A robot (2) having a movable robot arm (4) and a robot head (6) connected to the robot arm (4) at the end, in particular having an end effector (8) on the robot head (6) or as the robot head (6); a control unit (10) adapted to at least control the robot (2) and to move the robot; at least one touch display (12) adapted to visually output at least one operating menu (14) and to detect touch-sensitive inputs as operating inputs and to send the touch-sensitive inputs to the control unit (10) in order in particular to control the robot (2); It is characterized in that The at least one touch display (12) is rigidly fixed to the robot head (6) and moves together with the robot head.

2. The medical robot guidance system (1) according to claim 1, characterized in that: The robot guidance system (1) is implemented in the form of a surgical microscope having a microscope head connected to the robot arm (4) as a robot head (6), or the robot guidance system is implemented in the form of a navigation system having a camera system, in particular a laser system, connected to the robot arm (4).

3. The medical robot guidance system (1) according to any one of the preceding claims, characterized in that The robot guidance system (1) is adapted to output a robot control menu as an operation menu (14) via the touch display (12) so as to control the robot (2) by operation input, in particular to control the robot movement in six degrees of freedom.

4. The medical robot guidance system (1) according to any one of the preceding claims, characterized in that The robot guidance system (1) is adapted to output a visualization control menu as an operating menu (14) via a touch display (12) in order to control settings of a visualization system, preferably a surgical microscope, in particular zoom, focus and / or illumination intensity.

5. The medical robot guidance system (1) according to any one of the preceding claims, characterized in that The robot guidance system (1) is adapted to switch between at least two different control menus in the touch display (12), preferably between the robot control menu and the visualization control menu, so as to control at least two different functions of the robot guidance system, in particular to control the movement of the robot (2) as a first function and the visualization as a second function.

6. The medical robot guidance system (1) according to any one of the preceding claims, characterized in that The touch display (12) fixed to the robot head (6) has a sterile outer cover, which is preferably designed to be replaceable in order to provide a sterile barrier relative to the sterile surgical site.

7. The medical robot guidance system (1) according to any one of the preceding claims, characterized in that In addition, an inertial measurement unit is arranged on or in the robot head (6), in particular on or in the touch display (12) or fastened to the end effector (8) to detect the position and / or location and / or orientation of the touch display (12), and the robot guidance system (1), in particular the control unit (10) is adapted to adapt the orientation of the visual output of the touch display (12) based on the detected position and / or orientation, in particular to adapt the output so that the output is always displayed in a constant horizontal orientation.

8. The medical robot guidance system (1) according to any one of the preceding claims, characterized in that The touch display (12) has a radio connection module in order to establish a wireless data connection with the control unit, in particular via WLAN or Bluetooth.

9. The medical robot guidance system (1) according to any one of the preceding claims, characterized in that The touch display (12) has a display diagonal or a display diameter of at least 4 cm and / or at most 20 cm.

10. The medical robot guidance system (1) according to any one of the preceding claims, characterized in that The robot head (6) has at least one operating button (22), preferably three operating buttons (22), and the touch display (12) is arranged directly adjacent to the operating button (22) and is adapted to display the current functional status of the operating button (22).

11. The medical robot guidance system (1) according to any one of the preceding claims, characterized in that The robot guidance system (1) is constructed in the form of a surgical microscope and has a microscope head (6) connected to the robot arm (4), wherein the optical outlet forms one side of the microscope head and the touch display (12) is arranged on the end side opposite to the side, in particular, the orthogonal line on the touch display (12) is parallel to the optical axis.

12. The medical robot guidance system (1) according to any one of the preceding claims, characterized in that The touch display (12) has a rounded shape, in particular a rounded shape, and preferably when the robot (2) is designed as a robot-guided surgical microscope, the center point of the rounded touch display (12) is located in the extension of the optical axis.

13. A medical robot operation method, in particular for a medical robot guidance system (1) according to any one of the preceding claims, characterized in that step: outputting (S1) at least one visual operating menu (14) as a visual presentation via a touch display (12), the touch display being rigidly fastened to a robot head (6) of the robot (2); detecting (S2) a touch-sensitive input as an operation input via the touch display (12); sending (S3) the operation input to a control unit (10), the control unit being adapted to at least control the robot (2); The control unit (10) controls functions, in particular the movement of the robot (2), based on the operation input (S4).

14. The medical robot operation method according to claim 11, characterized in that: The method further comprises the steps of: Detecting (S1.1) the position and / or orientation of the robot head (6) by means of an inertial measurement unit; sending (S1.2) the detected position and / or orientation to the control unit; Calculating (S1.3) an orientation of the visual presentation, in particular the visual operating menu (14), adapted to the position and / or orientation; An adapted visual presentation, in particular an adapted visual presentation of the operating menu (14), is output (S1.4).

15. A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to implement the method steps of the robot operation method according to any one of claims 11 or 12.

Images