A medical robotic guide system that integrates touch displays and operation methods

Abstract

The present invention relates to a medical robotic guidance system (1) for surgical intervention on a patient (P), comprising a robot (2) having a movable robotic arm (4), an end effector (8) terminally connected to the robotic arm (4), in particular on or as a robot head (6), a control unit (10) configured to control and move at least the robot (2), and at least one touch display (12) configured to visually output at least one operation menu (14) and to detect touch-sensitive inputs as operation inputs and transmit them to the control unit (10), in particular for controlling the robot (2), the at least one touch display (12) being fixedly fixed to the robot head (6) and moving therewith. Furthermore, the present invention relates to a robot operation method and a computer-readable storage medium according to the adjacent claims.

Description

[Technical Field]

[0001] The present invention relates to a medical, particularly surgical, robotic guidance system for performing medical procedures, particularly surgical interventions, on a patient. For this purpose, the robotic guidance system comprises a robot having a robot arm movably connected to a robot base and a robot head terminally connected to the robot arm. In particular, the robot head has an end effector, or the end effector itself forms the robot head. Furthermore, the robotic guidance system comprises a (central) control unit (in particular a processor and memory unit) configured to control and move at least the robot, in particular the robot arm and the robot head (and in particular the end effector). Furthermore, the robotic guidance system comprises at least one touch display / touch screen / touch-sensitive screen configured to visually output at least one operation menu (for control) and to detect touch-sensitive inputs as operation inputs and transmit them to the control unit, in particular for controlling the robot. Furthermore, the present invention relates to a (robot) operation method and a computer-readable storage medium according to the preambles of the adjacent claims. [Background technology]

[0002] Surgical guide systems are being used more and more frequently for interventions, especially minimally invasive interventions. The number of functions of medical systems is rapidly increasing due to technological developments and the increasing specialization of the various (sub-system) systems with their corresponding integration.

[0003] As the guidance system expands over time to include a variety of functions, it becomes increasingly difficult to provide the user with a central operating function or mode that allows them to maintain an overview of functions and controls.

[0004] For the operation of robotic guidance systems, currently state-of-the-art touch displays are provided, for example on the base of medical trolleys, medical towers or medical (surgical) microscopes, allowing medical professionals to provide (operational) inputs for the corresponding controls via guidance via (operational) menus.

[0005] However, the problem here is that the touch display is not centrally located in the interventional area, but rather at a distance. This makes it difficult to access the touch display on the one hand, and on the other hand, it makes it difficult to meet hygiene requirements, as the touch display is not sterile. Due to the distance on the one hand and the sterility requirements on the other hand, the interventional surgeon is usually unable to operate the touch display himself.

[0006] For example, US2005 / 0041282A1 discloses a surgical microscope as a robotic guidance system with a touch display firmly attached to the base / trolley of the surgical microscope. The touch-sensitive display allows different functions to be controlled from different areas of the display. However, this configuration requires the surgeon to instruct another medical professional to control the functions, who then performs the controls. Summary of the Invention

[0007] It is therefore an object of the present invention to avoid or at least mitigate the drawbacks of the prior art, and in particular to provide a medical robotic guidance system and method that provides a user, such as a medical professional, especially a surgeon, with intuitive and clearly organized operational controls with an intuitive user interface that allows full control of the system or systems used during a medical procedure, such as an intervention. In particular, it is desirable that the surgical controls be provided in an area that centralizes the various surgical modalities and is easily accessible to the surgeon.

[0008] The object of the invention is solved for a universal robot guide system according to the invention by the features of claim 1, for a universal operating method according to the invention by the features of claim 13 and for a computer-readable storage medium according to the invention by the features of claim 15.

[0009] Therefore, the basic idea of the present disclosure is to provide an operation mode, such as control of the robot, in the area of the head or end effector of the robot. In contrast to the prior art, the touch display as an input unit and output unit is not provided in a separate remote location such as a medical tower, but is provided directly on the moving part of the robot, i.e., on the head of the terminal robot, and therefore in the area of the end effector.

[0010] In other words, at least one touch display is firmly connected / attached / fixed to the robot head, in particular the end effector, and moves therewith. Instead of providing a static touch display on the robot base, e.g., the carriage of a robotic surgical microscope, the present invention dynamically moves the touch display on the robot head. Since the end effector is located on the robot head or the end effector itself forms the robot head, the operating modality in the form of a touch display is located directly in the area of the end effector and is easily and safely accessible to the surgeon. The surgeon has the option of using the dynamically moving touch display to centrally control various functions of the surgical device, so to speak, or to centrally control at least one movement of the robot, i.e., the robot head, in the area of the end effector. In particular, the surgeon can control various functions via the touch display, for example, by displaying operating menus related to the corresponding functions.

[0011] In other words, the present disclosure describes a touch display / touch screen / touch-sensitive screen integrated into a robot-assisted / robotic guide system for medical intervention. The touch display is connected to a central control unit (as a software execution unit) by data technology, allowing for flexible control of various functions, such as non-sterile and sterile environments, and visualization of information. The robotic guide system may include, for example, both visualization-based guides (such as surgical microscopes) and instrument guides (such as trocars).

[0012] In other words, a touch display on the robot end effector or on or embedded as the robot head is proposed herein, allowing the user to flexibly provide information and various control options (e.g., via an external monitor) in both sterile and non-sterile environments. The touch display is firmly connected to the robot end effector and oriented to provide a good viewing angle for the surgeon, particularly in the most common surgical positions (specific poses) of the guide system. The touch display may display both interactive content (e.g., operation menus) and non-interactive content (e.g., annotations). The visualized content can, for example, display the same content as the surgeon's larger main monitor (not located on the robot arm) or the same content as another control display not connected to the control device. Alternatively or additionally, independent content that may be context-dependent is also preferably displayed by the touch display.

[0013] The term "position" refers to a geometric location in three-dimensional space, especially specified using coordinates in a Cartesian coordinate system. In particular, a position can be specified by three coordinates: X, Y, and Z.

[0014] The term "orientation" refers to a location (e.g., position) in space. Orientation can also be said to refer to a location along with an indication of direction or rotation in three-dimensional space. In particular, three angles can be used to specify a direction:

[0015] The term "pose" covers both position and orientation. In particular, a pose can be specified using six coordinates: three position coordinates X, Y, Z, and three angular coordinates of orientation.

[0016] The operation input is, for example, a control command associated with a selection of an operation menu, and is sent to the control unit, which then executes the corresponding function directly or indirectly, for example, via a control unit of a sub-system. For example, a central control unit may indirectly control a visualization system via its sub-controllers, and may also indirectly control a navigation system via its sub-controllers, such as to set waypoints.

[0017] Advantageous embodiments are set forth in the dependent claims and are particularly described below.

[0018] According to one embodiment, the robotic guidance system can be in the form of a surgical microscope with a microscope head connected to a robotic arm as the robot head, or in the form of a navigation system with a camera system, in particular with a laser system, connected to the robotic arm. The microscope head can be actively controlled and moved, and the surgeon can give direct operational input to the microscope head via a touch display, for example, regarding zoom, alignment and / or illumination and / or movement / displacement (orientation change) of the microscope head. For example, in the case of a navigation system, the display on the touch display or the display on the navigation monitor can be changed and adjusted to get a better overview or better track the instruments.

[0019] In particular, the robot arm of the robot may be configured to adjust both the position and orientation, i.e., pose, of the robot head (or have six degrees of freedom / 6DOF). In particular, the robot arm may have at least first and second robot arm segments connected to each other via joints, the robot head may be connected to the robot arm via a further joint, and the robot arm may be connected to the robot base via an additional joint. In particular, the joints of the first and second robot arm segments may have a rotational degree of freedom for rotating about a rotation axis, and this rotation axis may be located at a kinematic position of the robot arm, particularly in a horizontal direction (i.e., perpendicular to the up-down direction), to provide a kind of cantilever (similar to the arm of an excavator). In particular, the robot may be configured in the form of an articulated arm robot.

[0020] In particular, the robot head is rigidly constructed and connected to the robot arm via bearings or joints.

[0021] Preferably, the robot arm has at least three robot arm segments, each connected to another via a joint. In particular, the robot (or the robot's control unit) can control the position and orientation of the robot head via a multi-link drive kinematic system.

[0022] Preferably, the surgical microscope is a digital microscope that creates a digital microscope image via a sensor such as a CMOS sensor and provides it digitally.

[0023] Preferably, the touch display may be located on a lateral side of the robot head (i.e., on a side that is not rearward in line with the longitudinal axis of the robot head). In particular, the touch display may be positioned on the robot head such that the normal to the display surface is substantially perpendicular to the longitudinal axis of the robot head, such as the viewing axis of a surgical microscope.

[0024] In particular, in an embodiment of the microscope head as a robot head, the touch display may be arranged laterally (i.e., not on the rear side) in an area opposite the optical output. Alternatively or additionally, the touch display in the microscope head may be arranged on the side away from the optical output, i.e., approximately opposite (one front surface is adapted to the optical output, in particular the optical system lens, and the opposite front surface is the touch display). It can also be said that at least one touch display is arranged on the side and / or upper side of the microscope (camera) head "upper," and the optical output is provided "below." Optional operating or actuating buttons, or input means in the form of a joystick or 3D mouse, can be arranged axially between the optical output (lower) and the touch display (upper), in particular directly below the touch display, when viewed in the longitudinal direction of the robot head. Preferably, the robot arm can also be connected to the robot head via a joint between the optical output (lower) and the touch display (upper), when viewed in the longitudinal direction of the robot head. In particular, the touch display represents the "top" or most distal element or component provided on the side and / or front of the robot head.

[0025] Preferably, the touch display may have a rounded outer contour or shape, in particular a circular outer contour or shape.

[0026] According to one embodiment, a robotic head in the form of a microscope head has a cylindrical / cylindrical base body, one circular front surface forming an optical output for a digital microscope or microscope camera, and the opposite circular front surface is provided with a circular touch display.

[0027] In particular, additional input means, for example in the form of a joystick or a 3D mouse, can be provided on the opposite side of the connection to the robot arm, or at the same height. If the robot head is connected to one side of the robot arm via a joint, in particular a pivot joint, the input means are provided on the opposite side, starting from the connection on an extension of an axis perpendicular to the optical axis (which can also be said to be perpendicular to the longitudinal axis of the robot head).

[0028] In particular, the robotic guidance system may be adapted to display a menu structure for accessing various functions via the touch display. Representative uses of the touch display are listed below. Thus, the touch display can not only display a single operation menu or a single indicator, but can also provide the surgeon with multiple operation menus for controlling various functions. In particular, it is possible to switch from a top menu structure to multiple sub-menu structures and vice versa.

[0029] Preferably, the robot guide system may be adapted to output a robot control menu as an operation menu via the touch display in order to control the robot via operation inputs (movements), in particular to control the robot's movements in six degrees of freedom, for example in six translational directions (in each case opposite directions of a Cartesian coordinate system +X / -X, +Y / -Y, +Z / -Z) and / or six rotational directions (rotating clockwise or counterclockwise around the respective axes). In other words, the touch display may be adapted to display a (operation) menu structure for accessing functions in particular adapted to control the robot's movements in six degrees of freedom.

[0030] In particular, the robotic guidance system may be adapted to output a visualization control menu as an operation menu via the touch display in order to control / change the settings of the visualization system, in particular the zoom, focus, and / or lighting intensity (as settings). In other words, the touch display may be adapted to display a menu structure for accessing the setting functions of the visualization system (such as a surgical microscope), such as zoom, focus, and light intensity, in particular via a touch bar / slider.

[0031] According to a further embodiment, the robotic guide system is adapted to switch between at least two different control menus, in particular at least a robot control menu and a visualization control menu, allowing to control at least two different functions of the robotic guide system, in particular robot movement as a first function and visualization as a second function.

[0032] In particular, the robotic guide system can be adapted to output a visualization control menu as an operation menu via a touch display, which allows control of the various light / image modes of the visualization system, in particular the fluorescence of ICG (indocyanine green) and 5-ALA (5-aminolevulinic acid).

[0033] In particular, the robotic guidance system may also be adapted to output a navigation control menu as an operation menu via the touch display, which allows waypoints and / or robot configurations and / or navigation positions to be saved in relation to the patient (or the patient's position), and in particular provides a so-called "hold and drive" function whereby a long press on one of the saved and displayed data will move to the saved point or predefined position.

[0034] In particular, the robotic guidance system can also be adapted to output a navigation control menu as an operation menu via the touch display, which allows or provides control of the navigation sequence, in particular the digitization and verification of points for patient registration and the calibration / activation of tools.

[0035] In particular, the robot guide system can be adapted to output an equipment control menu as an operation menu via the touch display, which allows the user to control equipment guide functions such as moving the robot or equipment as an end effector along a target trajectory and switching equipment functions on and off.

[0036] In particular, the robotic guidance system can be further adapted to output a media control menu as an operational menu via the touch display, thereby allowing control of the recording of images (e.g., current snapshots) and / or video data of the visualization system, allowing the surgeon to, for example, create images at an initial time point using operational inputs and later output the images to the touch display or an external monitor, for example, for before-and-after comparisons or to recall information about the intervention site.

[0037] In particular, the robotic guide system can be further adapted to output a media control menu as an operation menu via the touch display, which allows control of playback and management of recorded media data, and the played video is particularly displayed on another display (in addition to or instead of the touch display).

[0038] In particular, the robotic guidance system can be further adapted to output a navigation control menu as an operation menu via the touch display, which allows control of the settings of the information displayed on the visualization monitor, in particular to show and hide planned trajectories, operation targets, or other navigation information.

[0039] In particular, the robotic guide system can also be adapted to output a navigation control menu as an operation menu via the touch display, which provides controls for switching between different monitor layouts on the main visualization monitor.

[0040] In particular, the display of additional context-dependent content based on information from the control unit / controller can save the user time when navigating the operation menus. In particular, the following context-dependent content (depending on the robot configuration, the current status of the operation plan, etc.) can be displayed individually or in selectable combinations: - Visualization system settings are now highlighted after the guide system is repositioned, allowing you to quickly adjust visualization parameters to the new position; -When navigation device movement is detected, the device guidance settings may be highlighted; - Visualize the current state / current function of multipurpose hardware buttons placed near the display; - Safety stop button on the touch display to interrupt the automated robot operation; -Displaying notifications to the user (information, warnings, errors, etc.) with or without requesting feedback from the user; - Visualize distance to target for instrument guidance with depth tracking / depth indication; -Rendering of images of the visualization system for (rough) positioning of the system; -In particular, the indicator mode of the touch display can be switched to the so-called "trackpad" mode; in this mode, A mouse icon is activated on the main visualization screen, especially on the surgical monitor, allowing the user to operate the visualization screen as a computer screen with a mouse / laptop trackpad (via the touch display), or The user can scroll through navigation views / slices (segmentations) especially on the main visualization screen on the OR monitor.

[0041] In particular, the touch display fixed to the robot head may have a sterile sheathing, preferably designed to be replaceable, to provide a sterile barrier to the sterile intervention site. For example, to operate in a sterile environment, the touch display is covered with a transparent medical (surgical) drape and is provided with connection / coupling points, especially for the surgical drape.

[0042] According to a further embodiment, an inertial measurement unit (IMU) may be further provided on or within the robot head, particularly on or within the touch display, or attached to the end effector, to detect the position and / or orientation of the touch display or robot head, and the robot guidance system, particularly the control unit, may be adapted to align the visual output of the touch display based on the detected position and / or orientation, particularly so that the output (such as an operation menu) is always displayed with a constant horizontal alignment. In other words, the inertial measurement unit (IMU) may be optionally connected to the touch display, the robot head, or the end effector to enable repositioning of the visualized content at various positions and orientations of the guidance system, particularly at extreme positions. Alternatively or additionally, the position and / or orientation of the touch display or robot head may be preferably detected via the kinematics of the robot or via real-time configuration of the robot with the robot arm segment and the robot head or connected end effector as the end effector. Furthermore, the position and / or orientation of the touch display or robot head may be preferably detected via a navigation camera of the navigation system and corresponding data processing. In particular, the control unit can adapt the indicators based on the detected pose of the touch display so that they are optimally displayed relative to a predetermined position in space, i.e., the position where the surgeon's head is located (this position can be detected, for example, by a navigation camera). In particular, this includes rotating the view of the touch display so that it appears approximately horizontal relative to the operating room floor and therefore relative to the surgeon, and stretching or compressing the view to provide as neutral and natural a view as possible at the detected position of the surgeon (similar to arrows or labels or lane markings on the road that are adjusted (stretched) to the driver so that the driver can perceive the information as best as possible).In particular, the control unit or touch display may be adapted to output such a projection via the touch display onto a virtual plane, which plane is perpendicular to the line of sight between the surgeon and the touch display, so that the surgeon does not see a distorted representation (if the touch display is at an angle to the line of sight), but rather sees a representation similar to what the surgeon would see if he were looking at the touch display perpendicularly.

[0043] Preferably, the touch display is connected to a wireless LAN, in particular to a WLAN or Bluetooth (registered trademark) The touch display may comprise a wireless communication module for (independently) establishing a wireless data connection to a control unit via a touch display control device (e.g., a computer system) via a data cable. In other words, in order to control the various functions of the guidance system for the display of context-sensitive content, the touch display can be connected to a control unit (e.g., a computer system) via a touch display control device via a cable or wirelessly (using a wireless communication module). In particular, the touch display itself comprises an independent sub-control unit as the computer system and forms an independent control component. This sub-control unit can be integrated into the overall system and connected to the (central) control unit using data technology.

[0044] More preferably, the touch display may have a display diagonal or diameter of at least 4 cm and / or at most 20 cm. That is, the size of the touch display is greater than or equal to 4 cm and less than or equal to 20 cm. In particular, the touch display may have a square, rectangular, or circular form factor. The size and shape may be determined, for example, based on the number of functions and the size of the connected robot effector. The size and shape may advantageously allow the touch display to be integrated into the area of the end effector.

[0045] According to an embodiment, the robot head has at least one (physical) actuation button (hardware button), preferably three actuation buttons, and a touch display may be arranged adjacent to the button and adapted to display the current function state ( / current operating mode) of the button. This creates a multi-purpose or multi-function button that can detect different operation inputs for different operation menus. This may improve safety in particular in cases where, for example, input via the touch display fails or is inaccurate due to dirty surgical gloves.

[0046] In particular, the touch display is equipped with a color display that displays colored operation menus and information.

[0047] In further embodiments, in addition to the touch display on the robot head, touch displays may be provided on the robot arm and / or robot base.

[0048] In particular, the robotic guidance system can be configured as a robotic guidance unit, and all components can be integrated into this unit or a single module. In particular, the robotic guidance system can be configured to be mobile and can be installed and moved autonomously within the operating room, for example, as a mobile surgical microscope with a touch display located on the microscope head (as the robotic head and end effector).

[0049] The object of the present invention is solved by a method for controlling a medical robot, and in particular a robotic guidance system according to the present disclosure, by the following steps: outputting at least one visual operation menu via a touch display firmly attached to the robot head of the robot; detecting a touch-sensitive input as an operation input via the touch display; transmitting the operation input to a control unit adapted to control at least the robot; and controlling functions, in particular the movement of the robot, by the control unit based on the operation input. As mentioned above, this step provides a flexible and centralized operation method, allowing the surgeon to perform various functions, such as moving the robot and setting up the visualization system.

[0050] Preferably, the (operating) method further comprises the steps of: Detecting (changes in) the position and orientation of the robot head, in particular by an inertial measurement unit (IMU) or detected robot kinematics, or a navigation camera of a navigation system; transmitting the detected position and orientation to a control unit; calculating an alignment of a visual operation menu adapted to the position and orientation; and outputting an adapted visual representation of the operation menu. In this way, the surgeon can obtain the best or optimally adapted visual output possible, and in particular does not need to turn his head to read labels or information, or change his position to easily recognize information on the display, such as text, when looking at a touch display from an angle.

[0051] With regard to the computer-readable storage medium, the object is solved by including instructions that, when executed by a computer, cause the computer to perform the method steps of the operating method according to the present disclosure. In particular, a control unit of a robotic guidance system may be equipped with such a computer-readable storage medium.

[0052] Any disclosure relating to the robotic guide system of the present disclosure also applies to the method of operation of the present disclosure, and vice versa.

[0053] The present invention will now be described in more detail with reference to preferred embodiments with reference to the accompanying drawings, in which: [Brief explanation of the drawings]

[0054] [Figure 1] FIG. 1 shows a schematic front view of a robotic guide system according to a preferred embodiment of the present invention. [Figure 2] FIG. 2 shows a schematic diagram of the functional relationships between the touch display, the control unit, and the IMU of the robotic guidance system according to a further preferred embodiment. [Figure 3a] Figure 3a shows an example view of the operations menu in tabular form. [Figure 3b] FIG. 3b shows an example view of the operations menu in a circular format. [Figure 4] FIG. 4 shows a schematic diagram of an exemplary submenu of the visualization control menu for setting light intensity. [Figure 5] FIG. 5 shows a schematic diagram of exemplary submenus of the media control menu for recorded video management and playback functions. [Figure 6] Figure 6 shows a schematic diagram of an operation menu with only one stop button as interactive input. [Figure 7] Figure 7 shows a schematic front view of a robot head or end effector with three buttons and a touch display. [Figure 8] FIG. 8 shows a schematic front view of a robotic guide system according to a further preferred embodiment of the present disclosure. [Figure 9] Figure 9 shows various views of the robotic guidance system with a robotic head equipped with a touch display placed on the top of the microscope head opposite the optical output. [Figure 10]Figure 10 shows various views of the robotic guidance system with a robotic head equipped with a touch display placed on the top of the microscope head opposite the optical output. [Figure 11] Figure 11 shows various views of the robotic guidance system with a robotic head equipped with a touch display placed on the top of the microscope head opposite the optical output. [Figure 12] FIG. 12 shows a flow chart of the operational procedure according to the preferred embodiment.

[0055] The figures are merely schematic and are intended only to facilitate understanding of the invention. Identical elements are provided with the same reference symbols. Features of the various embodiments are interchangeable. DETAILED DESCRIPTION OF THE INVENTION

[0056] FIG. 1 shows a schematic front view of a medical robotic guide system 1 (hereinafter referred to simply as the guide system) for surgical intervention on a patient P. The guide system 1 comprises a robot 2 having a movable robot arm 4 and a robot head 6 terminally connected to the robot arm 4. In this embodiment, the guide system 1 is designed in the form of a robot-guided surgical microscope. The microscope head is provided as a robot head on the robot arm 4, which comprises multiple robot arm segments. Therefore, the robot head or the entire microscope head can also be referred to as the end effector 8 of the surgical microscope.

[0057] For controlling the robot 2 and its robot arm 4, and for positioning and orienting the microscope head as end effector 8, the guiding system 1 is provided with a control unit 10. The control unit 10 can be designed as a central control unit, for example a computer system, which can process and control various functions, or it can comprise subsystems of a robot control unit for appropriate control of the robot.

[0058] Furthermore, the guidance system 1 comprises a touch display 12 as an input / output unit, which is adapted to visually output different operation menus 14 and indicators for different functions, and to detect and send touch-sensitive inputs as operation inputs to the control unit 10 for controlling the robot 2 and configuring the visualization system.

[0059] However, unlike the prior art, the control touch display is not attached to a static base of the guide system 1, but rather the touch display 12 is rigidly attached directly to the robot head 6 or end effector 8 and moves with it.

[0060] In this way, the touch display 12 moves within the intervention area, allowing the surgeon to visualize various operation menus 14 in a unified manner and flexibly control various functions of various subsystems via a single touch display. In this embodiment, the surgeon can select from at least one robot control menu and one visualization control menu, or can switch between individual control menus as the operation menu 14. Using the robot control menu, the surgeon can, for example, perform instrument guidance or move the robot head 6 so that the end effector 8 is in a pose suitable for the intervention. In this case, the microscope head is moved to a pose that provides better visibility as the end effector via operation input on the touch display 12, either manually or automatically.

[0061] Further images can be output via an external (operating room) monitor 15 and the media output can be controlled by further media control menus via the touch display 12. For example, real-time images from a microscope can be output via the operating monitor 15 and brightness and color contrast can be set via the media control menu.

[0062] In the first embodiment of the robot guide system 1, the surgeon is provided with at least three control menus: a robot control menu, a visualization control menu, and a media control menu, which can be operated centrally to flexibly and safely control various functions.

[0063] 2 further shows a schematic functional diagram of the interactions between the individual modules of the guidance system 1 according to the second preferred embodiment. Here, an inertial measurement unit (IMU) 16 is placed directly on the touch display 12 to detect movement (or changes in movement due to acceleration) and determine the new position and orientation of the touch display 12 after the robot's movement based on the initial pose. Data from the IMU is sent directly to the central control unit 12. Furthermore, the central control unit 10 receives data from the end effector control unit 18 (sub-control unit). Furthermore, the central control unit 12 has a bidirectional data connection with the touch display control unit (as a sub-control unit). This central control unit 12 calculates a visual output based on this data and outputs it to the touch display 12.

[0064] In particular, the view most favorable to the surgeon can be calculated in this manner and output accordingly. Unlike the prior art, the touch display is not statically positioned but dynamically moves with the end effector 8, so the indicators change alignment with the surgeon. For example, if the robotic arm 4 in FIG. 1 is rotated upwards by 90 degrees, the visual field of view of the touch display 12 will also rotate counterclockwise by 90 degrees relative to the surgeon without compensation.

[0065] However, the control unit 12 calculates such alignment so that the surgeon can continue to easily read the visual indicator. In the above example, the indicator is rotated 90 degrees clockwise relative to the touch display, so that the rotation and counter-rotation cancel each other out and are intuitively readable by the surgeon. In other words, during the movement of the robot 2, the control unit calculates an arrangement of the indicator that appears as constant as possible to the surgeon, allowing, to some extent, for a constant relationship of the indicator on the touch display to the surgeon.

[0066] In one embodiment, for example, when changing from a front view (i.e., a vertical view) to an oblique view on the touch display 12, the visual output may be distorted accordingly (similar to road markings in the form of letters being elongated and thinned to provide the driver with the best possible visibility).

[0067] Figures 3a and 3b are exemplary illustrations of operation menus presented by a touch display, with Figure 3a showing a table-like arrangement with rectangular symbol-labeled (touch) buttons and Figure 3b showing a circular arrangement of symbol-labeled (touch) buttons that a user can use to select submenus, for example as shown in Figures 4 and 5 and described below.

[0068] Figure 4 shows the submenus of the lighting control menu when the symbol-labeled light bulb is selected in the main menu. In this embodiment, the user can set the brightness of the white light, as well as the intensity of the ultraviolet and infrared light.

[0069] When the user selects the video symbol in the main menu of Figure 3a or 3b, the menu jumps to the video function submenu shown in Figure 5, where the user can select functions for managing and playing back the video data. The video can be played back on an external monitor (not shown here) in the operating room.

[0070] FIG. 6 shows a schematic diagram of a (digital) stop button (a kind of emergency button) on the touch display 12 for stopping the movement of the moving robot 2 or robot arm 4.

[0071] 7 shows a front view of the end effector of another preferred embodiment of the robotic guide system 1. In this embodiment, three physical push / actuation buttons 22 are provided on the end effector 8, equidistant from one another and arranged in a straight line. Parallel to and above the linear arrangement of the three actuation buttons 22, a touch display 12 is mounted on the end effector, displaying the function status above the corresponding actuation button 22. In this way, multiple functions can be assigned to the three actuation buttons 22 and visualized accordingly by the touch display.

[0072] For example, the control unit 12 can assign different functions to actuation buttons for different steps in the operation plan and output them via the touch display.

[0073] FIG. 8 shows a schematic front view of another embodiment of the guide system 1. The robot 2 comprises a fixed robot base 24 to which a robot arm 4 is movably connected. A robot head 6 in the form of a (digital) microscope head is connected to the end of the robot arm 4 as an end effector 8, which can adjust both the position and orientation (i.e., spatial pose) of the microscope head to achieve the appropriate imaging pose to create a digital microscope image. The (vertical) dashed line indicates the optical axis of the microscope head, which passes through the optical system (not shown) and then through the CMOS sensor for the digital image. By configuring both the position (3 degrees of freedom) and the orientation (3 degrees of freedom), 6 degrees of freedom / 6 DOF (degrees of freedom) can be configured (within the robot's kinematics).

[0074] In this embodiment, the robot arm 4 has multiple robot arm segments 26, each connected to another via a joint 28. The robot head 6 with optics is also connected to the robot arm 4 via another joint 28, and the robot arm 4 is connected to the robot base 24 via a joint 28. In particular, the joints of the first and second robot arm segments 26 can have a rotational degree of freedom for rotation about an axis of rotation, which in the kinematic position of the robot arm 4 shown in FIG. 8 can be positioned in a particularly horizontal direction (i.e., perpendicular to the up-down direction) to provide a kind of cantilever.

[0075] The robot head 6 is of rigid construction (or has a rigid robot head housing in which the optics, downstream CMOS sensor, and other electronics are housed), so that the robot 2 (or the robot's control unit) can control the position and orientation of the robot head 6 via a multi-link drive kinematic system and adjust the optical axis accordingly.

[0076] In this embodiment, the touch display 12 is positioned on a lateral side (i.e., on the non-rear side in extension of the longitudinal axis of the robot head) of the robot head 6. In particular, the touch display can be positioned on the robot head such that the normal to the display surface is substantially perpendicular to the longitudinal axis of the robot head, such as the viewing axis of a surgical microscope.

[0077] The touch display 12 is arranged on the top of the robot head 6, as shown in FIG. 8. As shown in FIG. 8, the lower side forms the optical aperture of the digital microscope's optical system, and the touch display is arranged in the upper region. It can also be said that the optical output is arranged in a first lower region of the robot head 6, and the touch display is arranged in a second upper region of the robot head, with the first and second regions forming end regions facing opposite each other. Thus, the touch display 12 is arranged so that the laterally arranged touch display is particularly clearly visible and operable by medical personnel during normal surgery, when the surgical microscope or microscope head creates a top-down image of the patient. The actuation button 22 is arranged below the touch display 12, as shown in FIG. 8, so that the touch display forms the topmost element. Thus, when viewed along the longitudinal axis of the robot head 6, the following elements are arranged in the axial order: the touch display 12, the actuation button, and the optical output. The robot arm 4 is connected to the side of the robot head 6 via joint 28, and the three dashed rotational indications are intended to indicate that the robot head can be oriented around three axes, or has three rotational degrees of freedom.

[0078] 9 to 12 show side, top, and isometric perspective views of a medical robotic guide system 1 according to a further preferred embodiment. The robot 2 is designed in the form of a robot-guided surgical microscope, and its robot head 6 forms an end effector 8 as a microscope head, the position and orientation of which can be adjusted via the robot arm 4. The optical output is provided on the underside as shown in FIG. 9, and the optical axis is indicated by a dashed line. The touch display 12 is disposed on the opposite side of the optical output. In this embodiment, the touch display 12 is circular, and the center of the circular touch display 12 is substantially on the extension of the optical axis, i.e., it is concentric with the optical axis, so to speak. It can also be said that the normal of the display is parallel to the optical axis, particularly on the optical axis (the optical system and the touch display are symmetrical to each other to a certain extent).

[0079] The touch display 12 forms a flat surface at the top of the front face and is spaced apart (i.e., protruding) from the rest of the microscope head (and even from the robot arm 4) to allow for easy operation. Looking from top to bottom along the longitudinal or optical axis of the robot head, the touch display 12 is provided first, followed by an input means 30 in the form of a 3D mouse or 3D space mouse, and finally an optical output (e.g., the last lens in an optical system). The input means 30 is also located on the extension of the longitudinal axis of a cylindrical arm with a pivot joint 28 to a robot arm segment 26 opposite the connection to the robot arm 4. The zero axis of the 3D mouse is concentric with the axis of the robot arm segment 26.

[0080] Additionally, the actuation buttons 22 can also be used to control movement and / or zoom and / or other functions, depending on the state of the functions in particular.

[0081] The robot head 6 is connected to the robot arm 4 (or robot arm segment 26) via a pivot joint as joint 28. The orthogonal line to the touch display 12 is also perpendicular to the rotation axis, where the orthogonal line to the touch display 12 is parallel to the optical axis.

[0082] FIG. 12 shows a flow chart of the method of operation according to the preferred embodiment.

[0083] In the first step S1, the operation menu 14 is output by the touch display 12.

[0084] Specifically, in this embodiment of the operating method, the output of the operating menu is solved by the following sub-steps: In a first sub-step S1.1, the position and orientation (i.e. pose) of the touch display 12 is detected, in particular by the inertial measurement unit (IMU) 16 (preferably also indirectly via the robot head 6). Alternatively, the pose may be detected via detected (mechanical) robot kinematics or by a tracking / navigation camera of the navigation system.

[0085] Next, in substep S1.2, the pose of the touch display 12 is passed to the control unit 12.

[0086] An adaptive alignment of the output is then calculated in substep S1.3, and this adapted representation is output via the touch display 12 in substep S1.4.

[0087] In the following second step S2, an input to the touch display 12 is detected.

[0088] In a third step S3, the detected operation input is transmitted to the control unit 12.

[0089] Finally, in step S4, the corresponding function displayed on the touch display 12 and selected is controlled.

[0090] Specifically, it may be the control of the robot 2. [Explanation of symbols]

[0091] 1 Medical robotic guide system 2. Robot 4. Robotic Arm 6 Robot Head 8 End Effector 10. Control Unit 12 Touch Display 14 Operation Menu 15 External Monitor 16 Inertial Measurement Unit (IMU) 18 End effector control unit 20 Stop button 22 Operation button 24 Robot Base 26 Robot Arm Segments / Robot Arm Links 28 joints 30 Input Methods P patient Steps to output the operation menu via the S1 touch display S1.1 Steps to detect position and orientation using IMU S1.2 Steps to send to the control unit S1.3 Steps for calculating adaptive alignment S1.4 Step to output adaptive representation Steps to detect input on the S2 touch display S3: Transmitting operation input to the control unit Steps to control S4 functions

Claims

1. A medical robotic guided system (1) for surgical intervention on a patient (P), comprising: a movable robot arm (4) comprising a robot head (6) terminated on the robot arm (4), in particular comprising a robot (2) with an end effector (8) on or as the robot head (6); a control unit (10) adapted to control and move at least the robot (2); at least one touch display (12) adapted to visually output at least one operation menu (14) and to detect touch-sensitive inputs as operation inputs and transmit them to the control unit (10) in particular for controlling the robot (2); At least one touch display (12) is fixedly attached to the robot head (6) and moves with the robot head (6). Medical robotic guide system (1).

2. the robotic guidance system (1) is in the form of a surgical microscope with a microscope head connected to the robotic arm (4) as a robotic head (6) or in the form of a navigation system with a camera system, in particular with a laser system, A medical robotic guide system (1) according to claim 1.

3. The robot guide system (1) is configured to output a robot control menu as an operation menu (14) via a touch display (12) in order to control the robot (2) through operation input, particularly to control the robot's movement with six degrees of freedom. A medical robot guide system (1) according to claim 1 or 2.

4. the robotic guidance system (1) is configured to output a visualization control menu as an operating menu (14) via a touch display (12) in order to control the settings of the visualization system, preferably of a surgical microscope, in particular the zoom, focus and / or illumination intensity; A medical robot guide system (1) according to any one of claims 1 to 3.

5. the robotic guidance system (1) is configured to switch between at least two different control menus in the touch display (12), preferably a robot control menu and a visualization control menu, in order to control at least two different functions of the robotic guidance system, in particular the movement of the robot (2) as a first function and the visualization as a second function; A medical robot guide system (1) according to any one of claims 1 to 4.

6. the touch display (12) fixed to the robot head (6) has a sterile sheath, preferably designed to be replaceable, in order to provide a sterile barrier to the sterile intervention site; A medical robot guide system (1) according to any one of claims 1 to 5.

7. an inertial measurement unit is further provided on or in the robot head (6), in particular on or in the touch display (12) or attached to the end effector (8) to detect the position and / or orientation of the touch display (12), and the robot guidance system (1), in particular the control unit (10), is characterized in that it adapts the alignment of the visual output of the touch display (12) based on the detected position and / or orientation, in particular adapting the output so that it is always displayed with a constant horizontal alignment; A medical robot guide system (1) according to any one of claims 1 to 6.

8. the touch display (12) is characterized in that it comprises a wireless communication module for establishing a wireless data connection with the control unit, in particular via WLAN or Bluetooth; A medical robot guide system (1) according to any one of claims 1 to 7.

9. The touch display (12) has a display diagonal or a display diameter of at least 4 cm and / or at most 20 cm. A medical robot guide system (1) according to any one of claims 1 to 8.

10. the robot head (6) has at least one activation button (22), preferably three activation buttons (22), and the touch display (12) is arranged adjacent to the activation button (22) and is configured to display the current function status of the activation button (22). A medical robot guide system (1) according to any one of claims 1 to 9.

11. The robotic guidance system (1) is configured in the form of a surgical microscope with a microscope head connected to a robot arm (4) as a robot head (6), characterized in that an optical output is formed on one side of the microscope head and, on the opposite side to this side, a touch display (12) is arranged in front, in particular in a direction perpendicular to the touch display (12) parallel to the optical axis. A medical robot guide system (1) according to any one of claims 1 to 10.

12. the touch display (12) is circular, in particular circular, and preferably, when the robot (2) is designed as a robot-guided surgical microscope, the center point of the circular touch display (12) is on the extension of the optical axis; A medical robot guide system (1) according to any one of claims 1 to 11.

13. A medical robot control method for a medical robot guide system (1) according to any one of claims 1 to 4, comprising the following steps: outputting (S1) at least one visual operation menu (14) as a visual representation via a touch display (12) firmly attached to the robot head (6) of the robot (2); Detecting a touch-sensitive input as an operational input via the touch display (12) (S2); sending (S3) an operational input to a control unit (10) configured to control at least the robot (2); Controlling (S4) the function, in particular the operation of the robot (2), by the control unit (10) based on the operation input; A medical robot control method.

14. 12. The method for controlling a medical robot according to claim 11, further comprising the steps of: Detecting the position and / or orientation of the robot head (6) by means of an inertial measurement unit (S1.1); transmitting the detected position and / or direction to a control unit (S1.2); Calculating the placement of the visual representation, in particular the visual manipulation menu (14), adapted to the position and / or orientation (S1.3); outputting (S1.4) an adapted visual representation, in particular an adapted visual representation of an operation menu (14), A method for controlling a medical robot.

15. A computer-readable storage medium containing instructions which, when executed by a computer, cause the computer to perform the method steps of the robot control method according to claim 11 or 12.

Images