Medical robotic system

The medical robot system addresses collision detection challenges by using a movable robot arm with an optical recording device and display that highlights environmental changes during movements, improving safety by maintaining surgeon attention on the surgical field.

DE102024112788A1Pending Publication Date: 2025-11-13B BRAUN NEW VENTURES GMBH
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
DE102024112788
Authority / Receiving Office
DE · DE
Patent Type
Applications
Current Assignee / Owner
Filing Date
2024-05-07
Publication Date
2025-11-13

AI Technical Summary

Technical Problem

Existing medical robot systems face challenges in reliably detecting collisions due to continuous changes in the operating environment, leading to potential safety risks for patients and surgeons, and visual monitoring systems can distract surgeons, reducing their attention to the surgical field.

Method used

A medical robot system with a movable robot arm equipped with an optical recording device that generates wide-angle recordings, a display to image these recordings, and a trigger controller to display the environment only during arm movements, alerting the user to potential collisions.

Benefits of technology

The system reduces collision risks by ensuring the surgeon's attention is focused on the surgical field by displaying environmental changes only during arm movements, enhancing patient safety.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

The present disclosure relates to a medical, preferably surgical, robotic system (2), comprising a robot (4) having at least one movable robotic arm (6), preferably for positioning surgical instruments (18) and / or a visualization system (19), an optical recording device (8) attached to the robotic arm (6), preferably an end effector (14) of the robotic arm (6), and configured to produce a preferably wide-angle optical image (16), a display (10) configured to image the optical image (16), and a trigger control (12) configured to control the imaging of the optical image (16) on the display (10) depending on a movement of the robotic arm (6).
Need to check novelty before this filing date? Find Prior Art

Description

[0001] The present disclosure relates to a medical, preferably surgical, robotic system, comprising a robot having at least one movable robotic arm, preferably for positioning surgical instruments and / or a visualization system, a (first) optical recording device attached to the robotic arm, preferably an end effector of the robotic arm, and configured to produce a (first) optical image, and a display configured to show the (first) optical image. Background of the Revelation

[0002] The use of medical robotic systems is particularly widespread in surgery to support precise, for example, minimally invasive procedures. A robot with at least one movable robotic arm can be used, in particular, to position surgical instruments in space and / or to perform individual surgical operations. For positioning, the movable robotic arm, especially its end effector, which holds the surgical instrument, can be moved within the space. One risk here is that a collision between the robotic arm or its end effector and the environment, such as medical personnel, especially surgeons, or a patient, can occur during movement.

[0003] To mitigate the risk of collisions, medical robotic systems are equipped with collision detection systems. As disclosed, for example, in patent CN 1 10 613 511 A, collision detection in a medical robotic system can be implemented, similar to collision detection in vehicles, using ultrasound sensors, lidar sensors, or 3D point clouds for distance control. However, such collision detection is not very reliable due to constant changes in the robot's collision space, such as those caused by the surgeon's movements. An alternative is to model the robot's environment, particularly the patient's position, using navigation and pre-operative data. However, such a static method does not account for changes to the patient during surgery. Furthermore, it does not allow for collision detection with objects that are not modeled.Due to the lack of reliable collision detection, safety depends on how carefully the robot is moved.

[0004] Furthermore, there are medical robotic systems that feature a recording device / camera attached to the robot arm, particularly the end effector, for creating an optical image and a display to show the image in order to indicate the (immediate) surroundings and, in particular, a potential collision area of ​​the robot arm. This camera monitoring or optical display of the surroundings thus constitutes a passive camera system that serves to enable and / or simplify the control of the robot arm's movement and, in particular, to avoid collisions of the robot arm through careful control and knowledge of the immediate surroundings.

[0005] However, a disadvantage of this approach is that – especially when the robot arm is controlled by the surgeon – the constant visual display for monitoring the surroundings can lead to the surgeon paying less and less attention to it over time, thus undermining the goal of reducing the risk of collisions. Furthermore, it can distract the surgeon's attention from the surgical field, which in turn compromises patient safety. Brief description of the Revelation

[0006] In view of the problems described above, it is therefore a task of disclosure to provide a suitable medical robotic system that increases (patient) safety and in particular helps a user, for example the surgeon, to avoid a collision of the medical robotic system, especially with a patient.

[0007] This problem is solved by a medical robot system according to claim 1. Advantageous embodiments are the subject of the dependent claims and / or described below.

[0008] The disclosed medical robot system comprises a robot having at least one movable robot arm, a (first) optical recording device attached to the robot arm and configured to create a (first) optical image, a display configured to image the (first) optical image, and a trigger control configured to control the imaging of the (first) optical image on the display depending on a movement of the robot arm.

[0009] The advantages of this disclosure feature are that the medical robotic system reduces the risk of collision by displaying the initial optical image on the screen in relation to the robot arm's movement. For example, if the robot arm begins to move, this can trigger a change in the display or rendering of the initial optical image. This alerts the user, such as a surgeon, to the movement. This reduces the risk of collision and increases patient safety. If the initial optical image is only displayed and / or highlighted during robot arm movement, and not otherwise, the user will not be distracted by the initial optical image when the robot arm is stationary. This also increases patient safety.In other words, the user's attention is preferably drawn to the first visual image only when necessary or when safety is increased. The robot system thus primarily assists the user in preventing collisions by capturing the robot's surroundings as the first visual image and displaying it on the screen.

[0010] In other words, the present disclosure relates to a medical, preferably surgical, robotic system comprising a movable robotic arm (in an operating environment), a recording device / camera mounted on the robotic arm (and moving accordingly with the robotic arm) for capturing, in particular, wide-angle images of the operating environment, and a display for showing the operating environment captured by the recording device (i.e., the optical image). The display is configured (or the medical robotic system has a trigger control / recording device control / display control that is set up) to display / image / show the operating environment captured by the recording device on the display depending on a movement of the robotic arm. In particular, the display is set up or controlled to show the operating environment (preferably only) when the robotic arm is moving.This means that the operating environment is only displayed during movement, or not before the start of the movement and / or after the end of the movement, i.e., when the robot arm is not yet moving and / or is no longer moving.

[0011] Preferably, the medical robotic system is a medical, and in particular a surgical, robotic system. In other words, the robotic system is preferably used in surgery. However, the disclosure is not limited to this. The medical robotic system can also be used in other medical specialties, such as dermatology or ophthalmology.

[0012] Preferably, the robot is a six-axis robot with six degrees of freedom, particularly preferably a six-axis articulated robot or a six-axis articulated robot. Preferably, the robot has a robotic arm that is suitable and / or configured to position surgical instruments and / or a visualization system. The robot can then assist a surgeon during an operation. A surgical microscope and / or an endoscope are preferred examples of such a visualization system. An optical imaging device, preferably one of those mentioned herein, is particularly preferred. In other words, the visualization system can be an optical imaging device.

[0013] Preferably, the robot has an end effector, i.e., a device at a (free) end of the robot arm or a final / end-ended kinematic robot arm element, which is configured to interact with the environment and / or the patient. The end effector can, for example, be at least one gripper or holder that can, for instance, pick up and / or hold a visualization system and / or surgical instruments. The end effector can also itself be a visualization system and / or an optical recording device, such as one of those mentioned herein.

[0014] The first optical recording device is preferably a camera. Preferably, the first optical recording device is mounted on the robot arm, particularly preferably on the end effector. Preferably, the first optical recording device is wide-angle. This has the advantage that a particularly large field of view can be captured, which allows the recording to provide a better / broader overview of the scene, especially at shorter distances between the recording device and the area to be captured.

[0015] As described above, the robotic system is preferably used / applied in surgery to assist a surgeon. Preferably, the first optical imaging device is aligned with and / or focused on a surgical field. That is, preferably, the first optical imaging device is aligned with and / or focused on, for example, an operating table and a patient located thereon (and the patient's surroundings). The first optical image is preferably an image of the robot's immediate surroundings. Particularly preferably, the first optical image is an image of the surgical field, especially of a patient who is undergoing or is to undergo surgery and / or at least of the body part or section on which the surgery is being or is to be performed. However, the disclosure is not limited to this.Particularly when the medical robot system is used in other medical, non-operative fields, for example, the first optical recording device can be directed at an area, such as a treatment / examination area, and the first optical image can then be an image of that area.

[0016] A representation can be understood as a (visual) display and / or depiction.

[0017] It should be understood that the display can be a single unit or multiple units. For example, the display can be a single screen, but it can also consist of several screens that (together) display the same or different content.

[0018] Preferably, the robot system and / or the display and / or the first optical recording device is configured to display the first optical recording as a live transmission, live stream, or real-time transmission on the display. In other words, it is preferred that when the display displays the first optical recording, it shows what the first optical recording device sees and / or records as the first optical recording at (near) that moment.

[0019] The preferred display setting is to show multiple images, videos, optical recordings, etc. simultaneously in various ways, for example side by side, inside each other as picture within a picture, etc.

[0020] A movement of the robot arm can result, for example, from a control / regulation system, a movement command entered by a user via a joystick, or from a user touching and moving the robot arm.

[0021] Preferably, the trigger control is configured to initiate the display of the first optical image as a function of the robot arm starting its movement, particularly preferably at the start of the robot arm's movement. For example, the first optical recording device can be switched on (only) at this point or activated from a standby mode. Alternatively or additionally, a (data) transmission of the first optical image to the display can also occur (only) at this point, and / or the display can be switched on (only) at this point or activated from a standby mode. For example, the display can initially show something else, such as another image or a still image, which is then replaced or supplemented by the first optical image as a function of the robot arm starting its movement, particularly preferably at the start of the robot arm's movement.

[0022] Preferably, the trigger control is configured to terminate the display of the first optical image when the robot arm stops moving, particularly preferably when the robot arm's movement has ceased. This can be achieved, for example, by switching off the first optical recording device or putting it into standby mode. Alternatively or additionally, the (data) transmission of the first optical image to the display can be prevented, and / or the display can be switched off or put into standby mode. The display can also show something other than the first optical image, such as a still image or (without the first optical image) at least one other image.

[0023] The trigger control can (also) be configured to change the way the first optical image is displayed on the screen depending on when the robot arm starts moving, particularly preferably at the start of the robot arm's movement. Preferably, the first optical image can be visually highlighted, for example, with a marker or indicator, by being enlarged or displayed larger / largest, displayed in the foreground, and / or in some other way, whereby the image / displaying method is changed and attention is drawn to it more strongly. For example, depending on when the robot arm starts moving, particularly preferably at the start of the robot arm's movement, the first optical image can (now) be displayed larger, in the foreground, and / or with a marker or similar on the screen.

[0024] The trigger control can (then) (also) be configured to change the display mode of the first optical image depending on whether the robot arm's movement stops, particularly preferably when the robot arm's movement ends. Preferably, the highlighting of the first optical image is ended / undone.

[0025] Preferably, a movement or the start of a movement of the robot arm (from a previous standstill / rest / immobility) triggers the start of an image being displayed, or a different / modified image being displayed, of the first optical image on the screen. Preferably, the cessation of a movement of the robot arm (standstill after a previous movement) triggers the cessation of the image being displayed, or the different image being displayed, of the first optical image on the screen. Preferably, the first optical image is displayed on the screen during a movement of the robot arm and not otherwise (when there is no movement), or, for example, the first optical image is displayed differently on the screen during a movement than otherwise.

[0026] The trigger control can be configured to control / change the rendering of the first optical image on the display, preferably as described above, depending on the movement of the robot arm. For example, the rendering of the first optical image on the display can be controlled / changed depending on a distance changing due to the movement, e.g., to another object, a patient, or user of the robot system in the collision area, a collision imminent due to the movement, preferably as described below, the speed and / or acceleration of the robot arm, preferably the end effector and / or devices attached to it.For example, the first visual impression can be emphasized more strongly, preferably as described above, the smaller the distance, the greater the likelihood of a collision, the greater the speed, and / or the greater the acceleration. A scale, preferably displayed on the screen alongside the first visual impression, can also visualize, for example, a distance, a collision risk, a speed, or an acceleration, and change depending on this or the movement, thus emphasizing the first visual impression.

[0027] A movement therefore preferentially leads to a change in the image (the first optical image) on the display, with the trigger control being set up to control the display (accordingly).

[0028] Such a change on the display can then be noticeable to a user and draw their attention. This way, the user is made more aware that the robot arm is moving. This can increase safety and reduce the risk of collision.

[0029] Furthermore, the user can preferably use the first image to see the (changing) view / perspective of the first optical recording device, preferably from the end effector, of, for example, the patient, and thus estimate a distance (which may change with movement). This can also increase safety and reduce the risk of collision.

[0030] Preferably, the first optical image is not displayed or highlighted on the screen if the robot arm is not moving. This allows the user to concentrate on the operation and / or another optical image displayed on the screen without being distracted by the first image. This can also increase patient safety.

[0031] The first optical recording device can be a 2D or a 3D camera.

[0032] Preferably, the first optical recording device is configured to capture 3D bodies and / or 3D surfaces, preferably with / as stereo reconstruction, point clouds, and / or lidar, preferably as the first optical image. The display is preferably configured accordingly, preferably three-dimensionally, to depict such bodies and / or surfaces (possibly further processed). This allows for a better spatial understanding, and in particular, enables a user to better estimate depth and distances. The robot system can also calculate distances for collision avoidance.

[0033] Preferably, the robot system is configured to detect an impending collision (based on the movement of the robot arm) based on the captured 3D surfaces and to take appropriate measures. An impending collision can be detected, for example, based on the captured 3D surfaces and / or the motion control of the robot arm. A minimum distance and / or a maximum speed (preferably distance-dependent) and / or a maximum acceleration (preferably distance- and / or speed-dependent) can also be set as a limit value, for example (within the workspace) for the end effector relative to the / specific captured 3D surfaces.

[0034] Such a measure could be, for example, a warning, preferably issued on / from the display, braking, stopping, redirecting, suppressing a directional component of the robot arm's movement and / or highlighting the first optical image, for example as described above.

[0035] An impending collision (detected due to the captured 3D surfaces) can be imminent and / or detected between elements of the robot system, such as the robot arm, the end effector and / or devices attached to the end effector, such as the first optical recording device or surgical instruments, and other objects in the robot's collision space, such as the patient, and preferably trigger at least one action as described above.

[0036] Preferably, the robot system includes a further (second) optical recording device configured to create a further (second) optical image. Preferably, the further (second) optical recording device is mounted on the robot arm, particularly preferably on the end effector. Preferably, the further (second) optical recording device is a microscope and the further (second) optical image is a microscopic image.

[0037] Such an image can, for example, support a user during an operation by (magnifying) the surgical area (body part / body area) being captured / can be captured and / or displayed.

[0038] Preferably, the second optical imaging device is aligned with and / or focused on the surgical field. That is, preferably, the second optical imaging device is aligned with and / or focused on, for example, a body part to be operated on. However, the disclosure is not limited to this. In particular, when the medical robotic system is used in other medical, non-surgical fields, the second optical imaging device can, for example, be directed at an area, such as a treatment / examination area, and the second optical image can then be an image of this area.

[0039] Preferably, the display is configured (optionally or depending on the robot arm movement) to show the first and second optical images together (and / or simultaneously) and without displaying each other. When the first and second optical images are displayed together (and / or simultaneously), a user can preferably view the overall scene (e.g., operating table, patient, etc.) from the first optical image and, simultaneously, the surgical area (in more detail) from the second optical image. Preferably, the display is a monitor or screen, but it can also have multiple monitors or screens. Preferably, the first optical image is displayed on the same monitor or screen as the second optical image.This has the advantage that, for example, a user who is already looking at the display because of the second optical image will then notice the display of the first optical image and can perceive both images simultaneously. However, the disclosure is not limited to this; the first and second optical images can also be displayed (simultaneously) on different monitors or screens.

[0040] Preferably, the robot system and / or the display and / or the second optical recording device is configured such that the second optical recording is displayed as a live stream or real-time transmission on the display. In other words, preferably, when the display shows the second optical recording, it shows what the first optical recording device sees and / or records as the second optical recording at (near) that moment.

[0041] The display can be configured to emphasize either the first or the second optical image more strongly when the first and second optical images are displayed simultaneously, preferably as described above. Preferably, the display is configured to show the first optical image smaller than the second optical image when they are displayed simultaneously. Preferably, the first optical image is displayed within the second optical image. Thus, the first optical image is preferably displayed within the second optical image as a picture-in-picture or in the foreground. This can draw attention to the first optical image, and the second optical image, preferably a microscopic one, can be displayed larger / large enough so that, for example, the surgical field is clearly visible.

[0042] The display can be configured, particularly during robot arm movement, to prominently show the first optical image, preferably as described above, for example, even more prominently than the second optical image. Thus, for example, preferably during robot arm movement, the first optical image can be displayed larger (than when the robot arm is stationary) and / or with a marker, scale, or warning. The display can also be configured to show the first optical image depending on the robot arm's movement, preferably during movement, without / instead of the second optical image, and / or (only, otherwise) to show the second optical image when the robot arm is stationary.Depending on the movement of the robot arm, the first optical image can also be displayed on a first (main) screen of the display (on which, for example, the second optical image is otherwise displayed), and the second optical image can then be displayed on a second screen.

[0043] The movement of the robot and / or robot arm can be detected and / or verified, for example, by means of a change in the joint angle of a joint of the robot arm, with navigation markers on the robot arm, by means of a control signal / movement command to the robot and / or robot arm, for example via a joystick, and / or by means of a (temporary) change in perspective from the first optical recording and / or the second optical recording or a change in the orientation of the first and / or second optical recording device.

[0044] Preferably, the first and / or the second optical recording device is rotatably and / or movably mounted on the robot arm, preferably the end effector. The robot system and / or the first optical recording device and / or the second optical recording device can be configured to perform a counter-movement of the respective optical recording device(s) and thus compensate for a movement of the robot arm, so that the orientation of the respective optical recording device(s) or the perspective of the respective optical image(s) is maintained. This can be advantageous, for example, if a user wants to move the robot arm out of the way but does not want to change the first or second optical image. A movement of the robot arm can then also be detected by means of such a counter-movement.

[0045] Preferably, the first and / or second optical recording device has an autofocus. Brief description of the characters

[0046] The disclosure is explained in more detail below with reference to preferred embodiments and the accompanying figures. Fig. Figure 1 shows a medical robotic system according to the present disclosure; Fig. 2 shows the medical robot system from Fig. 1 with a robot arm at rest according to the present disclosure; Fig. 3 shows the medical robot system from Fig. 1 and Fig. 2 with the robot arm in motion according to the present disclosure; Fig. Figure 4 shows a flowchart and illustrates how mapping on a display according to the present disclosure can depend on a movement; Fig. 5 shows an example of highlighting a first optical image according to the present disclosure; Fig. 6 shows another example of highlighting a first optical image in accordance with the present disclosure; Fig. 7 shows another example of highlighting a first optical image in accordance with the present disclosure; Fig. Figure 8 shows another example of highlighting a first optical image in accordance with the present disclosure; Fig. Figure 9 shows a flowchart and illustrates how, according to the present disclosure, for example, an impending collision can be detected and a response can be taken; Fig. Figure 10 illustrates by way of example how a movement of a robot arm can be detected according to the present disclosure.

[0047] The figures are schematic and serve only to illustrate the revelation. The features of the different versions can be interchanged. Detailed description of preferred embodiments

[0048] Fig. Figure 1 shows a medical robot system 2. In this case, it is a surgical robot system. The robot system 2 comprises a robot 4 with a movable robot arm 6, a first optical recording device 8, a display 10, and a trigger control 12.

[0049] The robot arm 6 has an end effector 14. The first optical recording device 8 is attached to the robot arm 6, in this case to the end effector 14, and configured to acquire a first optical image 16. The first optical recording device 8 can be a 2D camera 8' or a 3D camera 8". In particular, the recording device 8 can be wide-angle. The first optical recording device 8 can be directed towards a surgical field or a patient 24.

[0050] The robot 4 is configured to position surgical instruments 18 and a visualization system 19. In this case, the robot system 2 has a second optical receiving device 20, here a microscope 20'. Here, the microscope 20 is therefore the visualization system 19. The second optical receiving device 20 is attached to the robot arm, in this case also to the end effector 14. The second optical receiving device 20' is configured to create a second optical image, in this case a microscopic image 22.

[0051] Display 10 is configured to show the first optical image 16. Display 10 can also be configured to show the second optical image 22. For example, display 10 can be configured to show the first optical image 16 and the second optical image 22, each without the other, and also both together / simultaneously. The second optical imaging device 20 can be aligned with the patient 24.

[0052] The trigger control 12 is configured to control the display of the first optical image 16 on the display 10 depending on a movement of the robot arm 6. In particular, the trigger control 12 can be configured to begin displaying the first optical image 16 on the display 10 depending on the start of the movement of the robot arm 6, preferably at the beginning of the movement of the robot arm 6, and / or to end displaying the first optical image 16 on the display 10 depending on the stop of the movement of the robot arm 6, preferably at the end of the movement of the robot arm 6.

[0053] Fig. Figure 2 shows the medical robot system 2 from Fig. 1 and a user 26 of whom, in this case a surgeon. Fig. Figure 2 shows the robot arm 6 at rest / stationary, i.e., not in motion. This is illustrated by the dot above the end effector 14. The first optical image 16 is not shown on the display 10. The second optical image 22 is shown on the display 10.

[0054] Fig. Figure 3 shows the medical robot system 2. Fig. 1 and Fig. 2, but now the robot arm 6 is in motion. This is illustrated by the arrows above the end effector 14. The display 10 shows the first optical image 16. The display 10 also shows the second optical image 22. The image of the first optical image 16 is smaller and displayed in the foreground, or as a picture-in-picture, i.e., highlighted, within the image of the second optical image 22. Because the first optical image 16 is displayed in the foreground, the user's attention 26 is drawn to it.

[0055] Fig. Figure 4 shows a flowchart illustrating when or how preferentially the display 10 shows the first optical image 16 and / or the second optical image 22. The sequence shown is, for example, carried out by the robot system 2. Fig. 1. First, it is checked whether the robot arm 6 begins a movement. If the robot arm 6 begins a movement, condition B1 is met (Yes). If the robot arm 6 does not begin a movement, or if condition B1 is not met (No), the process is terminated. If the robot arm begins a movement, in step S1 the mapping of the first optical image 16 onto the display 10 begins, or the first optical image 16 is mapped onto the display 10, as is the case, for example, in Fig. 3 is shown.

[0056] Condition B2 then checks whether the movement of the robot arm 6 has ended. Condition B2 is fulfilled (Yes) if the movement of the robot arm 6 has ended. If the movement of the robot arm 6 has not ended, or if condition B2 is not fulfilled (No), the process continues in step S1, and the first optical image 16 is (continued to) be displayed. If the movement of the robot arm 6 has ended, or if condition B2 is fulfilled (Yes), then in step S2 the display of the first optical image 16 on the screen 10 is stopped, or the first optical image 16 is no longer displayed on the screen 10 (see below). Fig. 2) The process then ends. The process is preferably repeated / permanently executed, for example, when robot system 2 is switched on.

[0057] Alternatively, condition B1 can / will check whether the robot arm 6 is in motion. Accordingly, the first optical image 16 is then captured, and condition B2 checks whether the robot arm 6 is no longer in motion.

[0058] Alternatively or additionally, in step S1 the first optical image 16 can be highlighted or highlighting can be started. Similarly, in step S2 the first optical image 16 can no longer be highlighted or the highlighting can be stopped.

[0059] Fig. Figure 5 shows an example of highlighting a first optical image 16, which is taken from Fig. 3 corresponds. The first optical image 16 is highlighted (when the robot arm 6 moves) by being displayed on a large part of a (the display 10 from Fig. The corresponding display (1) is shown in display 10. Display 10 shows, in particular, the first optical image 16 larger than the second optical image 22.

[0060] Fig. 6 shows another example of Fig. 5 for highlighting the first optical image 16. The first optical image 16 is highlighted (when the robot arm 6 is moved) by mapping or outlining the first optical image 16 with a marker 28.

[0061] Fig. Figure 7 shows another example of highlighting the first optical image 16. The first optical image 16 is highlighted (during movement of the robot arm 6) by adding a scale 30 to the first optical image 16. The scale 30 can, for example, indicate / visualize the speed of a movement of the robot arm 6.

[0062] Fig. Figure 8 shows another example of highlighting the first optical image 16. The first optical image 16 is highlighted (when the robot arm 6 is moved) by displaying the first optical image 16 alone (in particular without the second optical image 22) on the display 10.

[0063] Another preferred example of highlighting the first optical image 16 is in Fig. Figure 3 shows the first optical image 16 being highlighted (when the robot arm 6 is moved) by displaying the first optical image 16 in the foreground (of the second optical image 22) or as a picture-in-picture.

[0064] Display 10 is set up to display the first optical image 16 and / or the second optical image 22 in the manner shown.

[0065] Fig. Figure 9 shows a flowchart and illustrates how, according to the present disclosure, for example, an impending collision 32 can be detected and a reaction can be initiated, or what measures can be taken. The sequence shown is carried out, for example, by the robot system 2. Fig. 1 executed. In step S1, the first optical recording device 8 captures 3D surfaces 34, for example of a patient 24, as shown in Fig. Figure 3 shows the process. The 3D surfaces 34 are captured / obtained, for example, using stereo reconstruction, point clouds, and / or lidar. This can happen, for example, before or during a movement of the robot arm 6. Subsequently, for example, during / for a movement of the robot arm 6, condition B1 is checked. Using the captured 3D surfaces 34, it is checked whether there is an impending collision 32 between the captured 3D surfaces 34 and, for example, the moving robot arm 6 or objects attached to it, such as surgical instruments 18. If a collision is imminent (Yes), the process continues in S2. If no collision is imminent (No), the process ends. In S2, a reaction is triggered by the impending collision 34. A warning 36 can be displayed on the display 10 of the robot system 2. The marker 28 from Fig. 6 can be such a warning 36, and flashes red, for example. This alerts a user 26 to the impending collision 34. Furthermore, the movement of the robot arm 6 can be stopped.

[0066] Fig. Figure 10 shows how a movement of the robot arm 6 of a robot system 2 is performed as shown. Fig. 1 can be / will be recognized. The robot system 2, and in particular the robot 4, is simplified here and shown with only two segments of the robot arm 6. The robot arm 6, or rather the end effector 14, moves from P1 to P2. The movement is detected by a change in a joint angle 38 (Δφ) of a joint 40 of the robot arm 6, with navigation markers 42 on the robot arm 6, and by a control signal 44 (φ). soll ) to the robot 4 or robot arm 6, and / or by means of a change in perspective 46 from the first optical recording 16 and / or the second optical recording 22. List of reference symbols 2 medical robotic systems 4 robots 6 robot arm 8 first optical recording device 8' 2D camera 8" 3D camera 10 ads 12 Trigger control 14 End effector 16 first optical recording 18 surgical instruments 19 Visualization system 20 second optical recording device 20' microscope 22 second optical recording 24 patients 26 users 28 Marking 30 scale 32 Imminent collision 34 3D surface 36 Warning 38. Changing a joint angle 40 (robot) joint 42 navigation markers 44 Control signal 46 Changing a perspective QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] CN 1 10 613 511 A

[0003]

Claims

[1] Medical, preferably surgical, robotic system (2), with a robot (4) comprising at least one movable robot arm (6), preferably for positioning surgical instruments (18) and / or a visualization system (19), an optical recording device (8) which is attached to the robot arm (6), preferably an end effector (14) of the robot arm (6) and is configured to produce a preferably wide-angle optical recording (16), and a display (10) which is set up to display the optical image (16), characterized by a trigger control (12) which is set up to control the display of the optical image (16) on the screen (10) depending on a movement of the robot arm (6). [2] Medical robot system (2) according to claim 1, characterized by, that the trigger control (12) is set up to begin displaying the optical image (16) on the display (10) depending on the start of the movement of the robot arm (6), preferably with the start of the movement of the robot arm (6). [3] Medical robot system (2) according to claim 1 or 2, characterized by , that the trigger control (12) is set up to terminate the display of the optical image (16) on the display (10) depending on a stopping of the movement of the robot arm (6), preferably with termination of the movement of the robot arm (6). [4] Medical robot system (2) according to any one of claims 1 to 3, characterized by , that the optical recording device (8) is a 2D camera (8') or a 3D camera (8"). [5] Medical robot system (2) according to any one of claims 1 to 4, characterized by, that the optical recording device (8) is configured to capture 3D surfaces (34), preferably with stereo reconstruction, point clouds and / or lidar. [6] Medical robot system (2) according to claim 5, characterized by , that the robot system (2) is configured to detect an impending collision (32) with the robot system (2) based on the detected 3D surfaces (34) and / or to issue a warning (36), preferably on the display (10), and / or to stop the movement of the robot arm (6). [7] Medical robot system (2) according to any one of claims 1 to 6, characterized by , that the robot system (2) has a further optical recording device (20), preferably a microscope (20'), which is configured to produce a further optical, preferably microscopic, image (22). [8] Medical robot system (2) according to claim 7, characterized by, that the display (10) is set up to show the further optical image (22) together with and without the optical image (16). [9] Medical robot system (2) according to claim 8, characterized by , that the display (10) is configured to image the optical image (16) and the further optical image (22) together, making the optical image (16) smaller than the further optical image (22), preferably the optical image (16) in the further optical image (22). [10] Medical robot system (2) according to any one of claims 1 to 9, characterized by , that the movement of the robot arm (6) is detected by means of a change in a joint angle (38) of a joint (40) of the robot arm (6), with navigation markers (42) on the robot arm (6), by means of a control signal (44) to the robot (4) and / or robot arm (6), and / or by means of a change in a perspective (46) of the optical recording (16).

Citation Information

Patent Citations

  • Multi-panel graphical user interface for a robotic surgical system

    WO2019117926A1

  • AU002019202697B2

  • AU2019202697B2