Medical robot-assisted visualisation system and control method for a visualisation system

Abstract

The invention relates to a medical robot-assisted visualisation system (1) for visualisation, in particular during a surgical procedure on a patient (P), comprising a robot (2) with a robot base (4), preferably a moveable medical cart as a mobile robot base (4), and a movable robot arm (6) connected to the robot base (4), a visualisation unit (8) as end effector of the robot (2), which is mounted or attached to the movable robot arm (6) and can be spatially positioned, in particular arranged in a position and orientation, by a movement of the robot arm (6), and a control unit (10), which is designed to control at least one position and / or orientation of the visualisation unit (8). The robot (2) is a seven-axis robot and / or the robot arm (6) has seven axes and can be rotated, in particular, about seven axes of rotation, via which the robot arm (6) can be moved into different robot arm positions while the position and / or orientation of the visualisation unit (8) remains constant. The invention also relates to a control method for a medical robot-assisted visualisation system (1), a computer-readable storage medium and a computer program according to the associated claims.

Description

[0001] Medical robot-assisted visualization system and control method for a visualization system

[0002] Description

[0003] Technical area

[0004] The present disclosure relates to a medical robot-assisted visualization system for visualization, in particular during a surgical procedure on a patient, comprising a robot having a robot base, preferably a displaceable medical cart as a mobile robot base, and a movable robot arm connected to the robot base.The visualization system further comprises a visualization unit (for optical visualization, whether digital or optical analog) as the (controllable) end effector of the robot, which is (movably) mounted or fixed / attached to the movable robot arm and can be spatially positioned (for example, relative to the patient) by a movement of the robot arm (via a configuration of the robot arm segments of the robot arm), in particular arranged in a position and orientation (i.e., location). Furthermore, the system comprises a control unit adapted to control at least one position and / or orientation of the visualization unit. Furthermore, the present disclosure relates to a control method for a medical robot-assisted visualization system, a computer-readable storage medium, and a computer program according to the preambles of the independent claims.

[0005] Technical background of the revelation

[0006] To assist surgeons during operations, visualization systems such as a boom-mounted optical surgical microscope or, even more advantageously, robot-assisted visualization systems such as robot-assisted surgical microscopes are used. These improve the surgeon's view of the area of ​​the patient's body being treated, particularly during minimally invasive procedures. A visualization unit is moved as an end effector by a robot arm. The robot arm can, for example, be movably attached to a robot base, which is a movable medical cart to which the movable robot arm is attached. The robot arm allows the visualization unit to be spatially positioned or arranged. The robot arm can be controlled by the user using buttons, a joystick, or by manually moving the end effector to a desired position.

[0007] While there are various medical robot-assisted visualization systems on the market, these well-known visualization systems all have a robot arm with six (6) joints, meaning that well-known medical robot-assisted visualization systems have a 6-axis robot arm. Such 6-axis robots are familiar from a wide variety of applications and are characterized by their simplicity and robustness. These conventional 6-axis robots are kinematically determined, meaning that the position of the end effector and the robot arm position are coupled. Thus, the robot arm cannot be moved without changing the position of the end effector.

[0008] This has a number of disadvantages. If the robot arm is positioned incorrectly, the robot arm, or individual robot arm segments or limbs of the robot arm, can block the surgeon's view of the operating screen. Furthermore, the robot arm can collide with people or objects in the robot's vicinity when the end effector moves into a different position or orientation. In the worst case, no movement may occur at all, as the objects in the operating room may be arranged in such a way that the robot arm cannot move out of view without colliding. An unfavorable combination of the robot arm position and the position of the end effector can also force the surgeon or user into an unergonomic posture, which is particularly stressful for the user during long operations.

[0009] A predetermined or maintained end-effector position therefore leads to a very limited number of robot arm configurations. Typically, only one robot arm position can be used during the procedure. The end-effector is constantly moved during the operation to achieve different viewing directions. With each movement, the robot arm necessarily assumes a new position or configuration. With the 6-axis robot, the robot arm position is determined for each end-effector position, meaning only one configuration of the robot arm segments is possible.

[0010] If the visualization unit is an (analog) microscope with an eyepiece system, the surgeon looks directly into the microscope's optics. Thus, the surgeon's field of view is not blocked by the microscope. However, a new generation of surgical microscopes utilizes digital optics to provide visualization and even better images for all professionals in the operating room. They often use a 3D screen as an output device to display the image captured by the microscope to the surgeon.

[0011] With such known solutions, the surgeon can view the image captured by the visualization unit on a screen. However, with such visualization systems, the limited kinematic configuration options mean that the robot arm blocks the surgeon's view when the cart is positioned in front of the surgeon. This means that the robot arm is located between the surgeon and the 3D screen. In this case, the robot arm can block the surgeon's view, and the disadvantageous robot arm position cannot be changed for a specific end-effector position, even for kinematic reasons, because the robot arm position and end-effector position are necessarily coupled. This leads to a significant impairment of ergonomics for the surgeon.Alternatively, if the cart is positioned behind the surgeon and the robot arm extends over the surgeon's shoulder, the limitations of a particular robot arm position for a particular end-effector position can lead to collisions between the robot arm and the surgeon and / or force the surgeon to assume a non-ergonomic position. While conventional optical microscopes projecting downward from a ceiling generally do not block the surgeon's field of view, the limited number of robot arm positions can still cause ergonomic problems, or parts of the system can collide with other systems in the operating room. Summary of the Disclosure.

[0012] It is therefore the object of the present disclosure to overcome or at least mitigate the disadvantages of the prior art and, in particular, to provide a visualization system, a control method, a computer-readable storage medium, and a computer program that enables flexible adaptation of a visualization unit with an optimal configuration of the robot arm. One sub-objective can be seen in the fact that a robot arm does not obscure a surgeon's line of sight to an output unit. Another sub-objective is to provide a stable configuration of the robot arm so that the visualization unit is arranged very stably and performs the smallest possible elastic movements. A further object is to provide the user with a visualization system that is easily movable and controllable according to their requirements, for example during input, and that preferably avoids collisions.

[0013] These objects of the present disclosure are achieved according to the invention with regard to a medical robot-assisted visualization system by the features of claim 1, with regard to a medical control method by the features of claim 14, with regard to a computer-readable storage medium by the features of claim 16 and with regard to a computer program by the features of claim 17.

[0014] The present disclosure relates to a medical robot-assisted visualization system for visualization, in particular during a surgical procedure. The medical robot-assisted visualization system comprises a robot with a robot base, preferably a medical movable cart as a mobile robot base, and a movable robot arm connected to the robot base. The visualization system further comprises a visualization unit as a (controllable and movable) end effector of the robot, which is mounted or attached to the movable robot arm and can be spatially positioned (for example, via three coordinates x, y, z) by a movement of the robot arm, in particular with regard to a position and orientation (i.e., a position), and a control unit adapted to control at least the position and / or orientation of the visualization unit.According to the disclosure, the robot is a seven-axis robot (7-axis robot) and / or the robot arm has (at least) seven joints or (different joint) axes, through which the robot arm can be moved / configured into different robot arm positions while maintaining a constant position and / or orientation of the visualization unit. In particular, the robot arm has seven axes of rotation, in particular exactly seven axes of rotation, around which individual robot arm segments can be rotated.

[0015] In other words, the robot arm has seven joints, each with one degree of freedom, so that the robot arm has a total of seven degrees of freedom. The robot arm is therefore a 7-axis robot with a medical visualization unit as the end effector. The control unit moves the robot arm into a variety of different robot arm positions, while the position and / or orientation of the visualization unit remains unchanged, which is possible thanks to the seven joints or axes. In summary, the core of the present disclosure lies in decoupling the position and / or orientation of the visualization unit from the robot arm position via the at least seven axes of the robot, as far as possible. Thus, a seven-axis robot is proposed as a visualization system, in particular, which has a visualization unit as the end effector.

[0016] This allows the robot arm to be moved or the robot arm position to be varied without the end effector or the visualization unit being forcibly moved during the movement of the robot arm or the position and / or orientation of the visualization unit being varied. The robot arm can therefore execute or perform a so-called zero-space movement. This allows arm links or robot arm segments of the robot arm to be moved from unfavorable positions to other, more favorable positions without the visualization unit having to be moved. This can, for example, prevent robot arm segments from being arranged in the user's line of sight. Furthermore, collisions between the robot arm and the user can be avoided. The user is no longer forced into an ergonomic position or posture by the position of the visualization unit and / or the robot arm position.The visualisation system according to the disclosure thus combines in particular the advantages of a visualisation unit into which the user looks directly and thus has an unobstructed view of the object to be observed, with the advantages of solutions with a screen in which the user is not forced into an uncomfortable position.

[0017] Specifically, the visualization system can be designed as a robotic microscope with a mobile carriage, a robot arm with seven or more joints, and a visualization unit, for example a microscope, connected, in particular fixed, as an end effector. An output unit, such as a screen for displaying an image captured by the visualization unit, can be mounted on the same carriage or on a separate stand. The control unit can be adapted, in particular, to determine the positioning of the surgeon / user and the positioning of the screen relative to the end effector using a microscope-specific algorithm or a microscope-specific model. The model or algorithm can be used by the control unit to optimize a robot arm configuration for a (pre)determined or predefined end effector position.For example, blocking the surgeon's field of view can be avoided by rotating an "elbow," i.e., an angled joint of the robot arm downward (i.e., toward the floor), out of the surgeon's line of sight. In particular, a "elbow" of the robot arm can be described as a central joint of the robot arm that allows two adjacent tubular robot arm segments to be angled toward each other and therefore protrudes significantly from the base extension of the robot arm in certain robot arm positions.

[0018] The object of the present disclosure is further achieved by a control method for the robot-assisted visualization system, in particular according to the present disclosure. The control method comprises the following steps. The visualization unit is moved or displaced by the robot arm to a predetermined target position and / or target orientation. A sensor, in particular an optical sensor, in particular a camera, detects a spatial position of a user's head, in particular an eye position of the user. Furthermore, a position of an output unit, in particular the screen, is determined by the control unit. The control unit calculates a line of sight / visual axis or a straight line between the determined position of the output unit and the detected head position of the user. Preferably, a tolerance range can be drawn around the calculated line of sight, in particular in the form of a tunnel or tube with a predefined diameter.The calculated line of sight corresponds to the user's view of the output unit. The control unit also detects the robot arm position. This means that the control unit detects the position and / or orientation of the individual arm segments.

[0019] Robot arm segments of the robot arm. The control unit detects or determines a possible blockage of the calculated line of sight, for example by a

[0020] Robot arm segment of the robot arm. In particular, the control unit can detect if the robot arm enters the tolerance range. If the control unit detects a blockage of the line of sight, the control unit moves the robot arm, keeping the target position and / or target orientation of the visualization unit constant.

[0021] In particular, the sensor or sensor system may comprise an optical sensor system (such as a camera) and / or a radar and / or an ultrasound and / or a LIDAR to detect a spatial position and / or orientation.

[0022] Sensor technology can be implemented as or in an optical camera (e.g., with a CMOS sensor). However, the sensor or sensor technology can also be implemented as a radar, a LIDAR (Light Detection and Ranging), and / or an ultrasonic sensor, which are particularly suitable for detecting spatial structures and distances.

[0023] In principle, various sensors can be used to spatially detect the position of the user's head.

[0024] So if the control unit determines that the user does not have a clear view of the output unit because the robot arm is in the line of sight, the control unit moves the robot arm to a different robot arm position in such a way that only the robot arm is moved, while the visualization unit as the end effector remains in its position and / or orientation.

[0025] The robot arm can be moved, in particular, by an input signal via a joystick, a button, and / or manually (force-sensitive) by the user, who pushes or pulls the robot arm into the desired position. In particular, when moving a microscope as an end effector via the robot arm (using a joystick and / or force or button control), the robot arm can also optimize its position to obstruct the surgeon's view as little as possible.

[0026] Using the proposed control method, the control unit can detect whether the user's line of sight is blocked by the robot arm. If the control unit determines that a blockage exists, it can then move the robot arm until the blockage is removed. This results in an unobstructed view of the output unit for the user without requiring the user to stand in an uncomfortable or non-ergonomic position.

[0027] Preferably, the control unit can detect the robot arm position already during the movement of the visualization unit and thus determine already during the movement of the visualization unit whether the robot arm blocks the calculated line of sight of the user and accordingly configure the robot arm segments during movement in such a targeted manner that in the target position and / or target orientation of the visualization unit the robot arm does not impair the view.

[0028] Advantageous further developments of the present disclosure are the subject of the dependent claims and are explained in particular below.

[0029] According to an optional feature of the present disclosure, the individual, respectively adjacent or serially following axes of rotation of the robot arm are each arranged perpendicularly or orthogonally to one another. In this context, adjacent axes of rotation can be axes of rotation that are arranged one after the other in a force flow or torque flow that starts or ends at the robot base. Due to the respectively perpendicular or orthogonal axes of rotation, the robot arm can have seven degrees of freedom. As a result, the position and / or orientation of the visualization unit and the robot arm position can be decoupled. The robot arm can thus be moved or the robot arm position can be varied without moving the end effector of the robot arm. The robot can have a number of axes of rotation and a number of

[0030] Robot arm segments or arm links. In this context, individual arm links should be separated from one another by a joint (with one degree of freedom) or an axis of rotation. The robot arm can have a first axis of rotation to the robot base. The first axis of rotation can be arranged such that a rotation of a first robot arm segment or arm link and thus of the entire robot arm relative to the robot base (if no other joints are changed) is possible. Preferably, the first axis of rotation is arranged vertically or in the direction of gravity. The terms vertical, horizontal, top and bottom are to be seen in relation to an operating room. In this case, “top” is a ceiling and “bottom” is a floor, which connects the “vertical”. Likewise, the floor of the operating room extends “horizontally”.A second axis of rotation of the robot arm can be arranged such that a pivoting, tilting, or pitching of a second robot arm segment relative to the first robot arm segment is possible. In particular, the second axis of rotation can be perpendicular to the first axis of rotation. A third axis of rotation can be arranged such that a rotation of a third robot arm segment relative to the coaxially arranged second robot arm segment is possible. In this case, coaxial means that the longitudinal axes of the two robot arm segments are arranged coaxially in the region of the connection. In this case, a longitudinal axis of the second robot arm segment can be coaxial with a longitudinal axis of the third robot arm segment or merge into it. This means that the third axis of rotation can, in particular, be parallel or coaxial with the longitudinal axis of the second and third robot arm segments.As a result, a rotation of the third robot arm segment about the common longitudinal axis of the second and third robot arm segments can be carried out. The degree of freedom of the third robot arm segment may not be present, in particular in conventional robots. Preferably, the robot arm can have a fourth axis of rotation, which is arranged such that a pivoting of the third robot arm segment relative to a fourth robot arm segment can be carried out. In particular, the fourth axis of rotation is perpendicular to the third axis of rotation. The configuration of the robot arm with at least the three, in particular four, axes of rotation, which are orthogonal to one another, means that the joint with the fourth axis of rotation can be moved independently of a position and / or orientation of the end effector of the robot arm. Thus, the joint with the fourth axis of rotation can be moved or shifted without the end effector being moved.In particular, the degree of freedom of the third rotation axis, through which the third robot segment can be rotated relative to the second robot segment, can enable the decoupling (of the position and / or orientation) of the end effector from the robot arm position.

[0031] In particular, vertical axes of rotation can be arranged parallel to the direction of gravity, while horizontal axes of rotation can be aligned perpendicular to the direction of gravity.

[0032] According to a further optional feature of the present disclosure, the visualization system comprises a sensor, in particular an optical one, in particular a camera. The control unit can be adapted, in particular, to detect a spatial position of a user's head, in particular an eye position of a user, via detection by the sensor, in particular an optical one. The control unit can further be adapted to determine a position of an output unit, in particular a screen. The control unit can further be adapted to calculate a line of sight or visual axis, which is in particular a straight line, between the detected position of the user and the determined position of the output unit. In this case, a tolerance range around the calculated line of sight can be included, which, for example, is a type of hose or cylinder with a predefined tolerance diameter around the calculated straight line.The tolerance range around the calculated line of sight can be predefined or variably adjusted, for example, through user input. In particular, in addition to a cylindrical tolerance "tunnel," the user can also select and use a different cross-sectional geometry, such as a triangle or a quadrilateral (for a block).

[0033] The spatial position of the user's head can therefore be determined in particular via the preferably optical sensor or sensor technology. However, the spatial position of the head can alternatively or additionally (as two redundant detection methods) also be determined via a stored data set or a heuristic. The data set can in particular be a historical data set from a past / previous procedure in which data on the previous procedure (such as a specific knee endoprosthesis procedure) is stored, in particular spatial positions of a surgeon's head relative to the output device. Based on the stored data from the previous procedure, the heuristic can use a calculated / estimated position of the head to determine whether the user's line of sight is obscured and then keep it clear accordingly by (as a precautionary measure) moving the robot arm out of the surgeon's field of vision.The dataset may further include a model of the entire operating room, in which the positions of the user and / or objects in the operating room are stored or taken into account. This model can be provided to the control unit as a spatial basis for boundary conditions, and the control unit can be adapted to move the robot arm based on the model (e.g., for collision avoidance).

[0034] The control unit can in particular be further adapted to detect a robot arm position or a robot arm configuration and to determine whether a section of the robot arm, in particular a robot arm segment, lies in the line of sight or the tolerance range. The control unit then determines whether a direct line of sight between the head, in particular the eyes of the user, and the position of the output unit is blocked by a robot arm segment of the robot arm. Using the sensor, in particular an optical sensor, the visualization system can detect a position of the user or surgeon, in particular the position of the user's eyes. From the detected head position of the user, the position of the output unit, and the arm position, the control unit can calculate whether the user's view is blocked. The control unit can thereby detect or determine whether the user's line of sight to the output unit is clear or obstructed.This can be the basis for subsequent movement control by the control unit.

[0035] Preferably, the control unit can be adapted to move the robot arm out of the line of sight upon determining or detecting the blockage of the line of sight in such a way that the blockage of the robot arm segment is resolved, wherein the position and / or orientation of the visualization unit as the end effector is maintained, in particular a section of the robot arm with the joint of the fourth rotation axis is moved out of the line of sight. The end effector can be moved by the robot arm until the control unit determines that the blockage of the line of sight has been resolved, i.e., until the line of sight between the output unit and the user is clear.

[0036] The control unit can preferably be adapted to control the robot arm such that the individual robot arm segments, in particular a robot arm elbow, do not exceed a predefined height (in space). The predefined height can be a coordinate (e.g., a Z_max coordinate) in a coordinate system stored in the control unit.

[0037] The control unit can be particularly adapted to control the robot arm based on a predefined plane, which extends horizontally parallel to the ground, in particular at a maximum height, such that none of the robot arm segments penetrates this plane. The plane can be arranged in such a way that the probability of blocking the line of sight above the plane is reduced. In particular, the predefined plane can be spanned by two stored vectors and define a kind of spatial ceiling for the robot arm.

[0038] The blockage of the line of sight is solved in particular by moving the section of the robot arm with the joint of the fourth rotation axis out of the line of sight. This is made possible in particular by the rotation of the joint with the third rotation axis. This degree of freedom gives the robot arm greater "flexibility" than conventional robots. This means that the robot arm can have different positions without the end effector changing its position and / or orientation. In particular, the joint of the second rotation axis can be rotated by 90° to 120°. This would move the entire robot arm downwards in the direction of gravity. However, if the joint of the third rotation axis is simultaneously rotated by approximately 180°, the end effector can remain in its position. The robot arm can therefore perform its zero-space movement.

[0039] According to a further embodiment of the present disclosure, the dispensing unit can be movable relative to the robot base; in particular, the dispensing unit can be displaceable on a second medical cart. The control unit can be adapted to detect the position of the movable dispensing unit using a tracking system. In particular, a rigid body with optical markers can be arranged on the dispensing unit, which is detected via an external camera of the tracking system. The optical sensor can detect the optical markers, and the control unit can thereby determine the (variable) position of the dispensing unit.

[0040] In particular, the tracking system can be arranged on the second medical cart, for example rigidly attached, and adapted (aligned) to detect the robot, in particular designed as an (external) camera into whose field of view the robot falls. The tracking system can thus detect and spatially localize the robot and thus determine a position of the dispensing unit with respect to / relative to the robot(s). Alternatively, a tracking system in the form of a camera system with at least one camera each can preferably be arranged on both the movable dispensing unit and the robot, which in particular detect each other and thereby detect a relative position to each other. The tracking system can in particular have a sensor designed as a camera, radar, lidar and / or an ultrasound sensor.

[0041] Alternatively or additionally, the dispensing unit can be movably mounted on the first medical cart and can be moved by an electric motor. The control unit can determine the position of the dispensing unit relative to the cart by controlling the electric motor or applying a control current to the electric motor.

[0042] The output unit can, in particular, be a screen or a (touch) display. The position of the output unit can be calculated by the control unit. For example, the output unit can be adjusted by a stepper motor. By controlling the stepper motor, the control unit can detect the position of the screen.

[0043] The dispensing unit can preferably be mounted alternatively or additionally at a fixed location, for example, on a wall in the operating room. The robot arm with the visualization unit could be movable relative to the dispensing unit via the carriage. The control unit could determine the position of the dispensing unit relative to the carriage using the optical sensor.

[0044] The control unit preferably detects objects and / or people in the environment of the visualization system via the optical sensor. Furthermore, the control unit can be adapted to detect a collision of the robot arm with the detected objects and / or people when the robot arm is moving and to prevent this by (slowing down or) stopping the movement of the robot arm. Using the optical sensor, the control unit can create a digital (virtual) model of the entire environment of the visualization system, in particular of an entire operating room. This model can contain all objects and / or people. If there is a risk of collisions between the robot arm and detected objects and / or people, the control unit can stop movements of the robot arm and / or issue alarm signals.

[0045] The control unit can preferably also predict expected / predictable movements of the detected persons and / or the robot arm based on historical data and adapt the movement profile or the robot arm position of the robot arm accordingly.

[0046] According to a further optional feature of the present disclosure, the control unit is adapted to adjust / configure the robot arm such that the robot arm segments / arm links lie substantially in a vertical plane, and thus are aligned parallel to the direction of gravity, in order to increase the rigidity of the visualization unit. A vertical plane can be understood as a plane that extends or is aligned parallel to the direction of gravity. This can increase the rigidity of the robot arm, especially when a load in the direction of gravity increases rapidly.

[0047] Preferably, the control unit is adapted to adjust the robot arm such that the robot arm segments are arranged as far away as possible (as far away as possible) from the user, so that the visualization unit is easily accessible by the user and, in particular, no robot arm segment is arranged between the user's position and the visualization unit. This can increase the accessibility of the visualization unit. In particular, the joint with the fourth axis of rotation can be variably aligned. In particular, an orientation of this fourth joint towards the user is disadvantageous, while an orientation of the fourth joint away from the user can be advantageous. The control unit can therefore increase the accessibility of the visualization unit by controlling the configuration of the individual robot arm segments.

[0048] A robot arm position in which the visualization unit is easily accessible to the user can be understood in particular as a robot arm position with respect to the user in which no part of the robot arm or no robot arm segment is arranged between the user and the visualization unit.

[0049] According to a further optional feature of the present disclosure, the control unit is adapted to adjust the robot arm or the configuration of the robot arm segments such that singularities in an end position of the robot arm are avoided.

[0050] In this context, the singularity of the robot arm should be understood as positions of the robot arm, especially when the robot arm is fully extended, in which two joints or the rotation axes of two joints are redundant to each other and thus one degree of freedom of the robot arm is lost.

[0051] Thanks to the additional degree of freedom of the seven-axis robot, the control unit can adjust the robot arm position so that no two joints overlap.

[0052] Preferably, the control unit is adapted to weight individual parameters of collision avoidance, singularity avoidance, increasing reachability, and increasing rigidity, and to adapt a configuration of the robot arm according to the weighting in order to achieve an optimum robot arm position. Collision avoidance, in particular, can have the greatest weighting. The control unit can therefore, to a certain extent, weigh up which of the individual (target) parameters is to be regarded as more important in the event of a conflict of objectives. The individual (optimization) parameters of the robot arm position can be, as described above, collision avoidance, singularity avoidance, increasing reachability of the visualization unit, and increasing the rigidity of the robot arm. The weighting of the individual parameters can be predefined.Furthermore, the control unit can learn the weighting of the individual parameters using an artificial intelligence (AI) system, in particular reinforcement learning. The control unit can also preferably be adapted to always favor the robot position that serves to avoid collisions in the event of a conflict of objectives, for example, between increasing rigidity and avoiding collisions. This can result, for example, from the expected consequences, which are more negative in the case of a collision between the robot arm and a person than in the case of a (minimal) movement of the end effector in the direction of gravity.In particular, the parameters to be optimized can be: increasing the stiffness of the robot arm by suitable robot arm position (when applying pressure to the microscope or trocar) and / or avoiding singularities and / or increasing the accessibility of the visualization unit for the user and / or offering the possibility to move the 'elbow' of the robot (in particular the joint of the fourth rotation axis) into different positions.

[0053] The weighting factors can be applied particularly during operation. The control unit can calculate alternative possible arm positions for a given position and / or orientation of the end effector and steer the robot arm to an optimized position with respect to the parameters.

[0054] Preferably, an algorithm of the control unit can also be used to train the control unit to maintain a specific joint position combination or to optimize it within a predetermined or trained range of one or more joints.

[0055] In particular, the visualization system can be implemented by the adapted control unit in addition to the parameter for avoiding visual obstructions, as well as additional restrictions / limitations or optimization parameters, in particular one parameter or a combination of the following parameters: keeping a space (volume of space) free for the assisting surgeon, maximizing the accessibility of the robot, maximizing the manipulability of the robot arm, considering user preferences for joint positions (especially the elbow of the robot arm), avoiding singularities, optimizing end-effector stiffness via robot arm stiffness, or other motion parameters. If multiple optimization parameters / criteria are used, these can be combined by the control unit, in particular via weighting or weighting factors.The weighting factors can, in particular, be applied on-the-fly. An algorithm of the control unit can, in particular, be used to calculate alternative arm positions for a given end-effector position and move the robot arm to a position that optimizes the defined constraints. The algorithm can also be used to train the robot arm to maintain a specific joint position combination or to optimize it within a predefined or trained range of one or more joints. For example, during surgery, the surgeon manually moves the robot elbow to a specific position, and the robot arm maintains this elbow position when the end-effector is moved again.It is possible to either optimize only a null space (fixed end effector position) or to optimize the joint configuration while the end effector is moving (sending a target position and / or target orientation for the end effector while simultaneously sending a preferred angle for one or more joints). With multiple joints, the joints can be weighted against each other, particularly using relevance factors (similar to weightings).

[0056] Preferably, the user can manually move the joint with the fourth rotation axis into an advantageous position and the control unit can store the robot arm position with the advantageous position of the joint and control it automatically.

[0057] The control unit can maintain a specified position and / or orientation of the end effector at any robot arm position as well as maintain specified angles for individual joints of the robot arm.

[0058] The visualization system preferably has an input unit, in particular a button or a touch display, by actuating which the user can manually request a movement of the robot arm without the position and / or orientation of the visualization unit being varied. In particular, the joint with the fourth axis of rotation can be rotatable by 90° about a horizontal axis or a longitudinal axis of the robot arm. A longitudinal axis of the robot arm can be a general longitudinal extension of the robot arm. In particular, the control unit can be adapted to rotate the elbow of the robot arm (in particular with the fourth joint or the fourth axis) by 45° about a substantially horizontal axis upon input from a user (interaction option) via an input unit such as a button, or to rotate the elbow until no further input is present.

[0059] The input unit can be located, for example, on the robot base and can also be a (touch) display or a joystick that allows the user to control the robot arm. The joystick can preferably be located either on the movable carriage or on the end effector.

[0060] Preferably, the control unit can be adapted to adjust the robot arm position even when the end effector is moved manually by the user in such a way that, for example, collisions with persons and / or objects are avoided.

[0061] The user can grasp the end effector and move or pull it to a desired position. The robot arm offers only minimal resistance, allowing the user to move or pull the end effector effortlessly to the desired position. During manual movement of the end effector, the control unit can adjust the robot arm to set or optimize parameters such as increased stiffness, singularity avoidance, collision avoidance, and increased reachability.

[0062] According to a further optional feature of the present disclosure, the visualization unit is a digital microscope, in particular a stereomicroscope / stereoscopic microscope, an (analog) optical microscope, in particular a stereomicroscope, and / or an endoscope. The images recorded by the visualization unit can be displayed, in particular, to the user via the output unit. Areas in the patient's body orifices can also be displayed through the endoscope. The end effector can further comprise an instrument guide, in particular a trocar, attached to the visualization unit. In this case, the control unit can optimize or adjust the position and / or orientation of the visualization unit such that the instrument guide is suitably aligned.In particular, stiffness optimization can be prioritized in this case so that the end effector does not perform any unwanted movement when subjected to external force, for example pressure from above.

[0063] The robot arm can also have a secondary end effector attached to the visualization unit. The secondary end effector can be, for example, an endoscope, a camera, or a surgical tool for contact with the patient. If a conflict of objectives arises between the configurations of the individual end effectors, the individual end effectors can be weighted. The weighting can be varied during robot operation or can be predefined.

[0064] The control unit can be arranged, in particular, in the robot base. The carriage, as a movable robot base, can have a CPU, a storage unit such as a hard disk or RAM, and a power supply, in particular a power supply.

[0065] The objects of the present disclosure are achieved with regard to the computer-readable storage medium and with regard to the computer program in that they comprise instructions which, when executed by a computer, cause the computer to carry out the control method according to the present disclosure.

[0066] Short description of the characters

[0067] The disclosure is explained in more detail below using preferred embodiments with the aid of figures. They show:

[0068] Fig. 1 is a schematic view of a visualization system according to the prior art; Fig. 2 is a schematic view of a visualization system according to an embodiment of the present disclosure;

[0069] Fig. 3 is a schematic view of the visualization system of Fig. 1 with a schematically shown blocked line of sight;

[0070] Fig. 4 is a schematic view of the visualization system of Fig. 1 with a schematically shown free line of sight; and

[0071] Fig. 5 is a flowchart of a control method according to a preferred embodiment of the present disclosure.

[0072] The figures are schematic in nature and are intended only to aid understanding of the disclosure. Like elements are designated by like reference numerals. The features of the various embodiments may be interchanged.

[0073] Detailed description of the characters

[0074] Fig. 1 shows a visualization system 100 according to the prior art. The visualization system 100 comprises a robot 102 with a movable robot arm 106, to which a visualization unit 108 is attached as an end effector. In this case, the visualization unit 108 is an analog microscope into which a user looks directly. Due to the configuration of the robot arm, a position and / or orientation of the visualization unit 108 is necessarily linked to a robot arm position. In Fig. 1, the robot arm position is unfavorable because a user is obstructed by the robot arm 106. This problem is solved by a visualization system according to the present disclosure, as explained below.

[0075] Fig. 2 shows a medical robot-assisted visualization system 1 according to a preferred embodiment of the present disclosure. The visualization system 1 has a robot 2 with a robot base 4. The robot base 4 is in this case a movable medical cart that is used in an operating room. A robot arm 6 is movably connected to the robot base 4. The visualization system 1 has a visualization unit 8 as the end effector of the robot 2, which is attached to the movable robot arm 6. The visualization unit 8 can be spatially positioned by moving the robot arm 6. The visualization system 1 further comprises a specially configured control unit 10 that is adapted to control the position and / or orientation of the visualization unit 8.The robot arm 6 has seven (different and spaced-apart) axes A1 to A7, by which the robot arm 6 can be moved into different robot arm positions / robot arm configurations, particularly while maintaining a constant position and / or orientation of the visualization unit 8. In this embodiment, the robot 2 is a seven-axis robot with a surgical microscope (as the visualization unit 8) as the end effector. This special configuration enables the robot 2 to flexibly adjust the arrangement of the robot arm 6 without forcing any movement of the end effector.

[0076] The visualization system 1 further comprises an (OP) screen 12 as an output unit, which is attached to the robot base 4. The visualization system 1 has a camera 14 as an optical sensor, which detects a spatial position of a user and, if necessary, can detect a position of another screen (not shown).

[0077] The robot arm 6 in this case has seven axes of rotation A1 to A7 and six robot arm segments S1 to S6. The individual robot arm segments 18 or S1 to S6 are each separated from one another by a joint or one of the axes of rotation A1 to A7. A first axis of rotation A1 is arranged such that a rotation of a first robot arm segment S1, and thus of the robot arm 6, can be carried out relative to the robot base 4. The first axis of rotation A1 is vertically positioned on the robot base 4. A second axis of rotation A2 is perpendicular to the first axis of rotation A1 and arranged horizontally (this can be rotated about the first axis of rotation A1 while maintaining the horizontal position). The second axis of rotation A2 is arranged such that a pivoting / tilting of a second robot arm segment S2 relative to the first robot arm segment can be carried out. The second axis of rotation A2 is further arranged perpendicular to a longitudinal axis of the second robot arm segment S2.A third rotation axis A3 is arranged such that a third robot arm segment S3 can be rotated relative to the coaxially arranged second robot arm segment S2. This means that the longitudinal axes of the second robot arm segment S2 and the third robot arm segment S3 coincide (are coaxial) and extend along the third rotation axis A3.

[0078] A fourth axis of rotation A4, in turn, is arranged such that pivoting of the third robot arm segment S3 relative to a fourth robot arm segment S4 is possible. The fourth axis of rotation A4 is in turn perpendicular to the third axis of rotation A3 and to a fifth axis of rotation A5. Thus, the fourth axis of rotation A4 is aligned parallel to the second axis of rotation A2. The fifth axis of rotation A5 extends along the longitudinal axis of the fourth robot arm segment S4. Thus, the fifth axis of rotation A5 enables a rotary movement of a fifth robot arm segment S5 about the longitudinal axis of the fourth robot arm segment S4. When the robot arm 6 is fully extended, the fifth axis of rotation A5 can lie on the third axis of rotation A3. A sixth axis of rotation A6 is arranged perpendicular to the fifth axis of rotation A5 and in particular parallel to the fourth axis of rotation A4. The sixth axis of rotation A6 thus enables pivoting of a sixth robot arm segment S6 relative to the fifth robot arm segment S5.The visualization unit is rotatably mounted on the sixth robot arm segment S6 as an end effector. The visualization unit 8 is rotatable about a seventh rotational axis A7 relative to the sixth robot arm segment S6. The seventh rotational axis A7 is perpendicular to the sixth rotational axis A6. All rotational axes A1 to A7 are perpendicular or orthogonal to the adjacent rotational axis. For example, the third rotational axis A3 is perpendicular to both the adjacent second rotational axis A2 and the adjacent fourth rotational axis A4.

[0079] The user can enter the desired position and / or orientation of the visualization unit 8 via an input unit 20. The input unit 20 can be a (touch) display, a button on the robot base 4, and / or a joystick on the visualization unit 8.

[0080] Fig. 3 shows a robot arm position of the robot arm 6 or a configuration of the robot arm segments 18. In the position shown, a joint of the robot arm, which has the fourth rotation axis A4, is arranged such that a user's line of sight 16 to the screen 12 is blocked by a robot arm segment 18 or by the joint with the fourth rotation axis A4. This has the disadvantage that the user either cannot see the entire screen 12 or has to change their posture to see the entire screen 12. This could force the user into a non-ergonomic posture, and the advantages of the visualization system with screen 12 are reduced.

[0081] The control unit 10 uses the camera 14 to detect the position of the user's head and, in this case, even the user's eye position. Furthermore, the control unit 10 determines the position of the screen 12. The control unit 10 calculates the line of sight 16 as a straight line between the detected eye position and the determined screen position. Since the control unit 10 can also detect the robot arm position, the control unit 10 can thus determine whether a part or section of the robot arm is in the line of sight 16 and blocking it. The control unit 10 thus determines whether the user has a clear view of the screen 12.

[0082] The individual robot arm segments 18 are moved relative to each other, for example, by (electric) servomotors (not shown). Using the control signal to the servomotors, the control unit 10 can calculate the robot arm position or, using the robot's internal kinematics, determine the current configuration of the robot arm segments.

[0083] The screen 12 can, for example, be moved relative to the robot base 4. The screen 12 can be moved up and down relative to the robot base 4, for example, using an electric motor (not shown) at the push of a button. The control unit 10 can calculate the position of the screen 12 from a control signal to the electric motor.

[0084] If the screen 12 or another screen is arranged on a second movable carriage and can be moved relative to the robot 2, the camera 14 can detect the position of the external screen 12. Fig. 4 shows a robot arm position of the robot arm 6 in which the joint with the fourth axis of rotation A4 is moved such that the user's line of sight 16 to the screen 12 is unobstructed. The core idea here is that the robot arm 6 is moved into a different robot arm position when the control unit determines that the line of sight 16 is blocked, as shown in Fig. 3. By designing it as a 7-axis robot, the robot arm 6 can be moved without the position and / or orientation of the end effector being varied. Therefore, the position and / or orientation of the end effector 8 in Fig. 4 is identical to Fig. 3. Only the robot arm position has been varied.

[0085] In particular, the control unit 10 can be adapted to calculate such a joint configuration of the robot arm 6 based on a spatial geometric situation in the operating room, with the focus or primary emphasis on the surgeon's free line of sight. An environmental model provides the input for the optimization algorithm. The environmental model can consider or integrate the patient's anatomy and / or consider predictable movements of the personnel and / or predictable movements of the robot taking the surgical situation into account and / or consider the entire geometry of the operating room to avoid collisions.

[0086] In particular, the optimization can have a single or any combination of two or more of the following constraints or restrictions as parameters:

[0087] - the user’s view is obstructed as little as possible by the robot arm 6,

[0088] - the rotation of the elbow of the robot arm 6 into a specific position,

[0089] - a manually defined restriction during use, which is observed for the following movements of the end effector, in particular a spatial arrangement of a joint in an area,

[0090] - Avoiding collisions with objects or anatomical parts of the patient or staff. The above restrictions are used in particular by the control unit 10 to optimize the robot arm movement and / or robot arm position to optimize one of the following parameters or a combination thereof:

[0091] - an increase in rigidity

[0092] - an improvement in the manipulability of the robot arm 6

[0093] - increasing the accessibility of the robot arm 6

[0094] - avoiding singularities of the robot arm 6

[0095] - Optimization of the movement of robot arm 6 (e.g., optimization so that the robot arm can accelerate quickly from a given position). Multiple parameters can be combined using weighting factors for the boundary conditions. Weighting factors and / or optimization criteria can also be changed during operation. The optimization of the position of robot arm 6 occurs after a new end effector position has been reached or on-the-fly during movement to a new position.

[0096] The visualization system 1 is in particular equipped with a navigation system that tracks the position of the visualization unit 8.

[0097] The visualization system 1 can be equipped with a secondary (or tertiary...) effector (not shown here), such as an instrument guide, such as a trocar, attached to the visualization unit. In this case, the optimization criteria are applied to consider the surgical needs of the instrument guide, with particular emphasis on stiffness optimization. If the secondary end effector is used, the needs of the (two) end effectors can be balanced against each other to find an optimal joint position, or one end effector can be prioritized over another (changing the prioritization can be done during operation).

[0098] Fig. 5 shows a flowchart of a control method according to the disclosure for the medical robot-assisted visualization system 1 of a preferred embodiment. In step 1, the robot arm 6 moves or displaces the visualization unit 8 to a predetermined target position and / or target orientation. The predetermined target position and / or target orientation can be entered or specified by the user via an input unit 20. In step 2, the control unit 10 detects a spatial position of the user's head, in particular the user's eye position, via the camera 14 as an optical sensor. Furthermore, the control unit 10 determines the position of the screen 12 as an output unit. In step 3, the control unit calculates the line of sight 16 or visual axis between the determined position of the screen 12 and the detected head position of the user. Preferably, a tolerance range can be drawn around the calculated line of sight 16.

[0099] The tolerance range can be used to form a cylinder with a predetermined diameter along the straight line of sight 16. The diameter of the (tolerance) cylinder can be, for example, 20 cm or particularly preferably 50 cm. In step 4, the control unit 10 detects the robot arm position. This means that the control unit 10 detects the position and / or orientation of the individual arm links / robot arm segments 18 of the robot arm 6. In step 5, the control unit 10 detects a possible blockage of the calculated line of sight 16, for example, by an individual robot arm segment 18. In particular, the control unit 10 detects an intrusion of the robot arm 6 into the tolerance range. If a blockage of the line of sight 16 was detected by the control unit 10, the control unit 10 moves the robot arm 6 in step 6, wherein the target position and / or target orientation of the visualization unit 8 is kept constant.

[0100] The control unit 10 further includes an environmental model in which the position of the user or surgeon relative to the visualization system 1 and the position of the screen 12 are recorded. Furthermore, the environmental model includes the robot arm position. The environmental model can capture the entire operating room and is recorded by the camera 14. Based on the environmental model, the control unit 10 controls the robot arm 6.

[0101] The environmental model may in particular include one or more of the following elements:

[0102] - an anatomy of the patient;

[0103] - predictable movements of the user(s);

[0104] - predictable movements of the robot arm 6;

[0105] - the entire geometry of the operating room; Using the environmental model, the control unit 10 prevents, for example, collisions between the robot arm 6 and detected users, patients, and / or objects in the operating room. Furthermore, the control unit 10 optimizes the robot arm position to increase the rigidity of the robot arm 6, the accessibility of the visualization unit 8, and the avoidance of singularities of the robot arm 6.

[0106] List of reference symbols

[0107] 1 visualization system

[0108] 2 robots

[0109] 4 Robot base

[0110] 6 Robot arm

[0111] 8 Visualization unit

[0112] 10 Control unit

[0113] 12-inch screen

[0114] 14 optical sensor (camera)

[0115] 16 Line of sight

[0116] 18 robot arm segment

[0117] 20 input unit

[0118] 100 visualization system

[0119] 102 robots

[0120] 106 Robot arm

[0121] 108 Visualization Unit

[0122] A1 first axis of rotation

[0123] A2 second axis of rotation

[0124] A3 third axis of rotation

[0125] A4 fourth axis of rotation

[0126] A5 fifth axis of rotation

[0127] A6 sixth axis of rotation

[0128] A7 seventh axis of rotation

[0129] S1 first robot arm segment

[0130] S2 second robot arm segment

[0131] S3 third robot arm segment

[0132] S4 fourth robot arm segment

[0133] S5 fifth robot arm segment

[0134] S6 sixth robot arm segment

Claims

Claims 1. Medical robot-assisted visualization system (1) for visualization, in particular during a surgical procedure on a patient (P), comprising: a robot (2) with a robot base (4), preferably a displaceable medical cart as a mobile robot base (4), and a movable robot arm (6) connected to the robot base (4);a visualization unit (8) as the end effector of the robot (2), which is mounted or attached to the movable robot arm (6) and can be spatially positioned, in particular arranged in a position and orientation, by a movement of the robot arm (6), a control unit (10) which is adapted to control at least one position and / or orientation of the visualization unit (8), characterized in that the robot (2) is a seven-axis robot and / or the robot arm (6) has at least seven axes, in particular is rotatable about seven axes of rotation, by means of which the robot arm (6) can be moved into different robot arm positions with a constant position and / or orientation of the visualization unit (8); 2. Medical robot-assisted visualization system (1) according to claim 1, characterized in that the axes are rotation axes (A1, A2, A3, A4, A5, A6, A7) and the individual, respectively adjacent rotation axes (A1, A2, A3, A4, A5, A6, A7) of the robot arm (6) are each arranged perpendicular to one another.

3. Medical robot-assisted visualization system (1) according to claim 1 or 2, characterized in that a first rotation axis (A1) is arranged such that a rotation of a first robot arm segment (S1) and thus of the robot arm (6) relative to the robot base (4) can be carried out, a second rotation axis (A2) is arranged such that a pivoting of a second robot arm segment (S2) relative to the first robot arm segment (S1) can be carried out, in particular the second axis of rotation (A2) is perpendicular to the first axis of rotation (A1), a third axis of rotation (A3) is arranged such that a rotation of a third robot arm segment (S3) relative to the coaxially arranged second robot arm segment (S2) can be carried out, and preferably a fourth axis of rotation (A4) is arranged such that a pivoting of a fourth robot arm segment (S4) relative to the third robot arm segment (S3) can be carried out, in particular the fourth axis of rotation (A4) is perpendicular to the third axis of rotation (A3).

4. Medical robot-assisted visualization system (1) according to one of claims 1 to 3, characterized by a, in particular optical, sensor (14), in particular a camera, and in that the control unit (10) is adapted to: detect a spatial position of a user's head, in particular an eye position of a user, by means of the, in particular optical, sensor (14), determine a position of an output unit (12), in particular a screen, calculate a line of sight (16) between the detected position of the user and the determined position of the output unit (12), in particular with a tolerance range around this line of sight (16), and detect a robot arm position and determine whether a section of the robot arm (6) lies in the line of sight (16), in particular in the tolerance range, and a blockage of a direct line of sight between the head, in particular eyes,of the user and the position of the output unit (12) by a robot arm segment (18) of the robot arm (6).

5. Medical robot-assisted visualization system (1) according to claim 4, characterized in that the control unit (10) is adapted to move the robot arm (6) out of the line of sight (16) upon detection of the blockage of the line of sight, so that the blockage of the robot arm segment (18) is resolved, wherein the position and / or orientation of the visualization unit (8) as an end effector is maintained, in particular a joint of the robot arm (6) with the fourth axis of rotation (A4) is moved out of the line of sight (16).

6. Medical robot-assisted visualization system (1) according to claim 4 or 5, characterized in that the output unit (12) is movable relative to the robot base (4), in particular is displaceable on a second medical carriage, wherein the control unit (10) is adapted to detect the position of the movable output unit (12) by means of a tracking system, in particular a rigid body with optical markers is arranged on the output unit (12), which is detected via an external camera of the tracking system.

7. Medical robot-assisted visualization system (1) according to one of claims 4 to 6, characterized in that objects and / or persons in the environment of the visualization system (1) are detected via the, in particular optical, sensor (14) and the control unit (10) is adapted to detect and prevent a collision of the robot arm (6) with the detected objects and / or persons when the robot arm (6) is moved by stopping the movement of the robot arm (6).

8. Medical robot-assisted visualization system (1) according to one of claims 1 to 7, characterized in that the control unit (10) is adapted to adjust the robot arm (6) such that the robot arm segments (18) lie substantially in a vertical plane and are thus aligned parallel to the direction of gravity in order to increase a rigidity of the robot arm (6).

9. Medical robot-assisted visualization system according to one of claims 1 to 8, characterized in that the control unit (10) is adapted to adjust the robot arm (6) such that each of the robot arm segments (18) is arranged at a maximum distance from the user, so that the visualization unit (8) is easily accessible to the user and in particular no robot arm segment (18) is arranged between the position of the user and the visualization unit (8).

10. Medical robot-assisted visualization system (1) according to one of claims 1 to 9, characterized in that the control unit (10) is designed to is adapted to adjust the robot arm (6) in such a way that singularities, in particular in an end position, of the robot arm (6) are avoided.

11. Medical robot-assisted visualization system (1) according to one of claims 1 to 10, characterized in that the control unit (10) is adapted to weight individual parameters of collision avoidance, singularity avoidance, increasing accessibility and / or increasing rigidity and to adapt a configuration of the robot arm (6) according to the weighting in order to achieve an optimum robot arm position, wherein in particular collision avoidance has the greatest weighting.

12. Medical robot-assisted visualization system (1) according to one of claims 3 to 11, characterized by an input unit (20), in particular a button or a touch display, by the actuation of which the user can manually request a movement of the robot arm (6), wherein the position and / or orientation of the visualization unit (8) is kept constant, wherein in particular upon actuation the joint with the fourth axis of rotation (A4) is rotated by 90° about a virtual horizontal axis or about a longitudinal axis of the robot arm (6).

13. Medical robot-assisted visualization system (1) according to one of claims 1 to 12, characterized in that the visualization unit (8) is a digital surgical microscope, an optical microscope and / or a medical endoscope.

14. Control method for a medical robot-assisted visualization system (1), in particular for a robot-assisted visualization system (1) according to one of the preceding claims, characterized by the following steps: - Method, by a control unit (10), a visualization unit (8) as an end effector of a robot (2) with a robot base (4) and a robot arm (6) of the visualization system (1) connected to the robot base (4) into a predetermined target position and / or target orientation by a robot arm (6) of the visualization system (1), wherein the robot (2) is a seven-axis robot and / or the robot arm (6) has at least seven axes, and in particular is rotatable about seven axes of rotation, by means of which the robot arm (6) can be moved into different robot arm positions with a constant position and / or orientation of the visualization unit (8); - detecting a spatial position of a user's head, in particular an eye position of the user, via a sensor (14), in particular an optical sensor, and determining a position of an output unit (12) by the control unit (10); - calculating a line of sight (16) between the detected position of the user and the determined position of the output unit (12); - detecting a robot arm position of the robot arm (6) by the control unit (10); - Detection of a possible blockage of the line of sight (16), in particular by a robot arm segment (18) of the robot arm (6), by the control unit (10); - If a blockage has been detected, the control unit (10) moves the robot arm (6), keeping the target position and / or target orientation of the visualization unit (8) constant.

15. Control method according to claim 14, characterized in that the step of detecting a robot arm position of the robot arm (6) by the control unit (10) takes place during the movement of the visualization unit (8) by the robot arm (6).

16. A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the steps of the control method according to one of claims 14 or 15.

17. A computer program comprising instructions which, when executed by a computer, cause the computer to carry out the steps of the control method according to one of claims 14 or 15.

Images