Method for controlling a movement of a robot

Abstract

It is presented a method for controlling a robot, comprising the steps of: detecting, by a first image sensor, first image information of a first field of view, providing the first image information to a data processing device, detecting, by a second image sensor, second image information of a second field of view, wherein the first field of view and the second field of view at least partially overlap, providing the second image information to the data processing device, determining, by the data processing device, first monocular depth information based on the first image information and / or determining, by the data processing device, second monocular depth information based on the second image information, determining, by the data processing device, stereo depth information based on the first image information and the second image information, determining, by the data processing device, control information for controlling the robot based on the first monocular depth information and / or the second monocular depth information and the stereo depth information, providing the control information to a control device, and controlling a movement of the robot by the control device in accordance with the control information.

Description

[0001] Our reference: A19283WO

[0002] Method for controlling a movement of a robot

[0003] 1. FIELD OF THE INVENTION

[0004] A method for controlling a robot, a robot, a computer-program product and a computer- readable medium.

[0005] 2. BACKGROUND

[0006] Autonomous mobile robots and certain types of collaborative industrial robots often operate in areas with people. In order to operate safely and to comply with the required industrial standards, they often comprise safety-rated means for detecting people and reacting appropriately in a correspondingly controlled movement. Today, personnel detection is often implemented using safety-rated laser scanners. They can detect obstacles in a configurable field and provide an input to the robot control system such that the appropriate action may be taken. The problem in such a solution is that safety-rated laser scanners are expensive and may solely be employed for safety purposes. Furthermore, laser scanners often cannot identify the nature of an obstacle, e.g. if a person and non-person object like another robot or a wall is detected. Hence, to retain safety, the robots are usually controlled to slow down or stop for every obstacle, which is highly inefficient in an industrial setting. Other solutions attempt include the employment of image sensors in the navigation, which are however usually complex and expensive, in particular in the field of industrial applications. Existing systems regularly propose only a single channel for personnel detection, which renders it difficult or even impossible to achieve required safety levels in particular in complex mixed industrial settings.

[0007] Thus, there is a need for an improved method of controlling robots.

[0008] 3. SUMMARY OF THE INVENTION

[0009] The object is achieved by the present invention according to the independent claims. Preferred embodiments of the invention are provided in the dependent claims, the description and the accompanying figures. The present invention relates to a method for controlling a robot.

[0010] The method comprises the step of detecting, by a first image sensor, first image information of a first field of view.

[0011] The method further comprises the step of providing the first image information to a data processing device.

[0012] The method further comprises the step of detecting, by a second image sensor, second image information of a second field of view, wherein the first field of view and the second field of view at least partially overlap.

[0013] The method further comprises the step of providing the second image information to the data processing device.

[0014] The method further comprises the step of determining, by the data processing device, first monocular depth information based on the first image information.

[0015] Additionally or alternatively, the method further comprises the step of determining, by the data processing device, second monocular depth information based on the second image information.

[0016] The method further comprises the step of determining, by the data processing device, stereo depth information based on the first image information and the second image information.

[0017] The method further comprises the step of determining, by the data processing device, control information for controlling the robot based on the first monocular depth information and / or the second monocular depth information and the stereo depth information.

[0018] The method further comprises the step of providing the control information to a control device.

[0019] The method further comprises the step of controlling a movement of the robot by the control device in accordance with the control information.

[0020] Accordingly, an improved method for operating a robot in a safe and reliable manner may be provided. Furthermore, the proposed method could be implemented in a cheap and simple manner. Advantageously, the method according to the present invention may allow to use dense stereo depth information and at the same time employ monocular depth information of the first and second image sensors to enhance reliability of the stereo depth information. For instance, the stereo depth information may be synchronized with or corrected by the monocular depth information, which may include the first monocular depth information and / or the second monocular depth information, and which may accordingly allow for a particular safe and reliable depth estimation and respective robot control.

[0021] An image sensor may generally include any detection device or arrangement that is adapted to acquire optical image data, such as one or more image sensors, e.g. digital image sensors, or arrays thereof. The first image sensor may be of the same type as the second image sensor or may be different to the second image sensor.

[0022] A field of view of the respective image sensor may be understood as an area or space observable by the respective sensor. The image sensors and the respective fields of view may be for instance fixed or static. In other embodiments, the image sensors and the respective fields of view may be dynamically adjustable, such that the orientation of the image sensors could be adapted to change a view of the image sensors, as desired. Of course, also more than two image sensors may be provided. The respective further fields of view of the additional sensors may then be accordingly considered in the same way as described herein for the first image sensor and the second image sensor.

[0023] An overlapping field of view may be understood as an area or space that is covered by both, the first field of view and the second field of view. The first image sensor and the second image sensor may be provided at different positions, such that the first field of view and the second field of view may not fully correspond. Furthermore, the first image sensor and the second image sensor may have different angles of view. An overlapping field of view may accordingly enable detecting a certain area or space from different views or angles. Suitable algorithms executed by the data processing device may be employed to determine a respective stereo depth information.

[0024] To acquire stereo depth information, the first image sensor may enable acquiring a field of view that is least partially different in the view angle compared to the view angle of the field of view of the second image sensor. For instance, the first image sensor may be arranged at the robot at a different position compared to the second image sensor.

[0025] Image information may include any processable information, e.g. digital data, representing one or more images acquired by the respective image sensor. The data processing device may include any suitable digital data processing means and may comprise a corresponding processor and temporal and / or permanent data storage devices, such as random-access memory (RAM), read-only memory (ROM), or any other kind of volatile or non-volatile memory. The data processing device may be operatively coupled to the control device to transfer for instance control information to the control device or to receive respective data therefrom. Furthermore, the data processing device may be operatively coupled to one or more sensors of the robot, such that data information generated from the image sensors could be transferred to and received by the data processing device. The data processing device may be accordingly adapted to receive first image information and second image information from the first and second image sensors, respectively, and to process the digital image information subsequently in a desired manner. For instance, the data processing device may determine respective control information concerning a movement or change in a movement of the robot and may provide the control information to the control device. The control device may be accordingly adapted to receive the control information and may allow to control the robot based thereon.

[0026] The robot may include any kind of mobile or movable robot. “Mobile” or “movable” robots in this context may be understood such that the robot may assume different positions or locations on the ground. The position of the robot may be accordingly changed. For instance, the robot may be moved by respective means, e.g. a motor and propellable and steerable wheels, from a starting point to an end point. However, a movement of the robot may also include movements of other movable or steerable parts of the robot, such as a robot arm or an end effector attached thereto. The robot may also include stationary robots, such as industrial robots operating for instance in a collaborative environment.

[0027] The first image sensor and / or the second image sensor may for instance acquire images of an environment of the robot, for instance an area in front of the robot. However, also any other area around the robot may be acquired, such as a back side or a lateral side, as desired. An object of interest, such as an obstacle, being arranged in the movement path of the robot, may be accordingly determined in the image data provided by the first image sensor and / or the second image sensor via the data processing device executing corresponding algorithms. The data processing device may accordingly determine suitable control information to be supplied to the control device. This may include for instance a change in the velocity of the robot, i.e. a deceleration or braking of the robot to avoid a collision with the obstacle. Or respective steering information may be provided to the control device, to avoid a collision with the obstacle. Similarly, any other movable part of the robot, for instance the robot arm or the end effector may be controlled as desired based on the first detected image information and the second detected image information to take appropriate movements based on the image data acquired by the first and second image sensors. The robot may, for instance, automatically slow down or stop any movement if a person is within a certain range. In case of stationary robots, it may be accordingly not necessary to install fences and other protective equipment around the robot.

[0028] The control device may be arranged at the robot or at least partially outside and remote to the robot. The control device may comprise suitable computing and controlling means for controlling any elements of the robot, for instance an orientation or extension of the robot arm and / or a propelling speed, acceleration or a steering of, for instance, wheels of the robot. The control means may comprise respective sensors or detectors to determine a position, movement or acceleration of respective elements of the robot, such as of the robot arm, the robot base and further movable elements of the robot. The end effector may comprise one or more tools arranged at the distal end of the robot arm to perform respective manipulating actions, such as for instance picking and lifting an item and transporting and placing said item at a different position.

[0029] The stereo depth information may include information about a spatial extension, orientation and distance of an object of interest detected by the first and second image sensors. The stereo depth information may be calculated by employing image information of the object of interest shown in the first and second images at different angles. Hence, a distance of the object of interest to the robot acquired in the overlapping field of view of the two sensors may be calculated. Hence, a depth perception may be enabled by stereo vision thus enabling the determination of the imaged environment in three dimensions.

[0030] Contrary to this, monocular depth information may include information about a spatial extension, orientation and distance of an object of interest, derived from a single image sensor, e.g. derived from a single image. However, the present invention is not delimited thereto, and respective information could also be derived from a serious of images of a single sensor. In this respect, for instance size information or perspective information of objects may be used to determine a distance of the object of interest to the robot. Generally, respective algorithms executed by the data processing device may be employed to determine a respective monocular depth information. In a preferred embodiment, the step of determining control information for controlling the robot based on the first monocular depth information and / or the second monocular depth information and the stereo depth information further includes the steps of: validating, by the data processing device, the stereo depth information by the first monocular depth information and / or the second monocular depth information, wherein the validating step includes the step of comparing, by the data processing device, the stereo depth information with the first monocular depth information and / or the second monocular depth information, and determining, by the data processing device, faults in the stereo depth information based on the comparison of the stereo depth information with the first monocular depth information and / or the second monocular depth information.

[0031] Accordingly, a particularly quick and reliable method for acquiring depth information and controlling the robot accordingly may be obtained. A respective validation of stereo depth information obtained from the first and second image sensors may be achieved via a comparison of the stereo depth information with corresponding monocular depth information of the first and / or second image sensors. Hence, the stereo depth information including for instance a distance to an object may be confirmed or corrected by the respective monocular depth information. That is, if no confirmation of correctness of the stereo depth information could be determined, respective faults may be identified, and the stereo depth information may be accordingly corrected. Thus, the reliability of the stereo depth information may be in enhanced.

[0032] The present method may accordingly allow to verify or falsify stereo depth information, which may accordingly increase safety in the control of a robot based on the stereo depth information. For instance, if certain distance values initially determined in the stereo depth information could not be verified by the respective monocular depth information, the movement of the robot may be accordingly adapted to assume a safe condition considering the potential error in the distance derived from the stereo depth information. For instance, a respective steering or braking of the robot may be induced.

[0033] On the other hand, if respective stereo depth information is confirmed in its correctness by the comparison with the respective monocular depth information, this may consequently enhance the reliability of the control of the robot. Hence, it may be ensured that the robot is controlled in a correct manner and the risk of occurrences of hazardous situations could be minimized. Consequently, the robot may be employed in mixed working environments, which require a particular safety level since human operators or other persons are present in the working environment of the robot.

[0034] Accordingly, the presented method may allow to achieve an increased accuracy and robustness which may lead to lower tolerances and reduced false positives in the depth information determination. Furthermore, since the monocular depth estimation is not used to control a movement of the robot, but only used to verify the respective stereo depth information control, higher permitted response times compared to purely monocular depth information-based controls, may be achieved.

[0035] In a preferred embodiment, the method further comprises the steps of: providing a robot controlling architecture including a functional channel and a testing channel, wherein the steps of determining first monocular depth information based on the first image information, determining second monocular depth information based on the second image information, validating the stereo depth information by the first monocular depth information and / or the second monocular depth information, and determining faults in the stereo depth information based on the comparison of the stereo depth information with the first monocular depth information and / or the second monocular depth information is assigned to the testing channel, wherein the step of determining stereo depth information is assigned to the functional channel.

[0036] Thus, a particular failsafe operation and control of the robot may be achieved. The provision of a respective two-channel approach may accordingly improve reliability in the determination of objects and distances thereof in the environment of the robot, which may enhance safety in the control of the movement of the robot. Potential faults or errors in one channel, e.g. the functional channel, may be revealed, and in case a fault is detected, the control of the movement of the robot may be accordingly adapted. For instance, if a fault in one channel is detected, the robot may be forced to stop by the control device for safety reasons. A fault may be for instance a mismatch in the distance to an object determined by the stereo depth information and the respective distance to the same object determined by the monocular depth information.

[0037] Thus, functional safety compliance of a robot controlled according to the present invention may be enhanced by the provision of two generally independent channels. A functional safety criteria may be accordingly achieved by providing a functional channel and a testing channel, and the data processing device may be provided with a certain degree of independence in the handling of the first image information and the second image information. Furthermore, lower tolerances and fewer false positives may be achieved. In addition, the stereo depth estimation may require fewer computational resources, which may accordingly reduce complexity and costs with respect to computing power needed to control the robot.

[0038] In a preferred embodiment, the method further comprises the steps of: comparing, by the data processing device, the first monocular depth information with the second monocular depth information, and determining, by the data processing device, faults in the detection of the first image information and / or the second image information based on the comparison of the first monocular depth information with the second monocular depth information.

[0039] Accordingly, further enhanced reliability in the depth estimation and accordingly in the control of the robot may be achieved. The monocular depth information determination may be generally performed by suitable machine learning and / or Al based models and each sensor may use the same or different data sets for training the same or different one or more algorithms. Hence, faults or errors, e.g. in the detection of the image, may be identified by respective algorithms for each individual channel. The fault detection may be based on the acquired images. However, also different type of faults, such as errors in the control of the image sensors and / or data transfer errors may be detected. Thus, any faults based on hardware or software errors, may be detected. The determination of faults in the detection of the first image information and / or the second image information may also be based on the comparison of one or more of the first monocular depth information and the second monocular depth information with the stereo depth information.

[0040] If a fault is determined, the control of the movement may be accordingly adapted. For instance, if a fault in one (e.g. the first or the second) monocular depth information is detected, the robot may be forced to stop by the control device until the error is remedied. Or the respective faulty image sensor may be deactivated, or its data may be at least temporarily ignored to prevent false image data acquisition and an impaired control of the robot. In case that more than two image sensors are provided, the depth estimation and respective control of the robot could then be continued based on the remaining properly working sensors. Thus, a failsafe operation of the robot could be ensured.

[0041] In a preferred embodiment, the method further comprises the steps of: determining kinematic information of the robot, wherein the step of determining control information for controlling the robot is further based on the kinematic information of the robot. Furthermore, also the steps of determining monocular depth information and / or the step of determining stereo depth information may be based on the kinematic information. In a preferred embodiment, the kinematic information includes one or more of a positional information, a stance information, a velocity information and / or an acceleration information of the robot.

[0042] Accordingly, respective kinematic information may be considered in the control of the robot, which may enable a more sophisticated control of the robot. A disadvantageous positioning or movement of the robot or parts thereof, such as an arm, may be prevented. Accordingly, collisions with obstacle may be avoided and the robot may also be moved in a more efficient manner. Furthermore, when considering sequences of images for depth estimation, it may be useful to know how the camera itself has moved in between the individual image acquisitions. It may be then accordingly easier to detect how other objects in the images are moving and thus where they are located.

[0043] The respective kinematic information may be determined directly by sensors, e.g. image sensors or other sensors to determine a position and / or velocity of the robot or of an arm of the robot. For this, the respective kinematic information may be either determined directly by the first and second image sensors and / or additional sensors, which may be specifically configured to allow a detection of the respective kinematics.

[0044] The kinematic information could also be determined indirectly, e.g. via motor control characteristics such as a power consumption of a respective motor monitored. However, the present invention is not delimited thereto, and further and different means could be considered to determine the positional information, the stance information, the velocity information and / or the acceleration information of the robot.

[0045] The positional information may include the actual spatial position of the robot and may be determined for instance via a GPS sensor. The stance information may include a respective orientation of the robot and / or any further parts thereof, such as an arm. The velocity information may accordingly include information about the movement speed of the robot or any further parts thereof, such as an arm. The acceleration information may include information about a slowing down or speeding up of a movement of the robot and / or any further parts thereof, such as an arm. The kinematic information may be considered only in a single channel, e.g. the functional or the testing channel, or the kinematic information may be considered both channels. In a preferred embodiment, the method further comprises the steps of: determining a first detection time of the detection of the first image information and / or determining a second detection time of the detection of the second image information, wherein the step of determining control information for controlling the robot is further based on the first detection time and / or the second detection time.

[0046] Thus, time information may be additionally considered in the control of the robot, which may further improve the control of the robot. Determining respective detection times may allow to derive dynamic environmental information from the acquired images, which could be considered in the control of the robot accordingly. For instance, velocities and accelerations of detected moving objects could be determined and the control of the robot may be accordingly adapted thereto. Hence, a movement of such objects could be forecasted and a collision with such objects could be prevented. The detection time may include a single time instance or a time range. The detection time may be considered only in a single channel, e.g. the functional or the testing channel, or may be considered both channels.

[0047] In a preferred embodiment, the method further comprises the steps of: determining, by the data processing device, information about one or more objects of interest from the first image information and / or the second image information, wherein the step of determining control information for controlling the robot is further based on the determined information about the one or more objects of interest.

[0048] Accordingly, imaged objects may be identified, and the control of the robot may be suitably adapted. This may allow for an improved control in the movement of the robot. The data processing device may calculate from the image information provided by the first and second image sensor characteristics of a detected object. For instance, respective image analyzing algorithms may identify an object and determine if the object is person or not a person. Furthermore, the data processing device may identify and verify if the object is for instance moving towards or away from the robot. The robot may identify if the object is a known object, which may be for instance registered and therefore known to the robot, and for which a respective movement may be known or could be easily predicted, or in the object is an unknown object, which movement is unknown and could not easily predicted. Generally, any desired characteristic may be accordingly included in the determined information of the respective object of interest. Hence, the movement of the robot may be suitably adapted based on this additionally determined information.

[0049] In a preferred embodiment, the method further comprises the steps of: detecting, by at least one additional sensor, additional sensor information, wherein the at least one additional sensor information includes one or more of LIDAR sensor information, RADAR sensor information, position sensor information and / or a temperature sensor information, wherein the step of determining control information for controlling the robot is further based on the additional sensor information.

[0050] Thus, the control of the robot may be further enhanced. For instance, a respective distance information gathered from the image sensors may be further verified via the additional sensor information. For instance, the additional sensor may operate to detect objects that are not or at least not easily detectable by the image sensors. As an example, a line of sight to an obstacle may be blocked for the image sensors, and the additional sensor, which may not require a line of sight to detect said obstacle or which may be arranged at a different position such that the additional sensor has a line of sight, may provide a respective information about the presence of the obstacle. Accordingly, the control of the robot may be accordingly adapted.

[0051] The additional sensor may be arranged at the same position as the first image sensor and / or the second image sensor, or the additional sensor may be arranged at a different position. As an example, the additional sensor may be arranged at a top side of the robot, whereas the first and second image sensors may be arranged at a front side of the robot. However, also any other positioning of the image sensors and the additional sensor at the robot may be of course conceivable. Also more than one additional sensor may be provided, for instance an array of additional sensors. The one or more additional sensors may include one or more of a LIDAR sensor, a RADAR sensor, a position sensor and / or a temperature sensor. However, also any other sensors using different technology to acquire environmental information of the robot and in particular distance information to objects located in the environment of the robot may be employed. The additional sensor could also be an additional image sensor. The additional sensor may have a respective field of view, which may or may not overlap with the field(s) of view of the first image sensor and / or the second image sensor. The invention further relates to a robot, comprising: a data processing device, a control device for controlling a movement of the robot, a first image sensor adapted to detect first image information of a first field of view, a second image sensor adapted to detect second image information of a second field of view, wherein the first field of view and the second field of view at least partially overlap, wherein the data processing device is adapted to receive the first image information and the second image information, wherein the data processing device is adapted to provide control information to the control device, wherein the robot is adapted to be controlled according to the method according to the present invention.

[0052] Accordingly, an improved robot operable in a safe and reliable manner may be provided. Advantageously, the robot according to the present invention may employ dense stereo depth information and at the same time use monocular depth information of the first and second image sensors. For instance, the stereo depth information may be synchronized with or corrected by the monocular depth information, which may accordingly allow for a particular safe and reliable depth estimation. No further sensors may be necessary to achieve a proper control of the robot also in complex and safety rated environments. Thus, the proposed robot could be implemented in a cheap and simple manner.

[0053] It is noted that the above explanations regarding certain elements or advantages described with respect to the method of controlling a robot of the present invention accordingly apply to the robot described herein.

[0054] The robot may include any kind of at least partially automatically movable electromechanical machines. For instance, the robot may be a mobile manipulator. The robot may comprise a robot base and a robot arm attached at an outer exterior of the robot base. Also more than one robot arm may be provided or no robot arm may be provided such that the robot may only comprise a robot base. For instance, the robot may be an autonomous vehicle comprising one or more steerable wheels. The movement of the robot along the ground and the movement of the arm may be fully or semi-automatically controlled. The steerable wheels may allow for a particular well controlled movement over the ground. The wheels may be jointly or independently propellable or steerable. The robot may be for instance an industrial robot, a logistic robot, a medical robot, a laboratory robot or any other kind of robot, working in a respective industrial, logistic, medical or laboratory environment. However, the present invention is of course not delimited thereto, and the robot may be employed in any desired environment. Further, one or more manipulation arms may be arranged on the robot base to execute desired tasks.

[0055] The robot base may essentially define the core body of the robot, at which, for instance the manipulator arm may be attached. The manipulator base may house respective propelling or controlling means such as a power supply, one or more motors and / or computing or data processing devices, which may be enclosed by a housing to protect the robot elements arranged inside from negative impacts, such as dust and moisture.

[0056] The robot may be controlled to move along a floor or ground. The robot base may comprise accordingly means for moving such as wheels to contact the floor. However, in different embodiments, different contact elements for moving the robot along the floor or ground may be provided, e.g. legs etc. The movement of the wheels may be controlled by the control device, which may be arranged at the robot, e.g. at the robot base. However, it will be understood that in different embodiments, the control device can also be arranged at least partially outside of or remote from the robot and could be operatively coupled to the robot to enable a respective control thereof.

[0057] The control device may comprise suitable sensing, computing and controlling means for controlling the movable elements of the robot, for instance a propelling speed, acceleration and / or movement direction of the wheels.

[0058] The robot arm may be formed as a movable arm comprising multiple arm elements which are linked together to allow a flexible movement and extension or retraction of the arm. However, it will be understood that in different environments also a stiff robot arm may be provided consisting for instance only of a single stiff arm element movably attached to the robot base. The robot arm may be hinged at a proximal end of the robot arm with the base and may comprise an end effector at a distal end of the robot arm. For instance, the end effector may be formed as a gripper to allow a grip of an external item or load. The orientations of the robot arm may also be set and controlled by the control device of the robot. The control device may accordingly comprise suitable sensing, computing and controlling means for controlling a movement of the robot arm. A respective movement of the robot arm or the end effector, which may include for instance a translational or rotational movement, may be performed in any direction such as in a vertical direction, a horizontal direction or a combined movement in both directions as desired. The control device may be adapted to control any types of propelling motors and / or brakes or steering devices that allow to control movements of any movable or propellable part of the robot.

[0059] The image information acquired by the first and second image sensors could include for instance a two-dimensional image of an environment covered by the first field of view and the second field of view. The first field of view and the second field of view may at least partially overlap. Accordingly, stereo depth information may be obtained from the images acquired by the first image sensor and the second image sensor. Of course, also more than two image sensors could be provided.

[0060] The data processing device may include any suitable digital data processing means may include a corresponding processor and temporal and / or permanent data storage devices, such as random-access memory (RAM), read-only memory (ROM), or any other kind of volatile or non-volatile memory. The data processing device may be operatively coupled to the control device, such that data, for instance control information, could be transferred to and received from the control device. Furthermore, the data processing device may be operatively coupled to one or more sensors of the robot, such that data information generated by the sensors could be transferred to and received from the data processing device. The data processing device may be accordingly adapted to receive the first image information and the second image information from the first image sensor and the second image sensor, respectively, and to process the digital image information in a desired manner. For instance, the data processing device could determine respective control information concerning a movement or change in a movement of the robot and could provide the control information to the control device. The control device may be accordingly adapted to receive the control information and may allow to control the robot based thereon.

[0061] For instance, the first image sensor and / or the second image sensor may acquire images of an environment of the robot, for instance an area in front of the robot. An object of interest, such as an obstacle, being arranged in the movement path of the robot, may be accordingly determined by the data processing device in the image data provided by the first image sensor and / or the second image sensor. The data processing device may accordingly determine suitable control information to be supplied to the control device, which may include for instance a change in the velocity of the robot, e.g. a deceleration or braking of the robot to avoid a collision with the obstacle. Or respective steering information may be provided to the control device, to avoid a collision with the obstacle. Similarly, any other movable part of the robot, for instance the robot arm or the end effector may be controlled as desired, based on the first detected image information and the second detected image information.

[0062] In a preferred embodiment, the robot comprises at least one additional sensor adapted to detect additional sensor information, wherein the at least one additional sensor includes one or more of a LIDAR sensor, a RADAR sensor, a position sensor and / or a temperature sensor.

[0063] Thus, the control of the robot may be further enhanced. For instance, a respective distance information gathered from the image sensors may be further verified via the additional sensor information. For instance, the additional sensor may operate to detect objects that are not or at least not easily detectable by the image sensors. As an example, a line of sight to an obstacle may be blocked for the image sensors, and the additional sensor, which may not require a line of sight to detect said obstacle or which may be arranged at a different position such that the sensor has a line of sight, may provide a respective information about the presence of the obstacle. Accordingly, the control of the robot may be accordingly adapted.

[0064] The additional sensor may be arranged at the same position as the first image sensor and / or the second image sensor, or the additional sensor may be arranged at a different position. As an example, the additional sensor may be arranged at a top side of the robot, whereas the first and second image sensors may be arranged at a front side of the robot. However, also any other positioning of the image sensors and the additional sensor at the robot may be of course conceivable. Also more than one additional sensor may be provided, for instance an array of additional sensors. The one or more additional sensors may include one or more of a LIDAR sensor, a RADAR sensor, a position sensor and / or a temperature sensor. However, also any other sensors using different technology to acquire environmental information of the robot and in particular distance information to objects located in the environment of the robot may be employed. The additional sensor could also be an additional image sensor. The additional sensor may have a respective field of view, which may or may not overlap with the field(s) of view of the first image sensor and / or the second image sensor.

[0065] In a preferred embodiment, wherein the robot comprises a robot controlling architecture including a functional channel and a testing channel, wherein the robot is adapted to be controlled according to the method according to the present invention. In a preferred embodiment, the robot is compliant with category 2 according to the ISO 13849-1 standard or compliant with a hardware fault tolerance of 0 according to the IEC 61508 standard.

[0066] Thus, a robot that is operable in a particular failsafe and safety rated manner could be provided. The provision of two channels accordingly improves reliability in the determination of the environment of the robot and the control of the movement of the robot. Faults in one channel, e.g. the functional channel, may be revealed and, in case a fault is detected, the control of the movement may be accordingly adapted. For instance, if a fault in one channel is detected, the robot may be forced to stop by the control device. A fault may be for instance a mismatch in the distance to an object determined by the stereo depth information and the respective distance to the object determined by the monocular depth information.

[0067] Thus, functional safety compliance of a robot controlled according to the present invention may be enhanced by the provision of two, preferably independent, channels.

[0068] The invention further relates to a computer-program product comprising instructions, which, when executed by a data processing device, cause the data processing device to carry out and / or control the method according to the present invention.

[0069] The features and advantages outlined above in the context of the method for controlling the robot and the robot similarly apply to the computer program product described herein.

[0070] The features of the method according to the present invention may be implemented by respective suitable digital or computational means, which can include, for instance, one or more computers, apps and / or networks. The method may be at least partly computer- implemented, and may be implemented in software or in hardware, or in software and hardware. The data processing means may be any suitable computing means, such as a computer, an electronic control module etc., which may also be a distributed computer system. The data processing means may comprise one or more of a processor, a memory, a data interface, or the like.

[0071] The computer-program product may be stored / distributed on a suitable medium such as an optical storage medium or a solid-state medium, supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.

[0072] The invention further relates to a computer-readable medium comprising instructions which, when executed by a data processing device, cause the data processing device to carry out and / or control any the method according to the present invention.

[0073] The features and advantages outlined above in the context of the method for controlling the robot, the robot and the computer-program product similarly apply to the computer-readable medium described herein.

[0074] Any of the data processing device, the computer-program product and / or the computer-readable medium may be at least part of the robot or may be arranged remotely to the robot but operatively coupled thereto, e.g. via respective data transmission means, such as wired or wireless data transmission means.

[0075] Further features, examples, and advantages will become apparent from the following detailed description of preferred embodiments and the accompanying figures.

[0076] 4. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] For a better understanding of the present invention, and to illustrate its practicality, figures are provided in the following and reference is made thereto. It should be understood that the figures represent only exemplary embodiments and thus in no way limit the scope of the claimed invention. Identical or like-acting elements are indicated throughout by the same reference signs. Any reference signs in the claims should not be construed as limiting the scope of the claims. The figures are merely schematic representations and serve only to illustrate examples of the disclosure. Further, as used herein, a hyphenated form of a reference sign refers to a specific embodiment of an element and the unhyphenated form of the reference numeral refers to the collective element. For example, a device "21-1" or “21-2” refers to an instance of a device class that can be collectively referred to as device "21" and each of which can be generically referred to as device "21".

[0078] In the accompanying drawings,

[0079] Figure 1 is a flowchart illustrating an embodiment of a method for controlling a robot according to the present invention, Figure 2 is a flowchart illustrating an embodiment of a method for controlling a robot according to the present invention, and

[0080] Figure 3 schematically illustrates a robot according to an embodiment of the present invention.

[0081] 5. DESCRIPTION OF EMBODIMENTS

[0082] Figures 1 and 2 depict flowcharts each illustrating a method 100 for controlling a robot 1 according to an embodiment of the present invention and refer also to elements concerning the robot 1 , described in detail with respect to Figure 3.

[0083] As depicted, the method 100 for controlling the robot 1 comprises the step S1A of detecting, by a first image sensor 21-1 , first image information of a first field of view 23-1. The method 100 further comprises the step S2A of providing the first image information to a data processing device 17. The method 100 further comprises the step S1 B of detecting, by a second image sensor 21-2, second image information of a second field of view 23-2, wherein the first field of view 23-1 and the second field 23-2 of view at least partially overlap. The method 100 further comprises the step S2B of providing the second image information to the data processing device 17. The method 100 further comprises the step S3A of determining, by the data processing device 17, first monocular depth information based on the first image information. Additionally or alternatively to the step S3A, the method 100 further comprises the step S3B of determining, by the data processing device 17, second monocular depth information based on the second image information. The method 100 further comprises the step S4 of determining, by the data processing device 17, stereo depth information 31 based on the first image information and the second image information. The method 100 further comprises the step S5 of determining, by the data processing device 17, control information for controlling the robot 1 based on the first monocular depth information and / or the second monocular depth information and the stereo depth information 31 . The method 100 further comprises the step S6 of providing the control information to a control device 7. The method 100 further comprises the step S7 of controlling a movement of the robot 1 by the control device 7 in accordance with the control information.

[0084] As depicted, the step S5 of determining control information for controlling the robot 1 based on the first monocular depth informationand / or the second monocular depth information and the stereo depth information 31 further includes the step S8 of validating, by the data processing device 17, the stereo depth information 31 by the first monocular depth information and / or the second monocular depth information. Furthermore, the validating step S8 includes the step S9 of comparing, by the data processing device 17, the stereo depth information 31 with the first monocular depth information and / or the second monocular depth information. Furthermore, the validating step S8 includes the step S10 of determining, by the data processing device 17, faults in the stereo depth information 31 based on the comparison of the stereo depth information 31 with the first monocular depth information and / or the second monocular depth information.

[0085] As is further depicted in Figure 1 and in detail in Figure 2, the method 100 further comprises the step S11 of providing a robot 1 controlling architecture 80 including a functional channel 27 and a testing channel 29. The step S3A of determining first monocular depth information based on the first image information could be assigned to the testing channel 29. Further, the step S3B of determining second monocular depth information based on the second image information could be assigned to the testing channel 29. Further, the step S8 of validating the stereo depth information 31 by the first monocular depth information and / or the second monocular depth information could be assigned to the testing channel 29. Further, the step S9 of determining faults in the stereo depth information 31 based on the comparison of the stereo depth information 31 with the first monocular depth information and / or the second monocular depth information could be assigned to the testing channel 29.

[0086] The method 100 further comprises the step S12 of comparing, by the data processing device 17, the first monocular depth information with the second monocular depth information. The step S12 could be assigned to the testing channel 29.

[0087] Further, the method 100 further comprises the step S13 of determining, by the data processing device 17, faults in the detection of the first image information and / or the second image information based on the comparison of the first monocular depth information with the second monocular depth information.

[0088] Further, the method 100 comprises the step S14 of determining kinematic information of the robot 1 . As depicted, the step S5 of determining control information for controlling the robot 1 could be further based on the kinematic information of the robot 1. The kinematic information may include one or more of a positional information, a stance information, a velocity information and / or an acceleration information of the robot 1 . The method 100 further comprises the step determining S15A a first detection time of the detection of the first image information. Alternatively, or additionally, the method 100 further comprises the step S15B of determining a second detection time of the detection of the second image information. Accordingly, as depicted in Figure 1 , the step S5 of determining control information for controlling the robot 1 could be further based on the first detection time and / or the second detection time.

[0089] As depicted in Fig. 2, the step S1A of detecting, by the first image sensor 21-1 , first image information of a first field of view 23-1 , and the step S1 B of detecting S1 B, by a second image sensor 21-2, second image information of a second field of view 23-2, could also be assigned to the testing channel 29. In other words, the steps S1A and S1 B may be assigned to both, the functional channel 27 and the testing channel 29.

[0090] Likewise, also the step S14 of determining kinematic information of the robot 1 may be assigned to both, the functional channel 27 and the testing channel 29.

[0091] Furthermore, also the step S15A of determining a first detection time of the detection of the first image information and / or the step S15B of determining a second detection time of the detection of the second image information may be assigned to both, the functional channel 27 and the testing channel 29.

[0092] The method 100 further comprises the step S16 of determining, by the data processing device 17, information about objects of interest 19 from the first image information and / or the second image information, wherein the step S5 of determining control information for controlling the robot 1 could be further based on the determined information about one or more objects of interest 19.

[0093] The method 100 further comprises the step S17 of detecting, by at least one additional sensor 25, additional sensor information, wherein the at least one additional sensor information may include one or more of LIDAR sensor information, RADAR sensor information, position sensor information and / or a temperature sensor information. As depicted in Figure 1 , the step S5 of determining S5 control information for controlling the robot 1 could be further based on the additional sensor information.

[0094] With particular reference to Figure 2, it is depicted that the functional channel 27 may comprise the reception of image information (cf. steps S1A and S1 B) from the first and second image sensors 21-1 and 21-2, respectively. The image information of both image sensors 21-1 , 21-2 is then forwarded to the data processing device 17, which accordingly determines in step S4 a combined stereo depth information 31 from the detected image information of the first and second image sensors 21-1 and 21-2, respectively. The stereo depth information 31 is then forwarded to the control device 7, wherein a respective movement of the robot 1 could be controlled (cf. step S7).

[0095] As is further depicted in Figure 2, the detection of image information (cf. steps S1 A and S1 B) from the first and second image sensors 21-1 , 21-2, respectively, could also be assigned to the testing channel 29. As depicted, first image information detected by a first image sensor (cf. step S1A) could be used to determine first monocular depth information (cf. step S3A). Likewise, second image information detected by a second image sensor (cf. step S1 B) could be used to determine second monocular depth information (cf. step S3B).

[0096] Based on the first monocular depth information and the second monocular depth information, the stereo depth information 31 could be validated (cf. step S8). This includes comparing the stereo depth information 31 with the first monocular depth information and / or the second monocular depth information (cf. step S9). In other words, a cross comparison between the stereo depth information and the respective monocular depth information could be performed. Based thereon, faults in the stereo depth information 31 may be determined (cf. step S10), which additional information may be accordingly supplied to and considered by the control device 7 when controlling the robot 1. The monocular depth information may only be used to correct and accordingly improve the stereo depth information provided to the control device 7.

[0097] Furthermore, additional information may be used to be supplied either to the stereo depth estimation (cf. step S4) and / or the respective monocular depth estimation (cf. steps S3A and S3B). Such additional information may be for instance kinematic information (cf. step S14) and / or time measurements of the first and second image acquisitions (cf. steps S15A and S15B). As is depicted in Figure 2, respective additional information and corresponding sensors for acquiring said additional information may be assigned to one or both of the functional channel 27 and the testing channel 29.

[0098] Figure 3 depicts a robot 1 according to an embodiment of the present invention. In the depicted embodiment, the robot 1 is a mobile manipulator, wherein the robot 1 comprises a robot base 3 and a robot arm 5 attached at the top side of robot base 3. It will be understood that in different embodiments, also more than one robot arm 5 may be provided or even no robot arm 5 may be provided such that the robot 1 only comprises a robot base 3. For instance, the robot 1 may be an autonomous vehicle.

[0099] The robot 1 may be controlled to move along a floor or ground 11 , as depicted by arrow 50-1 . In the depicted embodiment, the robot base 3 comprises wheels 9 to contact the floor 11 . However, in different embodiments different contact elements for moving the robot along the floor or ground 11 may be provided. The movement of the wheels 9 are controlled by a control device 7 arranged at the robot base 3 of the robot 1 . However, it will be understood that in different embodiments, the control device 7 can also be arranged at least partially outside of or remote from the robot 1 and could be operatively coupled to the robot 1 to enable a respective control thereof. The control device 7 may comprise suitable sensing, computing and controlling means for controlling the movable elements of the robot 1 , for instance a propelling speed, acceleration and a movement direction of the wheels.

[0100] The robot arm 5 may be formed as a movable arm comprising multiple arm elements which are linked together to allow a flexible movement and extension or retraction of the arm. However, it will be understood that in different environments also a stiff robot arm may be provided consisting for instance only of a single stiff arm element movably attached to the robot base 3. The robot arm 5 is hinged at a proximal end of the robot arm 5 with the base 3 and comprises an end effector 13 at a distal end of the robot arm 5. In the depicted embodiment, the end effector 13 is formed as a gripper to allow a grip of an external item or load. The orientations of the robot arm 5 may also be set and controlled by the control device 7 of the robot. The control device 7 may accordingly comprise suitable sensing, computing and controlling means for controlling a movement of the robot arm 5, as schematically depicted by the arrow 50-2. A respective movement of the robot arm 5, which may include for instance a translational or rotational movement, may be performed in any direction such as in a vertical direction, a horizontal direction or a combined movement in both directions, as desired.

[0101] Furthermore, as indicated by the arrow 50-3, also the movement of the end effector 13 may be accordingly set and controlled by the control device 7. A respective movement of the end effector 13, which may include for instance a translational or rotational movement, may be performed in any direction such as in a vertical direction, a horizontal direction or a combined movement in both directions, as desired. The control device 7 may be accordingly adapted to control any types of propelling motors and / or brakes or steering arrangements that allow to control a movement 50 of any movable or propellable parts of the robot 1 .

[0102] The robot 1 comprises a first image sensor 21-1 adapted to detect first image information of a first field of view 23-1 . Further, the robot 1 comprises a second image sensor 21-2 adapted to detect second image information of a second field of view 23-2. The image information could include for instance a two-dimensional image of an environment covered by the first field of view 23-1 and the second field of view 23-2. As schematically indicated, the first field of view 23-1 and the second field of view 23-2 at least partially overlap at a certain area 23-3. Accordingly, stereo depth information could be obtained from the images acquired by the first image sensor 21-1 and the second image sensor 21-2. Of course, also more than two image sensors could be provided.

[0103] The robot 1 comprises the data processing device 17. The data processing device 17 may be any suitable digital data processing means may include a corresponding processor and temporal and / or permanent data storage devices, such as random-access memory (RAM), read-only memory (ROM), or any other kind of volatile or non-volatile memory. The data processing device 17 may be operatively coupled to the control device 7, such that data, for instance control information could be transferred to and received from the control device 7. Furthermore, the data processing device 17 may be operatively coupled to one or more sensors 21 , 25 of the robot 1 , such that data information generated from the sensors 21 , 25 could be transferred to and received by the data processing device 17. The data processing device 17 is accordingly adapted to receive the first image information and the second image information from the first image sensor 21-1 and the second image sensor 21- 2 and to process the digital image information in a desired manner. For instance, the data processing device 17 could determine control information concerning the movement or change in the movement of the robot 1 and could provide the control information to the control device 7. The control device 7 is adapted to receive the respective control information and to control a movement of the robot 1 accordingly.

[0104] For instance, the first image sensor 21-1 and / or the second image sensor 21-2 may acquire images of an environment of the robot 1 , for instance an area in front of the robot 1 , as depicted. An object of interest 19, such as an obstacle, being arranged in the way of the robot 1 , may be accordingly determined by the data processing device 17 in the image data provided by the first image sensor 21-1 and / or the second image sensor 21-2. The data processing device 17 may accordingly determine suitable control information to be supplied to the control device 7. This may include for instance a change in the velocity of the robot 1 , e.g. via a deceleration or braking of the robot 1 , to avoid a collision with the obstacle. Or respective steering information may be provided to the control device 7, to avoid a collision with the obstacle. Similarly, any other movable part of the robot 1 , for instance the robot arm 5 or an end effector 13 may be controlled as desired, based on the first detected image information and the second detected image information.

[0105] The robot 1 further comprises an additional sensor 25 adapted to detect additional sensor information. The additional sensor 25 may be arranged at the same position as the first image sensor 21-1 and / or the second image sensor 21-2 or the additional sensor 25 may be arranged at a different position. As depicted in the embodiment shown in Figure 3, the additional sensor 25 may be arranged for instance at a top side of the robot 1 , whereas the first and second image sensors 21-1 , 21-2 are arranged at a front side of the robot 1. It is clear that the sensors may be generally provided also at completely different positions at the robot 1.

[0106] Also more than one additional sensor 25 may be provided, for instance an array of additional sensors. The one or more additional sensors 25 may include one or more of a LIDAR sensor, a RADAR sensor, a position sensor and / or a temperature sensor. However, also any other sensors using different technology to acquire environmental information, and in particular distance information, may be employed. The additional sensor 25 could also be an additional image sensor. The additional sensor 25 may have a respective field of view 23- 4, which may or may not overlap with the field(s) of view of the first image sensor 23-1 and / or the second image sensor 23-2.

[0107] Further, a computer-program product 60 and a computer-readable medium 70 are shown, each comprising instructions, which, when executed by a data processing device 17, cause the data processing device 17 to carry out and / or control the method of any embodiments of the present invention, in particular the method 100 as illustrated in Figures 1 and 2.

[0108] In the depicted embodiment, the computer-program product 60 and the computer- readable medium 70 are depicted as external elements, whereas the data processing device 17 is depicted as an internal element of the robot 1. However, it will be understood that in different embodiments, any of the data processing device 17, the computer-program product 60 and the computer-readable medium 70 may be at least partly integrated in the robot 1 or may be provided remotely, but operatively coupled to the robot 1 , e.g. via respective means for wired or wireless data transfer.

[0109] It will be understood that the method according to the present invention is not limited to the above noted order of method steps. Quite to the contrary, the method steps may be also provided in a different order and one or more of the above noted method steps may be removed or further method steps may be added, as desired.

[0110] Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed disclosure, from the study of the drawings, the disclosure, and the appended claims. In the claims the word “comprising” or “including” does not exclude other elements or steps and the indefinite article “a” or “an” does not exclude a plurality. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0111] LIST OF REFERENCE SIGNS

[0112] 1 robot

[0113] 3 robot base

[0114] 5 robot arm

[0115] 7 control device

[0116] 9 wheels

[0117] 11 ground

[0118] 13 end effector

[0119] 17 data processing device

[0120] 19 object of interest

[0121] 21-1 first image sensor

[0122] 21-2 second image sensor

[0123] 23-1 first image sensor field of view

[0124] 23-2 second image sensor field of view

[0125] 23-3 overlapping field of view

[0126] 23-4 additional sensor field of view

[0127] 25 additional sensor

[0128] 27 functional channel

[0129] 29 testing channel

[0130] 31 stereo depth information

[0131] 50 robot movement

[0132] 60 computer program product

[0133] 70 computer readable medium

[0134] 80 robot controlling architecture

[0135] 100 method

[0136] S1 to S17 method steps

Claims

CLAIMS1 . A method (100) for controlling a robot (1), comprising the steps of: detecting (S1 A), by a first image sensor (21-1), first image information of a first field of view (23-1), providing (S2A) the first image information to a data processing device (17), detecting (S1 B), by a second image sensor (21-2), second image information of a second field of view (23-2), wherein the first field of view (23-1) and the second field (23-2) of view at least partially overlap, providing (S2B) the second image information to the data processing device (17), determining (S3A), by the data processing device (17), first monocular depth information based on the first image information and / or determining (S3B), by the data processing device (17), second monocular depth information based on the second image information, determining (S4), by the data processing device (17), stereo depth information (31) based on the first image information and the second image information, determining (S5), by the data processing device (17), control information for controlling the robot (1) based on the first monocular depth information and / or the second monocular depth information, and the stereo depth information (31), providing (S6) the control information to a control device (7), and controlling (S7) a movement of the robot (1) by the control device (7) in accordance with the control information.

2. The method (100) according to the preceding claim, wherein the step (S5) of determining control information for controlling the robot (1) based on the first monocular depth information and / or the second monocular depth information, and the stereo depth information (31) further includes the steps of: validating (S8), by the data processing device (17), the stereo depth information (31) by the first monocular depth information and / or the second monocular depth information,wherein the validating step (S8) includes the step (S9) of comparing, by the data processing device (17), the stereo depth information (31) with the first monocular depth information and / or the second monocular depth information, and determining (S10), by the data processing device (17), faults in the stereo depth information (31) based on the comparison of the stereo depth information (31) with the first monocular depth information and / or the second monocular depth information.

3. The method (100) according to the preceding claim, further comprising the steps of: providing (S11) a robot (1) controlling architecture (80) including a functional channel (27) and a testing channel (29), wherein the steps of determining (S3A) first monocular depth information based on the first image information, determining (S3B) second monocular depth information based on the second image information, validating (S8) the stereo depth information (31) by the first monocular depth information and / or the second monocular depth information, and determining (S9) faults in the stereo depth information (31) based on the comparison of the stereo depth information (31) with the first monocular depth information and / or the second monocular depth information are assigned to the testing channel (29), and wherein the step (S4) of determining stereo depth information (31) is assigned to the functional channel (27).

4. The method (100) according to one of the preceding claims, further comprising the steps of: comparing (S12), by the data processing device (17), the first monocular depth information with the second monocular depth information, and determining (S13), by the data processing device (17), faults in the detection of the first image information and / or the second image information based on the comparison of the first monocular depth information with the second monocular depth information.

5. The method (100) according to one of the preceding claims, further comprising the step of: determining (S14) kinematic information of the robot (1), wherein the step of determining (S5) control information for controlling the robot (1) is further based on the kinematic information of the robot (1).

6. The method (100) according to the preceding claim, wherein the kinematic information includes one or more of a positional information, a stance information, a velocity information and / or an acceleration information of the robot (1).

7. The method (100) according to one of the preceding claims, further comprising the steps of: determining (S15A) a first detection time of the detection of the first image information and / or determining (S15B) a second detection time of the detection of the second image information, and wherein the step of determining (S5) control information for controlling the robot (1) is further based on the first detection time and / or the second detection time.

8. The method (100) according to one of the preceding claims, further comprising the step of: determining (S16), by the data processing device (17), information about one or more objects of interest (19) from the first image information and / or the second image information, wherein the step of determining (S5) control information for controlling the robot (1) is further based on the determined information about the one or more objects of interest (19).

9. The method (100) according to one of the preceding claims, further comprising the step of: detecting (S17), by at least one additional sensor (25), additional sensor information,wherein the at least one additional sensor information includes one or more of LIDAR sensor information, RADAR sensor information, position sensor information and / or a temperature sensor information, wherein the step of determining (S5) control information for controlling the robot (1) is further based on the additional sensor information.

10. A robot (1), comprising: a data processing device (17), a control device (7) for controlling a movement (50) of the robot (1), a first image sensor (21-1) adapted to detect first image information of a first field of view (23-1), a second image sensor (21-2) adapted to detect second image information of a second field of view (23-2), wherein the first field of view (23-1) and the second field of view (23-2) at least partially overlap (23-3), wherein the data processing device (17) is adapted to receive the first image information and the second image information, wherein the data processing device (17) is adapted to provide control information to the control device (7), wherein the robot (1) is adapted to be controlled according to the method (100) of one of the preceding claims 1 to 9.

11. The robot (1) according to the preceding claim, wherein the robot (1) comprises at least one additional sensor (25) adapted to detect additional sensor information, wherein the at least one additional sensor (25) includes one or more of a LIDAR sensor, a RADAR sensor, a position sensor and / or a temperature sensor.

12. The robot (1) according to one of the preceding claims 10 or 11 , wherein the robot (1) comprises a robot controlling architecture (80) including a functional channel (27) and a testing channel (29), and wherein the robot (1) is adapted to be controlled according to the method (100) of one of the preceding claims 3 to 9.

13. The robot (1) according to one of the preceding claims 10 to 12,wherein the robot (1) is compliant with category 2 according to the ISO 13849- 1 standard or compliant with a hardware fault tolerance of 0 according to the I EC 61508 standard.

14. A computer-program product (60) comprising instructions, which, when executed by a data processing device (17), cause the data processing device (17) to carry out and / or control at least partly the method (100) of any of claims 1 to 9.

15. A computer-readable medium (70) comprising instructions which, when executed by a data processing device (17), cause the data processing device (17) to carry out and / or control at least partly the method (100) of any of claims 1 to 9.

Images