Method for operating a collaborative robot with a sensor on the head of a person and computer program

Abstract

Method for operating a collaborative robot (210) in an environment (100) in which a person (200) is located, using sensors (120, 121, 130, 132, 141, 143) present in the environment (100), of which at least one sensor (141) is arranged on a head of the person (200), wherein, using the at least one sensor (141) arranged on the head of the person (200), information relating to a position (P1) of at least one limb (201) of the person is acquired, wherein information relating to a position (P2) of at least one movable component (211) of the robot (210) is acquired using at least one selected one of the sensors (120, 121, 130, 132), wherein the information acquired by the at least one sensor (141) arranged on the head of the person (200) and the information (125) acquired by the at least one selected one of the sensors (120, 121, 130, 132) are transmitted to a computer (110), wherein, using the computer (110), positions (P1, P2) of the at least one limb (201) of the person and of the at least one movable component (211) of the robot are determined based on the information (125) detected and transmitted by the sensors, and wherein the at least one movable component (211) of the robot is controlled, if necessary, depending on the position (P1) of the at least one limb (201) of the person or on the positions (P1, P2) of the at least one limb (201) of the person and the at least one movable component (211) of the robot.

Description

[0001] The present invention relates to a method for operating a collaborative robot with a sensor on the head of a person and a computer program for carrying out the method. State of the art

[0002] In modern environments, especially factory environments such as production or assembly halls, more and more processes are being automated. Robots, for example, are being used for this purpose. This is also known as "Industry 4.0." Despite automation, it is still necessary for people to move around the environment and work or collaborate with the devices available there, such as the robots mentioned above. In the latter case, the robots are also referred to as collaborative robots.

[0003] To achieve this, it is desirable to obtain the most accurate information possible, particularly regarding the position of robots and people. Especially in the case of collaborative robots mentioned above, when a robot and a person work together, the positions of the person's limbs and moving components of the robot, as well as components of the workpiece and the wider environment, must also be taken into account. This is important, not least, to ensure safe operation of the robot for people.

[0004] In order to ensure the safety of people working with such robots, it is possible to provide touch sensors on the robots or their moving components to stop movement in the event of contact with a person.

[0005] US 2017 / 0 210 017 A1, DE 10 2015 215 407 A1, DE 103 45 743 A1, and US 2015 / 0 158 178 A1 disclose ways of operating robots in conjunction with people. US 2015 / 0 158 178 A1 specifically describes a system comprising a computer, a sensor, and a robot, where the robot may be controlled depending on people detected by the sensor. Disclosure of the invention

[0006] According to the invention, a method for operating a collaborative robot and a computer program for implementing the method are proposed, having the features of the independent patent claims. Advantageous embodiments are the subject of the dependent claims and the following description.

[0007] A method according to the invention serves to operate a collaborative robot in an environment in which a person is located, using sensors present in the environment, at least one of which sensor is arranged on the person's head. In particular, a factory environment such as a production or assembly hall comes into consideration as the environment. The at least one sensor arranged on the person's head can in particular be a device worn by the person on their head, preferably a helmet, a mask (or protective mask) or glasses, particularly preferably for use for augmented reality. The at least one sensor itself can in particular comprise a LIDAR sensor and / or a camera, preferably a stereo camera. LIDAR sensors in particular enable particularly precise and fast, three-dimensional detection of the immediate surroundings.

[0008] Using the at least one sensor arranged on the person's head, information relating to a position of at least one limb of the person is now recorded. Preferably, all of the person's limbs are recorded, i.e., arms, legs, hands, and feet; the person's torso can also be included. Furthermore, using at least one sensor, information relating to a position of at least one movable component of the robot is recorded. Here, too, it is preferred if, if possible, all movable components of the robot are recorded. It is also preferred if information relating to a position of one or more components of a workpiece (i.e., in particular, one that is being processed by the person and / or the robot) and / or objects and / or components of the environment is recorded.Here, the at least one sensor can be the at least one sensor on the person's head, but also (alternatively or additionally) one or more other sensors present in the environment. Sensors arranged on the robot and / or sensors located elsewhere in the environment can be used. These sensors also include cameras, in particular video cameras and stereo cameras, but also ultrasonic sensors, microphones, proximity sensors, RADAR units, LIDARs or LIDAR units, radio modules, in particular WLAN units and Bluetooth units, inertial sensors such as acceleration and / or yaw rate sensors, pressure sensors, torque sensors, and distance sensors.All these sensors are particularly common in environments and can be used to collect as much information as possible about the position of the people present and their limbs as well as at least one moving component of the robot.

[0009] The information acquired by the sensor(s) is then transmitted, preferably wirelessly, to a computer. Using the computer, the positions of at least one limb of the person and of at least one movable component of the robot are then determined based on the information acquired and transmitted by the sensor(s). These positions are then preferably also entered into an object map of the environment, in particular a 3D object map. Such a computer is preferably a server, in particular a so-called edge server, i.e. a very powerful computer that is preferably located in spatial proximity to the sensors. Determining the positions in real time is generally only possible with a powerful computer, in particular with very short communication times.

[0010] The at least one movable component of the robot is now controlled, if necessary, depending on the position of the at least one limb of the person and / or on the positions of the at least one limb of the person and the at least one movable component of the robot. It should be noted in this regard that the robot or its movable component can generally be controlled for its operation, either by the computer or by a separate control unit. However, the additionally known, very precise position of the at least one limb of the person now allows targeted and, in particular, preventative intervention in the control of the robot if, for example, a limb of the person comes too close to a movable component of the robot.

[0011] In this context, it is preferred, for example, if the control of the at least one movable component of the robot comprises moving the movable component more slowly than before, stopping it, or moving it in a different direction than before. This can prevent dangerous collisions between a person and the robot. In particular, this can occur when the position of the at least one limb of the person reaches a predetermined area, for example a predetermined area around the robot, or when the positions of the at least one limb of the person and the at least one movable component of the robot fall below a predetermined distance from one another. This means that control intervention can take place, in particular, before any possible contact between the person and the robot. Compared to the conventional way of using contact sensors, safety can thus be significantly increased.Nevertheless, contact sensors and / or proximity sensors can be used as a fallback solution to intervene in the control of the at least one moving component when necessary, as mentioned above.

[0012] Particularly when the at least one sensor is attached to a device for use with augmented reality, the possibility of worker guidance, as enabled by augmented reality—for example, the display of instructions and the like—and the safety of the person or worker can be combined particularly easily and effectively. In addition, other virtual objects can then also be displayed, for example, depending on the determined positions, in order to improve safety and / or worker guidance.

[0013] Furthermore, it is preferred if the control of the at least one movable component of the robot includes moving the movable component in response to a change in the position of at least one limb of the person. Thus, for example, the movable component of the robot can be used to follow the movement of a limb of the person, such as an arm or a hand. This makes it possible, in particular, to teach or train specific movements of the robot or its movable component without contact or direct guidance of the robot, and above all, very intuitively.

[0014] Advantageously, at least one additional sensor from the sensors present in the environment is used to capture information relating to a position of the at least one sensor arranged on the person's head, wherein the captured information is transmitted to the computer and used to determine the position of the at least one limb. This at least one additional sensor can be, in particular, an inertial sensor, which is preferably arranged on the person's head, and / or several radio sensors (for example WLAN units and / or Bluetooth units, which in particular enable triangulation), and / or a camera. In this way, the position of the sensor arranged on the head can be used as a reference, so that the position of the at least one limb of the person can be determined particularly accurately.

[0015] Preferably, a change in the position of at least one limb of the person is detected and made available for further processing.

[0016] For example, a text or similar message that the person writes in the air, or something they type on a virtual (e.g., a displayed) keyboard, can be captured and interpreted and further processed as an instruction or similar message, particularly for controlling the robot or its moving components. This is particularly easy and precise thanks to the use of the aforementioned sensor on the person's head.

[0017] A computing unit or a computer, in particular a server, is used with a collaborative robot and with sensors present therein, at least one of which is arranged on a person's head, with which information relating to a position of at least one limb of the person can be acquired, and wherein, using at least one selected one of the sensors, information relating to a position of at least one movable component of the robot can be acquired. The computer is then configured to carry out the steps executed by the computer in a method according to the invention.This includes, in particular, receiving the information acquired by the sensors, determining the positions of the limb and movable component, and controlling the robot directly (e.g., using the computer as the control unit) or indirectly (e.g., by transmitting signals and / or commands to a corresponding control unit) based on the position of the limb or the positions of the limb and movable component. For further advantageous embodiments and advantages, please refer to the explanations of the method.

[0018] A system comprising a computer, in particular a computer according to the invention, a plurality of sensors (at least one of which can be arranged on the head of a person) and a robot with a movable component, in particular the computer, is configured, in particular in terms of programming, to carry out a method according to the invention.

[0019] Implementing the method in the form of a computer program is also advantageous, as this results in particularly low costs, especially if an executing control unit is also used for other tasks and is therefore already present. Suitable storage media for the computer program include, in particular, magnetic, optical, and electrical storage devices, such as hard disks, flash memories, EEPROMs, DVDs, and others. Downloading a program via computer networks (Internet, intranet, etc.) is also possible.

[0020] Further advantages and embodiments of the invention will become apparent from the description and the accompanying drawings.

[0021] The invention is illustrated schematically in the drawing using an embodiment and is described below with reference to the drawing. Short description of the drawings Fig. 1 schematically shows an environment with a system in a preferred embodiment with which a method according to the invention can be carried out. Fig. 2 shows schematically a sequence of a method according to the invention in a preferred embodiment. Embodiment(s) of the invention

[0022] In Fig. Figure 1 schematically illustrates an environment 100 with a system in a preferred embodiment, with which a method according to the invention can be carried out. The environment can be, for example, a factory hall. The system comprises a computer 110, which is configured to carry out a preferred embodiment of the invention and which can in particular be a server, as well as various sensors 120, 121, 130, 132, and 140, by way of example.

[0023] The computer 110 here has a processor 111 and a memory 112, as well as a network connection 113, here in the form of a radio module, in particular WLAN or WiFi. The sensors designated 120 and 130 are, for example, cameras; the sensors designated 121 and 141 are LIDARs or LIDAR units (LIDAR stands for "Light Detection and Ranging"); the sensor designated 132 is a torque sensor; and the sensor designated 143 is an inertial sensor (e.g., an acceleration and / or yaw rate sensor).

[0024] Furthermore, a person 200 wearing a helmet 205, in particular a so-called "augmented reality helmet," and a collaborative robot 210 with a movable component 211 are now depicted in the environment 100. Of the above-mentioned sensors, sensors 120 and 121 are arranged or provided in the environment, sensors 130 and 132 are arranged or provided on the robot 210 or its movable component 211, and sensors 141 and 143 are arranged or provided on the helmet 205 and thus on the head of the person 200.

[0025] Using sensor 141, information relating to the position of a limb 201, here an arm, of person 210 can be captured and transmitted, for example, via WLAN to computer 110. Using sensor 143, information relating to the position of helmet 205 and thus also of sensor 130 can be captured and likewise transmitted to computer 110. Using the remaining sensors, in addition to additional information relating to the position of limb 201 of person 210, information relating to a position of movable component 211 of robot 210 can be captured and transmitted to computer 110.

[0026] In Fig. 2 schematically shows a sequence of a method according to the invention in a preferred embodiment. The individual sensors 120, 121, 130, 132, 141 and 143 can now receive information 150, in particular with respect to the positions of the limb 201 of the person 200 and the movable component 211 of the robot 210, as described with respect to Fig. 1, transferred to the computer 110.

[0027] Based on the information 150, the computer 110 can now determine a position P1 of the person's limb and a position P2 of the robot's movable component and enter them into an object map 160. This can be done continuously or quasi-continuously, or at specific time intervals.

[0028] Positions P1 and P2 can now, for example, be related to each other. If a distance D between these two positions P1 and P2 falls below a predefined distance threshold D', i.e., if the person, for example, has approached the robot's moving component with their arm to within this predefined distance D', the control of the robot or its moving component is actively intervened in such a way that this moving component is stopped or moved away from the arm, for example, in order to prevent possible injury to the person.

[0029] In addition, the position P1 of the limb, or its change, as mentioned above, can also be used to control the robot's moving component so that it follows the movement of the limb. This allows the robot to learn certain workflows very easily and intuitively.

[0030] In addition, as in Fig.As indicated in Figure 2, based on the determined positions, virtual objects can be superimposed into the person's field of vision using the augmented reality helmet 205. This allows, for example, the worker's guidance to be specified with particular precision depending on the current status of the robot.

Claims

[1] Method for operating a collaborative robot (210) in an environment (100) in which a person (200) is located, using sensors (120, 121, 130, 132, 141, 143) present in the environment (100), of which at least one sensor (141) is arranged on a head of the person (200), wherein, using the at least one sensor (141) arranged on the head of the person (200), information relating to a position (P1) of at least one limb (201) of the person is acquired, wherein information relating to a position (P2) of at least one movable component (211) of the robot (210) is acquired using at least one selected one of the sensors (120, 121, 130, 132), wherein the information acquired by the at least one sensor (141) arranged on the head of the person (200) and the information (125) acquired by the at least one selected one of the sensors (120, 121, 130, 132) are transmitted to a computer (110), wherein, using the computer (110), positions (P1, P2) of the at least one limb (201) of the person and of the at least one movable component (211) of the robot are determined based on the information (125) detected and transmitted by the sensors, and wherein the at least one movable component (211) of the robot is controlled, if necessary, depending on the position (P1) of the at least one limb (201) of the person or on the positions (P1, P2) of the at least one limb (201) of the person and the at least one movable component (211) of the robot. [2] Method according to claim 1, wherein, using the computer (110), the positions (P1, P2) of the at least one limb (201) of the person and of the at least one movable component (211) of the robot, determined on the basis of the information (125) detected and transmitted by the sensors, are entered into an object map (160) of the environment, in particular a 3D object map. [3] Method according to claim 1 or 2, wherein the control of the at least one movable component (211) of the robot comprises moving the movable component (211) more slowly than before, stopping it, or moving it in a different direction than before, in particular when the position (P1) of the at least one limb (201) of the person reaches a predetermined range, or when the positions (P1, P2) of the at least one limb (201) of the person and the at least one movable component (211) of the robot fall below a predetermined distance threshold (D') from one another. [4] Method according to one of the preceding claims, wherein the control of the at least one movable component (211) of the robot comprises moving the movable component (211) in dependence on a change in the position (P1) of the at least one limb (201) of the person. [5] Method according to one of the preceding claims, wherein the at least one sensor (141) arranged on the head of the person (200) is provided on a device (205) to be worn by the person (200) on their head, preferably a helmet, a mask or glasses, particularly preferably for use for augmented reality. [6] Method according to one of the preceding claims, wherein the at least one sensor (141) arranged on the head of the person (200) comprises a LIDAR sensor and / or a camera, preferably a stereo camera, and / or a RADAR sensor. [7] Method according to one of the preceding claims, wherein at least one further sensor (143) of the sensors present in the environment (100) is used to acquire information relating to a position of the at least one sensor (141) arranged on the head of the person (200), the acquired information being transmitted to the computer (110) and used to determine the position (P1) of the at least one limb (201). [8] Method according to claim 7, wherein the at least one further sensor (143) comprises an inertial sensor, preferably an acceleration and / or yaw rate sensor, which is preferably arranged on the head of the person (200), and / or a plurality of radio sensors, and / or a camera. [9] Method according to one of the preceding claims, wherein a change in the position (P1) of the at least one limb (201) of the person is detected and provided for further processing. [10] A computer program which causes a system comprising a computer (110), a plurality of sensors (120, 121, 130, 132, 141, 143) and a collaborative robot (210) with a movable component (211) to carry out a method according to any one of claims 1 to 9 when executed on the system, wherein the computer (110) is for use with the collaborative robot (210) in an environment (100) in which a person (200) is located and in which the sensors (120, 121, 130, 132, 141, 143) are present, of which at least one sensor (141) is arranged on a head of the person (200), wherein using the at least one sensor (141) arranged on the head of the person (200) information relating to a position (P1) of at least one limb (201) of the person can be acquired and using at least one selected one of the sensors (120, 121, 130, 132) information relating to a position (P2) of at least one movable component (211) of the robot (210) can be acquired, wherein the computer (110) is configured to to receive the information acquired by the at least one sensor (141) arranged on the head of the person (200) and the information (125) acquired by the at least one selected one of the sensors (120, 121, 130, 132), to determine positions (P1, P2) of the at least one limb (201) of the person and of the at least one movable component (211) of the robot based on the information (125) detected and transmitted by the sensors, and to control the at least one movable component (211) of the robot directly or indirectly as needed depending on the position (P1) of the at least one limb (201) of the person or on the positions (P1, P2) of the at least one limb (201) of the person and the at least one movable component (211) of the robot. Registered office: Stuttgart, Register court: Stuttgart District Court, HRB 14000;

Images