MANIPULATOR SYSTEM AND METHOD FOR CONTROLLING A ROBOTIC MANIPULATOR

Description

[0001] The invention relates to a manipulator system and a method for controlling a robotic manipulator.

[0002] State-of-the-art applications exist in which a robot operates in close proximity to a human without being secured, for example, behind a fence. In this case, the human operator can give the robot instructions. This can be done, for example, via a computer, tablet, or voice input.

[0003] It's also possible to use the robot itself as a haptic input device. The robot is configured to yield slightly when pressure is applied, so that, for example, pressure against the end effector is interpreted as an input for a specific action.

[0004] The object of the invention is to provide a manipulator system that offers simplified operation. Furthermore, the object of the invention is to provide a method for simple control of a robotic manipulator.

[0005] The publication WOSCH T ET AL: "Reactive motion control for human-robot tactile interaction", Proceedings / 2002 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION: MAY 11-15,2002, WASHINGTON, DC, IEEE SERVICE CENTER, PISCATAWAY, NJ, Vol. 4, May 11, 2002 (2002-05-11), page 3807, XP032882734, DOI: 10.1109 / ROBOT.2002.1014313, ISBN: 978-0-7803-7272-6 discloses a manipulator system with a robotic manipulator and tactile sensors for detecting a force that is applied externally to the manipulator arm. A display is used to mark points on the manipulator where the robotic manipulator is calibrated by applying an external force.

[0006] WO 2013 / 156468 A1 describes a manipulator system with an augmented reality display on which, for example, an arrow can be displayed pointing in a direction in which the manipulator arm would have to be moved in order to achieve a greater reach of the desired target workspace.

[0007] The object is achieved according to the invention by the features of claims 1 and 6.

[0008] The manipulator system according to the invention comprises a robotic manipulator having sensors for detecting a force and / or torque applied externally to the manipulator. By applying a force and / or torque externally to the manipulator, input commands for the manipulator to operate the manipulator system can be generated, as is already known from the prior art.

[0009] According to the invention, the manipulator system comprises an augmented reality display device that displays locations on the manipulator arm where a command for the robotic manipulator can be generated by applying a force and / or torque from the outside. In other words, the interaction options with the manipulator are displayed at these locations on the manipulator, i.e., in direct spatial proximity to the manipulator.

[0010] This allows the user to operate the robotic manipulator or manipulator system very easily. For example, it is possible to display the available interaction options in a context-dependent manner, so that different interaction options are displayed on the manipulator arm depending on the situation or operating state of the manipulator system.

[0011] It is preferred that the augmented reality display device is augmented reality glasses.

[0012] Furthermore, it is preferred that a direction be indicated in which the manipulator must be moved to generate a specific input command. This can be done, for example, by displaying an arrow at a specific location on the robotic manipulator.

[0013] According to the invention, the command generated by applying a force and / or torque at the corresponding point on the manipulator is displayed at the corresponding point. This can be done, for example, using symbols, pictograms, or short texts. This allows the user to immediately and very intuitively recognize the effects of applying a force and / or torque at a specific point on the manipulator. This makes operating the manipulator system very intuitive, without the need for an operating manual.

[0014] In a preferred embodiment, the augmented reality glasses have markers so that the absolute position of the augmented reality glasses in space is determined by a camera tracking system. Such methods are known from the prior art. Based on this, the position of the augmented reality glasses relative to the robotic manipulator can be calculated. This makes it possible to calculate the position and orientation of the interaction options relative to the manipulator in the augmented reality display device.

[0015] The invention further relates to a method for controlling a robotic manipulator that is part of a manipulator system. This manipulator system has sensors for detecting a force and / or torque applied externally to the manipulator. By applying a force and / or torque externally to the manipulator arm, input commands for the robotic manipulator are generated to operate the manipulator system.

[0016] According to the invention, at least one location on the manipulator is displayed on an augmented reality display device at which the application of a force and / or torque from the outside generates a command for the robotic manipulator.

[0017] The method according to the invention can have all the features described in connection with the manipulator system and vice versa.

[0018] It is preferred that the absolute position of the augmented reality display device in space and the joint positions of the robotic manipulator are determined, based on which the position of the augmented reality glasses relative to the robotic manipulator is calculated.

[0019] Furthermore, it is preferred that the display on the manipulator arm on the augmented reality display device be dependent on the operating state of the manipulator system. It is also possible for the display to be context- or status-dependent. In other words, the display is dependent on specific contexts or states in which the manipulator system is located. The state of the manipulator system can also be dependent on the program being executed and its execution state.

[0020] For example, the manipulator system can have at least three operating states, namely a first, ready operating state in which a program is started by applying a force and / or torque to a specific point on the robotic manipulator. Such a program is typically a sequence of (motion) commands.

[0021] There is also a hand-guided state, in which the manipulator can be freely moved by the user's hand. This hand-guided state allows for corrections to be made during execution. For example, the manipulator can be guided around an obstacle and then allowed to continue the movement.

[0022] Furthermore, a collision state is provided in which the application of a force from a first direction is interpreted as a command to continue. If a collision occurs during program execution, the program flow is interrupted (for safety reasons). The application of a force (e.g., from above) signals that the collision is confirmed and the program flow can continue at the interruption point. The application of a force from a second direction can open a gripper of the robotic manipulator, while the application of a force from a third direction, opposite to the second direction, closes the gripper. The application of a force from a fourth direction can transfer the manipulator system to the hand-held state.

[0023] Furthermore, it is preferred that, in the hand-held state, a virtual plane is displayed on the augmented reality display device at a defined height above the robot base, in particular parallel to the floor on which the manipulator system is located. When the end effector of the manipulator is brought to the height of this plane, the manipulator is transferred to the collision state.

[0024] Furthermore, it is preferred that the system continuously evaluates whether the manipulator is within the user's field of vision. If this is the case, the operating state of the manipulator system is detected. Depending on this operating state, the possible interaction variants are displayed in the augmented reality display device.

[0025] Furthermore, it is preferred that the external forces acting on the manipulator and its joint configuration be evaluated and checked for compliance with a specific interaction variant. If such an interaction variant is detected, the associated action can be executed.

[0026] In the following, a preferred embodiment of the invention is explained with reference to a figure.

[0027] The figure shows a representation of a robotic manipulator 12. An interaction option in the form of an arrow is displayed on the left side of the manipulator 12. This arrow, along with the corresponding text and the play symbol, is displayed in the augmented reality display and indicates that pressing the end effector from above, for example, triggers the "Start" action, which launches a specific program. This allows the user to interact with the manipulator system very intuitively.

Claims

1. Manipulator system comprising a robotic manipulator (12), wherein the manipulator comprises sensors for detecting a force and / or a torque applied onto the manipulator from outside, wherein by applying a force and / or a torque on the manipulator (12) from outside, input instructions for the manipulator (12) can be generated for the operation of the manipulator system, wherein the manipulator system further comprises an augmented reality display device via which locations on the manipulator (12) are displayed at which an instruction for the robotic manipulator (12) is generated by the application of a force and / or a torque from outside, wherein it is displayed at a corresponding location of the manipulator (12), which instruction is generated at the corresponding location by the application of a force and / or a torque, so that the user directly and intuitively realizes which effects the application of a force and / or a torque at a particular location of the manipulator (12) will have.

2. Manipulator system of claim 1, characterized in that the augmented reality display device are augmented reality glasses.

3. Manipulator system of claim 1 or 2, characterized in that a direction is displayed in which the manipulator (12) has to be moved to generate a specific input instruction.

4. Manipulator system of claims 1-3, characterized in that it is displayed at the corresponding location of the manipulator (12) which instruction is generated by the application of a force and / or a torque at the corresponding location.

5. Manipulator system of claims 2-4, characterized in that the augmented reality glasses have markers, so that the absolute position of the augmented reality glasses in space is determined by means of a camera tracking system, wherein the position of the augmented reality glasses relative to the robotic manipulator is calculated based thereon.

6. Method for controlling a robotic manipulator (12) which is part of a manipulator system, wherein the manipulator comprises sensors for detecting a force and / or a torque applied on the manipulator from outside, wherein by applying a force and / or a torque onto the manipulator (12) from outside, input instructions for the manipulator (12) can be generated for the operation of the manipulator system, wherein the manipulator system further comprises an augmented reality display device via which locations on the manipulator (12) are displayed at which an instruction for the robotic manipulator (12) is generated by the application of a force and / or a torque from outside, wherein it is displayed at a corresponding location of the manipulator (12), which instruction is generated at the corresponding location by the application of a force and / or a torque, so that the user directly and intuitively realizes which effects the application of a force and / or a torque at a particular location of the manipulator (12) will have.

7. Method of claim 6, characterized in that the absolute position of the augmented realty display device in space and the joint positions of the robotic manipulator (12) are determined, wherein the position of the augmented reality glasses relative to the robotic manipulator (12) is calculated based thereon.

8. Method of claim 6 or 7, characterized in that the display on the augmented reality display device is shown depending on a context and / or state which the manipulator system is in.

9. Method of one of claims 6 to 8, characterized in that the display at the manipulator arm on the augmented reality display device is shown depending on the operating state of the manipulator system, wherein the manipulator system having in particular at least three operating states, namely a ready operating state in which a program is started by applying a force and / or a torque onto a specific location of the robotic manipulator (12), a manually guided state in which the robotic manipulator (12) can be guided freely by hand, and a collision state in which the application of a force from a first direction is interpreted as an instruction to continue, an application of a force from a second direction opens a gripper of the robotic manipulator (12), an application of a force from a third direction opposite to the second direction closes the gripper, an application of a force from a fourth direction transfers the manipulator system to the manually guided state.

10. Method of claim 9, characterized in that in the manually guided state, the augmented reality display device displays a virtual plane on a defined level above the robot base, in particular in parallel to a ground, wherein when the end effector of the robotic manipulator system (12) is moved to the level of this plane, the manipulator system is transferred to the collision state.

11. Method of claims 6-10, characterized in that it is continuously evaluated, whether the manipulator (12) is in the field of view of the user, and if so, the operating state of the manipulator system is detected and, depending on this operating state, the possible interaction variants are displayed on the augmented reality display devoice.