METHOD FOR OPERATING A ROBOT ARM

Description

[0001] The invention relates to a method for operating a robot arm.

[0002] Various methods for controlling or generating input commands for a robot arm are known from the prior art. For example, it is known to use a tactile sensor to detect gesture commands on the robot structure. Furthermore, the use of force sensors to detect input commands for robot arms is known.

[0003] Information on the state of the art can be found in the following publications: [1] Tenebaum, J.B., De Silva, V. and Langford, J.C., 2000. A global geometric framework for nonlinear dimensionality reduction. science, 290(5500), pp.2319-232 [2] Carbune, V., Gonnet, P., Deselaers, T., Rowley, H.A., Daryin, A., Calvo, M., Wang, L.L., Keysers, D., Feuz, S. and Gervais, P., 2020. Fast multi-language LSTM-based online handwriting recognition. International Journal on Document Analysis and Recognition (IJDAR), pp.1-14. [3] https: / / ai.googleblog.com / 2019 / 03 / rnn-based-handwriting-recognition-in.html [4] Kubus, D., Muxfeldt, A., Kissener, K., Haus, J. and Steil, J., 2017, September. Robust recognition of tactile gestures for intuitive robot programming and control. In 2017 IEEE / RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 1643-1650). IEEE. [5] Hanyu, R., Tsuji, T. and Abe, S., 2010, October. Command recognition based on haptic information for a robot arm. In 2010 IEEE / RSJ International Conference on Intelligent Robots and Systems (pp. 4662-4667). IEEE. [6] DE 10 2020 102 160 A1 [7] DE 10 2020 134 260 A1

[0004] The object of the invention is to provide a simplified method for operating a robot arm and in particular for entering an input command.

[0005] The object is achieved according to the invention by the features of claim 1.

[0006] The method according to the invention for operating a robot arm comprises the following steps: detecting an input from a user, which the user applies in the form of a symbol to the surface of the robot arm, by the following steps: Detecting a force and / or a torque in at least one joint by force and / or torque sensors that are already present in the robot joint, wherein the force and / or the torque is caused by the user's input. Deriving a trajectory of the contact point on the three-dimensional surface of the robot arm. Converting the three-dimensional trajectory of the contact point into a two-dimensional trajectory. Deriving a symbol from the two-dimensional trajectory. Deriving an input command for the robot arm from one or more symbols.

[0007] The method according to the invention makes it particularly easy to generate an input command for a robot arm. No separate input devices are required for this. It is therefore not necessary to provide a haptic input device on the robot structure. Instead, force and / or torque sensors, which are already present in the robot joint, are used to derive the input command. Information about the structure of the robot arm, which may be known, for example, from a CAD model, can be used for this purpose.

[0008] The method according to the invention makes it possible, in particular, to generate an input command for a robot arm without requiring a flat input surface. Rather, the input command can be entered at any location on the robot structure. The robot structure can be convex at the relevant location.

[0009] Furthermore, it is preferred that, in order to derive the point of contact, the normal force that the user applies to the input surface is detected by a force sensor, and that the torque about an axis parallel to the vertical axis of the input surface, which is generated by the user when entering the gesture command, is detected by a first torque sensor in the joint.

[0010] Furthermore, a second torque sensor in the joint detects the torque around the transverse axis of the input surface generated by the user when entering the gesture command. This detection makes it possible to precisely determine the point at which the user touches the input surface to generate the gesture command over time.

[0011] In all embodiments of the present invention, the user can touch the robot arm with his finger or with any other input device, for example a stylus, to enter the gesture command.

[0012] Recording the touch point on the surface of the robot arm over time means that the position at which the user touches the input surface is recorded at different times, so that it is possible to determine the movement of this touch point on the input surface over time.

[0013] It is further preferred that the surface structure and / or the structure of the robot arm is taken into account when deriving the trajectory of the contact point on the three-dimensional surface.

[0014] Furthermore, it is preferred that a global map of the surface of the robot arm is generated, which maps the three-dimensional surface of the robot arm to a two-dimensional surface.

[0015] Furthermore, it is preferred that the detected forces and / or torques are filtered to generate a more uniform trajectory.

[0016] Furthermore, it is preferred that a section-by-section approximation of the three-dimensional trajectory is carried out using splines or Bezier curves.

[0017] Furthermore, it is preferred that the conversion of the three-dimensional trajectory of the contact point into a two-dimensional trajectory comprises a non-linear dimensionality reduction and in particular manifold learning.

[0018] Furthermore, it is preferred that the recognition of the symbol is carried out using an artificial neural network, in particular a convolutional neural network or a recurrent neural network.

[0019] Furthermore, it is preferred that a recognized symbol triggers a gesture command.

[0020] In the following, preferred embodiments of the invention are explained with reference to figures.

[0021] They show: Figure 1: A three-dimensional trajectory Figure 2: The conversion of the three-dimensional trajectory into a two-dimensional trajectory Figure 3: The derivation of a symbol from the two-dimensional trajectory Figure 4: Another embodiment of the present invention

[0022] In Figure 1 A three-dimensional trajectory is shown, where the symbol "8" is written on the three-dimensional surface of the robot arm. In Figure 1 the corresponding force vector is shown at different positions along the three-dimensional trajectory.

[0023] The conversion of this three-dimensional trajectory into a two-dimensional trajectory is Figure 2 The symbol "8" is now visible in the two-dimensional trajectory.

[0024] This is done, for example, according to Figure 3derived from the two-dimensional trajectory. Different layers of machine learning methods can therefore be used. The first layer is used to convert the three-dimensional trajectory of the touch point into a two-dimensional trajectory. The second layer is used to classify the handwritten trajectory and derive a symbol from it. In addition, a third layer can be used to derive an input command for the robot based on the recognized symbol or several symbols. This can be the command to perform a specific movement. However, an input command can also be understood as the input of one or more parameters for the operation of the robot arm.

[0025] In Figure 4A further embodiment is shown that can be used in conjunction with the method according to the invention. Here, a plate is provided that can be attached, for example, automatically to the distal end of the robot arm instead of a tool (e.g., a gripper). This plate can be used as an additional input surface for recognizing handwritten input commands. It can have a flat surface or a curved surface. Figure 4Two exemplary trajectories are shown, namely the symbols "B" and "P1." "P1" could, for example, be a point in Cartesian space to which the robot arm is to move. To record the forces and moments applied by the user during handwritten input on this input surface, a force / torque sensor (at the TCP, Tool Center Point) of the robot arm can be attached, for example. Alternatively, it is possible to use a robot arm with force / torque detection redundancy. This applies to all embodiments of the present invention. The embodiment according to Figure 4 is advantageous because it is possible to enter commands without using an additional input surface on the robot arm.

[0026] In a further embodiment, it is possible to detect a symbol of different line thickness depending on the force applied by the user. If the user presses harder on the input surface, the line thickness of the symbol increases, and vice versa.

[0027] A further advantage of the method according to the invention is that the user does not have to enter symbols in a predetermined orientation. Rather, the user can enter symbols at any location on the robot arm in any orientation, for example, even upside down. These are converted and processed accordingly by the method according to the invention.

Claims

1. A method for operating a robot arm, comprising the following method steps: detecting a user's input, which the user applies to the surface of the robot arm in the form of a symbol, by the following steps: detecting a force and / or a torque in at least one joint by force and / or torque sensors that are anyway included in the robot joint, wherein the force and / or the torque is generated by the user's input, deriving a trajectory of the point of contact on the three-dimensional surface of the robot arm, transforming the three-dimensional trajectory of the point of contact into a two-dimensional trajectory, deriving a symbol from the two-dimensional trajectory, deriving an input command for the robot arm from one or more symbols.

2. The method for operating a robot arm according to claim 1, characterized in that, when deriving the trajectory of the point of contact on the three-dimensional surface, the surface structure and / or the structure of the robot arm is considered.

3. The method for operating a robot arm according to claim 1 or 2, characterized in that a global map of the surface of the robot arm is generated, which includes a mapping of the three-dimensional surface of the robot arm onto a two-dimensional surface.

4. The method for operating a robot arm according to any one of claims 1 to 3, characterized in that the detected forces and / or torques are filtered for generating a uniform trajectory.

5. The method for operating a robot arm according to any one of claims 1 to 4, characterized in that the three-dimensional trajectory is approximated section by section using splines or Bezier curves.

6. The method for operating a robot arm according to any one of claims 1 to 5, characterized in that the transfer of the three-dimensional trajectory of the point of contact into a two-dimensional trajectory comprises a non-linear dimensionality reduction and in particular manifold learning.

7. The method for operating a robot arm according to any one of claims 1 to 6, characterized in that the symbol is detected by using an artificial neural network, in particular a convolutional neutral network or a recurrent neural network.

8. The method for operating a robot arm according to any one of claims 1 to 7, characterized in that a detected symbol triggers a gesture command.