Method for calibrating an input device for controlling a robot

Abstract

A method for calibrating an input device (10) for controlling a robot, the input device being fastened to a link (110) of the robot (100), and the input device having an inertial measuring device (12) for determining an orientation of the input device relative to a direction of gravity (g), comprises the steps of: moving (S20) the robot link into a plurality of measurement poses relative to a reference coordinate system; providing (S30), for each measurement pose, an orientation of the input device relative to the direction of gravity, said orientation being determined by means of the inertial measuring device of the input device fastened to the robot link, and an orientation of the robot link relative to the reference coordinate system; and calibrating (S50) a transformation between a coordinate system (D) fixed to the input device and a coordinate system (R) fixed to the robot link, on the basis of the provided orientations of the input device and of the robot link. The invention additionally relates to a method for controlling a robot (100) using the calibrated input device and to a system or computer program (product) for carrying out a method according to the invention.

Description

[0001] 2024P00033 WO 1 / 19 KUKA Deutschland GmbH

[0002] Description

[0003] Input device for controlling a robot

[0004] The present invention relates to a method and system for calibrating an input device attached to a limb of a robot for controlling the robot, the calibrated input device, a method and system for controlling a robot using the calibrated input device attached to a limb of the robot, and a computer program or computer program product for carrying out a method described herein.

[0005] One object of an embodiment of the present invention is to improve the control of a robot using an input device attached to a part of the robot or the calibration of the input device.

[0006] This problem is solved in particular by a method for calibrating the input device attached to a segment of a robot, comprising the features of claim 1. Claims 6, 7, 9-11 protect an input device calibrated according to a method described herein, a method for controlling a robot using an input device attached to a segment of the robot and calibrated according to a method described herein, a system, and a computer program or computer program product for carrying out a method described herein. The dependent claims relate to advantageous embodiments.

[0007] According to one embodiment of the present invention, a method for calibrating an input device attached to a limb of a robot (“robot limb”) for controlling the robot, which includes an inertial measuring device for determining an orientation of the input device relative to a direction of gravity, comprises the following steps:

[0008] - Adjusting the robot element into several measuring poses (i = 1, 2, ... n) relative to a reference coordinate system, preferably tilted relative to each other at least about one axis not aligned with the direction of gravity; 2024P00033 WO 2 / 19 KUKA Deutschland GmbH

[0009] - Providing an orientation of the input device relative to the direction of gravity for the respective measuring pose, wherein this orientation (in each case) is determined using the inertial measuring device of the input device attached to the robot limb, in a further development;

[0010] - Providing an orientation of the robot component relative to the reference coordinate system; and

[0011] - Calibration, in particular parameterization, of a transformation, in particular numerical, between an input device-fixed coordinate system and a coordinate system fixed relative to the robot element ("robot element-fixed") based on the provided orientations of the input device and robot element.

[0012] In one embodiment, the robot has a robot arm with at least three, in particular at least six, and in another embodiment at least seven, axes or joints adjustable by drives (of the robot), in a further embodiment rotary axes or joints, and / or a base, in a further embodiment stationary or mobile, and in a further embodiment a chassis. The present invention is particularly advantageous for such robots due to their operating conditions.

[0013] In one embodiment, in a further development, the input device for calibration and / or control is temporarily and / or, preferably non-destructively, detachably attached to the robot's limb.

[0014] - before calibration and / or (especially also) before control; and / or, preferably only or without intermediate loosening,

[0015] - after control and / or (especially also) after calibration (again, preferably non-destructively) released or removed.

[0016] Accordingly, a method according to the invention for calibrating the input device can comprise the initial attachment and, optionally only after controlling the robot according to the invention, the final release of the input device, and / or a method according to the invention for controlling the robot can comprise the initial attachment and final release of the 2024P00033 WO 3 / 19 KUKA Deutschland GmbH

[0017] The input device comprises a control method according to the invention, wherein the control method according to the invention may include a calibration method according to the invention for the input device and thus an initial attachment of the input device before adjusting the robot element to the measuring poses and the final release of the input device after calibration of the input device and control of the robot using the calibrated input device. This allows it to be advantageously used successively for several robots. In one embodiment, the robot element is a distal and / or structural element, and in a further embodiment, an end element or tool attachment element or flange of the robot. This allows the element or the tool arranged on the element to be guided particularly advantageously by hand using the input device.

[0018] The inertial measurement device can, in particular, comprise one or more IMUs (“Inertial Measurement Unit”) and / or one or more sensors, wherein the inertial measurement device, IMU, or sensor(s) measure, or are configured or used for, the direction of gravity or the direction of the gravity vector, preferably at rest or without acceleration of the inertial measurement device. The direction of gravity is hereby designated by g without loss of generality and can, in particular, comprise a vector in the input device-fixed coordinate system, especially its (og).The orientation of the input device relative to the direction of gravity for a measurement pose i is accordingly designated by og without loss of generality and can in particular comprise a vector, preferably scaled to a unit length, which points in the direction of gravity in the coordinate system fixed to the input device, in particular be

[0019] The reference coordinate system is preferably a coordinate or inertial system in which the direction of gravity is known, preferably constant, and / or a robot-base-fixed, or in one embodiment, environment-fixed, coordinate system; it is designated by W without loss of generality. 2024P00033 WO 4 / 19 KUKA Deutschland GmbH

[0020] The orientation of the robot element relative to the reference coordinate system is described here without loss of generality by W7R denotes and can in particular comprise a transformation matrix, in particular be that which transforms the robot-link fixed coordinate system (without loss of generality denoted by R) into the reference coordinate system kl / or (in particular as [ R Tw]' 1 ) the reference coordinate system is converted into the robot-limb-fixed coordinate system, or one of the two coordinate systems is rotated so that the respective axes of both coordinate systems are parallel to each other. In one embodiment, and in a further development, the orientation of the robot limb relative to the reference coordinate system for the respective measurement pose i is determined based on joint coordinates of the robot determined for this measurement pose and a numerical kinematic model of the robot, for example in the form of a corresponding transformation matrix ( w R (qi)).

[0021] Providing an orientation can, in particular, include determining, in a further development, sensorially measuring, and / or receiving and / or retrieving corresponding data, in particular, providing an orientation of the input device relative to the direction of gravity determined using the inertial measuring device of the input device attached to the robot limb, determining this orientation using the inertial measuring device, and / or providing an orientation of the robot limb relative to the reference coordinate system, determining this orientation based on joint coordinates of the robot determined for the respective measurement pose and a numerical kinematic model of the robot, in particular.

[0022] The transformation between input device-fixed and robot-limb-fixed coordinate systems is described here without loss of generality using R7D and may in particular comprise a transformation matrix, in particular be that which transforms the input device fixed coordinate system (without loss of generality denoted by D) into the robot link fixed coordinate system or in particular as [ R 7b]' 1 ) the robot link-resistant 2024P00033 WO 5 / 19 KUKA Deutschland GmbH

[0023] The coordinate system is transformed into the input device-fixed coordinate system, or one of the two coordinate systems is rotated so that the respective axes of both coordinate systems are parallel to each other. Calibrating this transformation can, in particular, include determining a corresponding transformation matrix. Calibrating the input device can, in particular, include calibrating a transformation between the input device-fixed and robot-limb-fixed coordinate systems.

[0024] The idea underlying the invention can thus be explained without limitation of generality as follows:

[0025] If the orientation is the same for different measuring positions i w If the TR(g) of the robot element relative to the reference coordinate system and the orientation D& of the input device relative to a gravitational direction and the gravitational direction in the reference coordinate system (without loss of generality denoted by wg) are provided using the inertial measuring device, then by minimizing the error between the transformation R 7b the orientation Dg of the input device transformed into the reference system and the direction of gravity wg, preferably scaled to the same length, in the reference coordinate system, in one embodiment a corresponding angle, via the several measuring poses the desired transformation R7D can be determined or calibrated, without restriction of generality, for example in the form: determine R 7D so that Li (error / angle between wg and w 7-R( i)- R 7b-Dgi) 2 i becomes minimal.

[0026] Of course, the expert can implement various deviations here, for example using the magnitude instead of the error or angle square and / or the reverse transformation directions, i.e., wg with [ R 7b]' 1 -[ w TR(qi)]' 1 Transform into the input device-fixed coordinate system or the like. 2024P00033 WO 6 / 19 KUKA Deutschland GmbH

[0027] Accordingly, in one implementation, the transformation is calibrated based on error minimization via the orientations provided for the multiple measurement poses.

[0028] With this approach or method, an input device can be calibrated particularly advantageously, especially more precisely, easily, reliably and / or quickly, and thus a robot can be controlled better, especially more precisely, easily and / or reliably, using an input device attached to a part of the robot.

[0029] The transformation can, in a manner known per se, for example of Euler or Cardan angles, comprise a series of three rotations or (corresponding) transformation matrices, in particular, again without restriction, be X(y)-Y(β)-Z(a) with the rotation Z about the Z-axis by the angle a, the (subsequent) rotation V about the Y-axis by the angle β, and the (subsequent) rotation X about the X-axis by the angle y. The gravitational direction is, for example, the negative Z-direction (here, too, it is clear to those skilled in the art that various deviations are possible, for example, other rotations or rotation sequences, a different orientation relative to the gravitational direction, or the like).

[0030] In one embodiment, the transformation is calibrated based on virtual reorientations of the input-device-fixed coordinate system based on the direction of gravity, preferably virtual reorientations of the input-device-fixed coordinate system such that it has a predefined orientation to the direction of gravity, for example, its negative Z-direction or axis is parallel to the direction of gravity. Such a virtual reorientation is denoted, without loss of generality, by 7j|, and can thus, in particular, denote the rotation or transformation matrix that rotates a provided gravity vector in the input-device-fixed coordinate system, purely by way of example, such that it is or becomes antiparallel to the Z-axis of the input-device-fixed coordinate system (Tw-og = [0, 0, - 2024P00033 WO 7 / 19 KUKA Deutschland GmbH).

[0031] 1] TThis allows the virtual reorientations to be explained as follows, without limiting generality:

[0032] The aforementioned sequence of three rotations is supplemented by a preceding virtual reorientation and a subsequent, preferably compensating or opposite, virtual reorientation – in intuitive terms, the gravitational vector in the input-device-fixed coordinate system is first rotated so that it has a predetermined orientation to the direction of gravity, for example, antiparallel to the Z-axis of the input-device-fixed coordinate system, where 7|| comprises parameters of the transformation to be calibrated; then it is rotated around the Z-axis, where this rotation angle a comprises another parameter of the transformation to be calibrated; then around the Y-axis and the X-axis, where these two further rotation angles β, y are theoretically equal to zero; and then rotated back again.

[0033] R TD = [7||]- 1 -X(y).y(ß).Z( a ).7|| (1 )

[0034] This makes it possible to use distributions assumed for β, y in one implementation, in particular informed priors, for example Gaussian distributions with a mean of zero, and thus to advantageously take measurement noise into account.

[0035] Accordingly, in one implementation the transformation is calibrated based on an assumed, preferably stochastic or statistical, distribution, in a further development a Gaussian distribution or an informed prior, for measurement errors.

[0036] The input device comprises, in one embodiment, a movable input element, preferably manually mounted, with at least three degrees of freedom; in a further embodiment, with more than three degrees of freedom; and in a further embodiment, with six degrees of freedom. It may, in particular, be a so-called 6D mouse or the like. The present invention, KUKA Deutschland GmbH (2024P00033 WO 8 / 19), is particularly advantageous for such input devices, especially due to its functionality and kinematics.

[0037] According to one embodiment of the present invention, a method for controlling a robot using an input device attached to a part of the robot, which is an input device calibrated (according to a method described herein), comprises the following steps:

[0038] - Determining input data based on an actuation of the input device, in a further development of its input element;

[0039] - Transforming the input data into a command coordinate system based on the calibrated transformation; and

[0040] - Controlling the robot based on the transformed input data.

[0041] In one embodiment, the method for controlling the robot may include a method for calibrating the input device or its steps, as described here.

[0042] In one embodiment, the input data includes target values ​​related to the input device's fixed coordinate system. In a further development, the input data includes movements of the input device or the robot component attached to it, particularly translational movements (components) in the direction of the coordinate axes of the input device's fixed coordinate system and / or rotational movements (components) about coordinate axes of the input device's fixed coordinate system, or the like. By transforming such input data into the command coordinate system, a robot controller that controls the robot based on target values ​​related to the command coordinate system can advantageously implement these values. For the sake of clarity, this control is also referred to as "controlling" within the meaning of the present invention.

[0043] The command coordinate system is, in one implementation, the reference coordinate system, whereby the input data is then further processed by multiplication with the transformation matrices. w 7R(qi)- R 7b into the command or 2024P00033 WO 9 / 19 KUKA Deutschland GmbH

[0044] The reference coordinate system W can be transformed. In another embodiment, the command coordinate system is the robot-link-fixed coordinate system, where the input data is then further processed by multiplication with the transformation matrix. R 7b can be transformed into the robot-limb-fixed or command coordinate system R. Of course, other command coordinate systems or transformations can also be used.

[0045] According to one embodiment of the present invention, a system for calibrating an input device for controlling a robot, in particular hardware and / or software, especially programming technology, is set up and / or comprises a method described herein, in particular according to one of claims 1-5:

[0046] - the input device which includes the inertial measuring device for determining the orientation of the input device relative to a direction of gravity;

[0047] - Means for providing an orientation of the input device relative to the direction of gravity for several measuring poses of the robot element relative to a reference coordinate system, determined by means of the inertial measuring device of the input device attached to the robot element, wherein these means may in particular be implemented partially or completely in the input device or a robot controller or a device connected thereto with a signal;

[0048] - Means for providing (each) an orientation of the robot element relative to the reference coordinate system for the multiple measuring poses, wherein these means may in particular be implemented partially or completely in a robot controller or a device connected thereto with a signal; and

[0049] - Means for calibrating a transformation between an input device-fixed coordinate system and a robot-link-fixed coordinate system based on the provided orientations of the input device and robot link, wherein these means may be implemented, in particular partially or completely, in a robot controller or a signal-connected device thereto. 2024P00033 WO 10 / 19 KUKA Deutschland GmbH

[0050] According to one embodiment of the present invention, a system for controlling a robot using an input device attached to a part of the robot, in particular by means of hardware and / or software, in particular by means of programming, is set up and / or comprises:

[0051] - the input device, which has the inertial measuring device for determining an orientation of the input device relative to a direction of gravity and is preferably (non-destructively) detachably attached to the robot element and is calibrated or is calibrated in an embodiment according to a method described herein;

[0052] - Means for transforming input data determined on the basis of an actuation of the input device into a command coordinate system based on the calibrated transformation, wherein these means may in particular be implemented partially or completely in the input device or a robot controller or a signal-connected device thereto; and

[0053] - Means, in particular a robot controller, for controlling the robot based on the transformed input data.

[0054] A means according to the present invention can be configured as hardware and / or software, in particular comprising at least one processing unit, preferably a microprocessor unit (CPU), graphics processing unit (GPU), or the like, preferably connected to a storage and / or bus system via data or signals, and / or comprising one or more programs or program modules. The processing unit can be configured to execute instructions implemented as a program stored in a storage system, to acquire input signals from a data bus, and / or to output signals to a data bus. A storage system can comprise one or more, in particular different, storage media, in particular optical, magnetic, solid-state, and / or other non-volatile media. The program can be configured to embody the methods described herein.is capable of executing, so that the processing unit can perform the steps of such procedures 2024P00033 WO 11 / 19 KUKA Deutschland GmbH and thus, in particular, calibrate an input device or control a robot. A computer program product may, in one embodiment, have a storage medium, in particular a computer-readable and / or non-volatile medium, for storing a program or instructions, or with a program or instructions stored thereon. In one embodiment, the execution of this program or these instructions by a system or a controller, in particular a computer or an arrangement of several computers, causes the system or the controller, in particular the computer(s), to execute a procedure described herein or one or more of its steps, or the program or instructions are configured for this purpose.

[0055] In one embodiment, the system or its means includes means for calibrating the transformation based on error minimization via the orientations provided for the multiple measurement poses and / or based on virtual reorientations of the input device-fixed coordinate system based on the direction of gravity and / or based on an assumed distribution of measurement errors.

[0056] In one embodiment, one or more, in particular all, steps of the procedure are fully or partially computer-implemented, or one or more, in particular all, steps of the procedure are fully or partially automated, in particular by the system or its means.

[0057] In one embodiment, a system according to the invention for calibrating an input device for controlling a robot or a system according to the invention for controlling a robot comprises the robot.

[0058] Further advantages and features will become apparent from the dependent claims and the exemplary embodiments. These are shown, in part schematically:

[0059] Fig. 1: A system for controlling a robot using an input device attached to a limb of the robot and for calibrating the input device according to an embodiment of the present invention; and 2024P00033 WO 12 / 19 KUKA Deutschland GmbH

[0060] Fig. 2: a method for calibrating the input device attached to the robot element and for controlling the robot using the calibrated input device according to an embodiment of the present invention.

[0061] Fig. 1 shows a system for controlling a robot 100 using an input device 10 attached to a distal end element 110 of the robot 100 and for calibrating the input device 10 according to an embodiment of the present invention.

[0062] The input device 10 has a 6D movable input element 11 and is detachably or temporarily attached to the end member 110 of the robot 100 in a step S10 (see Fig. 2) for calibration and control. Reference is also made to the KUKA ready2_pilot technology or 6D mouse, which is known per se.

[0063] The robot 100 moves the robot end element 110 successively in one step S20 to one of several measuring poses i (i= 1 , 2, ... ).

[0064] According to the invention, the input device 10 has an inertial measuring device 12 for determining an orientation of the input device relative to a gravitational direction, which (for this purpose) determines the gravitational (unit) vector g in the respective measuring pose i in a coordinate system D fixed to the input device (step S30).

[0065] Robot 100 and input device 10 are signal-connected to a robot controller 200, which, for the respective measuring pose i, in step S30, derives the orientation of a robot-limb-fixed coordinate system R relative to a robot-base-fixed reference coordinate system W in the form of a corresponding transformation matrix from the joint coordinates, in the exemplary embodiment joint angles. w TR( ) determined.

[0066] In the robot control 200, integrated or signal-connected computer means 210 determine, after passing through (S40: “Y”) all measuring positions i in one step S50 by minimizing the sum Li (error / angle 2024P00033 WO 13 / 19 KUKA Deutschland GmbH between wg and w 7R(qi)- R 7b-Dg) 2 i the transformation matrix RTb, where wg denotes the gravitational vector in the reference coordinate system Wund D denotes the gravitational vector in the input device-fixed coordinate system D for the respective measurement pose i (in the embodiment of Fig. 1, for example, wg = [0, 0, -1 ] T and D = [-1 / 2, 0, -1 / ^2] T and 7|| = [[1 / 2, 0, 1 / 2] T , [0, 1 , 0] T , [-1A / 2, 0, 1 / 2] T ] be). The transformation matrix R 7b can in particular be parameterized according to equation (1 ), where assumed distributions can be used for ß, y.

[0067] After this calibration of the input device or transformation R 7b are used to control the robot 100, for example according to the aforementioned KUKA ready2_pilot technology, based on an actuation of the input element 11 of the input device, input data in the form of target movements of the robot end member 110 are determined (step S60), by multiplication withw 7R(qi)- R 7b is transformed into the Cartesian workspace or the reference or command coordinate system_W (step S70) and the robot is controlled to implement these target movements (step S80).

[0068] After this control, the input device 10 can be detached from the robot element 110 (step S90) in order to use it with another robot.

[0069] In the present disclosure, "has an X" does not generally imply an exhaustive list, but is a shorthand for "has at least one X" and also includes "has two or more X" as well as "has Y in addition to X". Although exemplary implementations were explained in the preceding description, it should be noted that a multitude of variations are possible. Furthermore, it should be noted that the exemplary implementations are merely examples and are not intended to limit the scope of protection, applications, or structure in any way.Rather, the preceding description provides the skilled person with a guide for the implementation of at least one exemplary embodiment, whereby various modifications, in particular with regard to the function and arrangement of the described components, can be made, 2024P00033 WO 14 / 19 KUKA Deutschland GmbH without leaving the scope of protection as it results from the claims and these equivalent combinations of features.

[0070] 2024P00033 WO 15 / 19 KUKA Deutschland GmbH

[0071] List of reference signs

[0072] 10 Input device

[0073] 11 movable input element 12 inertial measuring device

[0074] 100 robots

[0075] 110 robot link

[0076] 200 robot controllers

[0077] 210 computer tools

[0078] D input device-fixed coordinate system g Gravitation vector / direction

[0079] R robot-link fixed coordinate system Reference / command coordinate system

Claims

2024P00033 WO 16 / 19 KUKA Deutschland GmbH Patent claims 1. A method for calibrating an input device (10) attached to a link (110) of a robot (100) for controlling the robot, wherein the input device has an inertial measuring device (12) for determining an orientation of the input device relative to a gravitational direction (g); wherein the method comprises the steps: - Adjusting (S20) the robot element into several measuring poses relative to a reference coordinate system; - Deploy (S30) - an orientation of the input device relative to the direction of gravity, determined using the inertial measuring device of the input device attached to the robot limb and - an orientation of the robot limb relative to the reference coordinate system for the respective measuring pose; and - Calibration (S50) of a transformation between an input device fixed coordinate system (D) and a robot limb fixed coordinate system (R) based on the provided orientations of the input device and robot limb.

2. Method according to claim 1, characterized in that the transformation is calibrated on the basis of error minimization over the orientations provided for the multiple measuring poses.

3. Method according to one of the preceding claims, characterized in that the transformation is calibrated on the basis of virtual reorientations of the input device-fixed coordinate system on the basis of the direction of gravity. 2024P00033 WO 17 / 19 KUKA Deutschland GmbH 4. Method according to one of the preceding claims, characterized in that the transformation is calibrated on the basis of an assumed distribution of measurement errors.

5. Method according to one of the preceding claims, characterized in that the input device has a movable input element (11) with at least three degrees of freedom.

6. Input device (10) for controlling a robot (100), which has an inertial measuring device (12) for determining an orientation of the input device relative to a gravitational direction (g) and is calibrated according to a method according to one of the preceding claims.

7. Method for controlling a robot (100) using an input device (10) attached to a limb (110) of the robot according to the preceding claim, wherein the method comprises the steps: - Determining (S60) input data based on an operation of the input device; - Transforming (S70) the input data into a command coordinate system based on the calibrated transformation; and - Controlling (S80) the robot based on the transformed input data.

8. Method according to one of the preceding claims, characterized in that the input device is detachably attached to the robot element (S10) before calibration and / or control and / or is detached from the robot element (S90) after calibration and / or control.

9. System for calibrating an input device (10) for controlling a robot (100), which is configured and / or comprises a method according to one of the preceding claims: - the input device (10) which has an inertial measuring device (12) for determining an orientation of the input device relative to a gravitational direction (g); 2024P00033 WO 18 / 19 KUKA Deutschland GmbH Means to provide - an orientation of the input device relative to the direction of gravity, determined using the inertial measuring device of the input device attached to the robot limb and - an orientation of the robot element relative to the reference coordinate system for multiple measuring poses of the robot element relative to a reference coordinate system; and - Means for calibrating a transformation between an input device-fixed coordinate system (D) and a robot limb-fixed coordinate system (R) based on the provided orientations of the input device and robot limb.

10. System for controlling a robot (100) using an input device (10) attached to a limb (110) of the robot, wherein the system is configured and / or comprises a method according to one of the preceding claims: - the input device (10) attached to the robot limb, calibrated according to a method according to one of the preceding claims; - Means for transforming input data determined on the basis of an actuation of the input device into a command coordinate system based on the calibrated transformation; and - Means of controlling the robot based on the transformed input data.

11. Computer program or computer program product, wherein the computer program or computer program product, in particular stored on a computer-readable and / or non-volatile storage medium, contains instructions which, when executed by one or more computers or a system according to claim 9 or 10, cause the computer(s) or the system to execute a method according to one of the claims. 1 to 5 and / or one of claims 7 to 8.

Images