Device and method for training a machine learning model to control a robot
By training a machine learning model with simulated data that reflects real-world uncertainties in camera and object poses, the method enhances robot control precision and flexibility in handling objects with varying orientations.
Patent Information
- Application Number
- DE102021202759
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-03-22
- Publication Date
- 2024-11-07
- Estimated Expiration
- 2041-03-22
AI Technical Summary
Existing robot control systems struggle to accurately handle objects in varying poses within a workspace due to uncertainties in camera calibration, leading to inconsistent and imprecise control parameters.
A method for training a machine learning model that accounts for uncertainties in camera and object poses by generating simulated training data using camera and object pose uncertainties, incorporating calibration parameters and physically realistic variations to enhance the model's prediction accuracy.
The approach enables precise and robust control parameter predictions, allowing the robot to handle objects consistently across different poses and orientations, improving manufacturing flexibility and accuracy.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[0001] Various embodiments generally relate to a device and a method for training a machine learning model for controlling a robot. Aspects of systems for controlling robots are known from DE 10 2020 114 583 A1, DE 10 2019 122 790 A1, DE 10 2019 121 889 B3, DE 10 2019 002 065 A1, DE 10 2019 206 444 A1, and DE 20 2018 103 922 U1.
[0002] In order to enable flexible production or processing of objects by a robot, it is desirable for the robot to be able to handle an object regardless of the pose in which the object is placed in the robot's workspace. Therefore, a control device should be able to correctly generate control parameters for the robot for the respective pose (i.e. position and orientation) of the object, so that the robot can, for example, grasp the object at the correct location in order to attach it to another object, or treat the object in some other way (welding, painting, etc.) at the correct location. This means that the control device should be able to generate suitable control parameters from sensor information that reflects the pose of the object, for example from one or more camera images recorded by a camera attached to or near the robot.
[0003] According to various embodiments, a method for training a machine learning model for controlling a robot is provided, comprising determining a camera pose in a robot cell and an uncertainty region around the determined camera pose, determining an object region in the robot cell that contains positions for an object to be handled by the robot, generating training camera images, wherein for each training camera image, a training camera image camera pose is randomly determined in the uncertainty region around the determined camera pose, a training camera image object pose is randomly determined in the object region, and the training camera image is generated such that it shows the object with the training camera image object pose from the viewpoint of a camera with the training camera image camera pose, generating training data from the training camera images,wherein each training camera image is assigned one or more training robot control parameters for handling the object in the training camera image object pose of the training camera image, and training the machine learning model using the training data, wherein the machine learning model is trained to output an indication of one or more robot control parameters for handling the object from a camera image showing the object.
[0004] The approach described above uses properties of the robot arrangement as it is used in practice (in particular the camera position) to generate a training dataset for a robot controller (e.g. for grasping an object). This ensures that the simulated camera images are very close to those as they occur or would occur in practice (for corresponding object poses). Using the camera images for a training dataset for a machine learning model thus enables the machine learning model to be trained so that it is able to generate precise predictions for control parameters, e.g. a grasping position for a scenario in which a robot is to pick up an object from an object area (e.g. a box).
[0005] The object of the invention with the features of the independent claims is therefore to generate precise predictions for control parameters.
[0006] Various examples of implementation are given below.
[0007] Embodiment 1 is a method for training a machine learning model for controlling a robot as described above.
[0008] Embodiment 2 is the method according to embodiment 1, wherein the uncertainty range of the determined camera pose has an uncertainty range for the determined camera position and / or an uncertainty range for the determined camera orientation.
[0009] This allows both positioning and orientation uncertainties to be considered during camera calibration. Since both uncertainties typically occur in practice, this allows for the generation of training data for the machine learning model that is robust to both types of uncertainty.
[0010] Embodiment 3 is the method according to embodiment 2, wherein the random selection of the training camera image camera pose is carried out according to a normal distribution corresponding to the uncertainty range for the determined camera position and / or according to a normal distribution corresponding to the uncertainty range for the determined camera orientation.
[0011] The normal distribution provides a physically realistic variation around the determined camera pose, as typically occurs in the case of calibration errors.
[0012] Embodiment 4 is the method according to any one of embodiments 1 to 3, comprising determining an uncertainty region around the determined object region pose, and wherein setting the training camera image object pose comprises randomly selecting a camera image object region pose in the object region uncertainty region and randomly setting the training camera image object pose in the object region at the camera image-selected object region pose.
[0013] This also takes into account uncertainties in the calibration of the object area.
[0014] Embodiment 5 is the method according to embodiment 4, wherein the random selection of the training camera image object region pose is carried out according to a normal distribution corresponding to the uncertainty range for the determined object region pose.
[0015] As in the case of the camera, the normal distribution provides a physically realistic variation around the determined object region pose.
[0016] Embodiment 6 is the method according to one of the embodiments 1 to 5, wherein the determination of the training camera image object pose is carried out by determining a training camera image object pose according to a uniform distribution of object poses in the object region.
[0017] This allows for consistent training of the machine learning model for all object poses encountered during operation.
[0018] Embodiment 7 is the method according to one of the embodiments 1 to 6, wherein the setting of the training camera image object pose is carried out by setting a training camera image object pose according to a transformation into the object domain.
[0019] Within the object domain, the training camera image object pose (position and orientation) can be chosen randomly, e.g., uniformly distributed. This enables uniform training of the machine learning model for all object orientations that may occur during operation.
[0020] Embodiment 8 is the method for controlling a robot comprising training a machine learning model according to any one of embodiments 1 to 7, receiving a camera image showing an object to be handled by the robot, supplying the camera image to the machine learning model, and controlling the robot according to the robot control parameters specified by the output of the machine learning model.
[0021] Embodiment 9 is a device configured to carry out the method according to any one of embodiments 1 to 8.
[0022] Embodiment 10 is a computer program comprising program instructions that, when executed by one or more processors, cause the one or more processors to perform a method according to any one of embodiments 1 to 8.
[0023] Embodiment 11 is a computer-readable storage medium having stored thereon program instructions that, when executed by one or more processors, cause the one or more processors to perform a method according to any one of embodiments 1 to 8.
[0024] Embodiments of the invention are illustrated in the figures and will be described in more detail below. In the drawings, like reference characters generally refer to the same parts throughout the several views. The drawings are not necessarily to scale, with emphasis instead generally placed upon illustrating the principles of the invention. Fig. 1 shows a robot. Fig. 2 shows components of a robot cell with regard to simulation-based generation of training camera images. Fig. 3 shows a flowchart for a method for generating training camera images. Fig. 4 shows a flowchart for a method for training a neural network for controlling a robot.
[0025] The various embodiments, in particular the embodiments described below, can be implemented using one or more circuits. In one embodiment, a "circuit" can be understood as any type of logic-implementing entity, which can be hardware, software, firmware, or a combination thereof. Therefore, in one embodiment, a "circuit" can be a hard-wired logic circuit or a programmable logic circuit, such as a programmable processor, for example, a microprocessor. A "circuit" can also be software implemented or executed by a processor, for example, any type of computer program. Any other type of implementation of the respective functions described in more detail below can be understood as a "circuit" in accordance with an alternative embodiment.
[0026] According to various embodiments, an industrial robot system is used, consisting of an industrial robot with a gripping end effector, a camera, and a box with gripping objects. An example is described below with reference to Fig. 1 and Fig. 2. The type and design of the robot kinematics is not specified. (It could, for example, be serial or parallel kinematics with different types and numbers of joints.)
[0027] Fig. 1 shows a robot 100.
[0028] The robot 100 includes a robot arm 101, for example, an industrial robot arm for handling or assembling a workpiece (or one or more other objects). The robot arm 101 includes arm elements 102, 103, 104 and a base (or mount) 105 by means of which the arm elements 102, 103, 104 are supported. The term "arm element" refers to the movable components of the robot arm 101, the actuation of which enables physical interaction with the environment, e.g., to perform a task. For control, the robot 100 includes a (robot) controller 106 designed to implement the interaction with the environment according to a control program.The last component 104 (which is furthest from the base 105 in the kinematic chain) of the arm elements 102, 103, 104 is also referred to as the end effector 104 and may include one or more tools, such as a welding torch, a gripping instrument, a painting device or the like.
[0029] The other arm elements 102, 103 (located closer to the support 105) can form a positioning device, so that, together with the end effector 104, the robot arm 101 is provided with the end effector 104 at its end. The robot arm 101 is a mechanical arm that can provide functions similar to a human arm (possibly with a tool at its end).
[0030] The robot arm 101 may include joint elements 107, 108, 109 that connect the arm elements 102, 103, 104 to each other and to the support 105. A joint element 107, 108, 109 may have one or more joints, each of which can provide rotational movement (i.e., rotary movement) and / or translational movement (i.e., displacement) for associated arm elements relative to each other. The movement of the arm elements 102, 103, 104 may be initiated by actuators controlled by the controller 106.
[0031] The term "actuator" can be understood as a component configured to effect a mechanism or process in response to its drive. The actuator can implement instructions generated by the controller 106 (the so-called activation) into mechanical movements. The actuator, e.g., an electromechanical converter, can be configured to convert electrical energy into mechanical energy in response to its drive.
[0032] The term "controller" can be understood as any type of logic-implementing entity, which may, for example, include a circuit and / or a processor capable of executing software, firmware, or a combination thereof stored in a storage medium, and which can issue instructions, e.g., to an actuator in the present example. The controller can, for example, be configured by program code (e.g., software) to control the operation of a system, a robot in the present example.
[0033] In the present example, the controller 106 includes one or more processors 110 and a memory 111 that stores code and data based on which the processor 110 controls the robot arm 101. According to various embodiments, the controller 106 controls the robot arm 101 based on a machine learning model 112 stored in the memory 111.
[0034] According to various embodiments, the machine learning model 112 is configured and trained to specify (or predict) control parameters for the robot arm 101 for one or more camera images supplied to it.
[0035] The robot 100 and / or the robot cell in which the robot 100 is arranged may, for example, be equipped with one or more cameras 114 that are configured to capture images of the workspace of the robot 100 and of an object 113 (or several objects) arranged in the workspace.
[0036] The machine learning model is, for example, a neural convolutional network to which one or more images (e.g., depth images) are fed. Before practical use, the convolutional network is trained with training data so that the control device specifies suitable control parameters for different poses (positions and orientations) of the object 113.
[0037] The training data comprises, for example, a plurality of training data elements, each training data element containing a camera image (or multiple camera images) and associated (i.e., ground truth) control parameters. The control parameters comprise, for example, a position that the end effector 104 should assume to treat the object 113. The neural network can then be trained using supervised learning.
[0038] According to various embodiments, different training data images are generated for different object poses and associated control parameters. For example, if a specific random object pose is assumed for a training data image, the correct control parameters can be calculated directly from it (e.g., because the object pose indicates where the end effector 104 should be moved or where the object should be grasped). Thus, the main task for generating a training data element can be seen as generating a suitable camera image (for a respective camera in the case of multiple cameras) for a specific (random) object pose, so that if the object had the object pose in practice, the (respective) camera 114 would record this image.
[0039] According to various embodiments, an approach is provided for calculating the transformation between camera pose and object pose to generate a camera image (e.g., for a training data set with virtual depth images for input to a convolutional neural network for controlling the grasping of the object by the robot arm 101).
[0040] According to various embodiments, calibration parameters (typically available for a robot cell in practice) are used, and uncertainties (e.g., the camera pose) are taken into account in a physically plausible manner. This provides a method for simulation-based generation of a training dataset that is hardware-aware, i.e., takes the properties of the existing hardware (such as the camera) into account.
[0041] The object can have any geometry. The camera images are, for example, depth images. The camera is, for example, a pinhole camera, and the camera images are corresponding images.
[0042] According to various embodiments, a robustness metric is also calculated during the training data generation process, which represents the probability of a successful grasp (pickup) for the specific end effector (i.e., gripper type) of the robot. The position and orientation of the gripper for successful grasping is encoded in the camera image (or multiple camera images) by the respective translation, rotation, and size of the image content (i.e., specifically the object to be picked up) in the camera image. During training, the neural network (e.g., a deep convolutional network) is trained, for example, to predict the values of the robustness metric for different grasping positions for the camera images. These values of the robustness metric can be viewed as an indication of robot control parameters, for example, the grasping position for which the neural network outputs the highest value of the robustness metric.
[0043] After training, i.e., during operation (practical application), the trained neural network is used for inference, i.e., to predict the robustness values for various access options (e.g., end-effector poses) from a real camera image. The controller can then, for example, select the access option with the maximum robustness value and control the robot accordingly.
[0044] When generating (simulated) camera images for a camera for various object poses, the calibration results for the camera in the real robot setup (i.e., robot cell) are used. Camera calibration is typically performed during the setup of a robot system in a robot cell or prior to a processing operation to be performed by a robot to increase the accuracy of the knowledge of the camera pose and thus also of the robot actions (robot control) determined from the images generated by the camera.
[0045] According to various embodiments, extrinsic camera calibration values are used in the generation of training camera images, namely three-dimensional (e.g., Cartesian) displacement and rotation values of the camera with respect to the world coordinate system of the robot cell (i.e., a global reference coordinate system of the robot cell). These parameter values can be obtained, for example, through typical optimization-based calibration methods. Furthermore, according to various embodiments, uncertainty parameters of the camera calibration, which can be determined during the calibration process, are used. The uncertainty parameters represent the accuracy of the displacement values and rotation values determined during calibration.In addition, intrinsic camera parameters (camera matrix and noise values) of the real camera used in the robot cell can be used to make the generation of virtual camera images in the simulation as close as possible to the images from the real camera. Furthermore, information about the pose and size of the object area in which object 113 can be located (e.g., a box) is determined and used as input for generating the training camera images.
[0046] Fig. 2 shows components of a robot cell 200 with regard to simulation-based generation of training camera images.
[0047] A camera 201 has a coordinate system K K , an object 202 has a coordinate system K O , an object area (e.g. a box or crate) 203 has a coordinate system K B , the robot has a coordinate system K Rin its base, the end effector (gripper) moved by the robot has a coordinate system K G and the robot cell 200 has a global coordinate system K W , which represents a fixed reference system in the robot cell 200. The coordinate systems of components 201, 202, 203 are each fixed for the respective component and thus define the pose of the respective component within the robot cell. To generate training camera images, the camera is modeled as a virtual pinhole camera with a perspective projection that measures the surface of an object surface model.
[0048] The camera 201 has a field of view 204.
[0049] Fig. 3 shows a flowchart 300 for a method for generating training camera images.
[0050] The method can be performed by the control device 106 or by an external data processing device that generates the training data. The control device 106, or in turn an external data processing device, can then train the neural network 112 using the training data.
[0051] The input data for generating a camera image are the external camera calibration parameter values p K = (x K , y K , e.g. K , A K , B K , C K ), which determines the pose (position and orientation) of the camera coordinate system K K in the global coordinate system K W as well as associated uncertainties of the displacement values and the rotation values, e.g. u K = (t K , r K ), where for simplicity the uncertainty in the displacement t K and the uncertainty in the rotation r Kis assumed to be the same for all coordinates or angles in this example. These uncertainties are typically available as a result of the camera calibration process. The angles A K , B K , C K Examples include Euler angles, but other forms of representation can also be used for spatial orientation.
[0052] Additional input data are the intrinsic camera parameters in the form of a camera matrix K, which can typically be read out from the camera after an intrinsic camera calibration, as well as the noise value in the camera viewing direction σ K , which is typically specified by the camera manufacturer.
[0053] Further input data are the expected object area position p B = (x B , y B , e.g. B , A B , B B , C B), which specifies the position of the center of the cuboid object area in the global coordinate system, and the size of the object area b B = (h B , w B , d B ). These parameter values are typically determined by the configuration of the robot cell.
[0054] In 301 an initial camera transformation TKW, based on the camera calibration parameter values p K calculated, ie a translation and a rotation that converts a standard camera pose to the position determined by the calibration parameter values p K given camera pose.
[0055] In 302 an additional camera rotation becomes red K which calculates the uncertainty in the rotation during camera calibration. The rotation Rot KFor example, it is calculated based on the convention of rotation axis and angle of rotation, which assumes that every rotation in three-dimensional space can be represented by a suitable rotation axis and a rotation angle around the rotation axis. To calculate the rotation, Rot K The rotation axis is sampled from a uniform distribution of unit vectors in three-dimensional space. The rotation angle is calculated from a normal distribution with zero mean and standard deviation r K (ie the uncertainty in the orientation of the camera) is sampled. Then a transformation TKW"=TKW*RotK calculated.
[0056] In 303 an additional camera shift Trans Kwhich takes into account the uncertainty in the displacement of the camera calibration. For this purpose, a displacement direction is sampled from a uniform distribution of unit vectors in three-dimensional space. The length of the displacement is determined from a normal distribution with zero mean and standard deviation t K (ie the uncertainty in the camera position) is sampled. Then the final camera transformation TKW=TKW"*TransK calculated.
[0057] In 304 an object space transformation TBW based on the object area pose p B calculated, ie a translation and a rotation that sets a standard object area pose to the value determined by the parameter values p B maps the given object region pose. As with the camera pose, uncertainties in the object region pose can also be taken into account.
[0058] In 305 an additional object rotation becomes red O calculated. For this purpose, a rotation is sampled from a uniform distribution of rotations in order to account for a random rotation of the object 202 in the object area 203. This is relevant, for example, in the case where objects are arranged in clusters, in which each object can have a random rotation in three-dimensional space. If it can be expected that the object always lies on a flat surface (and not in clusters), a fixed orientation can also be used. In addition, an object displacement Trans O calculated by sampling from a uniform distribution over the object area U([-h B / 2, +h B / 21 × [-w B / 2, +w B / 2] × [d B / 2, +d B / 2]) to take into account that the object 202 can have a random position within the object area 203. In the case of object clusters, the object rotation and object translation of the individual objects in the cluster relative to the object area can alternatively be determined by a physics simulation. Object rotation and object translation thus result in the transformation TOB.
[0059] In 306, the object 202 is transformed with the pose of the object area 203. The object transformation is carried out with respect to the world coordinate system by means of TOW=TBW*TOB calculated.
[0060] In 307, the camera transformation with respect to the object coordinate system is calculated as TKO=(TOW)−1*TKW. This transformation is then used to position the virtual camera in the simulation space. The object (i.e., the surface model of the object) is positioned at the origin of the simulation space. The camera image is generated through a virtual measurement by rendering the simulation scene thus generated (in particular, with the positioned virtual camera) (e.g., to generate a depth image). For this purpose, the camera matrix K is used and the measurement data is filtered with a normally distributed noise with the noise parameter σ K applied.
[0061] The generation of a camera image in this manner is performed several times to generate a plurality of camera images for a training data set 308.
[0062] Since the camera pose and the object pose used physically represent parameter values from the real robot cell with associated uncertainties, this results in a training dataset that corresponds to the real physical configuration of the robot cell.
[0063] In summary, according to various embodiments, a method is provided as described below with reference to Fig. 4 is described.
[0064] Fig. 4 shows a flowchart 400 for a method for training a neural network for controlling a robot.
[0065] In 401, a camera pose in a robot cell and an uncertainty region around the determined camera pose are determined.
[0066] In 402, an object area in the robot cell is determined which contains positions for an object to be treated by the robot.
[0067] In 403, training camera images are generated.
[0068] For each training camera image, a training camera image camera pose is randomly determined in 404 in the uncertainty area around the determined camera pose, a training camera image object pose is randomly determined in the object area in 405, and the training camera image is generated in 406 such that it shows the object with the training camera image object pose from the perspective of a camera with the training camera image camera pose.
[0069] In 407, training data is generated from the training camera images, wherein each training camera image is assigned one or more training robot control parameters for handling the object in the training camera image object pose of the training camera image.
[0070] In 408, the neural network is trained using the training data, wherein the neural network is trained to output an indication of one or more robot control parameters for handling the object from a camera image showing the object.
[0071] The process generates synthetic sensor data. This can be sensor data from various optical sensors such as stereo cameras, time-of-flight cameras, and laser scanners.
[0072] Embodiments can be used to train a machine learning system and autonomously control a robot to achieve various manipulation tasks under different scenarios. In particular, embodiments are applicable for controlling and monitoring the execution of manipulation tasks, e.g., in assembly lines. They can, for example, be seamlessly integrated into a traditional GUI for a control process.
[0073] A “robot” can be any physical system (with a mechanical part whose movement is controlled), such as a computer-controlled machine, a vehicle, a household appliance, a power tool, a manufacturing machine, a personal assistant, or an access control system.
[0074] The neural network can be a convolutional network for regression or classification of data.
[0075] Various embodiments can receive and use sensor signals from various optical sensors, e.g., to obtain sensor data about demonstrations or system states (robot and object), as well as configurations and scenarios. The sensor data can be processed. This can involve classifying the sensor data or performing semantic segmentation of the sensor data, e.g., to detect the presence of objects (in the environment in which the sensor data was acquired). Embodiments can be used to train a machine learning system and autonomously control a robot to achieve various manipulation tasks under different scenarios. In particular, embodiments are applicable for controlling and monitoring the execution of manipulation tasks, e.g., in assembly lines. They can, e.g., be seamlessly integrated into a traditional GUI for a control process.
[0076] According to one embodiment, the method is computer-implemented.
Claims
[1] A method for training a machine learning model (112) for controlling a robot (100), comprising: Determining a camera pose in a robot cell and an uncertainty region around the determined camera pose; Determining an object area in the robot cell containing positions for an object (113, 202) to be treated by the robot (100); Generating training camera images, where for each training camera image a training camera image camera pose is randomly determined in the uncertainty range around the determined camera pose; a training camera image object pose is randomly set in the object space; the training camera image is generated so that it shows the object (113, 202) with the training camera image object pose from the perspective of a camera (114, 201) with the training camera image camera pose; Generating training data from the training camera images, wherein each training camera image is assigned one or more training robot control parameters for handling the object (113, 202) in the training camera image object pose of the training camera image; and Training the machine learning model (112) using the training data, wherein the machine learning model (112) is trained to output an indication of one or more robot control parameters for handling the object (113, 202) from a camera image showing the object (113, 202). [2] The method according to claim 1, wherein the uncertainty range of the determined camera pose comprises an uncertainty range for the determined camera position and / or an uncertainty range for the determined camera orientation. [3] Method according to claim 2, wherein the random selection of the training camera image camera pose is carried out according to a normal distribution corresponding to the uncertainty range for the determined camera position and / or according to a normal distribution corresponding to the uncertainty range for the determined camera orientation. [4] The method of any one of claims 1 to 3, comprising determining an uncertainty region around the determined object region pose, and wherein setting the training camera image object pose comprises randomly selecting a camera image object region pose in the object region uncertainty region and randomly setting the training camera image object pose in the object region at the selected camera image object region pose. [5] The method of claim 4, wherein the random selection of the training camera image object region pose is performed according to a normal distribution corresponding to the uncertainty range for the determined object region pose. [6] Method according to one of claims 1 to 5, wherein the setting of the training camera image object pose comprises setting a training camera image object position according to a uniform distribution of object positions in the object area. [7] Method according to one of claims 1 to 6, wherein the setting of the training camera image object pose comprises setting a training camera image object pose according to a transformation into the object domain. [8] Method for controlling a robot (100) comprising: Training a machine learning model (112) according to one of claims 1 to 7; Receiving a camera image showing an object (113, 202) to be treated by the robot (100); Feeding the camera image to the machine learning model (112); and Controlling the robot (100) according to the robot control parameters specified by the output of the machine learning model (112). [9] Apparatus (106) arranged to carry out the method according to one of claims 1 to 8. [10] A computer program comprising program instructions which, when executed by one or more processors (110), cause the one or more processors (110) to perform a method according to any one of claims 1 to 8. [11] A computer-readable storage medium (111) having stored thereon program instructions which, when executed by one or more processors (110), cause the one or more processors (110) to perform a method according to any one of claims 1 to 8.
Citation Information
Patent Citations
Machine learning device, robot control device and robot vision system that uses a machine learning device, and machine learning method
DE102019002065A1
Automation system and procedures for handling products
DE102019121889B3
Robot control system
DE102019122790A1
Machine learning for object recognition using a robot-guided camera
DE102019206444A1
Computing device, machine learning method and storage medium
DE102020114583A1
