Learning data set generation device and learning data set generation method

Abstract

A learning data set generation device (1) comprises: a memory storing three-dimensional CAD data of a workpiece (W) and a container (X); and one or more processors including hardware, wherein the processors use the three-dimensional CAD data of the workpiece (W) and the container (X) stored in the memory to generate a plurality of objects in a virtual three-dimensional space in which a plurality of workpieces (W) are randomly stacked in different forms in the container (X), measure each of the generated objects using a virtual three-dimensional measuring machine arranged in the virtual three-dimensional space, acquire a plurality of virtual distance images, receive at least one teaching position for each of the acquired virtual distance images, and associate the received teaching position with each of the virtual distance images, thereby generating a learning data set.

Description

Technical Field

[0001] The present invention relates to a learning data set generation device and a learning data set generation method. Background Art

[0002] The following method is known: using a three-dimensional measuring machine to acquire distance images of multiple workpieces, and storing a label map indicating teaching positions in association with teaching data consisting of the acquired distance images to generate a data set for machine learning (for example, see Patent Document 1).

[0003] The method of Patent Document 1 is used, for example, to generate a learned model that estimates the positions of multiple workpieces randomly stacked in a container when they are taken out one by one by a hand mounted on a robot. In order to generate a high-precision learned model, a large number of data sets need to be prepared. That is, each time a data set is generated, multiple workpieces need to be randomly stacked in different shapes and a distance image needs to be acquired using a three-dimensional measuring machine.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2019-58960 Summary of the invention

[0007] Problem that the invention aims to solve

[0008] Since each workpiece carried by the robot is very heavy, it takes a lot of time and effort to randomly stack multiple workpieces into different shapes every time each data set is generated. Therefore, it is desirable to generate a learning data set without randomly stacking actual workpieces.

[0009] Solutions for solving problems

[0010] One embodiment of the present invention is a device for generating a learning data set, comprising: a memory that stores three-dimensional CAD data of a workpiece and a container; and one or more processors that include hardware, wherein the processor uses the three-dimensional CAD data of the workpiece and the container stored in the memory to generate, in a virtual three-dimensional space, a plurality of objects in which a plurality of the workpieces are randomly stacked in different forms in the container, measure each of the generated objects using a virtual three-dimensional measuring machine configured in the virtual three-dimensional space, acquire a plurality of virtual distance images, receive at least one teaching position for each of the acquired virtual distance images, and associate the received teaching position with each of the virtual distance images, thereby generating a learning data set. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a diagram showing the overall configuration of a robot system to which a learning data set generating device and a learning data set generating method according to an embodiment of the present invention are applied.

[0012] Figure 2 It is shown Figure 1 Block diagram of a device for generating a dataset for learning.

[0013] Figure 3 It indicates the use of Figure 2 Flow chart of a learning data set generation method of a learning data set generation device.

[0014] Figure 4 Yes Description Figure 3 Flowchart of a variation of the method for generating a learning data set. DETAILED DESCRIPTION

[0015] Hereinafter, a learning data set generating device 1 and a learning data set generating method according to an embodiment of the present invention will be described with reference to the drawings.

[0016] like Figure 1 As shown, the learning data set generating device 1 of this embodiment is applied to a robot system 100, which comprises: a robot 110 having a hand 111 at the front end; and a three-dimensional measuring machine 120, which is positioned with high precision relative to the robot 110 and has a measuring range vertically below.

[0017] The robot system 100 uses a learned model to estimate the pick-up position for the robot 110 to pick up the workpieces W one by one from a container X storing a large number of workpieces W in a randomly stacked state within the measurement range of the three-dimensional measuring machine 120 .

[0018] The learning data set generating device 1 according to the present embodiment is a device that generates a large number of learning data sets for generating the learned model.

[0019] like Figure 2 As shown, the learning data set generating device 1 is a computer having a memory 2, a processor 3, a monitor 4, and a keyboard and a mouse as input devices 5. The memory 2 stores three-dimensional CAD data of a workpiece W actually processed and three-dimensional CAD data of a container X actually used.

[0020] The processor 3 generates a plurality of virtual subjects, which are a plurality of workpieces W having three-dimensional CAD data randomly stacked in different forms in a container X of three-dimensional CAD data stored in the memory 2. By randomly switching the positions and postures of the workpieces W of the three-dimensional CAD data, a plurality of virtual subjects randomly stacked in different forms can be easily generated in a short time.

[0021] The processor 3 arranges each object of the generated three-dimensional CAD data in a virtual three-dimensional space, and acquires a distance image of each object using a virtual three-dimensional measuring machine set in the same virtual three-dimensional space by a known method.

[0022] Every time a distance image is acquired, the processor 3 displays the acquired distance image on the monitor 4 , allowing the user to select a workpiece W that can be taken out by the robot 110 .

[0023] That is, the user who sees the distance image displayed on the monitor 4 specifies the workpieces W that are considered to be able to be taken out using the mouse or keyboard as the input device 5. As a result, the processor 3 receives the positions (for example, the center of gravity positions) of one or more workpieces W specified by the user as one or more teaching positions of the robot 110 for taking out these workpieces W.

[0024] Then, the processor 3 associates the received teaching position with the acquired range image to generate a learning data set, and stores it in the memory 2. By repeating the same process for each subject, a large amount of learning data sets can be easily generated in a short time.

[0025] A method of generating a learning data set using the learning data set generating device 1 of the present embodiment having the above-described configuration will be described below.

[0026] First, in the virtual three-dimensional space defined by the processor 3 , the three-dimensional model of the virtual robot and the virtual three-dimensional measuring machine are set in a positional relationship consistent with the positional relationship of the robot 110 and the three-dimensional measuring machine 120 set in the three-dimensional real space.

[0027] In addition, the three-dimensional CAD data of the workpieces W actually processed and the three-dimensional CAD data of the container X storing the workpieces W in a randomly stacked state are stored in advance in the memory 2 .

[0028] Then, if Figure 3 As shown, the parameters of random stacking are set (step S1). The parameters are, for example, the number of workpieces W stored in the container X, and the positions and postures of the plurality of workpieces W arranged on the bottom surface of the container X.

[0029] Then, a plurality of workpieces W of the three-dimensional CAD data are stored in a random stacked state using the set parameters in the container X of the three-dimensional CAD data. Thus, a virtual object (virtual object) composed of the three-dimensional CAD data is generated in which the plurality of workpieces W are stored in a random stacked state in the container X (step S2).

[0030] The virtual object composed of the generated three-dimensional CAD data is measured by a virtual three-dimensional measuring machine set in the virtual three-dimensional space, and a distance image (virtual distance image) of the virtual object is acquired (step S3).

[0031] Then, the acquired distance image is displayed on the monitor 4 (step S4), and the user is asked to designate a workpiece W that can be taken out with respect to the displayed distance image (step S5). Thus, the designated center of gravity position of the workpiece W is received as the teaching position.

[0032] It is determined whether the designation of the workpiece W that can be taken out is completed (step S6). If not, the designation of the workpiece W in step S5 is repeated. If it is completed, one or more received teaching positions are associated with the distance image to generate a learning data set (step S7). When a learning data set is generated for one subject, the user is asked to input whether to end the generation (step S8). If not, the parameters are changed (step S9), and the process starting from step S2 is repeated.

[0033] Thus, there is an advantage that a learning data set can be easily generated in a short time for a large number of subjects in which a plurality of workpieces W are stored in a state of being randomly stacked in different forms in a container X. That is, the user does not need to handle the actual heavy workpieces W and randomly stack them in different forms each time a learning data set is generated, which can reduce the burden on the user and shorten the time required to generate the learning data set.

[0034] Furthermore, by performing machine learning using the generated large amount of learning data sets, a highly accurate learned model can be generated.

[0035] When the robot 110 takes out the workpiece W from the container X, the three-dimensional measuring machine 120 arranged in the three-dimensional real space acquires a distance image of the container X in which the actual workpieces W are stored in a randomly stacked state and arranged in the measurement range of the three-dimensional measuring machine 120. Then, the acquired distance image is input into the learned model, so that the take-out position of at least one workpiece W that can be taken out by the robot 110 can be estimated.

[0036] In addition, in this embodiment, it is assumed that the positional relationship between the virtual three-dimensional measuring machine arranged in the virtual three-dimensional space and the container X is highly consistent with the positional relationship between the three-dimensional measuring machine 120 arranged in the three-dimensional real space and the container X. However, it is difficult to position the container X with respect to the three-dimensional measuring machine 120 with high accuracy in the three-dimensional real space.

[0037] Therefore, if Figure 4As shown, before generating a learning data set, the three-dimensional measuring machine 120 is fixed in the three-dimensional real space, the container X is arranged within the measurement range of the three-dimensional measuring machine 120, and an actual distance image (actual distance image) of the container X is obtained (step S10). Based on the obtained actual distance image of the container X, the relative position (actual relative position) between the three-dimensional measuring machine 120 and the container X is calculated (step S11).

[0038] Then, at least one of the virtual three-dimensional measuring machine and the container X is moved in the virtual three-dimensional space. Thus, the relative position (virtual relative position) between the virtual three-dimensional measuring machine and the container X is made consistent with the actual relative position between the three-dimensional measuring machine 120 and the container X in the three-dimensional real space (step S12), and the process is transferred to step S1.

[0039] By performing such processing in advance before generating the learning data set, the virtual distance image displayed on the monitor 4 for the user to specify the workpiece W that can be taken out can be made close to the actual distance image acquired by the actual three-dimensional measuring machine 120. Therefore, there are advantages that the accuracy of the teaching position associated with the virtual distance image can be improved, and the learning data set can be generated, which can improve the accuracy of estimating the removal position based on the actual distance image.

[0040] Description of reference numerals:

[0041] 1. Learning Dataset Generation Device

[0042] 2 Memory

[0043] 3 Processor

[0044] 120 3D measuring machine

[0045] X Container

[0046] W Workpiece

Claims

1. A learning data set generation device, characterized in that: have: A memory storing three-dimensional CAD data of the workpiece and the container, and a distance image obtained by photographing the container in a three-dimensional real space; and One or more processors, which include hardware, The processor is used to Using the three-dimensional CAD data of the workpiece and the container stored in the memory, a plurality of subjects are generated in a virtual three-dimensional space, in which a plurality of the workpieces are randomly stacked in different forms in the container, According to the distance image, the configuration of the container and the virtual three-dimensional measuring machine is set, The virtual three-dimensional measuring machine configured in the virtual three-dimensional space measures each of the generated objects to obtain a plurality of virtual distance images. receiving a teaching position corresponding to the center of gravity position of at least one workpiece for each of the acquired virtual distance images, The received teaching position is associated with each of the virtual distance images to generate a learning data set.

2. The learning data set generating device according to claim 1, characterized in that: The processor uses the distance image acquired by a three-dimensional measuring machine set in the three-dimensional real space to set the position of the virtual three-dimensional measuring machine relative to the container in the virtual three-dimensional space at a position consistent with the configuration of the three-dimensional measuring machine relative to the container.

3. A method for generating a learning data set, characterized in that: Using three-dimensional CAD data of a workpiece and a container, a plurality of subjects are generated in a virtual three-dimensional space, in which a plurality of the workpieces are randomly stacked in different forms in the container. According to the distance image obtained by photographing the container in the three-dimensional real space, the configuration of the container and the virtual three-dimensional measuring machine is set, The virtual three-dimensional measuring machine configured in the virtual three-dimensional space measures each of the generated objects to obtain a plurality of virtual distance images. receiving a teaching position corresponding to the center of gravity position of at least one workpiece for each of the acquired virtual distance images, The received teaching position is associated with each of the virtual distance images to generate a learning data set.

4. The method for generating a learning data set according to claim 3, wherein: Using the distance image acquired by a three-dimensional measuring machine set in the three-dimensional real space, the position of the virtual three-dimensional measuring machine relative to the container in the virtual three-dimensional space is set at a position consistent with the configuration of the three-dimensional measuring machine relative to the container.

Images