Machine learning method and robot system

Abstract

This machine learning method is a machine learning method for learning the actions of a robot (10) having a hand (11). The hand (11) takes out a workpiece from a container (C) that houses a plurality of workpieces (W) in a bulk state and sets the workpiece to a determined set state. This machine learning method includes the following steps: a step of learning a reverse process action of removing the workpiece in the determined set state after the setting is completed by using the hand; and a step of learning the setting order of the workpieces based on the learning result of the reverse process action of removing the workpiece.

Description

Technical Field

[0001] The present invention relates to a machine learning method and a robot system, and more particularly to a machine learning method and a robot system for learning the actions of a robot. The robot uses its hand to pick up a workpiece from a container that houses a plurality of workpieces in a bulk state, and uses its hand to place the picked-up workpiece. Background Art

[0002] Conventionally, there has been known a machine learning method for learning the actions of a robot. The robot uses its hand to pick up a workpiece from a container that houses a plurality of workpieces in a bulk state, and uses its hand to place the picked-up workpiece. Such a method is disclosed, for example, in Japanese Patent Application Laid-Open No. 2017-30135.

[0003] In the above-mentioned Japanese Patent Application Laid-Open No. 2017-30135, there is disclosed a machine learning method for learning the actions of a robot. The robot uses its hand to pick up a workpiece from a container that houses a plurality of workpieces in a bulk state, and uses its hand to place the picked-up workpiece on a conveyor or a workbench. In this machine learning method, the optimal action for picking up a workpiece from a container using a hand is learned.

[0004] Prior Art Documents

[0005] Patent Documents

[0006] Patent Document 1: Japanese Patent Application Laid-Open No. 2017-30135 Summary of the Invention

[0007] Problems to be Solved by the Invention

[0008] However, in the machine learning method described in the above-mentioned Japanese Patent Application Laid-Open No. 2017-30135, while learning the optimal action for picking up a workpiece from a container using a hand, the action of placing (loading) the workpiece using a hand is not considered. Therefore, there may be a case where the workpiece picked up using a hand is re-held. As a result, there is a problem that it is difficult to improve the efficiency of the setting operation after the workpiece is picked up.

[0009] The present invention has been made to solve the above-described problems, and one of the objects of the present invention is to provide a machine learning method and a robot system that can improve the efficiency of the setting operation after the workpiece is picked up.

[0010] Means for Solving the Problems

[0011] To achieve the above object, the machine learning method according to the first aspect of the present invention is a machine learning method for learning the actions of a robot equipped with a hand. The hand takes out workpieces from a container that houses a plurality of workpieces in a bulk state and sets the workpieces to a determined set state. The machine learning method includes: a step of learning the reverse process action of removing the workpiece in the determined set state after the setting is completed by using the hand; and a step of learning the setting order of the workpieces based on the learning result of the reverse process action of removing the workpieces.

[0012] In the machine learning method according to the first aspect of the present invention, the following steps are provided: a step of learning the reverse process action of removing the workpiece in the determined set state after the setting is completed by using the hand; and a step of learning the setting order of the workpieces based on the learning result of the reverse process action of removing the workpieces. Thus, it is possible to determine the setting order of the workpieces in consideration of the actions of the workpieces after being taken out. As a result, it is possible to suppress the occurrence of re-holding of the workpieces taken out by the hand. That is, it is possible to directly set (place) the workpieces taken out by the hand without re-holding them. Thus, it is possible to achieve the efficiency improvement of the setting operation after the workpieces are taken out.

[0013] In the machine learning method according to the above first aspect, preferably, the step of learning the reverse process action of removing the workpieces includes the following steps: repeatedly performing the reverse process action of removing the workpieces by using the hand until all or a part of the workpieces in the determined set state disappear. If configured in this way, it is possible to learn the reverse process action of the workpieces by repeatedly performing the reverse process action of removing the workpieces by using the hand until all or a part of the workpieces in the determined set state disappear. As a result, it is possible to fully consider the actions of the workpieces after being taken out to determine the setting order of the workpieces.

[0014] In this case, preferably, the step of learning the reverse process action of removing the workpieces includes the following steps: while changing the order of removing the workpieces, repeatedly performing the reverse process action of removing the workpieces by using the hand until all or a part of the workpieces in the determined set state disappear. If configured in this way, it is possible to learn multiple setting orders. As a result, it is possible to increase the options of the setting order and to grasp the highly efficient setting order.

[0015] In the above-described configuration in which the reverse process operation of removing the workpiece is repeated while changing the order of removing the workpiece, preferably, the step of learning the setting order of the workpiece includes the following steps: learning a plurality of setting orders based on the learning result of the reverse process operation of removing the workpiece, and learning the priorities of the plurality of setting orders. If configured in this way, it is possible to easily select an efficient setting order based on the priorities of the plurality of setting orders. As a result, it is possible to easily achieve the efficiency of the setting operation after the workpiece is taken out.

[0016] In the above-described configuration in which the reverse process operation of removing the workpiece is repeated, preferably, the step of learning the reverse process operation of removing the workpiece includes the following steps: in a state where a part of the order of removing the workpiece is set, repeatedly performing the reverse process operation of removing the workpiece with the hand until all or a part of the workpiece in the determined setting state disappears. If configured in this way, it is possible to limit the order of removing the workpiece corresponding to a part of the order of removing the workpiece that is set. As a result, it is possible to shorten the time required for learning corresponding to the limitation of the order of removing the workpiece.

[0017] In the machine learning method according to the first aspect described above, preferably, it further includes the following steps: learning the holding position of the workpiece held by the hand based on the learning result of the reverse process operation of removing the workpiece. If configured in this way, it is possible to determine the holding position of the workpiece in consideration of the operation of setting the workpiece. As a result, it is possible to further suppress the occurrence of re-holding of the workpiece taken out by the hand. Thus, it is possible to further achieve the efficiency of the setting operation after the workpiece is taken out.

[0018] In this case, preferably, the step of learning the holding position of the workpiece includes the following steps: learning the holding position of the workpiece in consideration of at least one of the constraints including the holding prohibited part of the workpiece and the obstacles near the workpiece. If configured in this way, it is possible to learn the position that can avoid the holding prohibited part of the workpiece and the obstacles near the workpiece as the holding position of the workpiece. As a result, it is possible to learn an appropriate holding position of the workpiece.

[0019] In the machine learning method according to the first aspect described above, preferably, it further includes the following steps: notifying the user to re-check at least one of the hand and the fixture tray for setting the workpiece based on the success probability of the reverse process operation of removing the workpiece. If configured in this way, the user can re-check the hand and the fixture tray for setting the workpiece, etc. As a result, it is possible to suppress the situation of continuing to use an inappropriate hand and a fixture tray for setting the workpiece, etc.

[0020] In the machine learning method of the first aspect described above, preferably, the step of learning the reverse process actions for removing the workpiece includes the following steps: learning the reverse process actions for removing various workpieces using the hand. With such a configuration, even when dealing with various workpieces, it is possible to improve the efficiency of the setting operation after the workpiece is taken out.

[0021] To achieve the above object, a robot system according to a second aspect of the present invention includes: a robot having a hand that takes out a workpiece from a container that houses a plurality of workpieces in a bulk state and sets the workpiece to a determined set state; a machine learning device that learns the actions of the robot; and a control device that controls the actions of the robot based on the learning results of the machine learning device. The machine learning device is configured to learn the reverse process actions for removing the workpiece in the determined set state after the setting is completed using the hand, and to learn the setting order of the workpieces based on the learning results of the reverse process actions for removing the workpiece.

[0022] In the robot system according to the second aspect of the present invention, the machine learning device is configured to learn the reverse process actions for removing the workpiece in the determined set state after the setting is completed using the hand, and to learn the setting order of the workpieces based on the learning results of the reverse process actions for removing the workpiece. Thus, similarly to the machine learning method of the first aspect described above, it is possible to improve the efficiency of the setting operation after the workpiece is taken out.

[0023] In the robot system according to the second aspect described above, preferably, the machine learning device is configured to learn the holding position of the workpiece held by the hand based on the learning results of the reverse process actions for removing the workpiece. With such a configuration, it is possible to determine the holding position of the workpiece in consideration of the action of setting the workpiece. As a result, it is possible to further suppress the occurrence of re-holding of the workpiece taken out by the hand. Thus, it is possible to further improve the efficiency of the setting operation after the workpiece is taken out.

[0024] In this case, preferably, the machine learning device is configured to select the holding position of the workpiece that can take out the workpiece from the container and can set the workpiece. With such a configuration, it is possible to reliably suppress the occurrence of re-holding of the workpiece taken out by the hand. Thus, it is possible to reliably improve the efficiency of the setting operation after the workpiece is taken out.

[0025] In the structure where the above-described selection can take out the workpiece from the container and can set the holding position of the workpiece, preferably, the machine learning device is configured to extract the holding position of the workpiece that can take out the workpiece from the container based on the extraction success probability of the workpiece, and extract the holding position of the workpiece that can set the workpiece based on the setting success probability of the workpiece. If configured in this way, it is possible to easily extract the holding position of the workpiece that can take out the workpiece from the container and can set the workpiece.

[0026] Advantageous Effects of the Invention

[0027] According to the present invention, as described above, it is possible to provide a machine learning method and a robot system that can improve the efficiency of the setting operation after taking out the workpiece. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a block diagram of a robot system showing one embodiment of the present invention.

[0029] Figure 2 It is a schematic diagram for explaining the setting of a workpiece by a robot according to one embodiment of the present invention.

[0030] Figure 3 It is a flowchart showing an example of the reverse process operation according to one embodiment of the present invention.

[0031] Figure 4 It is a schematic diagram for explaining the learning of the setting order of a workpiece based on the learning result of the reverse process operation according to one embodiment of the present invention.

[0032] Figure 5 It is a schematic diagram for explaining the learning of the holding position of a workpiece based on the learning result of the reverse process operation according to one embodiment of the present invention.

[0033] Figure 6 It is a schematic diagram for explaining the case where the success probability of the reverse process operation according to one embodiment of the present invention is low.

[0034] Figure 7 It is a schematic diagram for explaining the notification to the user in the case where the success probability of the reverse process operation according to one embodiment of the present invention is low.

[0035] Figure 8 It is a flowchart for explaining the learning and execution operations according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0036] Hereinafter, embodiments for embodying the present invention will be described based on the drawings.

[0037] Refer to Figure 1, the structure of the robot system 100 according to an embodiment of the present invention will be described.

[0038] (Structure of the robot system)

[0039] The robot system 100 of the present embodiment is configured to take out the workpiece W from a container (cabinet) C that houses a plurality of workpieces W in a bulk state, and set the workpiece W on the jig tray P1 to achieve a determined setting state S. The workpiece W is not particularly limited, and is, for example, a small component such as a screw and a nut. As Figure 1 shown, the robot system 100 includes a robot 10, a machine learning device 20, a control device 30, and a camera device 40.

[0040] As Figure 2 shown, the robot 10 is a robotic arm that performs operations on the workpiece W. Specifically, the robot 10 is a vertical multi-joint robot. The robot 10 includes a hand (end effector) 11. The hand 11 is configured to take out the workpiece W from the container C and set the taken-out workpiece W on the jig tray P1. The hand 11 is configured to set the workpiece W on the jig tray P1 to achieve a determined setting state S by repeatedly taking out the workpiece W from the container C and setting the taken-out workpiece W on the jig tray P1. The hand 11 has a plurality of claw portions 11a for holding (gripping) the workpiece W.

[0041] As Figure 1 shown, the machine learning device 20 is a device that learns (machine learning) the actions of the robot 10. Specifically, the machine learning device 20 is a personal computer including a processor and a memory. The machine learning performed by the machine learning device 20 is not particularly limited, and for example, supervised learning, unsupervised learning, and reinforcement learning can be adopted. In the present embodiment, reinforcement learning is adopted as the machine learning performed by the machine learning device 20.

[0042] The machine learning device 20 includes a state quantity observation unit 21, an action result acquisition unit 22, a learning unit 23, a decision-making unit 24, and a display unit 25. In addition, the state quantity observation unit 21, the action result acquisition unit 22, the learning unit 23, and the decision-making unit 24 are illustrated as software functional blocks. The state quantity observation unit 21, the action result acquisition unit 22, the learning unit 23, and the decision-making unit 24 may be constituted by one hardware circuit (such as a GPU (Graphics Processing Unit)), or may be constituted by a plurality of hardware circuits.

[0043] The state quantity observation unit 21 is configured to observe the state quantities of the robot 10 and the workpiece W. Specifically, the state quantity observation unit 21 observes the state quantities of the robot 10 and the workpiece W based on the output data from the imaging device 40. The state quantities observed by the state quantity observation unit 21 include, for example, the position of the hand 11, the posture of the hand 11, and the position of the workpiece W. The action result acquisition unit 22 is configured to acquire the action result of the robot 10. Specifically, the action result acquisition unit 22 is configured to acquire the action result of the robot 10 based on the output data from the imaging device 40. The action results acquired by the action result acquisition unit 22 include, for example, the action results of the reverse process action (described later) of removing the workpiece W using the hand 11, etc.

[0044] The learning unit 23 is configured to learn the actions of the robot 10. Specifically, the learning unit 23 is configured to learn the actions of the robot 10 based on the observation results of the state quantity observation unit 21 and the acquisition results of the action result acquisition unit 22. In addition, the details of the learning of the learning unit 23 will be described later. The decision-making unit 24 is configured to decide the actions of the robot 10. Specifically, the decision-making unit 24 is configured to decide the actions of the robot 10 based on the learning results of the learning unit 23. The display unit 25 is configured to display a setting screen, a notification screen, etc. The display unit 25 is, for example, a liquid crystal display unit including a liquid crystal panel.

[0045] The control device 30 is a device including a control circuit for controlling the actions of the robot 10. Specifically, the control device 30 is configured to control the actions of the robot 10 based on the learning results of the machine learning device 20. The control device 30 is configured to control the actions of the joints and the hand 11 of the robot 10 based on the output data from the decision-making unit 24. In addition, the control device 30 is configured to control the actions of the imaging device 40.

[0046] The imaging device 40 is a device including a camera for imaging the robot 10, the workpiece W, etc. The imaging device 40 is provided as a three-dimensional data acquisition device for acquiring three-dimensional data including the three-dimensional position information of the robot 10, the workpiece W, etc. The imaging device 40 is configured to image, for example, the hand 11 and the workpiece W during the reverse process action of removing the workpiece W (described later), the hand 11 and the workpiece W during the action of taking out the workpiece W from the container C, etc. In addition, as Figure 2 shown, the imaging device 40 is provided at the front end of the arm of the robot 10. In addition, the imaging device 40 is provided near the hand 11.

[0047] (Workpiece setting action)

[0048] Next, the workpiece setting action will be described.

[0049] As Figure 2As shown, the robot 10 performs the setting operation of the workpiece W based on the instruction of the control device 30 (refer to Figure 1 ). First, the robot 10 uses the hand 11 to take out the workpiece W from the container C. Specifically, the robot 10 uses the hand 11 to hold the workpiece W in the container C and take it out of the container C.

[0050] Then, the robot 10 uses the hand 11 to perform the operation of setting the taken-out workpiece W on the fixture tray P1. Specifically, the robot 10 uses the hand 11 to move the workpiece W to the determined position on the fixture tray P1 and set the workpiece W at the determined position on the fixture tray P1.

[0051] At this time, when the workpiece W cannot be directly set on the fixture tray P1 from the container C in the holding posture, the robot 10 uses the hand 11 to perform the operation of temporarily placing the workpiece W on the temporary placement tray P2 and re-holding it. The temporary placement tray P2 is a tray that serves as a buffer for temporarily placing the workpiece W. The temporary placement tray P2 is set near the fixture tray P1.

[0052] Then, the robot 10 repeatedly performs the operation of taking out the workpiece W from the container C using the hand 11 and the operation of setting the workpiece W taken out using the hand 11 on the fixture tray P1 until the fixture tray P1 reaches the determined set state S. Then, when the fixture tray P1 reaches the determined set state (final state) S, the fixture tray P1 containing the workpiece W in the determined set state S is transferred to the next process.

[0053] In addition, in Figure 2 , for convenience, an example in which four workpieces W are set on the fixture tray P1 one by one is shown. However, it is not limited to this example. For example, multiple of one type of workpiece W can be set on the fixture tray P1, or one or more of multiple types of workpieces W other than the four types can be set on the fixture tray P1.

[0054] (Learning of the robot's operation)

[0055] Next, the learning of the operation of the robot 10 will be described.

[0056] The machine learning device 20 learns the operation of the robot 10, which is adapted to take out the workpiece W from the container C that houses a plurality of workpieces W in a bulk state and set the workpiece W on the fixture tray P1 to reach the determined set state S.

[0057] Here, in the present embodiment, as Figure 3As shown, the machine learning device 20 is configured to learn the reverse process of removing the workpiece W in the determined set state (final state) S after the setup is completed from the fixture tray P1 using the hand 11. The machine learning device 20 is configured to learn the reverse process of removing multiple workpieces W from the fixture tray P1 using the hand 11 when the robot 10 processes multiple workpieces W. In addition, the machine learning device 20 is configured to learn the reverse process of removing one workpiece W from the fixture tray P1 using the hand 11 when the robot 10 processes one workpiece W.

[0058] In addition, the machine learning device 20 is configured to repeatedly perform the reverse process of removing the workpiece W from the fixture tray P1 until all the workpieces W in the determined set state S disappear. In addition, the machine learning device 20 is configured to learn the setup order of the workpiece W based on the learning results of the reverse process of removing the workpiece W. In addition, the reverse process of removing the workpiece W from the fixture tray P1 includes the action of holding the workpiece W set on the fixture tray P1 using the hand 11 and the action of separating the workpiece W held by the hand 11 from the fixture tray P1. In addition, the reverse process of removing the workpiece W from the fixture tray P1 means the opposite action of the forward process of setting the workpiece W on the fixture tray P1.

[0059] Here, with reference to Figure 3 an example of learning the reverse process of removing the workpiece W from the fixture tray P1 and learning the setup order based on the learning results of the reverse process will be described. Here, an example of removing four workpieces W, namely workpiece W A, workpiece W B, workpiece W C, and workpiece W D, will be described.

[0060] First, in step S1, the machine learning device 20 obtains the success probability of the reverse process of each workpiece W in the state (set state S) where all the workpieces W (workpiece W A, workpiece W B, workpiece W C, and workpiece W D) are present on the fixture tray P1. That is, the machine learning device 20 obtains the success probability of the reverse process of workpiece W A, workpiece W B, workpiece W C, and workpiece W D. The acquisition of this success probability can be performed by using simulation using three-dimensional data, the actions of the actual robot 10, and a combination of them.

[0061] Then, in step S2, the machine learning device 20 removes one workpiece W B with a high success probability from the fixture tray P1 using the hand 11 and obtains the success probability of the reverse process of each workpiece W in the state where the remaining three workpieces W (workpiece W A, workpiece W C, and workpiece W D) are present on the fixture tray P1. That is, the machine learning device 20 obtains the success probability of the reverse process of workpiece W A, workpiece W C, and workpiece W D.

[0062] Then, in step S3, the machine learning device 20 uses the hand 11 to remove one C workpiece W with a high success probability from the jig tray P1, and obtains the success probability of the reverse process action of each workpiece W in the state where the remaining two workpieces W (A workpiece W and D workpiece W) are present in the jig tray P1. That is, the machine learning device 20 obtains the success probability of the reverse process action of the A workpiece W and the D workpiece W.

[0063] Then, in step S4, the machine learning device 20 uses the hand 11 to remove one A workpiece W from the jig tray P1, and obtains the success probability of the reverse process action of the last workpiece W (D workpiece W) in the state where the last workpiece W is present in the jig tray P1. That is, the machine learning device 20 obtains the success probability of the reverse process action of the D workpiece W.

[0064] In Figure 3 the example shown, based on the success probability of the reverse process action, the workpieces W are removed from the jig tray P1 in the order of the B workpiece W, the C workpiece W, the A workpiece W, and the D workpiece W. In this case, the machine learning device 20 learns the reverse order of the setting order that is suitable for setting the workpiece W on the jig tray P1 to become the determined setting state S. That is, the machine learning device 20 learns the case of setting the workpiece W on the jig tray P1 in the order of the D workpiece W, the A workpiece W, the C workpiece W, and the B workpiece W as the setting order that is suitable for setting the workpiece W on the jig tray P1 to become the determined setting state S.

[0065] Refer to Figure 4 to illustrate in more detail the learning of the reverse process action of removing the workpiece W and the learning of the setting order based on the learning result of the reverse process action. In addition, Figure 4 the sequences 1 to 4 respectively correspond to Figure 3 steps S1 to S4. In addition, in Figure 4 the illustration of the case where the D workpiece W is selected in sequence 1 is omitted for the sake of easy illustration.

[0066] As Figure 4 shown, the machine learning device 20 is configured to repeatedly perform the reverse process action of removing the workpiece W from the jig tray P1 using the hand 11 while changing the order of removing the workpiece W until all the workpieces W in the determined setting state S disappear. At this time, the machine learning device 20 is configured not to learn the workpiece W whose success probability of the reverse process action is below the threshold. Thereby, the learning efficiency can be improved. The threshold is not particularly limited, but in Figure 4 the example shown, it is set to 50%.

[0067] In addition, the machine learning device 20 is configured to be able to set a part of the order in which the workpiece W is removed from the jig tray P1. In this case, the machine learning device 20 is configured to repeatedly perform the reverse process operation of removing the workpiece W from the jig tray P1 by the hand 11 until all the workpieces W in the determined setting state S disappear, in a state where a part of the order in which the workpiece W is removed from the jig tray P1 is set. The set order can be, for example, any of the first, last, and middle ones. The set order can be specified by the user.

[0068] For example, when the workpiece W of A is set as the workpiece W to be removed from the jig tray P1 first (that is, the workpiece W finally set on the jig tray P1), the machine learning device 20 satisfies the condition of removing the workpiece W of A from the jig tray P1 first, and repeatedly performs the reverse process operation of removing the workpiece W from the jig tray P1 by the hand 11 until all the workpieces W in the determined setting state S disappear. Similarly, when the workpiece W of B is set as the workpiece W to be removed from the jig tray P1 last (that is, the workpiece W initially set on the jig tray P1), the machine learning device 20 satisfies the condition of removing the workpiece W of B from the jig tray P1 last, and repeatedly performs the reverse process operation of removing the workpiece W from the jig tray P1 by the hand 11 until all the workpieces W in the determined setting state S disappear.

[0069] In addition, the machine learning device 20 is configured to learn multiple setting orders based on the learning results of the reverse process operation of removing the workpiece W, and learn the priorities of the multiple setting orders. Specifically, the machine learning device 20 is configured to obtain scores of multiple setting orders based on the success probability of the reverse process operation, and learn the priorities of the multiple setting orders based on the obtained scores of the multiple setting orders. The scores of the setting orders are not particularly limited. For example, the product and sum of the success probabilities of each reverse process operation of the setting order can be used. In this case, the larger the score, the higher the priority. In addition, the setting order with a higher priority is the setting order that is more suitable for setting the workpiece W on the jig tray P1 to become the determined setting state S.

[0070] In addition, the machine learning device 20 is configured to learn the setting order with the highest priority as the default setting order. In addition, in Figure 4 In the example shown, there are multiple (four) setting orders with the largest scores (highest priorities). In this case, the machine learning device 20 is configured to, for example, based on the user's specification, learn any one of the multiple setting orders with the highest priority as the default setting order.

[0071] In addition, in the present embodiment, as Figure 5As shown, the machine learning device 20 is configured to learn the holding position of the workpiece W held by the hand 11 based on the learning result of the reverse process action of removing the workpiece W from the jig tray P1. In addition, the holding position of the workpiece W held by the hand 11 is a concept that includes the holding position of the workpiece W and the holding posture of the hand 11. That is, learning the holding position of the workpiece W held by the hand 11 means learning the holding position of the workpiece W and the holding posture of the hand 11.

[0072] Specifically, the machine learning device 20 is configured to learn the holding position of the workpiece W in consideration of the constraint conditions including the holding prohibited part of the workpiece W and the obstacles near the workpiece W. The holding prohibited part of the workpiece W is, for example, the part of the workpiece W inserted into the jig tray P1 (the part where the workpiece W cannot be set on the jig tray P1 if held), and the part where holding itself is originally prohibited, etc. In addition, the obstacles near the workpiece W are, for example, other workpieces W near the workpiece W and the walls near the workpiece W.

[0073] The machine learning device 20 is configured to learn the holding position of the workpiece W that can be held by the hand 11 and the holding position of the workpiece W that cannot be held by the hand 11 in the reverse process action based on the learning result of the reverse process action of removing the workpiece W from the jig tray P1. In other words, the machine learning device 20 is configured to learn the holding position of the workpiece W that can set the workpiece W on the jig tray P1 in the forward process action and the holding position of the workpiece W that cannot set the workpiece W on the jig tray P1 in the forward process action based on the learning result of the reverse process action of removing the workpiece W from the jig tray P1.

[0074] In addition, in the present embodiment, as Figure 6 and Figure 7 shown, the machine learning device 20 is configured to notify the user to re-check the hand 11 and the jig tray P1 based on the success probability of the reverse process action of removing the workpiece W from the jig tray P1. Specifically, the machine learning device 20 is configured to notify the user to re-check the hand 11 and the jig tray P1 when the success probability of the reverse process action of all workpieces W is below the threshold value in the case of initially removing the workpiece W from the jig tray P1. The threshold value is not particularly limited, and can be set to 100%, for example. In addition, the machine learning device 20 is configured to notify the user of the re-check of the hand 11 and the jig tray P1 by displaying a notification on the display unit 25.

[0075] (Learning and execution actions)

[0076] Next, referring to Figure 8, the learning and execution actions of the robot system 100 will be described based on the flowchart. Here, an example of learning through simulation using three-dimensional data will be described.

[0077] As Figure 8 shown, first, in step S11, the creation of three-dimensional data is performed. Specifically, the three-dimensional data of the workpiece W, the hand 11, and the fixture tray P1 in the state where the workpiece W is not set are created. The creation of the three-dimensional data is not particularly limited. For example, it can be performed by reading CAD data and measuring the shape using a three-dimensional shape measuring device. In addition, as the three-dimensional shape measuring device, a dedicated measuring device can be prepared, but the imaging device 40 of the robot system 100 can also be used.

[0078] Then, in step S12, the three-dimensional data of the fixture tray P1 after the workpiece W is set is created. That is, the three-dimensional data of the fixture tray P1 with the workpiece W set in the set state S is created. The creation of the three-dimensional data can be performed in the same manner as in step S11, by reading CAD data and measuring the shape using a three-dimensional shape measuring device. In addition, as in step S11, the imaging device 40 of the robot system 100 can also be used as the three-dimensional shape measuring device.

[0079] Then, in step S13, learning is performed using the reverse process action of removing the workpiece W from the fixture tray P1. In step S13, as described above, the following steps are performed: the step of the reverse process action of removing the workpiece W in the determined set state S after being set from the fixture tray P1 using the hand 11; the step of learning the setting order of the workpiece W based on the learning result of the reverse process action of removing the workpiece from the fixture tray P1; and the step of learning the holding position of the workpiece W based on the learning result of the reverse process action of removing the workpiece W from the fixture tray P1. These learnings are performed through simulation using the three-dimensional data created in steps S11 and S12.

[0080] In addition, when learning is performed not through simulation but by actually moving the robot 10, a fixture tray P1 with the workpiece W set in the set state S is actually prepared. Then, learning is performed by performing the reverse process action on the prepared fixture tray P1. At this time, the reverse process action is performed while imaging with the imaging device 40. Thus, based on the output data from the imaging device 40, the state quantity observation unit 21 can observe the position of the hand 11, the posture of the hand 11, and the position of the workpiece W during the reverse process action. In addition, based on the output data from the imaging device 40, the action result acquisition unit 22 can obtain the action result (success or failure) of the reverse process action of removing the workpiece W using the hand 11.

[0081] Then, in step S14, the setting of the workpiece W based on the learning result of the machine learning device 20 is performed. Specifically, in step S14, the following steps are carried out: The workpiece W is taken out of the container C by the hand 11, and the workpiece W is set on the jig tray P1 to be in the determined setting state S. At this time, the machine learning device 20 selects the setting order based on the learning result of the reverse process action. Basically, the machine learning device 20 selects the default setting order as the setting order to be executed. However, when the setting cannot be performed in the default setting order, the machine learning device 20 selects a setting order other than the default setting order in the learned setting order as the setting order to be executed. In this case, the machine learning device 20 selects a setting order with a high priority and capable of being set among the setting orders other than the learned default setting order as the setting order to be executed.

[0082] In addition, the machine learning device 20 selects the holding position of the workpiece W when taking out the workpiece W from the container C based on the learning result of the reverse process action and the accommodation state of the workpiece W in the container C. The accommodation state of the workpiece W in the container C can be obtained based on the output data from the imaging device 40 obtained by imaging the workpiece W in the container C. The machine learning device 20 selects a holding position of the workpiece W that can take out the workpiece W from the container C and can set the workpiece W on the jig tray P1.

[0083] Specifically, the machine learning device 20 extracts the holding positions of the workpieces that can take out the workpiece W from the container C based on the extraction success probability of the workpiece W, and extracts the holding positions of the workpiece W that can set the workpiece W based on the setting success probability of the workpiece W. The extraction success probability of the workpiece W can be obtained based on the output data from the imaging device 40 obtained by imaging the workpiece W in the container C. In addition, the taking-out action of the workpiece W from the container C can be learned in advance. If learned in advance, the extraction success probability of the workpiece W can be obtained with high accuracy. In addition, the setting success probability of the workpiece W can be obtained based on the success probability of the learned reverse process action. The machine learning device 20 selects a holding position of the workpiece W that can take out the workpiece W from the container C and can set the workpiece W on the jig tray P1 from among the extracted holding positions of the workpiece W that can take out the workpiece W from the container C and the extracted holding positions of the workpiece W that can set the workpiece W.

[0084] In addition, the machine learning device 20 sends the selected information to the control device 30. Then, the control device 30 controls the operation of the robot 10 based on the information from the machine learning device 20. Then, based on the instruction of the control device 30, the robot 10 takes out the workpiece W from the container C that houses a plurality of workpieces W in a bulk state, and sets the workpiece W on the jig tray P1 to be in the determined setting state S.

[0085] In addition, the learning result of the machine learning device 20 may be further corrected based on the result of the forward process action of taking out the workpiece W from the container C and placing the workpiece W on the fixture pallet P1. For example, when the action of taking out the workpiece W from the container C and placing the workpiece W on the fixture pallet P1 fails, the information of the holding position of the workpiece W based on the learning result of the reverse process action may be corrected based on the result of the forward process action. In addition, for example, when the action of taking out the workpiece W from the container C and placing the workpiece W on the fixture pallet P1 fails, the information of the score (priority) of the setting order based on the learning result of the reverse process action may be corrected based on the result of the forward process action.

[0086] (Effects of this embodiment)

[0087] In this embodiment, the following effects can be obtained.

[0088] In the present embodiment, as described above, the following steps are provided in the machine learning method: a step of learning the reverse process action of removing the workpiece W in the determined setting state S after the setting is completed by the hand 11; and a step of learning the setting order of the workpiece W based on the learning result of the reverse process action of removing the workpiece W. Thus, the setting order of the workpiece W can be determined by considering the action of setting the workpiece W after it is taken out. As a result, the situation of re-holding the workpiece W taken out by the hand 11 can be suppressed. That is, the workpiece W taken out by the hand 11 can be directly set (loaded) without re-holding. Thus, the efficiency of the setting operation after the workpiece W is taken out can be achieved.

[0089] In addition, in the present embodiment, as described above, the step of learning the reverse process action of removing the workpiece W includes the following steps: repeatedly performing the reverse process action of removing the workpiece W by the hand 11 until all or part of the workpiece W in the determined setting state S disappears. Thus, the reverse process action of the workpiece W can be learned by repeatedly performing the reverse process action of removing the workpiece W by the hand 11 until all or part of the workpiece W in the determined setting state S disappears. As a result, the setting order of the workpiece W can be determined by fully considering the action of setting the workpiece W after removal.

[0090] In addition, in the present embodiment, as described above, the step of learning the reverse process of removing the workpiece W includes the following steps: while changing the order of removing the workpiece W, repeatedly perform the reverse process of removing the workpiece W using the hand 11 until all or a part of the workpiece W in the determined set state S disappears. Thus, multiple setting orders can be learned. As a result, the options of the setting order can be increased, and an efficient setting order can be grasped.

[0091] In addition, in the present embodiment, as described above, the step of learning the setting order of the workpiece W includes the following steps: based on the learning result of the reverse process of removing the workpiece W, learn multiple setting orders and learn the priorities of the multiple setting orders. Thus, based on the priorities of the multiple setting orders, an efficient setting order can be easily selected. As a result, the efficiency of the setting operation after removing the workpiece W can be easily achieved.

[0092] In addition, in the present embodiment, as described above, the step of learning the reverse process of removing the workpiece W includes the following steps: in a state where a part of the order of removing the workpiece W is set, repeatedly perform the reverse process of removing the workpiece W using the hand 11 until all or a part of the workpiece W in the determined set state S disappears. Thus, the order of removing the workpiece W can be limited corresponding to the setting of a part of the order of removing the workpiece W. As a result, the time required for learning can be shortened corresponding to the limitation of the order of removing the workpiece W.

[0093] In addition, in the present embodiment, as described above, the machine learning method includes the following steps: based on the learning result of the reverse process of removing the workpiece W, learn the holding position of the workpiece W held by the hand 11. Thus, the holding position of the workpiece W can be determined considering the operation of setting the workpiece W. As a result, the situation where the workpiece W taken out by the hand 11 is re-held can be further suppressed. Thus, the efficiency of the setting operation after removing the workpiece W can be further achieved.

[0094] In addition, in the present embodiment, as described above, the step of learning the holding position of the workpiece W includes the following steps: considering the constraint conditions including the holding prohibited part of the workpiece W and the obstacles near the workpiece W, learn the holding position of the workpiece W. Thus, the position that can avoid the holding prohibited part of the workpiece and the obstacles near the workpiece can be learned as the holding position of the workpiece. As a result, an appropriate holding position of the workpiece W can be learned.

[0095] In addition, in the present embodiment, as described above, the machine learning method includes the following steps: Based on the success probability of the reverse process action of removing the workpiece W, the user is notified to re-check the hand 11 and the jig tray P1 for setting the workpiece W. As a result, the user can re-check the hand 11 and the jig tray for setting the workpiece W. As a result, it is possible to suppress the situation of continuously using an inappropriate hand 11 and the jig tray for setting the workpiece W and the like.

[0096] In addition, in the present embodiment, as described above, the step of learning the reverse process action of removing the workpiece W includes the following steps: learning the reverse process action of removing various workpieces W using the hand 11. As a result, even when processing various workpieces W, it is possible to improve the efficiency of the setting operation after the workpiece W is taken out.

[0097] In addition, in the present embodiment, as described above, the machine learning device 20 is configured to select a holding position of the workpiece W that can take out the workpiece W from the container C and can set the workpiece W. As a result, it is possible to reliably suppress the occurrence of re-holding of the workpiece W taken out by the hand 11. As a result, it is possible to reliably improve the efficiency of the setting operation after the workpiece W is taken out.

[0098] In addition, in the present embodiment, as described above, the machine learning device 20 is configured to extract the holding position of the workpiece W that can take out the workpiece W from the container C based on the take-out success probability of the workpiece W, and extract the holding position of the workpiece W that can set the workpiece W based on the setting success probability of the workpiece W. As a result, it is possible to easily extract the holding position of the workpiece W that can take out the workpiece W from the container C and can set the workpiece W.

[0099] (Modification example)

[0100] In addition, it should be considered that the embodiments disclosed this time are illustrative in all aspects and not restrictive. The scope of the present invention is not shown by the description of the above embodiments, but is shown by the scope of claims, and includes all changes (modification examples) within the meaning and scope equivalent to the scope of claims.

[0101] For example, in the above embodiment, an example of setting a workpiece on a jig tray to become a determined setting state is shown, but the present invention is not limited thereto. In the present invention, the workpiece may also be set on an assembled product to become a determined setting state (assembled state). In this case, the machine learning device learns the reverse process action of removing the workpiece in the determined setting state (assembled state) from the assembled product.

[0102] In addition, in the above-described embodiment, an example in which the robot is a vertically articulated robot is shown, but the present invention is not limited thereto. In the present invention, any robot may be used as long as it can pick up a workpiece from a container and place the picked-up workpiece. For example, the robot may also be a horizontally articulated robot or the like other than a vertically articulated robot.

[0103] In addition, in the above-described embodiment, an example in which the imaging device is provided as a three-dimensional data acquisition device is shown, but the present invention is not limited thereto. In the present invention, a three-dimensional laser scanning device or the like other than the imaging device may be provided as the three-dimensional data acquisition device.

[0104] In addition, in the above-described embodiment, an example of a structure in which the imaging device is provided at the tip of the arm of the robot is shown, but the present invention is not limited thereto. In the present invention, the imaging device may also be provided at a position other than the robot. For example, a support member for supporting the imaging device may be provided near the working area of the robot. In this case, it is preferable that the imaging device is supported by the support member so as to be able to image the robot and the container from above.

[0105] In addition, in the above-described embodiment, an example in which the hand is configured to have a plurality of claw portions and hold the workpiece is shown, but the present invention is not limited thereto. In the present invention, the hand may be configured in any manner as long as it can hold the workpiece. For example, the hand may be configured to adsorb the workpiece using negative pressure from a negative pressure generating device, or may be configured to adsorb the workpiece using the magnetic force of an electromagnet.

[0106] In addition, in the above-described embodiment, an example in which the machine learning device is configured to repeatedly perform the reverse process operation of removing the workpiece with the hand until all the workpieces in the determined set state disappear is shown, but the present invention is not limited thereto. In the present invention, the machine learning device may also be configured to repeatedly perform the reverse process operation of removing the workpiece with the hand until a part of the workpieces in the determined set state disappears.

[0107] In addition, in the above-described embodiment, an example in which the machine learning device is configured not to learn a workpiece whose success probability of the reverse process operation is below a threshold is shown, but the present invention is not limited thereto. In the present invention, the machine learning device may also be configured to learn all workpieces regardless of the success probability of the reverse process operation.

[0108] In addition, in the above-described embodiment, an example in which the machine learning device is configured to be able to set a part of the order of removing the workpieces is shown, but the present invention is not limited thereto. In the present invention, the machine learning device may not be configured to be able to set a part of the order of removing the workpieces.

[0109] In addition, in the above-described embodiment, an example is shown in which the machine learning device learns the holding position of the workpiece in consideration of both the holding-prohibited part of the workpiece and the obstacle near the workpiece, but the present invention is not limited thereto. In the present invention, the machine learning device may also be configured to learn the holding position of the workpiece in consideration of only one of the holding-prohibited part of the workpiece and the obstacle near the workpiece.

[0110] In addition, in the above-described embodiment, an example is shown in which the machine learning device notifies the user of a re-inspection of both the hand and the jig tray based on the success probability of the reverse process operation for removing the workpiece, but the present invention is not limited thereto. In the present invention, the machine learning device may also be configured to notify the user of a re-inspection of only one of the hand and the jig tray based on the success probability of the reverse process operation for removing the workpiece.

[0111] Reference Signs Explanation

[0112] 10 Robot

[0113] 11 Hand

[0114] 20 Machine Learning Device

[0115] 30 Control Device

[0116] 100 Robot System

[0117] C Container

[0118] P1 Jig Tray

[0119] S Determined Setting State

[0120] W Workpiece

Claims

1. A machine learning method is a machine learning method for learning the actions of a robot with a hand. The hand takes out the workpieces from a container that houses multiple workpieces in a bulk state and sets the workpieces to a determined set state. Wherein, The machine learning method includes the following steps: A step of learning the reverse process action of removing the workpiece in the determined set state after the setting is completed by using the hand; and A step of learning the setting order of the workpiece to become the determined set state based on the learning result of the reverse process action of removing the workpiece.

2. The machine learning method according to claim 1, Wherein, The step of learning the reverse process action of removing the workpiece includes the following steps: repeatedly performing the reverse process action of removing the workpiece by using the hand until all or a part of the workpiece in the determined set state disappears.

3. The machine learning method according to claim 2, Wherein, The step of learning the reverse process action of removing the workpiece includes the following steps: while changing the order of removing the workpiece, repeatedly performing the reverse process action of removing the workpiece by using the hand until all or a part of the workpiece in the determined set state disappears.

4. The machine learning method according to claim 3, Wherein, The step of learning the setting order of the workpiece includes the following steps: learning multiple setting orders based on the learning result of the reverse process action of removing the workpiece, and learning the priorities of the multiple setting orders.

5. The machine learning method according to any one of claims 2 to 4, Wherein, The step of learning the reverse process action of removing the workpiece includes the following steps: in a state where a part of the order of removing the workpiece is set, repeatedly performing the reverse process action of removing the workpiece by using the hand until all or a part of the workpiece in the determined set state disappears.

6. The machine learning method according to any one of claims 1 to 4, Wherein, It further includes the following steps: learning the holding position of the workpiece held by the hand based on the learning result of the reverse process action of removing the workpiece.

7. The machine learning method according to claim 6, Wherein, The step of learning the holding position of the workpiece includes the following steps: learning the holding position of the workpiece in consideration of at least one of the constraints including the holding prohibited part of the workpiece and the obstacles near the workpiece.

8. The machine learning method according to any one of claims 1 to 4, Wherein, It further includes the following steps: notifying the user to re-check at least one of the hand and the jig tray for setting the workpiece based on the success probability of the reverse process action of removing the workpiece.

9. The machine learning method according to any one of claims 1 to 4, Wherein, The step of learning the reverse process actions for removing the workpiece includes the following steps: learning the reverse process actions for removing various workpieces using the hand.

10. A robot system comprising: A robot having a hand that takes out the workpiece from a container that houses a plurality of workpieces in a bulk state and sets the workpiece to a determined set state; A machine learning device that learns the actions of the robot; and A control device that controls the actions of the robot based on the learning results of the machine learning device, The machine learning device is configured to learn the reverse process actions for removing the workpiece in the determined set state after the setting is completed using the hand, and to learn the setting order of the workpiece to the determined set state based on the learning results of the reverse process actions for removing the workpiece.

11. The robot system according to claim 10, wherein, The machine learning device is configured to learn the holding position of the workpiece held by the hand based on the learning results of the reverse process actions for removing the workpiece.

12. The robot system according to claim 11, wherein, The machine learning device is configured to select the holding position of the workpiece that can take out the workpiece from the container and can set the workpiece.

13. The robot system according to claim 12, wherein, The machine learning device is configured to extract the holding position of the workpiece that can take out the workpiece from the container based on the extraction success probability of the workpiece, and to extract the holding position of the workpiece that can set the workpiece based on the setting success probability of the workpiece.

Images