Robot operation learning device, robot operation learning system, robot, and robot operation learning method

Abstract

To provide a robot operation learning method that enables relearning of a learning model in a short time and at a low hardware cost.SOLUTION: A robot operation learning device, which assists learning for executing an operation by a robot, comprises: one operation model; the other operation model; an error calculation part which calculates an error between the one operation model and the other operation model by using an operation execution result of the one operation model and an operation execution result of the other operation model; a collection range determination part which determines a collection range of learning data collected from peripheral environment of the robot by using the error calculated by the error calculation part; a data collection part which collects learning data by using the collection range determined by the collection range determination part; and a learning part which makes it learn at least any one of the one operation model and the other operation by using learning data collected by the data collection part.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technology that is effective when applied to a robot motion learning device, a robot motion learning system, a robot, and a robot motion learning method, which re-learn a learning model based on learning data collected from the robot's surrounding environment. [Background technology]

[0002] Learning-based robot control models are often used to achieve manipulation tasks with a high success rate in various different environments. The usual training method for learning-based models is to train in a lab environment and then retrain using data extracted from the field. This allows a learning-based model adapted to the field while maintaining high accuracy. However, since the range of training data that needs to be acquired from the field is not clear, the range of data acquisition is wide, which leads to problems such as extended learning time and hardware deterioration.

[0003] To solve this problem, for example, Patent Document 1 proposes an information processing system that can automatically switch learning without external instruction input by executing a model that has undergone reinforcement learning in advance and updating the learning model during execution based on the magnitude of the difference with existing environmental information and reward information. This allows the model to be learned only when learning is necessary, and furthermore, by using reinforcement learning, data collection is unnecessary, reducing the time spent collecting unnecessary learning data and reducing hardware deterioration. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2022 / 201796 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the information processing system of Patent Document 1 does not consider which range of the environment should be used for re-learning when it is determined that the difference with the existing environmental information or reward information is large. Therefore, learning may be performed over a wide range of the environment, and it may take a long time until the difference with the existing environmental information or reward information becomes small.

[0006] Therefore, an object of the present invention is to provide a robot motion learning device, a robot motion learning system, a robot, and a robot motion learning method that are capable of re-learning a learning model in a short time and with low hardware costs. [Means for solving the problem]

[0007] In order to solve the above problems, the present invention provides a robot motion learning device that assists a robot in learning to execute motion, comprising: an error calculation unit that calculates an error between a certain motion model and the other motion model using a certain motion model, another motion model, and a motion execution result of the certain motion model and the motion execution result of the other motion model; a collection range determination unit that determines a collection range of learning data to be collected from the surrounding environment of the robot using the error calculated by the error calculation unit; a data collection unit that collects learning data using the collection range determined by the collection range determination unit; and a learning unit that learns at least one of the certain motion model and the other motion model using the learning data collected by the data collection unit.

[0008] The present invention also provides a robot motion learning method for assisting a robot in learning to execute motions, comprising an error calculation step of calculating an error between a certain motion model and the other motion model using a motion execution result of the certain motion model and a motion execution result of the other motion model, a collection range determination step of determining a collection range of learning data to be collected from the surrounding environment of the robot using the error, a data collection step of collecting learning data using the collection range, and a learning step of learning at least one of the certain motion model and the other motion model using the learning data collected in the data collection step. Effect of the Invention

[0009] According to the present invention, it is possible to realize a robot motion learning device, a robot motion learning system, a robot, and a robot motion learning method that are capable of re-learning a learning model in a short time and with low hardware costs.

[0010] This enables the robot to perform manipulation tasks with a high success rate while reducing training time and hardware costs.

[0011] Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0012] [Figure 1] 1 is a block diagram showing a basic configuration of a robot action learning system according to a first embodiment of the present invention. [Diagram 2] 1 is a diagram showing a schematic configuration of a robot system according to a first embodiment of the present invention. [Diagram 3] 3 is a flowchart showing a robot movement learning method according to the first embodiment of the present invention. [Figure 4] FIG. 4 is a diagram illustrating a method of collecting learning data in the imitation learning of FIG. 3. [Diagram 5] FIG. 4 is a diagram illustrating a method of collecting learning data in the reinforcement learning of FIG. 3. [Figure 6] 4 is a flowchart showing a modification of FIG. 3. [Figure 7] FIG. 11 is a diagram showing a schematic configuration of a robot system according to a second embodiment of the present invention. [Figure 8] 10 is a flowchart showing a robot movement learning method according to a second embodiment of the present invention. [Figure 9] FIG. 11 is a diagram illustrating a method for generating actions for learning according to a second embodiment of the present invention. [Figure 10] FIG. 11 is a block diagram showing a configuration of a robot action learning system according to a third embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the drawings, the same components are designated by the same reference numerals, and detailed description of overlapping parts will be omitted. EXAMPLES

[0014] First Embodiment A robot motion learning method, a robot motion learning device, and a robot motion learning system according to a first embodiment of the present invention will be described with reference to FIGS.

[0015] 1 is a block diagram showing the basic configuration of a robot motion learning system 100 according to this embodiment. The robot motion learning system 100 according to this embodiment includes, as its main component, a robot motion learning device (learning support device) 101 that supports learning of a learning model of a motion command to be sent to a robot to be controlled. The robot motion learning device includes a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), a processor, etc., and is configured as a computer having a memory and peripheral circuits.

[0016] The robot motion learning device (learning support device) 101 includes an error calculation unit 104 that calculates an error between a certain motion model and the other motion model using a certain motion model, another motion model, and a motion execution result of the certain motion model and a motion execution result of the other motion model, a collection range determination unit 105 that determines a collection range of learning data to be collected from the robot's surrounding environment using the error calculated by the error calculation unit 104, a data collection unit 109 that collects learning data using the collection range determined by the collection range determination unit 105, and a learning unit 110 that learns at least one of the certain motion model and the other motion model using the learning data collected by the data collection unit 109.

[0017] In detail, the robot motion learning device (learning support device) 101 includes a first motion model 102, a second motion model 103 having a higher success probability of the robot motion compared to the first motion model 102, an error calculation unit 104 that calculates an error between the first motion model 102 and the second motion model 103 using a motion execution result of the first motion model 102 and a motion execution result of the second motion model 103, a collection range determination unit 105 that determines a collection range of learning data from the surrounding environment of the robot using the error calculated by the error calculation unit 104, and a collection range determination unit 106 that determines a collection range of learning data from the surrounding environment of the robot. The system includes a learning action generating unit 107 which generates a learning action based on the collection range determined by the range determining unit 105 and sends an action command to cause the action executing unit 106 to execute the action of the control target, an environmental data acquiring unit 108 which acquires environmental information obtained by the action execution unit 106 executing the action, a data collecting unit 109 which collects the environmental information from the environmental data acquiring unit 108 and the action command from the learning action generating unit 107 as learning data, and a learning unit 110 which extracts the learning data from the data collecting unit 109 and learns the first action model 102. Note that there may be a plurality of first action models and second action models.

[0018] Although FIG. 1 shows an example in which the environmental data acquisition unit 108 is disposed in the robot to be controlled, it may be disposed in the robot motion learning device (learning support device) 101 as described above.

[0019] In addition, while FIG. 1 shows an example of a configuration in which the robot motion learning device (learning support device) 101 and the robot to be controlled are separate, the robot motion learning device (learning support device) 101 may be built into the robot and configured as an integrated robot.

[0020] The error calculation unit 104 calculates, as an error, a difference between a motion position when executed by the first motion model 102 and a motion position when executed by the second motion model 103. The error calculation unit 104 may also calculate, as an error, a difference between a motion trajectory when executed by the first motion model 102 and a motion trajectory when executed by the second motion model 103. The error of the motion position, motion trajectory, etc. calculated by the error calculation unit 104 is the difference between the motion execution results according to each motion model.

[0021] The collection range determination unit 105 determines, as a collection range, a region in the vicinity of where an error occurs when the robot executes an operation, from among the environmental information acquired by the environmental data acquisition unit 108. The environmental information is information about the environment around the robot.

[0022] The data collection unit 109 has a learning data collection function for collecting data related to the operation of the robot within the collection range determined by the collection range determination unit 105 .

[0023] The learning unit 110 sends update information such as gradients to update the first operation model 102. The first operation model 102 and the second operation model 103 can cause the robot to execute a task by sending an operation command to the operation execution unit 106 of the controlled object.

[0024] Using the robot motion learning device (learning support device) 101 of this embodiment, it is possible to learn the first motion model 102 for application to a new environment in a short time and with low hardware costs.

[0025] A robot motion learning method using the robot motion learning device (learning support device) 101 of this embodiment will be described with reference to FIGS.

[0026] 2 is a diagram showing a schematic configuration of a robot system 200 according to this embodiment. In this embodiment, a robot motion learning system 100 is used for learning a control model of the robot system 200 shown in FIG.

[0027] The robot system 200 is composed of a robot arm 201, a belt conveyor 202, a plurality of fixed target parts 203 including four through holes, a plurality of base parts 204 including four screw holes, a plurality of screws 205, a screw feeder 206, a camera 207, and a camera fixing part 208. The robot arm 201 is composed of, for example, six joints (not shown), a force sensor 209 at the tip, and a gripper 210 with two fingers. The fixed target part 203 is transported near the robot arm 201 by the belt conveyor 202 and fixed to the base part 204 with the screws 205.

[0028] The task performed by the robot system 200 is an assembly task, and the operations required for this task will be described below. In this assembly task, after the belt conveyor 202 transports the base part 204 and the fixed target part 203, the robot arm 201 grasps the screw 205 of the screw feeder 206, and the robot arm 201 inserts the screw 205 into one hole of the fixed target part 203. After that, the joint closest to the tip of the robot arm 201 is rotated, and the screw 205 is screwed into the screw hole of the base part 204. By repeating this operation four times, the fixed target part 203 is fixed to the base part 204.

[0029] Since the positions of the robot arm 201 and the screw feeder 206 are fixed, the gripping of the screw 205 can be realized by a rule-based program. However, the insertion of the screw 205 into the hole of the fixing target part 203 is difficult to achieve by a rule-based program because the position of the part transported by the belt conveyor 202 is different each time.

[0030] Therefore, in the robot system 200 of this embodiment, in order to extract rough information about the hole position, the hole position is detected using an image processing algorithm such as template matching from the image from the camera 207, and the first movement model 102 or the second movement model 103, which is a data-driven learning model, generates the movement of the robot arm 201 based on the rough hole position information, and moves the robot arm 201 based on that, thereby realizing the insertion of the screw 205 into the hole in the fixed target part 203.

[0031] The motion in the air is generated based on image information from the camera 207, but the motion after the screw 205 comes into contact with the surface of the fixed target part 203 is generated based on not only the image information but also the contact information acquired by the force sensor 209. By using the information from the force sensor 209, the motion to search for the insertion hole can be generated more effectively.

[0032] In this assembly task, the robot control model that needs to be learned is the model used for inserting the screw 205, and in this embodiment, the first motion model 102 of the robot motion learning system 100 is assumed.

[0033] 3 is a flowchart showing a robot motion learning method according to the present embodiment. FIG 3 shows a learning method for the first motion model 102 when the robot system 200 is used in a new environment or for assembling a new part.

[0034] This situation is similar to when a learning model is transferred from a lab environment to the field. It is assumed that the first operating model 102 has been pre-trained in a previous environment similar to the current environment, and therefore, theoretically, can be used in the current environment without re-training. It is also assumed that the rule-based picking operation of the screw 205 is valid for the current system. It is further hypothesized that after the hole is found, the screwing of the screw 205 can be done by a rule-based program as well.

[0035] This robot motion learning method 300 begins with the motion execution unit 106 of the controlled object using a motion command from the first motion model 102 to attempt to insert a screw 205 into one hole in the fixed target part 203 using the robot arm 201 (step S301).

[0036] If the robot arm 201 succeeds in inserting the screw 205 (step S302a), this means that the first movement model 102 can find the hole with the robot arm 201 without re-learning the first movement model 102, and the learning flow is terminated.

[0037] On the other hand, if the robot arm 201 cannot insert the screw 205 (step S302b), the operation execution unit 106 searches for the hole using the operation command of the second operation model 103, which has a higher success rate for finding the hole (step S303).

[0038] If the hole cannot be found by second movement model 103 (step S304b), the actual position of the hole cannot be found, so re-learning of first movement model 102 is not possible, and the learning flow is terminated.

[0039] If a hole is found by second movement model 103 (step S304a), error calculation unit 104 detects an error between the movement from first movement model 102 and the movement from second movement model 103 (step S305).

[0040] Next, the collection range determination unit 105 determines the range of learning movement data for training of the first movement model 102 that should be collected for accomplishing the task, based on the movement error (step S306).

[0041] Then, the learning motion generator 107 generates a learning motion and sends this motion command to the motion execution unit 106, which then moves the robot arm 201 based on this motion command.

[0042] Next, the data collection unit 109 stores the operation command and the environmental information obtained during the operation (step S307).

[0043] Next, the learning unit 110 samples data from the data collecting unit 109 and re-learns the first action model 102 (step S308). This re-learning in step S308 is positioned as imitation learning or reinforcement learning, which will be described later.

[0044] Finally, after re-learning the first movement model 102, the first movement model 102 is used to try inserting the screw 205 again (step S301), and the robot movement learning method 300 is repeated until the first movement model 102 is able to accomplish the task.

[0045] The second operation model 103 may be either a rule-based or learning-based model, so long as it can realize a task with higher performance than the first operation model 102 .

[0046] An example of a rule-based model for the task of inserting a screw 205 is the spiral model, which searches for a hole by moving the robot arm 201 on a fixed spiral trajectory. This spiral model searches for a hole in a wide area where holes are likely to be found, so the probability of finding a hole, i.e., the success rate, is high. However, there is a problem in that it takes a long time to find a hole depending on the initial search position. By using a learning model such as the first operation model 102 used for environmental data, holes can be found in a shorter time, so the learning model is preferable in terms of the lead time for completing a field task.

[0047] The subtask assumed in this embodiment is to insert a screw 205 into a hole in a fixed target part 203, so the motion error can be determined as the difference between the center of the hole found by the second motion model 103 and the hole position predicted by the first motion model 102, that is, as a position error. This position error can be calculated using the Euclidean distance equation of formula (1).

[0048]

number

[0049] In this equation, d is the position error, x1 is the position of the first motion model 102 on the x-axis, x2 is the position of the second motion model 103 on the x-axis, y1 is the position of the first motion model 102 on the y-axis, and y2 is the position of the second motion model 103 on the y-axis.

[0050] There are two possible methods for collecting data and learning the first movement model 102 using the movement error (position error) in this embodiment.

[0051] <Imitation learning> FIG. 4 is a diagram showing a first example of a method for collecting data for training the first motion model 102. As shown in FIG.

[0052] This data collection method 400 is a method for contacting a screw 205 with the surface of a fixed target part 203 at multiple positions on a circle with a radius equal to a maximum distance error 402, sliding the screw 205 to the center of a hole 401 through a sliding motion trajectory 403 until the screw 205 is inserted into the hole 401, and collecting images from a camera 207, environmental data such as force and torque from a force sensor 209, and motion commands such as the joint angles of the robot during the motion.

[0053] The maximum distance error 402 is the same as the maximum value of the motion error described above. After collecting data, imitation learning is applied to the learning of the first motion model 102 by this data collection method 400. Imitation learning is a learning method that uses a mean square error loss function and a stochastic gradient descent optimizer to minimize the error between the motion (output) of the training data set and the motion generated by the first motion model 102, and is suitable for the data collection method 400.

[0054] 4 needs to be determined, and in some cases it may become huge, but since the amount of movement is limited by the maximum distance error 402, the amount of movement is smaller than sliding in a wide range. In other words, since the data range obtained from the maximum distance error 402 is smaller, the amount of learning data can be minimized, and it is possible to reduce the learning time and hardware costs, which are the goals of the present invention.

[0055] In this data collection method 400, the collection range determined by the collection range determination unit 105 is a circular area, the movement position obtained by the second movement model 103 is the center of the circular area, and the maximum distance error 402 is the radius of the circular area.

[0056] The data collected for learning the first movement model 102 is the environmental information and movement information obtained when moving from a position that is a distance of the maximum distance error 402 from the center of the circular area to the center of the circular area.

[0057] <Reinforcement Learning> FIG. 5 is a diagram showing a second example of a method for collecting data for training the first motion model 102. As shown in FIG.

[0058] In this data collection method 500, the robot arm 201 contacts the screw 205 at multiple data collection positions 502 of the hole 501, which are represented by circular marks, and within a limit position determined by a data collection limit position 503, which is represented by a triangular mark. The data collection limit position 503 is determined by the maximum distance error 402.

[0059] Similar to data collection method 400, the environmental data collected by data collection method 500 is the image from camera 207 or the force or torque from force sensor 209, and the action command collected is the robot joint angle. However, in data collection method 500, data is not collected at a set frequency (e.g., every 0.1 seconds), but is collected only at data collection position 502 and data collection limit position 503. By doing so, a map of the area around the hole can be created, and the map can be used to build an offline learning environment.

[0060] The offline learning environment can be used for reinforcement learning, which is suitable for the present data collection method 500. The offline learning environment can be used for a reinforcement learning agent to learn the first behavior model 102 and enable the first behavior model 102 to perform a task.

[0061] 5 needs to be determined in advance, and in some cases it may become huge, but since the number of data collection positions 502 is limited by the data collection limit position 503, it is less than collecting data in a wide range. In other words, since the data range obtained from the data collection limit position 503 is smaller, the amount of learning data can be minimized, and it is possible to reduce the learning time and hardware costs, which are the goals of the present invention.

[0062] In this data collection method 500, the collection range determined by the collection range determination unit 105 is a rectangular area, the motion position obtained by the second motion model 103 is the center of the rectangular area, and the error determines the data collection limit position 503 of the rectangular area.

[0063] The data obtained from the collection range is a plurality of discretized points within the collection range, and a reinforcement learning environment is constructed using a plurality of the discretized points, and the first operation model 102 is re-learned using the reinforcement learning environment.

[0064] As described above, by using the robot motion learning device (learning support device) 101 of this embodiment, it is possible to learn the first motion model 102 for application to a new environment in a short time and with low hardware costs.

[0065] <<Variations>> FIG. 6 is a flow chart showing a modification of FIG.

[0066] The robot motion learning method shown in FIG. 6 differs from the robot motion learning method shown in FIG. 3 in that it has two additional steps shown as step S606 and step S607 between step S305 and step S306 in the flow of FIG.

[0067] Similar to the robot motion learning method of FIG. 3, after the position error (motion error) is determined in step S605, in this variant, it is checked whether it is the first time that the first motion model 102 has failed to find the hole (step S606), and if so, as an adjustment step S607, the first motion model 102 is tested again for the same task in the same environment (step S601) simply by subtracting the distance error from the predicted position.

[0068] This is done because the prediction errors of the first motion model 102 may be systematic and therefore can be corrected by a simple calibration of the first motion model 102 without having to retrain the model.

[0069] The additional step of this modification in the update flow of the first operation model 102 can further accelerate the adaptation of the first operation model 102 to a new environment. EXAMPLES

[0070] Second Embodiment A robot motion learning method, a robot motion learning device, and a robot motion learning system according to a second embodiment of the present invention will be described with reference to FIGS.

[0071] 7 is a diagram showing a schematic configuration of a robot system 700 according to this embodiment. In this embodiment, a robot motion learning system 100 is used for learning a control model of the robot system 700 shown in FIG.

[0072] This robot system 700 is composed of substantially the same components as the robot system 200 of the first embodiment (FIG. 2). However, instead of the fixed target portion 203, the base portion 204, the plurality of screws 205, and the screw feed device 206, this robot system 700 is composed of an object to be grasped 701 and a grasped object collection device 702. In addition, the fingers of the gripper 210 include contact sensors 703.

[0073] The object-to-be-grasped collection device 702 may be a container that an operator periodically places therein, or a mobile robot that transports the collected objects to be grasped 701 to another location when the container is full of the objects to be grasped 701. When mobile robots are used, multiple mobile robots are required, and it is assumed that when one robot becomes full of the objects to be grasped 701 and moves to another location, another mobile robot moves to the same location instead.

[0074] The task performed by the robot system 700 is a pick-and-place task for palletizing, etc., and the operations required for this task will be described below. In this pick-and-place task, after the belt conveyor 202 transports the object to be grasped 701, the robot arm 201 grasps the object to be grasped 701, moves to the vicinity of an empty space on the object to be grasped collection device 702, and releases the object to be grasped 701.

[0075] The operation flow of this embodiment will be described with reference to Fig. 8 and Fig. 9. Fig. 8 is a flowchart showing the robot operation learning method of this embodiment, and shows the learning method of the first operation model 102 when the robot system 700 is used in a new environment or when a new part is picked and placed for use. Fig. 9 is a diagram showing a method of generating operations for learning.

[0076] The first motion model 102 is pre-trained in a previous environment similar to the current environment, and therefore, theoretically, it is assumed that it can be used in the current environment without re-learning. The robot motion learning method 800 of this embodiment starts with the motion execution unit 106 attempting to grasp (pick) the grasp target object 701 with the robot arm 201 using a motion command from the first motion model 102 (step S801).

[0077] If the robot arm 201 succeeds in grasping the object to be grasped 701 (step S802a), this means that the first movement model 102 is capable of grasping the object to be grasped 701 by the robot arm 201 without re-learning the first movement model 102, and the learning flow is terminated.

[0078] On the other hand, if the robot arm 201 cannot grasp the object to be grasped 701 (step S802b), the action execution unit 106 uses the action commands of the second action model 103, which can grasp the object to be grasped 701 with a higher success rate, to search for the position of the object to be grasped 701 using a contact sensor 703 or the like, and grasps the object to be grasped 701 (step S803).

[0079] If the second movement model 103 cannot grasp (find) the grasp target object 701 (step S804b), re-learning of the first movement model 102 is impossible, and therefore the learning flow is terminated.

[0080] When the second movement model 103 can grasp the object to be grasped 701 (step S804a), the error calculation unit 104 detects an error between the movement from the first movement model 102 and the movement from the second movement model 103 (step S805).

[0081] Next, the collection range determination unit 105 determines the range of learning movement data for training of the first movement model 102 that should be collected for accomplishing the task, based on the movement error (step S806).

[0082] Next, the learning motion generator 107 generates a learning motion and sends this motion command to the motion execution unit 106, which then moves the robot arm 201 based on this motion command (step S807).

[0083] This specific operation is shown in Fig. 9. After grasping the object to be grasped 701, the robot arm 201 moves the object to be grasped 701 along a random movement trajectory 902 into a data collection limit range 901 obtained from a maximum grasping position error, which is the difference between the position at which the object to be grasped 701 is grasped by the second operation model 103 and the position of the object to be grasped 701 predicted by the first operation model 102, and then releases it.

[0084] Next, the operation command and the environmental information are stored in the data collection unit 109 (step S808), and it is confirmed whether a predetermined target amount of data has been achieved (step S809).

[0085] If the target data amount has not been achieved (step S809b), the picking and moving operation of the object to be grasped 701 (step S807) is repeated. If the target data amount has been achieved (step S809a), the learning unit 110 re-learns the first action model 102 using the data stored by the data collection unit 109 (step S810).

[0086] Finally, after re-learning the first movement model 102, the first movement model 102 is again used to grasp a new object to be grasped 701 (step S801), and this learning flow 800 is repeated until the first movement model 102 is able to accomplish the task.

[0087] In addition, the method of finding the position of the object to be grasped 701 by the second operation model 103 of this embodiment in steps S803 and S804 is to move the gripper 210 to a height where it contacts the object to be grasped 701, and then move the gripper 210 on a rule-based trajectory such as a spiral trajectory, and when the contact sensor 703 installed on the gripper 210 detects contact, calculate the position of the object to be grasped 701 based on the specified dimensions of the object to be grasped 701.

[0088] In this embodiment, it is necessary to determine a target data amount, which may become huge in some cases, but since the position of the object to be grasped 701 is limited by the data collection limit range 901 obtained from the maximum grip position error, the amount of data required is smaller than if the object to be grasped 701 were positioned over a wide range. In other words, by using the data collection limit range 901 obtained from the grip position error, the amount of learning data can be minimized, making it possible to reduce the learning time and hardware costs, which are the goals of the present invention.

[0089] As described above, by using the robot motion learning device (learning support device) 101 of this embodiment, it is possible to learn the first motion model 102 for application to a new environment in a short time and with low hardware costs. EXAMPLES

[0090] Third Embodiment With reference to FIG. 10, a robot motion learning method, a robot motion learning device, and a robot motion learning system according to a third embodiment of the present invention will be described.

[0091] FIG. 10 is a block diagram showing the configuration of a robot motion learning system 100 of this embodiment, which shows yet another modification of the first embodiment (FIG. 1).

[0092] 10, a robot motion learning device (learning support device) 101 of this embodiment differs from that of the first embodiment (FIG. 1) in that, in addition to the configuration of the first embodiment (FIG. 1), it further includes a third motion model 111 that has a higher success probability of the motion of the robot arm 201 compared to the first motion model 102 and the second motion model 103. The other configurations are the same as those of the first embodiment (FIG. 1).

[0093] The error calculation unit 104 uses the operation execution results of the first operation model 102, the operation execution results of the second operation model 103, and the operation execution results of the third operation model 111 to calculate the error between the first operation model 102 and the second operation model 103, and the error between the first operation model 102 and the third operation model 111.

[0094] Then, if the error between the first operation model 102 and the third operation model 111 is greater than the error between the first operation model 102 and the second operation model 103, the collection range determination unit 105 uses the error between the first operation model 102 and the third operation model 111 to determine the collection range of learning data to be collected from the surrounding environment of the robot arm 201.

[0095] On the other hand, if the error between the first operation model 102 and the second operation model 103 is greater than the error between the first operation model 102 and the third operation model 111, the collection range determination unit 105 uses the error between the first operation model 102 and the second operation model 103 to determine the collection range of learning data to be collected from the surrounding environment of the robot arm 201.

[0096] As in this embodiment, by comparing the first action model 102 with a plurality of models, it becomes possible to re-learn the first action model 102 with a higher success rate.

[0097] In each of the above embodiments, the differences between the behavior models have been explained based on the probability of success of the robot's movement. However, each behavior model may be created based on other parameters besides the success probability, such as the length of time the robot takes to move.

[0098] Furthermore, the present invention is not limited to the above-mentioned embodiment, and various modifications are included. For example, the above-mentioned embodiment has been described in detail to clearly explain the present invention, and is not necessarily limited to those having all of the configurations described. Furthermore, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration. [Explanation of symbols]

[0099] 100...Robot motion learning system 101...Robot motion learning device (learning support device) 102...First operating model 103...Second operating model 104...Error calculation section 105...Collection range determination unit 106...Operation execution unit 107...Learning motion generation unit 108…Environmental Data Acquisition Section 109…Data Collection Section 110…Learning Department 111…Third operating model 200,700…Robot system 201...Robot arm 202...Conveyor belt 203...Fixed target part 204…Base section 205…Screw 206...Screw feed device 207…Camera 208…Camera fixing part 209... Force sensor 210…Gripper 300, 600, 800...Robot movement learning method 400...Data collection method for imitation learning 401...hole 402…Maximum distance error 403...Sliding motion trajectory 500…Data collection method for reinforcement learning 501...hole 502...Data collection location 503...Data collection limit position 701...Object to be grasped 702...Gripped object collection device 703...Contact sensor 901…Data collection limit range 902...Random movement orbit.

Claims

1. A robot motion learning device that supports learning for a robot to execute a motion, comprising: A certain operating model, The other operating model, an error calculation unit that calculates an error between the one behavior model and the other behavior model using an operation execution result of the one behavior model and an operation execution result of the other behavior model; a collection range determination unit that determines a collection range of learning data to be collected from the surrounding environment of the robot by using the error calculated by the error calculation unit; a data collection unit that collects learning data using the collection range determined by the collection range determination unit; a learning unit that learns at least one of the one behavior model and the other behavior model by using learning data collected by the data collection unit; A robot motion learning device comprising:

2. The robot motion learning device according to claim 1, the certain operating model is a first operating model, the other movement model is a second movement model having a higher success probability of the robot movement compared to the first movement model; the error calculation unit calculates an error between the first operation model and the second operation model using an operation execution result of a first operation model and an operation execution result of a second operation model; A robot motion learning device characterized in that the learning unit learns the first motion model based on learning data collected by the data collection unit.

3. The robot motion learning device according to claim 2, A robot motion learning device characterized in that the error calculation unit calculates the difference between the motion position when executed according to the first motion model and the motion position when executed according to the second motion model as an error.

4. The robot motion learning device according to claim 3, The robot motion learning device according to claim 1, wherein the collection range determination unit determines, as the collection range, an area of ​​the surrounding environment in the vicinity of where the error occurs when the robot executes a motion.

5. The robot motion learning device according to claim 4, A robot motion learning device characterized in that the data collection unit has a learning data collection function of collecting data related to the robot's motion within the collection range determined by the collection range determination unit.

6. The robot motion learning device according to claim 4, The collection range is a circular area, the motion position obtained by the second motion model is the center of the area; A robot motion learning device, wherein the error is a radius of the area.

7. The robot motion learning device according to claim 6, A robot motion learning device characterized in that the data collected for learning the first motion model is environmental information and motion information obtained when moving from a position away from the center of the area by the error distance to the center of the area.

8. The robot motion learning device according to claim 7, A robot motion learning device characterized in that the learning unit re-learns the first motion model by imitation learning using the collected learning data.

9. The robot motion learning device according to claim 4, The collection range is a rectangular area, the motion position obtained by the second motion model is the center of the area; A robot motion learning device, characterized in that the error defines the limits of the area.

10. The robot motion learning device according to claim 9, the data obtained from the collection range is a plurality of discretized points within the collection range; A robot motion learning device characterized in that the learning unit constructs a reinforcement learning environment using a plurality of the discretized points, and re-learns the first motion model using the reinforcement learning environment.

11. The robot motion learning device according to claim 4, A robot motion learning device, wherein the second motion model is a rule-based model.

12. The robot motion learning device according to claim 11, the second motion model is utilized for inserting a rod-shaped member into a hole by the robot; A robot motion learning device characterized in that the robot arm of the robot inserts the rod-shaped member into the hole by sliding the rod-shaped member around the hole on a trajectory that covers all of a fixed search area.

13. The robot motion learning device according to claim 11, the second operation model is utilized for picking an object by the robot; The robot motion learning device is characterized in that the robot moves its gripper through the air on a trajectory that covers all of a fixed search area, and detects the position of the object when contact with the object is detected by a contact sensor installed on the gripper.

14. The robot motion learning device according to claim 2, a third movement model having a higher success probability of the movement of the robot compared to the first movement model and the second movement model; The robot motion learning device is characterized in that the error calculation unit calculates the error between the first motion model and the second motion model and the error between the first motion model and the third motion model using the motion execution results of the first motion model, the motion execution results of the second motion model, and the motion execution results of the third motion model.

15. A robot motion learning method for supporting learning for a robot to execute a motion, comprising: an error calculation step of calculating an error between a certain behavior model and the other behavior model using an operation execution result of the certain behavior model and an operation execution result of the other behavior model; a collection range determination step of determining a collection range of learning data to be collected from the surrounding environment of the robot using the error; a data collection step of collecting learning data using the collection range; a learning step of learning at least one of the one behavior model and the other behavior model by using the learning data collected in the data collection step; A robot motion learning method comprising:

16. A robot motion learning system comprising the robot motion learning device according to any one of claims 1 to 14, Equipped with a robot, the robot includes a motion execution unit that executes a motion based on a motion command according to a motion model re-learned by the robot motion learning device; an environmental data acquisition unit that acquires environmental information obtained by the action execution unit; A robot motion learning system comprising:

17. A robot motion learning device according to any one of claims 1 to 14 is built in, a motion execution unit that executes a motion based on a motion command according to a motion model re-learned by the robot motion learning device; and an environmental data acquisition unit that acquires environmental information obtained by the action execution unit.

Images