Learning device, learning method, and recording medium

Abstract

To provide a learning device for generating a learning model capable of generating teaching data of an articulated robot.SOLUTION: The learning device includes a setting unit that sets a task including a first target value range for a tip position of the articulated robot and a second target value range for a direction in which the tip position is to be directed, a determination unit that determines whether first tip position information indicating the tip position of the articulated robot after the articulated robot is driven is within the first target value range and whether first direction information indicating a direction of the tip position of the articulated robot is within the second target value range, a learning unit that learns a correspondence relationship among a second tip position of the articulated robot before the articulated robot is driven by a drive control unit, second direction information indicating a direction in which the tip position is directed, a control value, the first tip position information, the first direction information, and a determination result, and an update unit that updates the task when one or more determination results by the determination unit satisfy a predetermined condition.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a learning device, a learning method, and a learning program that provide information processing for controlling an articulated robot. [Background technology]

[0002] Offline teaching of a robot usually requires human intervention, and the person conducting the teaching must have deep expertise and experience. Therefore, a method of operating a robot using a learning model has been devised. Patent Document 1 discloses a machine learning device that aims to automate the grasping and transporting of a workpiece by a robot hand with a simple configuration. The device learns a learning model that infers the three-dimensional coordinates of a robot hand in the real world by using a learning dataset that includes three-dimensional coordinate data of a robot hand when the robot hand on a simulator successfully grasps a workpiece and two-dimensional image data of the workpiece captured from a predetermined angle of view. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-82322 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in such learning models, depending on the target task, if the task is difficult, it may be difficult to obtain a judgment that the task has been achieved (success), and it may take a long time for learning to reach a solution.

[0005] Therefore, the present invention has been made in consideration of the above problems, and aims to provide a learning device, a learning method, and a learning program that will enable learning to lead to a solution and, in turn, enable learning to solve even difficult problems. [Means for solving the problem]

[0006] A learning device according to one aspect of the present invention includes a setting unit that sets a task including a first target value range for the tip position of a multi-joint robot and a second target value range for the direction in which the tip position should face; a drive control unit that assigns random control values ​​to at least each axis of the multi-joint robot; an acquisition unit that acquires first tip position information indicating the tip position of the multi-joint robot after assigning random control values ​​to the current values ​​of each axis of the multi-joint robot and first direction information indicating the direction in which the tip position is facing; a determination unit that determines whether the first tip position information is within the first target value range and whether the first direction information is within the second target value range; a learning unit that learns the correspondence between a second tip position of the multi-joint robot before driving by the drive control unit of the multi-joint robot and second direction information in which the tip position is facing, the random control values ​​of each axis assigned by the drive control unit, the first tip position information and the first direction information, and the determination results by the determination unit; and an update unit that updates the task when one or more determination results by the determination unit satisfy a predetermined condition.

[0007] Furthermore, a learning method according to one aspect of the present invention includes a setting step in which a computer sets a task including a first target value range for the tip position of the articulated robot and a second target value range for the direction in which the tip position should face; a drive control step in which a random control value is applied to at least each axis of the articulated robot; an acquisition step in which first tip position information indicating the tip position of the articulated robot after the random control value is applied to the current value of each axis of the articulated robot and first direction information indicating the direction in which the tip position is facing; a first determination step in which the computer determines whether the first tip position information is within the first target value range and whether the first direction information is within the second target value range; a learning step in which the computer learns the correspondence between the second tip position of the articulated robot before driving by the drive control step of the articulated robot and the second direction information in which the tip position is facing, the random control value of each axis applied by the drive control step, the first tip position information and the first direction information, and the determination result by the first determination step; and an update step in which the computer updates the task if one or more determination results by the first determination step satisfy a predetermined condition.

[0008] Furthermore, a learning program according to one aspect of the present invention has a setting function that sets a task including a first target value range for the tip position of the articulated robot and a second target value range for the direction in which the tip position should face in a computer; a drive control function that assigns random control values ​​to at least each axis of the articulated robot; an acquisition function that acquires first tip position information that indicates the tip position of the articulated robot after assigning random control values ​​to the current values ​​of each axis of the articulated robot and first direction information that indicates the direction in which the tip position is facing; a first determination function that determines whether the first tip position information is within the first target value range and whether the first direction information is within the second target value range; a learning function that learns the correspondence between the second tip position of the articulated robot before driving by the drive control function of the articulated robot and second direction information in which the tip position is facing, the random control values ​​of each axis assigned by the drive control function, the first tip position information and the first direction information, and the determination result by the first determination function; and an update function that updates the task if one or more determination results by the first determination function satisfy a predetermined condition.

[0009] In addition, in the above learning device, the first target value range may be information indicating the distance between the tip position indicated by the first tip position information and the tip position where the tip position should be, and the second target value range may be information indicating the allowable angle range of the angle between the direction indicated by the first direction information and the direction where the tip position should be.

[0010] In the above-described study device, the update unit may update the tasks so that the level of difficulty of the tasks increases.

[0011] In the learning device, the update unit may update the task so as to narrow at least one of the first target value range and the second target value range.

[0012] Furthermore, the above learning device may include a calculation unit that calculates a score indicating an evaluation of the drive by the drive control unit based on at least the distance from the tip position indicated by the first tip position information to the tip position indicated by the first target value range and the angle formed between the direction indicated by the first direction information and the direction indicated by the second target value range each time a judgment is made by the judgment unit, and the update unit may update the task when, as a predetermined condition, the cumulative value of the scores calculated by the calculation unit for the same task exceeds a predetermined threshold value.

[0013] In the learning device, the articulated robot may be a robot in a virtual space, and the drive control unit may drive the articulated robot that is a robot in the virtual space. [Effects of the Invention]

[0014] The learning device can generate a learning model that can ultimately achieve the target task by learning in stages, from easy tasks to more difficult tasks.Therefore, by using this learning model, teaching data for an articulated robot can be generated without human intervention. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 2 is a block diagram illustrating an example of the configuration of a learning device. [Figure 2] FIG. 1 is a diagram illustrating an example of the appearance of an articulated robot. [Figure 3] 10A and 10B are diagrams illustrating the tip position and the direction of the tip position. [Figure 4] 10 is an example of assignment information showing an example of an assignment. [Figure 5] 10 is a flowchart illustrating an example of the operation of the learning device. DETAILED DESCRIPTION OF THE INVENTION

[0016] A learning device according to an embodiment of the present invention will be described in detail below with reference to the drawings.

[0017] <Embodiment> <Configuration> The learning device 100 according to the embodiment shown in FIG. 1 is a computer system that can automatically generate a learning model capable of generating information for teaching the operation of an articulated robot or can re-learn and train an existing learning model. The learning device 100 may be a reinforcement learning device, and in particular, may be a device that trains a learning model by performing curriculum learning. Curriculum learning is a learning method that starts with an easy task, and each time the conditions for achieving that task are met, a more difficult task is set, and the learning model is trained to achieve the final goal by learning whether the task was achieved or not, as well as the process.

[0018] The learning device 100 controls the articulated robot based on the rotation angle of each axis constituting the articulated robot, and outputs the rotation angle of each axis as a numerical value. In the first embodiment, the learning device 100 controls the articulated robot in a virtual space (on a simulator), sets tasks to be executed by the articulated robot, and performs curriculum learning based on the simulation results. Note that the learning device 100 may also control an actual articulated robot.

[0019] Here, an articulated robot is a robot with multiple axes rotatably connected, such as an industrial manipulator, or more specifically, a welding robot. An articulated robot basically performs a predetermined task at its tip, and in order to perform the predetermined task, the tip is moved to a predetermined position in a predetermined posture. Tasks performed by an articulated robot include, but are not limited to, welding of an object, cutting of an object, picking and releasing of an object, etc. The object may be anything that the articulated robot processes, such as, but not limited to, a car body. The object may also be called a workpiece.

[0020] In this embodiment, a welding robot will be described as an example of an articulated robot. The articulated robot is a six-axis robot, but is not limited to six axes.

[0021] FIG. 2 illustrates an example of an articulated robot 1, specifically a six-axis robot that is a welding robot. While the articulated robot 1 will be described as a six-axis robot, it goes without saying that the articulated robot 1 is not limited to a six-axis robot. In a six-axis robot, as shown in FIG. 2, a first axis A1 rotates in the direction indicated by arrow R1, and a second axis A2 connected to the first axis A1 rotates in the direction indicated by arrow R2. When an arm M1 connected to the second axis A2 rotates around the second axis A2, a third axis A3 connected to the end of the arm M1 also moves. The third axis A3 rotates in the direction indicated by arrow R3. The arm M2 connected to the third axis A3 rotates around the third axis A3 in conjunction with the rotation of the third axis A3, and a fifth axis A5 connected to the end of the arm M2 also moves. Furthermore, the arm M2 rotates around the fourth axis A4 in the direction indicated by arrow R4. Accordingly, the fifth axis A5 also rotates around the fourth axis A4. The fifth axis A5 rotates in the direction indicated by arrow R5, and the arm M3 connected to the end of the fifth axis A5 also rotates in conjunction with this. The arm M3 rotates in the direction indicated by arrow R6 about the sixth axis A6, and in conjunction with this, the tip position of the articulated robot 1 also rotates. The articulated robot 1 is basically the same as a conventional robot, so a detailed description of its configuration will be omitted.

[0022] By rotating each of these first axis A1 to sixth axis A6, the articulated robot 1 can move the welding point at the tip to any position. When controlling an actual articulated robot, the learning device 100 may be connected to the articulated robot 1 so as to be able to communicate with it via wire or wirelessly, and may function as a control computer capable of controlling the articulated robot 1.

[0023] As shown in FIG. 1, a learning device 100 according to the embodiment includes a storage unit 120 and a control unit 130, and may also include a communication unit 110, an input unit 140, and an output unit 150.

[0024] The communication unit 110 is a communication interface that communicates with an external device via a network, either wired or wirelessly. The communication unit 110 receives information transmitted from the external device and transmits the information to the control unit 130. The communication unit 110 also transmits designated information to a designated destination in accordance with instructions from the control unit 130. For example, the communication unit 110 may receive information relating to a task to be performed by the articulated robot 1 from an external device and transmit the information to the control unit 130. For example, the communication unit 110 may transmit a learning model generated by the learning device 100 to an external device, or may transmit data (teaching data) for controlling the articulated robot to an actual articulated robot.

[0025] The storage unit 120 has the function of storing various programs and various data required for the operation of the learning device 100. The storage unit 120 can be realized by various recording media, such as, but not limited to, a hard disk drive (HDD), a solid state drive (SSD), or a flash memory. The storage unit 120 may store a program for simulating the operation of the articulated robot 1 in a virtual space, the simulation results, and a learning program for learning the task content, and may also store various control values ​​(signal values, parameter values, etc.) of the articulated robot 1. The storage unit 120 may also store a learning model 121 generated by the learning device 100 through reinforcement learning.

[0026] The learning model 121 may be a model that learns the correspondence between the tip position (first position) and orientation (first direction) of the articulated robot 1, the drive amount (rotation amount) of each axis of the articulated robot 1 while it is at that position, and the tip position (second position) and orientation (second direction) of the articulated robot 1 after movement (after driving) when the drive amount is applied to each axis of the articulated robot 1. The learning model 121 may be a model that receives input of the current tip position and orientation of the articulated robot and a target position and target orientation to which the tip position should reach, and estimates the amount of rotation to be applied to each axis of the articulated robot 1 so that the tip position reaches the target position and target orientation. Note that the tip position (first position, second position) and orientation (first direction, second direction) of the articulated robot 1 may be determined by coordinates and orientation in a virtual space in which the articulated robot 1 is simulated. Furthermore, the tip position (first position) and orientation (first direction) of the articulated robot 1 may be defined by the amount of rotation of each axis of the articulated robot 1 at that time.

[0027] Here, the tip position and direction of the articulated robot 1 and the target tip position and direction will be described with reference to FIG.

[0028] FIG. 3 is an enlarged view showing an example of the tip of the articulated robot 1 and an object to be welded. FIG. 3 illustrates the articulated robot 1, an object 30, and the target orientation of the articulated robot 1. The tip position of the articulated robot 1 in this specification refers to tip position 310 in FIG. 3, and the orientation of the tip position of the articulated robot 1 refers to orientation 320, which is on the aforementioned axis A5. Reference numeral 301 on the object 30 indicates a welding point, which may be set as the target position of the articulated robot 1 in this embodiment. The target position may be set not at the welding point but at a location slightly away from the welding point (e.g., position 303). Therefore, whether the distance d between the tip position 310 of the articulated robot 1 and the welding point 301 is within a predetermined distance is the first criterion for determining whether the task has been successfully completed.

[0029] Furthermore, a direction 302 (second direction) is set as a suitable direction for performing processing relative to the target position 301. Whether or not the angle θ formed by this direction 302 and the direction 320 of the articulated robot 1 is within a predetermined angle range is the second criterion for determining whether or not the task has been successful.

[0030] The distance d and angle θ shown in FIG. 3 are information that serve as indices used by learning device 100 to determine whether the task has been completed.

[0031] Control unit 130 is a processor that has the function of controlling each unit of learning device 100. Control unit 130 realizes the functions of learning device 100 by reading and executing various programs stored in storage unit 120.

[0032] The control unit 130 sets a task for the articulated robot 1, and if a predetermined condition is met as to whether the simulation of that task is successful, it sets a new, more difficult task and, by learning this process, generates a learning model 121 that can estimate control information (teaching data) for each axis of the articulated robot 1 to bring the articulated robot 1 from its current state to the target position and orientation.

[0033] The control unit 130 functions as a setting unit 131 , a drive control unit 132 , an acquisition unit 133 , a determination unit 134 , a learning unit 135 , a calculation unit 136 , and an update unit 137 .

[0034] The setting unit 131 sets a task including a first target value range for the tip position of the articulated robot 1 and a second target value range for the direction in which the tip position should face. As the task, at least an allowable distance from the target position of the tip position (distance 430 in FIG. 4 described later) and an angular difference between the orientation of the tip position and the target orientation (angular difference 440 in FIG. 4 described later) are set. The target position and target orientation themselves may be set each time, or may be fixed until one task is completed.

[0035] The drive control unit 132 provides a control value for each axis of the articulated robot 1. The drive control unit 132 generates a random control value for each axis of the articulated robot 1 within the movable range of each axis, and provides it to the articulated robot 1 on the simulator. Note that the random control values ​​may include control values ​​for the six axes that do not change from their current states. However, the drive control unit 132 generates a control value such that at least one axis changes from its current state. After generating the random control value for each axis of the articulated robot 1, the drive control unit 132 provides (drives) the axis of the articulated robot 1 on the simulator. The random control value generated by the drive control unit 132 may be a control value estimated using the learning model 121 (the learning model at the time of starting the simulation of the current task) generated by the learning unit 135.

[0036] The acquisition unit 133 acquires information indicating the tip position and the orientation of the tip position of the articulated robot 1 driven by the drive control unit 132. The acquisition unit 133 may acquire information indicating the tip position and the orientation of the tip position of the articulated robot 1 in accordance with the coordinate system on the simulator of the articulated robot 1 on the simulator.

[0037] The determination unit 134 determines whether or not the tip position and the orientation of the tip position of the articulated robot 1 acquired by the acquisition unit 133 are included within the target position and target orientation set by the setting unit 131, as well as within the respective ranges set as tasks. The determination unit 134 transmits the determination result to the learning unit 135.

[0038] The learning unit 135 performs learning using as training data information that associates the tip position and orientation of the tip position of the articulated robot 1 before the articulated robot 1 is driven by the drive control unit 132, the control values ​​of each axis provided by the drive control unit 132, the tip position and orientation of the tip position of the articulated robot 1 after the articulated robot 1 is driven by the drive control unit 132, and the determination result by the determination unit 134. By learning the correspondence between these pieces of information, the learning unit 135 can generate a learning model 121 that can estimate the control value to be provided to the articulated robot 1 by inputting information on the tip position and orientation of the tip position before the articulated robot 1 is driven and the target position and orientation. The learning unit 135 may perform learning using, for example, a neural network or a support vector machine, but is not limited to these.

[0039] The calculation unit 136 calculates a score to determine whether the task set by the setting unit 131 has been completed. The calculation method for the score may be arbitrary, but the score is calculated so that a positive score is obtained if the task is successful, and a negative score or a lower score than if the task is unsuccessful. The calculation unit 136 may set points for various situations, such as 0.2 points if the task is successful, -0.3 points if the task is unsuccessful, and 0.05 points if the tip position of the articulated robot 1 is within X mm (a distance separate from the task) from the target position (very close), and may calculate the score by adding up the points. Note that the score calculation example shown here is just an example.

[0040] Then, the calculation unit 136 calculates the cumulative value of the score calculated for the same task each time the articulated robot 1 is operated. The calculation unit 136 transmits the calculated cumulative value of the score to the update unit 137.

[0041] When the cumulative value of the scores calculated by the calculation unit 136 exceeds a threshold set for the task, the update unit 137 updates the content of the task currently set for the setting unit 131. The update unit 137 updates the task so that it is more difficult than the currently set task, and the setting unit 131 sets the task updated by the update unit 137. Note that the initial task may be generated by the setting unit 131 or the update unit 137.

[0042] In addition, the control unit 130 may further function as an estimation unit that uses the learning model 121 generated by the learning unit 135 to accept input of information indicating the current state of the articulated robot 1 (rotation amount of each axis) and information on the target position and target orientation of the tip position of the articulated robot 1, and estimates control information for each axis of the articulated robot 1.

[0043] The input unit 140 receives input from the user of the learning device 100 and transmits it to the control unit 130. For example, the input unit 140 may be implemented by an input interface (e.g., a keyboard, mouse, touch panel, etc.) connected to the learning device 100. For example, the input unit 140 may be implemented by a microphone that receives voice input. For example, the input unit 140 may receive input of information related to the task from the user and transmit it to the control unit 130. For example, the input unit 140 may receive information related to the target position and orientation of the articulated robot 1 from the user and transmit it to the control unit 130.

[0044] The output unit 150 outputs specified information in accordance with instructions from the control unit 130. The output unit 150 may be realized as a monitor provided in the learning device 100, or as an output port that outputs to a monitor connected to the learning device 100. Alternatively, the output unit 150 may be realized in the form of an audio notification via a speaker. For example, the output unit 150 may output the rotation amount (teaching data) of each axis to be applied to the articulated robot 1 estimated by the learning model 121. In this case, the output may be performed by displaying text data indicating the contents of the teaching data or by transmitting the teaching data to the articulated robot 1 as an operation command. For example, the output unit 150 may output (transmit) the generated learning model 121 to an external device via the communication unit 110.

[0045] This concludes the description of the configuration of learning device 100. Although not shown in Fig. 1, learning device 100 may also include an input unit that accepts input from an operator, and this input unit can be realized by devices such as a keyboard or a mouse.

[0046] <Example assignment> FIG. 4 is a task table 400 showing an example of a task. Note that the numerical values ​​in the task shown in FIG. 4 are merely examples and are not limited to these. Furthermore, the task shown in FIG. 4 is generated and set by learning device 100, but may also be input and set by a user of learning device 100 via input unit 140, or may be set by receiving information such as that shown in FIG. 4 from an external device capable of communicating with learning device 100. Task table 400 can also be said to be information indicating criteria for determining whether the operation of articulated robot 1 is successful or unsuccessful. In other words, the determination by determination unit 134 may be made by referring to such a task table.

[0047] The task table 400 is information in which at least a task number 410, a threshold value 420, a distance 430, and an angle difference 440 are associated with each other.

[0048] The assignment number 410 is information indicating the number of each assignment, and is a number used for convenience to distinguish it from other assignments.

[0049] The threshold 420 is a threshold used by the update unit 137 to determine whether or not to update the assignment, and is a threshold to be compared with the score calculated by the calculation unit 136. In the drawing, an example in which the threshold is set to 0.99 is shown, but the present invention is not limited to this numerical value. Furthermore, since the score is calculated as a value between 0 and 1, the threshold is also set to a value less than 1. The threshold 420 may be changed depending on the range of the score calculated by the calculation unit 136.

[0050] Distance 430 is information indicating an allowable range from the target position set by the setting unit, and is information indicating the distance range within which the determination unit 134 determines that the tip position is successful. Therefore, the tip position of the articulated robot 1 is successful if it is located inside a sphere whose center is the target position set by the setting unit and whose radius is the distance indicated by distance 430. In other words, if the distance d in FIG. 3 described above is equal to or less than the distance indicated by distance 430, it is determined to be successful.

[0051] The angle difference 440 is information indicating the allowable range of the angle between the orientation (second direction) of the tip position of the articulated robot 1 and the orientation set by the setting unit 131, and is information indicating the angle range within which the determining unit 134 determines that the orientation of the tip position is successful. That is, if the angle θ in Fig. 3 described above is equal to or smaller than the angle indicated by the angle difference 440, it is determined to be successful.

[0052] In curriculum learning, the difficulty of the task content is set to increase as the learning progresses. Here, increasing the difficulty may mean either a decrease in distance 430 or a decrease in angle difference 440, or both. If the difficulty of one of distance 430 and angle difference 440 increases, the difficulty of the other may be lower than in the previous task.

[0053] In the task table 400 shown in FIG. 4, the task becomes more difficult as the numerical value indicated by the task number 410 increases. In the task table 400 shown in FIG. 4, as shown by the change from task number 0 to task number 1, both the distance 430 and the angular difference 440 decrease, indicating an increase in the task difficulty. Furthermore, as shown by the change from task number 1 to task number 2, even if only the distance 430 decreases while the angular difference 440 remains the same, the task difficulty can be said to have increased. Furthermore, as shown by the change from task number 2 to task number 3, even if the distance 430 remains the same but the angular difference 440 decreases, the task difficulty can be said to have increased. Furthermore, in this embodiment, as shown by the change from task number 4 to task number 5, the angular difference 440 decreases from 80 to 60, while the distance 430 increases from 10 to 15. Even in this case, the task difficulty can be said to have increased at least in terms of the angular difference. Therefore, even in this case, the task difficulty is said to have increased.

[0054] <Operation> FIG. 5 is a flowchart showing an example of the operation of the learning device 100 in learning.

[0055] As shown in FIG. 5, the setting unit 131 of the learning device 100 sets the target position of the tip of the articulated robot 1 and the range (distance 430, angle difference 440) for the target orientation (step S501).

[0056] The drive control unit 132 generates and provides random control values ​​to each axis of the articulated robot 1 on the simulator (virtual space) (step S502).

[0057] The acquisition unit 133 acquires the tip position (second position) of the articulated robot 1 driven by the drive control unit 132 and the direction in which the tip position faces (second direction) (step S503).

[0058] The determination unit 134 determines whether the information acquired by the acquisition unit 133 is within the range of the task (step S504). That is, the determination unit 134 (i) determines whether the distance d between the tip position (second position) 310 of the articulated robot 1 acquired by the acquisition unit 133 and the target position 301 is within the distance 430 set as the task, and (ii) determines whether the angle θ between the direction (second direction) 320 of the tip position of the articulated robot 1 acquired by the acquisition unit 133 and the target direction 302 is within the angle difference 440 set as the task. The determination unit 134 determines that the task has been accomplished if both of these conditions are met, and determines that the task has not been accomplished (non-achievement, failure) otherwise.

[0059] If the determination unit 134 determines that the task has been accomplished (YES in step S504), the learning unit 135 learns by associating the state of the articulated robot 1 before control (the tip position (first position) and orientation (first direction) of the articulated robot 1), the state after control (the tip position (second position) and orientation (second direction) of the articulated robot 1), the control value by the drive control unit 132, and information indicating task accomplishment (step S505). That is, the learning model 121 is generated (updated).

[0060] Furthermore, if the judgment unit 134 judges that the task has not been accomplished (NO in step S504), the learning unit 135 learns by associating the state of the articulated robot 1 before control (the tip position (first position) and orientation (first direction) of the articulated robot 1), the state after control (the tip position (second position) and orientation (second direction) of the articulated robot 1), the control value by the drive control unit 132, and information indicating that the task has not been accomplished. That is, the learning model 121 is generated (updated). Thereafter, the learning device 100 returns to the processing of step S501, and the setting unit 131 sets a new target position and orientation (the distance 430 and the angle difference 440 are not changed).

[0061] Here, learning device 100 may determine whether to end learning (step S507). Learning device 100 may determine whether to end learning, for example, when learning device 100 receives input from the user of learning device 100 via input unit 140 indicating that learning should be ended, or when a learning end condition established at the start of learning (e.g., completion of task 10) is met. If learning is not to be ended (NO in step S507), update unit 137 updates the task so that the difficulty level of at least one of the task contents (distance 430, angular difference 440) is increased (in the case of distance 430, the distance is made shorter than the distance of the current task, and in the case of angular difference 440, the angular difference is made narrower than the angular difference of the current task) (step S508), and transmits the updated results to setting unit 131.

[0062] If the learning is to be ended (YES in step S507), the process ends. The above is the operation of the learning device 100 in learning.

[0063] <Summary> According to the learning device 100 shown in the above embodiment, random control values ​​are given to the articulated robot 1, and learning is performed starting with simple tasks such as approaching a target position and a target orientation even with random movement. By gradually increasing the difficulty of the tasks, it is possible to generate a learning model 121 that can output control values ​​that can control the articulated robot 1 so that it is ultimately positioned at a more difficult target (a target in which the tip position of the articulated robot 1 is close to the target position and in an orientation that is close to a preferred orientation). Therefore, since a learning model can be generated without manual assistance in learning, it is possible to reduce human effort, and by using the generated learning model, it is possible to generate teaching data, which normally requires expertise and experience, without relying on humans.

[0064] <Supplementary information> The learning device according to the above embodiment is not limited to the above embodiment, and may be realized by other methods. Various modifications will be described below.

[0065] (1) In the above embodiment, the learning device 100 is configured to set the next task and proceed to the new task when the score calculated by the calculation unit for one task exceeds a set threshold. However, the trigger for proceeding to the next task is not limited to this. For example, the user may proceed to the next task based on the number of times the set task is completed, or based on the percentage of attempts at the task that are determined to be successful. That is, as an example, the user may proceed to the next task if the set task is successfully completed a predetermined number of times, for example, 10 times (not limited to 10 times). As another example, the user may proceed to the next task if the percentage of attempts at the set task that are determined to be successful exceeds 80% (not limited to 80%) of the total number of attempts. In this case, the calculation unit 136 that calculates the score does not need to be configured in the control unit 130.

[0066] (2) In the above embodiment, in addition to the articulated robot 1 and the target object to be processed, an object that obstructs the driving of the articulated robot 1 (which in reality may be a room wall, a shelf, another robot, or another target object) may be placed on the simulator. If the articulated robot 1 comes into contact with the placed object due to driving based on a random control value for the articulated robot 1, the determination unit 134 may be configured to determine that the operation has failed. By making the object arrangement closer to the actual operational arrangement, the learning unit 135 can generate a learning model specialized for the actual operational site.

[0067] (3) In the above embodiment, an example was described in which processing returns to step S501 after processing in step S506. However, if the target position and target orientation of a task are fixed, processing returns to step S502.

[0068] (4) In the above embodiment, curriculum learning is performed by the processor of the learning device executing a program. However, this may also be realized by a logic circuit (hardware) or dedicated circuit formed on an integrated circuit (IC (Integrated Circuit) chip, LSI (Large Scale Integration)) or the like. These circuits may also be realized by one or more integrated circuits, and the functions of the multiple functional units shown in the above embodiment may be realized by a single integrated circuit. LSIs are sometimes referred to as VLSIs, super LSIs, ultra LSIs, etc., depending on the degree of integration. In other words, each functional unit constituting the learning device 100 shown in FIG. 1 may be realized by a physical circuit.

[0069] The learning program may be recorded on a processor-readable recording medium, which may be a "non-transitory tangible medium" such as a tape, disk, card, semiconductor memory, or programmable logic circuit. The learning program may also be supplied to the processor via any transmission medium capable of transmitting the learning program (such as a communications network or broadcast waves). The present invention may also be realized in the form of a data signal embedded in a carrier wave, in which the learning program is embodied by electronic transmission.

[0070] The learning program can be implemented using, for example, scripting languages ​​such as ActionScript and JavaScript (registered trademark), object-oriented programming languages ​​such as Objective-C, Java (registered trademark), and Python (registered trademark), and markup languages ​​such as HTML5.

[0071] (5) The configurations described in the above embodiment and supplementary notes may be combined as appropriate. [Explanation of symbols]

[0072] 1. Articulated robot 100 Learning Device 110 Communications Department 120 Storage section 130 Control Unit 131 Setting section 132 Drive control unit 133 Acquisition Department 134 Judgment section 135 Learning Department 136 Calculation Unit 137 Update Department 140 Input section 150 Output section

Claims

1. a setting unit that sets a task including a first target value range for a tip position of the articulated robot and a second target value range for a direction in which the tip position should face; a drive control unit that provides random control values ​​to at least each axis of the articulated robot; an acquisition unit that acquires first tip position information that indicates a tip position of the articulated robot after the random control value is applied to a current value of each axis of the articulated robot, and first direction information that indicates a direction in which the tip position is facing; a determination unit that determines whether the first tip position information is within the first target value range and whether the first direction information is within the second target value range; a learning unit that learns a correspondence relationship among a second tip position of the articulated robot before being driven by the drive control unit of the articulated robot and second direction information in which the tip position is facing, a random control value for each axis given by the drive control unit, the first tip position information and the first direction information, and a determination result by the determination unit; an update unit that updates the assignment when one or more determination results by the determination unit satisfy a predetermined condition; A learning device comprising:

2. the first target value range is information indicating a distance between a tip position indicated by the first tip position information and a tip position where the tip position should be, The second target value range is information indicating an allowable angle range of an angle between the direction indicated by the first direction information and the direction in which the tip position should be directed.

2. The learning device according to claim 1 .

3. The update unit updates the task so as to increase the difficulty level of the task.

2. The learning device according to claim 1 .

4. The update unit updates the assignment so as to narrow at least one of the first target value range and the second target value range.

4. The learning device according to claim 3.

5. a calculation unit that calculates, each time the determination unit makes a determination, a score indicating an evaluation of driving by the drive control unit based on at least a distance from a tip position indicated by the first tip position information to a tip position indicated by the first target value range and an angle formed between a direction indicated by the first direction information and a direction indicated by the second target value range, The update unit updates the assignment when the cumulative value of the scores calculated by the calculation unit for the same assignment exceeds a predetermined threshold as the predetermined condition.

2. The learning device according to claim 1 .

6. the articulated robot is a robot in a virtual space, The drive control unit drives the articulated robot, which is a robot in the virtual space.

6. The learning device according to claim 1, wherein the learning device is a learning device for learning a plurality of data.

7. The computer a setting step of setting a task including a first target value range for the tip position of the articulated robot and a second target value range for the direction in which the tip position should face; a drive control step of giving random control values ​​to at least each axis of the articulated robot; an acquisition step of acquiring first tip position information indicating a tip position of the articulated robot after the random control value is applied to a current value of each axis of the articulated robot, and first direction information indicating a direction in which the tip position is facing; a first determination step of determining whether the first tip position information is within the first target value range and whether the first direction information is within the second target value range; a learning step of learning a correspondence relationship between a second tip position of the articulated robot before driving by the drive control step of the articulated robot and second direction information in which the tip position is facing, a random control value for each axis given by the drive control step, the first tip position information and the first direction information, and a determination result by the first determination step; an updating step of updating the assignment when one or more determination results from the first determination step satisfy a predetermined condition; Learning how to do it.

8. On the computer, a setting function for setting a task including a first target value range for the tip position of the articulated robot and a second target value range for the direction in which the tip position should face; a drive control function that provides random control values ​​to at least each axis of the articulated robot; an acquisition function that acquires first tip position information that indicates a tip position of the articulated robot after the random control value is applied to a current value of each axis of the articulated robot, and first direction information that indicates a direction in which the tip position is facing; a first determination function that determines whether the first tip position information is within the first target value range and whether the first direction information is within the second target value range; a learning function that learns the correspondence between a second tip position of the articulated robot before being driven by the drive control function of the articulated robot and second direction information in which the tip position is facing, random control values ​​for each axis given by the drive control function, the first tip position information and the first direction information, and a determination result by the first determination function; an update function that updates the assignment when one or more determination results by the first determination function satisfy a predetermined condition; A learning program that makes this possible.

Images