Learning device, control device, learning method and program
Patent Information
- Application Number
- JP2024504055
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2022-03-01
- Filing Date
- 2022-03-01
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2042-03-01
AI Technical Summary
Existing systems for controlling robots to perform tasks efficiently struggle with determining when to stop learning metaparameter values, leading to unnecessary learning and inefficiencies in skill execution, especially when handling objects of varying shapes and sizes.
A learning device and method that evaluates the generalization error of a learning model to determine whether to continue learning metaparameter values, allowing for efficient skill learning and adaptation to different task parameters, including object shapes and sizes, by using a control unit to adjust robot movements accordingly.
Enables the robot to perform tasks with high precision and efficiency by optimizing learning processes, reducing unnecessary training and improving adaptability to diverse task conditions.
Abstract
Description
Learning device, control device, learning method, and storage medium
[0001] The present invention relates to a learning device, a control device, a learning method, and a storage medium.
[0002] When controlling a robot required to execute a task, a system has been proposed in which robot operations are modularized into skills for controlling the robot. For example, Patent Literature 1 discloses a system in which an articulated robot executes a given task, in which robot skills that can be selected depending on the task are defined as tuples, and parameters included in the tuples are updated through learning.
[0003] International Publication No. 2018 / 219943
[0004] When learning modularized skills for robot behavior, if differences in modules can be accommodated by learning the meta-parameter values of the learning model, it will be possible to have the robot execute multiple skills with a single model. In this way, when learning the meta-parameter values of a learning model, if it is possible to determine whether or not learning needs to continue, it is expected that unnecessary learning can be eliminated and learning can be carried out more efficiently.
[0005] An example of a purpose of this disclosure is to provide a learning device, a control device, a learning method, and a recording medium that can solve the above-mentioned problems.
[0006] According to a first aspect of the present invention, a learning device includes a metaparameter learning means that learns values of metaparameters that indicate a probability distribution in a learning model in which the values of parameters follow a probability distribution, based on training data that indicates the input and output in the learning model; a generalization error evaluation means that calculates an evaluation value that indicates an evaluation of the generalization error of the learning model; and a learning continuation determination means that determines whether or not learning of the values of the metaparameters needs to be continued based on the evaluation value.
[0007] According to a second aspect of the present invention, the control device includes a control means for controlling the robot in accordance with the shape of the object to be grasped so that the robot grasps each of the objects to be grasped having different shapes.
[0008] According to a third aspect of the present invention, a learning method includes a computer learning values of meta-parameters that indicate a probability distribution in a learning model in which the values of the parameters follow a probability distribution, based on training data that indicates the input and output in the learning model, calculating an evaluation value that indicates an evaluation of the generalization error of the learning model, and determining whether or not to continue learning the values of the meta-parameters based on the evaluation value.
[0009] According to a fourth aspect of the present invention, the recording medium is a recording medium that records a program for causing a computer to perform the following operations: learning values of meta-parameters that indicate a probability distribution in a learning model in which the values of the parameters follow a probability distribution, based on training data that indicates the input and output in the learning model; calculating an evaluation value that indicates an evaluation of the generalization error of the learning model; and determining whether or not to continue learning the values of the meta-parameters based on the evaluation value.
[0010] According to the present invention, when learning meta parameter values of a learning model, it is possible to determine whether or not the learning needs to continue.
[0011] 19 is a diagram illustrating an example of the configuration of a control system according to the first embodiment. FIG. 20 is a diagram illustrating an example of known task parameters according to the first embodiment. FIG. 21 is a diagram illustrating an example of unknown task parameters according to the first embodiment. FIG. 22 is a diagram illustrating an example of the hardware configuration of a learning device according to the first embodiment. FIG. 23 is a diagram illustrating an example of the hardware configuration of a robot controller according to the first embodiment. FIG. 24 is a diagram illustrating a robot that grasps an object according to the first embodiment, and an object to be grasped, in real space. FIG. 25 is a diagram illustrating the state shown in FIG. 6 in abstract space. FIG. 26 is a diagram illustrating an example of the configuration of a control system related to skill execution according to the first embodiment. FIG. 27 is a diagram illustrating an example of the functional configuration of a learning device related to updating of a skill database according to the first embodiment. FIG. 28 is a diagram illustrating an example of the configuration of a skill learning unit according to the first embodiment. FIG. 29 is a diagram illustrating an example of data input / output in the skill learning unit according to the first embodiment. FIG. 29 is a diagram illustrating an example of a skill database update process by the learning device according to the first embodiment. FIG. 29 is a diagram illustrating an example of data input / output in the skill learning unit according to the second embodiment. FIG. 29 is a diagram illustrating an example of a skill database update process by the learning device according to the second embodiment. FIG. 29 is a diagram illustrating an example of the configuration of a skill learning unit according to a third embodiment. FIG. 29 is a diagram illustrating an example of data input / output in the skill learning unit according to the third embodiment. FIG. 29 is a diagram illustrating an example of the configuration of a meta parameter processing unit according to the third embodiment. FIG. 29 is a diagram illustrating an example of data input / output in the meta parameter processing unit according to the third embodiment. FIG. 29 is a diagram illustrating a first example of the configuration of a meta parameter individual processing unit according to the third embodiment. FIG. 29 is a diagram illustrating an example of data input / 21 is a diagram showing a second example of the configuration of a meta parameter individual processing unit according to the third embodiment. FIG. 22 is a diagram showing an example of data input / output in the meta parameter individual processing unit shown in FIG. 21. FIG. 23 is a diagram showing an example of a skill database update process by the learning device according to the third embodiment. FIG. 24 is a diagram showing an example of a process by which the meta parameter processing unit according to the third embodiment calculates meta parameter values of a predictor. FIG. 25 is a diagram showing a first example of a process by which the meta parameter individual processing unit according to the third embodiment calculates meta parameter values for each predictor and determines whether or not learning of the meta parameter values needs to be continued.10 is a diagram showing a second example of processing in which a meta parameter individual processing unit according to a third embodiment calculates a meta parameter value for each predictor and determines whether or not learning of the meta parameter value needs to be continued. FIG. 11 is a diagram showing an example of the configuration of a learning device according to a fourth embodiment. FIG. 12 is a diagram showing an example of the configuration of a control device according to a fifth embodiment. FIG. 13 is a diagram showing an example of the processing procedure in a learning method according to a sixth embodiment.
[0012] Hereinafter, embodiments of the present invention will be described, but the following embodiments do not limit the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention. For the sake of convenience, in this specification, a character with an arbitrary symbol "x" above the arbitrary letter "A" will be referred to as "A". x " is expressed as ".
[0013] <First Embodiment> (1) System Configuration Fig. 1 is a diagram showing an example of the configuration of a control system according to the first embodiment. In the configuration shown in Fig. 1, the control system 100 includes a learning device 1, a storage device 2, a robot controller 3, a measurement device 4, and a robot 5. The learning device 1 performs data communication with the storage device 2 via a communication network or by direct wireless or wired communication. Furthermore, the robot controller 3 performs data communication with the storage device 2, the measurement device 4, and the robot 5 via a communication network or by direct wireless or wired communication.
[0014] The learning device 1 learns the behavior of the robot 5 for executing a given task by machine learning such as self-supervised learning (SSL), and also learns a set of states in which the behavior to be learned can be executed.
[0015] However, the object for which the learning device 1 learns the behavior is not limited to a specific object, and can be any control object that is controllable and whose control can be learned. Furthermore, the behavior of the control object, such as the robot 5, is not limited to one that involves a change in position. For example, the robot 5 may acquire sensor measurement data using a sensor, which may be set as one of the behaviors of the robot 5. This also applies to the following embodiments.
[0016] The state referred to here is the state of the target system including the robot 5 and the operating environment of the robot 5. The robot 5 and the operating environment of the robot 5 are collectively referred to as the target system, or simply as the system. When a task involves handling an object, such as a task of grasping an object, the object of the task is also included in the target system.
[0017] The state of the target system is called the system state, or simply the state. The system state at the completion of a task as defined in the task is also called the goal state of the task, or simply the goal state. Reaching the goal state of a task is also called accomplishing the task or succeeding in the task. If a task is accomplished by executing a skill, the state at the end of skill execution corresponds to the goal state. The system state at the start of a task is also called the initial state of the task.
[0018] The learning device 1 learns skills that modularize specific actions of the robot 5. In the embodiment, a task is assumed to be achieved by executing one skill for each task, and the learning device 1 learns skills so that the task can be achieved.
[0019] On the other hand, the robot controller 3 may execute a task by combining a plurality of skills. For example, the robot controller 3 may divide a given task into subtasks corresponding to skills, and combine skills for executing each subtask to plan the execution of the given task.
[0020] When learning a skill, the learning device 1 also learns a set of states in which the skill can be executed. The learning device 1 registers information about the learned skill in a skill database stored in the storage device 2. The information registered in the skill database is also called a skill tuple. The skill tuple includes various information necessary to execute the operation to be modularized. The learning device 1 generates a skill tuple based on detailed system model information, low-level controller information, and target parameter information stored in the storage device 2.
[0021] The storage device 2 stores information referenced by the learning device 1 and the robot controller 3. The storage device 2 stores, for example, detailed system model information, low-level controller information, target parameter information, and a skill database. The storage device 2 may be an external storage device such as a hard disk connected to or built into the learning device 1 or the robot controller 3, a storage medium such as a flash memory, or a server device that communicates data with the learning device 1 and the robot controller 3. The storage device 2 may also be composed of multiple storage devices, and may have the above-mentioned storage units distributed among them.
[0022] The detailed system model information is information that represents a model of the target system in real space. The model of the target system in real space is also called a detailed system model. To distinguish the detailed system model from an abstract system model, it is written as a "detailed" system model. The detailed system model information may be expressed as a differential or difference equation that represents the detailed system model. Alternatively, the detailed system model may be configured as a simulator that simulates the operation of the robot 5.
[0023] The low-level controller information is information about a low-level controller that generates inputs that control the actual movement of the robot 5 based on parameter values output by the high-level controller. For example, when the high-level controller generates a trajectory for the robot 5, the low-level controller may generate control inputs that cause the movement of the robot 5 to follow the trajectory. For example, the low-level controller may control the robot 5 using servo control using PID (Proportional Integral Differential) based on parameters output by the high-level controller.
[0024] The target parameter information is provided for each skill to be learned by the learning device 1, and includes, for example, initial state information, target state / known task parameter information, unknown task parameter information, execution time information, and general constraint information. Here, the variable parts of a task are referred to as task parameters.
[0025] Task parameters that are expressed as numerical values are called known task parameters. Examples of known task parameters include, but are not limited to, the size of an object in a task, such as the size of an object to be grasped when the task is to grasp an object, and the trajectory of the robot 5 for executing the task. Known task parameters can also be treated as parameters in skills. Known task parameters are examples of skill parameters.
[0026] 2 is a diagram showing examples of known task parameters. Fig. 2 shows an example in which the robot 5 executes a task of grasping a cylindrical object. In this case, the radius and height of the cylindrical object correspond to examples of known task parameters.
[0027] On the other hand, among the task parameters, those that are difficult to express numerically are referred to as unknown task parameters. Examples of unknown task parameters include, but are not limited to, the shape of an object in a task, such as the shape of an object to be grasped when the task is to grasp an object, and the type of movement of the robot 5 to execute the task, such as the skills required to execute the task.
[0028] 3 is a diagram showing examples of unknown task parameters. Fig. 3 shows an example in which the robot 5 executes a task of grasping objects of various shapes. In this case, the shapes of the objects correspond to an example of unknown parameters.
[0029] Furthermore, it is assumed that the control system 100 handles the system state as a numerical value, and the target state is expressed as a numerical value. For example, in the case of a task in which the robot 5 performs pick and place, the target state may be expressed as the coordinates of an object being within a predetermined range.
[0030] The initial state information is information that indicates a set of states in which a target skill can be executed. The state at the start of skill execution is also called the initial state of the skill, or simply the initial state. The set of initial states is also called the initial state set. The initial state is defined as x s or xsi Here, "i" is a positive integer representing an identification number for identifying the initial state. The time of the initial state is set to 0, and the initial state is represented by x 0 It may be expressed as:
[0031] The goal state / known task parameter information is information that indicates a set of combinations of possible values of a goal state, which is a state that can be reached by executing a target skill, and possible values of known task parameters that are treated as explicit parameters of the target skill. For example, in the case of a skill in which the robot 5 grasps an object, possible values of the goal state may include information on stable grasping conditions such as form closure and force closure. A combination of a goal state and a known task parameter value is referred to as a goal state / known task parameter value, and β g or β gi Here, "i" is a positive integer representing an identification number that identifies the target state / known task parameter value.
[0032] By treating the differences in the goal states of a task and the differences in the known task parameter values as parameters in a skill, tasks with different goal states, different known task parameter values, or both can be executed with a single skill.
[0033] For example, when the learning device 1 performs processing related to skill learning using a predictor, a target state and known task parameter values can be input to the predictor to obtain an output value corresponding to the target state and known task parameter values. Here, the predictor is configured using a learning model (a model in machine learning), such as a neural network or a Gaussian process (GP).
[0034] Note that some skills may not have known task parameters. In this case, the goal state / known task parameter information may be configured as a set of values that the goal state can take. Also, the goal state / known task parameter value β gmay indicate the target state.
[0035] The unknown task parameter information is information relating to unknown task parameters. For example, as will be described later in a third embodiment, the unknown task parameter information may indicate a probability distribution of data relating to the unknown parameters. If one skill has multiple unknown task parameters, the unknown task parameter information may indicate information relating to each unknown task parameter. Note that in the first and second embodiments, correspondence to the target state / known task parameter information will be described. In the first and second embodiments, values corresponding to the unknown task parameters may be indicated as fixed values.
[0036] The unknown task parameter value is set to τ or τ j Here, "j" is a positive integer representing an identification number that identifies the unknown task parameter value. Note that it is difficult to express the unknown task parameter numerically because it is difficult to systematically quantify its value, but it is possible to determine whether the unknown task parameter values are the same. For example, if the unknown task parameter represents the shape of an object, it is possible to determine whether the unknown task parameter values are the same by comparing the shapes of two objects. The control system 100 treats two tasks as the same task if the unknown task parameter values in the two tasks are the same, and treats the two tasks as separate tasks if the unknown task parameter values are different. τ or τ j The above "j" can also be considered as a positive integer representing an identification number that identifies the task.
[0037] The execution time information is information regarding the time limit for skill execution. For example, the execution time information may indicate the skill execution time (the time required to execute the skill), or the allowable condition value for the time from the start to the end of skill execution, or both. The general constraint information is information indicating general constraint conditions, such as conditions regarding limits on the range of movement of the robot 5, speed limits, and input limits.
[0038] The skill database is a database of skill tuples prepared for each skill. A skill tuple may include information about a high-level controller for executing the target skill, information about a low-level controller for executing the target skill, and information about a set of combinations of states (initial states for the skill) and target states / known task parameter values that allow the target skill to be executed. A set of states and target states / known task parameter values that allow the target skill to be executed is also referred to as an executable state set.
[0039] The feasible state set may be defined in an abstract space that abstracts the actual space. The feasible state set can be expressed, for example, using a level set function estimated by Gaussian Process Regression (GPR) or Level Set Estimation (LSE), or an approximation function of the level set function. In other words, whether the feasible state set includes a combination of a state and a target state / known task parameter value can be determined by whether the value (e.g., the mean value) of the Gaussian process regression for the combination of the state and the target state / known task parameter value, or the value of the approximation function for the combination of the state and the target state / known task parameter value, satisfies a constraint for determining feasibility. The following description will be given using, as an example, a case where a level set function is used as a function representing the feasible state set, but is not limited to this.
[0040] After the learning process by the learning device 1, the robot controller 3 formulates an operation plan for the robot 5 based on the measurement signals supplied by the measurement device 4, the skill database, etc. The robot controller 3 generates control commands (control inputs) for causing the robot 5 to execute the planned operation, and supplies the control commands to the robot 5.
[0041] For example, the robot controller 3 converts a task to be executed by the robot 5 into a sequence for each time step (time interval) of the task that can be accepted by the robot 5. Then, the robot controller 3 controls the robot 5 based on a control command corresponding to an execution command of the generated sequence. The control command corresponds to a control input output by a low-level controller.
[0042] The measurement device 4 is, for example, one or more sensors such as a camera, a range sensor, a sonar, or a combination thereof that detects the state within the workspace where the robot 5 executes a task. The measurement device 4 supplies the generated measurement signal to the robot controller 3. The measurement device 4 may be a self-propelled or flying sensor (including a drone) that moves within the workspace. The measurement device 4 may also include sensors provided on the robot 5 and sensors provided on other objects within the workspace. The measurement device 4 may also include a sensor that detects sound within the workspace. In this way, the measurement device 4 may include various sensors that detect the state within the workspace and that are provided at any location.
[0043] The robot 5 performs work related to a specified task based on a control command supplied from the robot controller 3. The robot 5 is a robot that operates, for example, in various factories such as an assembly factory or a food factory, or in a logistics site. The robot 5 may be a vertical articulated robot, a horizontal articulated robot, or any other type of robot. The robot 5 may supply a status signal indicating the status of the robot 5 to the robot controller 3. This status signal may be an output signal from a sensor that detects the status (position, angle, etc.) of the entire robot 5 or a specific part such as a joint, or may be a signal indicating the progress of the operation of the robot 5.
[0044] The configuration of the control system 100 shown in FIG. 1 is an example, and various modifications may be made to this configuration. For example, the robot controller 3 and the robot 5 may be configured as an integrated unit. In another example, at least two of the learning device 1, the storage device 2, and the robot controller 3 may be configured as an integrated unit. Furthermore, the control target of the control system 100 is not limited to a robot. Various control targets that the learning device 1 can learn to control can be the control targets of the control system 100.
[0045] (2) Hardware Configuration Fig. 4 is a diagram showing an example of the hardware configuration of the learning device 1. The learning device 1 includes, as hardware, a processor 11, a memory 12, and an interface 13. The processor 11, the memory 12, and the interface 13 are connected via a data bus 10.
[0046] The processor 11 executes a program stored in the memory 12 to function as a controller (arithmetic unit) that performs overall control of the learning device 1. The processor 11 is, for example, a processor such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a TPU (Tensor Processing Unit). The processor 11 may be composed of multiple processors. The processor 11 is an example of a computer.
[0047] Memory 12 is composed of various types of volatile and non-volatile memory, such as RAM (Random Access Memory), ROM (Read Only Memory), and flash memory. Memory 12 also stores programs for executing processes performed by learning device 1. Note that some of the information stored in memory 12 may be stored in one or more external storage devices (e.g., storage device 2) that can communicate with learning device 1, or may be stored in a storage medium that is detachable from learning device 1.
[0048] The interface 13 is an interface for electrically connecting the learning device 1 to other devices. These interfaces may be wireless interfaces such as network adapters for wirelessly transmitting and receiving data to other devices, or hardware interfaces for connecting to other devices via cables or the like. For example, the interface 13 may interface with input devices that accept user input (external input), such as a touch panel, buttons, keyboard, or voice input device, display devices such as a display or projector, or sound output devices such as speakers.
[0049] The hardware configuration of the learning device 1 is not limited to the configuration shown in Figure 4. For example, the learning device 1 may incorporate at least one of a display device, an input device, and a sound output device. The learning device 1 may also include a storage device 2.
[0050] 5 is a diagram showing an example of the hardware configuration of the robot controller 3. The robot controller 3 includes, as hardware, a processor 31, a memory 32, and an interface 33. The processor 31, the memory 32, and the interface 33 are connected via a data bus 30.
[0051] The processor 31 executes a program stored in the memory 32 to function as a controller (arithmetic unit) that performs overall control of the robot controller 3. The processor 31 is, for example, a processor such as a CPU, a GPU, or a TPU. The processor 31 may be composed of multiple processors.
[0052] The memory 32 is configured by various types of volatile and non-volatile memory, such as RAM, ROM, and flash memory. The memory 32 also stores programs for executing processes performed by the robot controller 3. Note that some of the information stored in the memory 32 may be stored in one or more external storage devices (e.g., the storage device 2) that can communicate with the robot controller 3, or may be stored in a storage medium that is detachable from the robot controller 3.
[0053] The interface 33 is an interface for electrically connecting the robot controller 3 to other devices. These interfaces may be wireless interfaces such as network adapters for wirelessly transmitting and receiving data to and from other devices, or may be hardware interfaces for connecting to other devices via cables or the like.
[0054] The hardware configuration of the robot controller 3 is not limited to the configuration shown in Fig. 5. For example, the robot controller 3 may incorporate at least one of a display device, an input device, and a sound output device. The robot controller 3 may also include the storage device 2.
[0055] (3) Abstract Space The robot controller 3 formulates an operation plan for the robot 5 in an abstract space based on the skill tuple. The abstract space targeted in the operation plan for the robot 5 will now be described.
[0056] Fig. 6 is a diagram showing in real space a robot (manipulator) 5 that grasps an object and an object to be grasped 6. Fig. 7 is a diagram showing in abstract space the state shown in Fig. 6.
[0057] Generally, formulating a motion plan for a robot 5 performing a pick-and-place task requires rigorous calculations that take into account the shape of the robot 5's end effector, the geometric shape of the object 6 to be grasped, the grasping position and posture of the robot 5, and the object characteristics of the object 6 to be grasped. In contrast, in this embodiment, the robot controller 3 formulates a motion plan in an abstract space in which the states of each object, such as the robot 5 and the object 6 to be grasped, are abstractly (simply) represented. In the example of FIG. 7 , the abstract space defines an abstract model 5x corresponding to the end effector of the robot 5, an abstract model 6x corresponding to the object 6 to be grasped, and a region in which the robot 5 can perform a grasping operation of the object 6 to be grasped (see dashed-line frame 60). Note that, as described above, the set of executable states in the abstract space is also represented as a set of combinations of initial states and target states / known task parameter values in which a skill can be executed. In the example of FIG. 7 , the set of combinations of initial states and target states / known task parameter values in which a grasping skill can be executed is illustrated as the grasp operation executable region indicated by the dashed-line frame 60. In this way, the state of the robot in the abstract space is expressed abstractly as the state of the end effector, etc. The state of each object corresponding to the operation target or environmental object is also expressed abstractly in a coordinate system based on a reference object such as a workbench.
[0058] In this embodiment, the robot controller 3 uses skills to formulate a motion plan in an abstract space that abstracts an actual system. This effectively reduces the computational cost required for motion planning, even in multi-stage tasks. In the example of Figure 7, the robot controller 3 formulates a motion plan to execute skills for performing grasping in a graspable area (dashed frame 60) defined in the abstract space, and generates control commands for the robot 5 based on the formulated motion plan.
[0059] Hereinafter, the state of the system in the real space may be represented as "x" and the state of the system in the abstract space as "x'" to distinguish between them. The state x' is represented as a vector (abstract state vector). For example, in the case of a task such as pick-and-place, the abstract state vector includes a vector representing the state of the object to be manipulated (e.g., position, posture, velocity, etc.), a vector representing the state of the end effector of the manipulable robot 5, and a vector representing the state of the environmental objects. In this way, the state x' is defined as a state vector that abstractly represents the state of some elements in the real system. Similarly, the target state / known task parameter value in the real space is expressed as "β g ”, and the goal state / known task parameter value in the abstract space is defined as “β g These are sometimes distinguished by writing ".
[0060] (4) Control System for Skill Execution Figure 8 is a diagram showing an example of the configuration of a control system for skill execution. The processor 31 of the robot controller 3 functionally comprises a motion planning unit 34, a high-level control unit 35, and a low-level control unit 36. The system 50 corresponds to an actual system (an actual system including the robot 5). The high-level control unit 35 is also referred to as a high-level controller, and π H The high-level control unit 35 corresponds to an example of a control means. The low-level control unit 36 is also called a low-level controller, and π L The robot controller 3 is an example of a control device that controls the robot 5.
[0061] 8, for the sake of convenience, a balloon showing a diagram (see FIG. 7) illustrating an abstract space targeted by the action planning unit 34 is displayed in association with the action planning unit 34, and a balloon showing a diagram (see FIG. 6) illustrating an actual system corresponding to the system 50 is displayed in association with the system 50. Similarly, in FIG. 8, a balloon showing information related to an executable state set of a skill is displayed in association with the high-level control unit 35.
[0062] The motion planning unit 34 formulates a motion plan for the robot 5 based on the state x' in the abstract system and the skill database. The motion planning unit 34 expresses the target state using, for example, a logical expression based on temporal logic. The motion planning unit 34 may express the logical expression using any temporal logic, such as linear temporal logic, MTL (Metric Temporal Logic), or STL (Signal Temporal Logic). The motion planning unit 34 converts the generated logical expression into a sequence (motion sequence) for each time step. This motion sequence includes, for example, information about the skills used in each time step.
[0063] The high-level control unit 35 recognizes the skill to be executed for each time step based on the action sequence generated by the action planning unit 34. Then, the high-level control unit 35 determines the high-level controller "π H ” to generate a parameter “α” to be input to the low-level control unit 36 .
[0064] The high-level control unit 35 calculates the state "x" in the abstract space at the start of the execution of the skill to be executed. 0 '" and the combination of goal state / known task parameter values are defined as the feasible state set "χ 0 If the parameter α belongs to the category ', the control parameter α is generated as shown in the following equation (1).
[0065]
[0066] As mentioned above, the state at the start of skill execution is also referred to as the initial state. The initial state is represented, for example, by a state in an abstract space. Also, the set of executable states of a skill, χ 0 If an approximation function of the level set function that can determine whether or not the state belongs to the state x' is defined as "g^", the robot controller 3 0 ' is the feasible state set χ 0 It is possible to determine whether the suffix "a" belongs to "a" group by determining whether the formula (2) is satisfied.
[0067]
[0068] Equation (2) can be said to represent a constraint that determines the feasibility of a skill from a certain state. Alternatively, the approximation function "g^" is 0 It can also be said that this is a model that can evaluate whether a goal state can be reached under known task parameter values based on '. The approximation function g^ is found by the learning device 1 through learning, as will be described later.
[0069] The set of goal states in the abstract space after the execution of the target skill is defined as χ' d " and the execution time of the target skill is denoted as "T". Furthermore, the state at the time when T time has elapsed since the start of skill execution is denoted as "x'(T)". By executing the skill using the low-level control unit 36, it is possible to realize equation (3).
[0070]
[0071] The low-level control unit 36 calculates the control parameter α generated by the high-level control unit 35, the state x of the current real system obtained from the system 50, and the target state / known task parameter value β g The low-level control unit 36 generates the input "u" based on the low-level controller "π L ", the input u is generated as a control command as shown in equation (4).
[0072]
[0073] In addition, the low-level controller π L is not limited to the above formula format, and may be a controller having various formats.
[0074] The low-level control unit 36 acquires, as a state x, the state of the robot 5 and the environment recognized using any state recognition technology based on the measurement signals (which may include signals from the robot 5) output by the measurement device 4. In Fig. 8, the system 50 is represented by a state equation shown in formula (5) using a function "f" with an input u to the robot 5 and the state x as arguments.
[0075]
[0076] operator" ・ " represents the differentiation with respect to time or the difference with respect to time.
[0077] (5) Overview of Skill Database Updates Figure 9 is a diagram showing an example of the functional configuration of the learning device 1 regarding updating of the skill database. Functionally, the processor 11 of the learning device 1 includes an abstract system model setting unit 14, a skill learning unit 15, and a skill tuple generation unit 16. Note that Figure 9 shows an example of data exchanged between each block, but is not limited to this. The same applies to the other figures.
[0078] The abstract system model setting unit 14 sets an abstract system model based on the detailed system model information. This abstract system model is a simplified version of the detailed system model specified by the detailed system model information. The detailed system model corresponds to the system 50 in FIG. 8.
[0079] The abstract system model is a model having, as a state, an abstract state vector x' constructed based on the state x in the detailed system model. The operation planning unit 34 formulates an operation plan using the abstract system model. The abstract system model setting unit 14 calculates the abstract system model from the detailed system model based on, for example, an algorithm stored in advance in the storage device 2 or the like.
[0080] Alternatively, information about the abstract system model may be stored in advance in the storage device 2, etc. In this case, the abstract system model setting unit 14 may acquire information about the abstract system model from the storage device 2, etc. The abstract system model setting unit 14 supplies information about the set abstract system model to the skill learning unit 15 and the skill tuple generation unit 16.
[0081] The skill learning unit 15 learns the control of skill execution based on the abstract system model set by the abstract system model setting unit 14, and the detailed system model information, low-level controller information, and target parameter information stored in the storage device 2. In particular, the skill learning unit 15 learns the control of skill execution based on the abstract system model set by the abstract system model setting unit 14, the detailed system model information, low-level controller information, and target parameter information stored in the storage device 2. H The low-level controller π L The skill learning unit 15 also learns the value of the control parameter α. The skill learning unit 15 also learns the level set function and acquires training data for learning the control parameter α, for example, by using an evaluation function that evaluates the prediction accuracy of the level set function.
[0082] The skill tuple generation unit 16 generates the executable state set χ 0 ' and the high-level controller π H The skill tuple generation unit 16 generates a skill tuple that includes information about the abstract system model set by the abstract system model setting unit 14, information about the low-level controller, and target parameter information. The skill tuple generation unit 16 then registers the generated skill tuple in a skill database. The data in the skill database is used by the robot controller 3 to control the robot 5.
[0083] Each component of the abstract system model setting unit 14, the skill learning unit 15, and the skill tuple generation unit 16 can be realized, for example, by the processor 11 executing a program. Alternatively, the necessary programs may be recorded on any non-volatile storage medium and installed as needed to realize each component. Note that at least a portion of these components may not necessarily be realized by software programs, but may also be realized by any combination of hardware, firmware, and software. Furthermore, at least a portion of these components may be realized using a user-programmable integrated circuit, such as an FPGA (Field-Programmable Gate Array) or a microcontroller. In this case, the integrated circuit may be used to realize a program consisting of the above components. Furthermore, at least a portion of each component may be configured using an ASSP (Application Specific Standard Produce), an ASIC (Application Specific Integrated Circuit), or a quantum computer control chip. In this way, each component may be realized by various hardware. The same applies to other embodiments described below. Furthermore, each component may be realized by the cooperation of multiple computers, for example, using cloud computing technology.
[0084] (6) Description of the Skill Learning Unit Fig. 10 is a diagram showing an example of the configuration of the skill learning unit 15 according to the first embodiment. Functionally, the skill learning unit 15 includes a search point set setting unit 210, a data acquisition unit 220, a prediction accuracy evaluation function learning unit 230, and a high-level controller learning unit 240.
[0085] The search point set setting unit 210 includes a search point set initialization unit 211 and a next search point set setting unit 212. The data acquisition unit 220 includes a system model setting unit 221, a problem setting calculation unit 222, and a data update unit 223. The prediction accuracy evaluation function learning unit 230 includes a level set function learning unit 231, a prediction accuracy evaluation function setting unit 232, and an evaluation unit 233.
[0086] As described above, the skill learning unit 15 controls the high-level controller π H and generate training data for learning the high-level controller π H The skill learning unit 15 also learns the level set function. The search point set setting unit 210 learns the high-level controller π H As a candidate for the task setting to be learned, the initial state x s and the target state / known task parameter value β g A plurality of combinations of the above are prepared. The search point set setting unit 210 selects, from the prepared plurality of candidates, a task setting that is to be the target for acquiring training data for learning the control of the robot 5 by the robot controller 3. The search point set setting unit 210 corresponds to an example of a search point setting means.
[0087] The search point set initialization unit 211 is a high-level controller π H Specifically, the search point set initialization unit 211 sets a set of candidate task settings for the initial state x s and the target state / known task parameter value β g Set a set whose elements are combinations of
[0088] The high-level controller π set by the search point set initialization unit 211 H The set of candidate task settings to be learned is called the search point set, and X ~ search The candidate for task setting is also called a search point. A search point is expressed as (x s , β g ) can be expressed as the search point (x s , β gOnce the search point (x) is determined, the task setting is determined, and the behavior of the robot 5 is also determined. s , β g ) can be said to represent the behavior of the robot 5 for each task.
[0089] The next search point set setting unit 212 sets the search point set X ~ search Each element of the subset extracted by the next search point set setting unit 212 is input to the high-level controller π H The next search point set setting unit 212 sets the search point set X ~ search The subset extracted from X is called the search point subset. ~ check The search point subset X ~ check The elements of X ~ or X ~ i Here, "i" is a positive integer representing an identification number that identifies an element of the search point subset. ~ check The elements of are also referred to as selected search points, or simply search points.
[0090] The data acquisition unit 220 acquires the search point subset X set by the next search point set setting unit 212. ~ check Element X of ~ For each, the high-level controller π H The system model setting unit 221 acquires training data for learning the search point X ~ For each experiment, a system model is set up to set up the optimal control problem.
[0091] The problem setting calculation unit 222 sets a solution search problem that indicates task execution by the robot 5 based on the settings made by the system model setting unit 221. The solution search problem here is a problem that seeks a solution that satisfies presented constraint conditions.
[0092] Specifically, the problem setting calculation unit 222 sets an optimal control problem including constraints, such as constraints related to the task and constraints related to the robot's operation, and an evaluation function for the possibility of reaching the goal state. The optimal control problem is a problem of determining a control input that maximizes the evaluation indicated by the evaluation function value, and can be considered as an optimization problem. The following describes an example in which a function indicating that the smaller the evaluation function value, the higher the evaluation is, is used as the evaluation function for the optimal control problem. In this case, when solving the optimal control problem, a solution is sought that minimizes the evaluation function value, such as the minimum value of the evaluation function. However, the learning device 1 may also use a function indicating that the larger the function value, the higher the evaluation is, as the evaluation function for the optimal control problem.
[0093] The problem setting calculation unit 222 solves the set optimal control problem and calculates a high-level controller π that minimizes the evaluation function value. H The problem setting calculation unit 222 calculates the output value of the search point X ~ The problem setting calculation unit 222 corresponds to an example of calculation means.
[0094] The data update unit 223 updates the data obtained by the problem setting calculation unit 222 solving the optimal control problem with the high-level controller π H and update the training data of the level set function to include it in the training data of the high-level controller π H The training data of the high-level controller π H The training data for the level set function is training data for learning the level set function. In particular, the high-level controller π H The parameter value α to be output * , high-level controller π H The training data for learning the optimal control problem can be used as training data for the level set function. Also, the information on whether the skill can be executed, which is indicated by the solution of the optimal control problem, can be used as training data for the level set function.~ j Includes:
[0095] High-level controller π H The training data is used by the robot controller 3 to H The data updating unit 223 corresponds to an example of a data acquiring means. H The set representing the training data of D is called the acquisition data set. opt It is expressed as:
[0096] The prediction accuracy evaluation function learning unit 230 uses the acquired data set D opt The level set function and the prediction accuracy evaluation function are learned using the above, and it is determined whether or not learning of the level set function needs to be continued. As described above, the level set function is a function that indicates a set of executable states, which are a set of combinations of states and target states / known task parameter values that can reach the target state. The prediction accuracy evaluation function is a function that indicates an evaluation of the estimation accuracy of combinations of states and target states / known task parameter values that can reach the target state using the level set function.
[0097] The training of the level set function is performed by the high-level controller π H The search point X selected as the target for obtaining training data ~ The problem setting calculation unit 222 calculates the high-level controller π H The prediction accuracy evaluation function is performed using data used for the training data of the above. It is considered that there is a positive correlation between the number of training data acquired by the data updating unit 223 and the estimation accuracy of the level set function. The prediction accuracy evaluation function can also be said to be a function that indicates an evaluation of the acquisition status of the training data.
[0098] The level set function learning unit 231 uses the acquired data set D opt For example, the level set function learning unit 231 uses the acquired data set D optFor each element of the equation, it is determined whether or not the goal state can be reached based on the evaluation function value calculated by the problem setting calculation unit 222. Then, the level set function learning unit 231 determines whether or not the goal state can be reached and the initial state x s and the target state / known task parameter value β g The level set function learning unit 231 is an example of a level set function learning means.
[0099] The prediction accuracy evaluation function setting unit 232 learns a prediction accuracy evaluation function for the level set function learned by the level set function learning unit 231. For example, the prediction accuracy evaluation function setting unit 232 learns a prediction accuracy evaluation function for the search point X ~ Search point X ~ Based on the distribution of candidates in the space, the search point X ~ The prediction accuracy evaluation function may be learned so that a subspace with a large number of or a subspace with a high density is highly evaluated. The prediction accuracy evaluation function setting unit 232 corresponds to an example of a prediction accuracy evaluation function setting means.
[0100] The prediction accuracy evaluation function is J g^ , or J g^j Here, "j" is a positive integer representing an identification number that identifies a task. As described above, if two tasks have different unknown task parameter values, the control system 100 treats them as separate tasks.
[0101] The evaluation unit 233 uses the prediction accuracy evaluation function to evaluate the high-level controller π H The evaluation unit 233 determines whether or not it is necessary to continue acquiring training data for the high-level controller π. H Whether or not it is necessary to continue acquiring training data can also be regarded as whether or not it is necessary to continue learning the level set function. The flag indicating the determination result of the evaluation unit 233 is also referred to as a learning continuation flag.
[0102] The high-level controller learning unit 240 is configured to have the evaluation unit 233 determine the high-level controller π H If it is determined that it is not necessary to continue acquiring the training data, the acquired data set D optUsing the high-level controller π H For example, the high-level controller learning unit 240 learns the acquired data set D opt Among the elements of the evaluation function, an element that indicates that the evaluation function value can reach the target state is used, and the state indicated by that element is transferred to the high-level controller π H When the input to the high-level controller π is H However, the high-level controller π H The learning method is not limited to a specific method.
[0103] 11 is a diagram showing an example of data input / output in the skill learning unit 15 according to the first embodiment. In the example of FIG. 11, the search point set initialization unit 211 initializes the search point set X ~ search For example, the search point set initialization unit 211 sets the initial state x based on the target parameter information. si and the target state / known task parameter value β g All possible combinations of ~ search The search point set initialization unit 211 may set the search point set X ~ search The setting of the search point set X ~ search This corresponds to the initial setting of the search point set X ~ search is updated by the next search point set setting unit 212.
[0104] The next search point set setting unit 212 sets the search point set X ~ search From the search point subset X ~ check Specifically, the next search point set setting unit 212 extracts the search point set X ~ search One or more elements are read from the search point subset X ~ check Then, the next search point set setting unit 212 sets the read elements as elements of the search point subset X~ check The elements set in are added to the search point set X ~ search Remove from the elements of .
[0105] When the prediction accuracy evaluation function setting unit 232 has learned the prediction accuracy evaluation function, the next search point set setting unit 212 sets the search point subset X ~ check In particular, the next search point set setting unit 212 sets the search point set X ~ search Among the elements of the search point subset X, the elements whose prediction accuracy evaluation function value indicates that the estimation accuracy of the level set function is lower than a predetermined condition are selected. ~ check Set it as an element of .
[0106] The method for determining whether the estimation accuracy is lower than the predetermined condition is not limited to a specific method. For example, if a larger prediction accuracy evaluation function value indicates lower accuracy, the estimation accuracy being lower than the predetermined condition may be, but is not limited to, that the prediction accuracy evaluation function value is greater than a predetermined threshold.
[0107] The system model setting unit 221 sets the search point subset X ~ check For example, the system model setting unit 221 sets the low-level controller π based on the detailed system model information, low-level controller information, and target parameter information stored in the storage device 2, and the abstract system model set by the abstract system model setting unit 14. l A system model, constraints on the parameters of the system model, and an evaluation function for the possibility of reaching the goal state are set.
[0108] The system model here refers to a model of the target system, such as a motion model of the target system. Constraints on the parameters of the system model are constraints on the values that the parameters of the system model can take, such as constraints on the specifications of the devices included in the target system and physical constraints. The system model and constraints on the parameters of the system model are used as part of the constraints in the optimal control problem handled by the problem setting calculation unit 222.
[0109] The system model setting unit 221 sets the low-level controller π l , a system model, parameters of the system model, an evaluation function of the possibility of reaching the goal state, and a search point X ~ i The execution time T and other information relating to the time limit for skill execution are output to the problem setting calculation unit 222.
[0110] The problem setting calculation unit 222 calculates the search point X based on the information from the system model setting unit 221. ~ i An optimal control problem is set for each parameter, and a solution to the set optimal control problem is searched for. As described above, the optimal control problem is, for example, a problem of determining a control input that minimizes the evaluation function value. Specifically, the optimal control problem here is a problem of determining a control input that minimizes the evaluation function value under constraints imposed by the operating environment and the like when an initial state and an evaluation function are given. The problem setting calculation unit 222 sets the evaluation function for the reachability of the target state as the evaluation function in the optimal control problem, and sets other various settings as constraints in the optimal control problem.
[0111] The problem setting calculation unit 222 calculates a high-level controller π that minimizes the evaluation function value under the constraints in the optimal control problem. H The problem setting calculation unit 222 calculates the output value of the search point X ~ i and the high-level controller π that minimizes the evaluation function value. H Output value α * i And the evaluation function value g *i Combination with (X ~ i , g * i , α * i ) to the data update unit 223.
[0112] For example, the problem setting calculation unit 222 may use, as the evaluation function for the optimal control problem, an evaluation function g such that the state x' is the target state when equation (6) holds.
[0113]
[0114] When equation (6) is satisfied, the state x' is the target state, as expressed by equation (7).
[0115]
[0116] x d ' represents the target state set. If the mapping from the state x of the detailed system model to the state x' of the abstract system model is represented by γ, then equation (8) can be obtained from equation (7).
[0117]
[0118] Minimizing the value of the evaluation function g in the optimal control problem is expressed as in equation (9).
[0119]
[0120] As mentioned above, T represents the time required to perform the skill. g(γ(x(T)), β g ) represents the evaluation function value for the state x(T) at the end of the skill. If this evaluation function value becomes 0 or less, it can be determined that the target state can be reached by executing the skill. As mentioned above, α is the value of the high-level controller π H Equation (9) represents the output of the high-level controller π that minimizes the value of the evaluation function g. H The system model in the optimal control problem can be expressed as in equation (10).
[0121]
[0122] As mentioned above, τ j represents an unknown task parameter. Time t is expressed as in equation (11).
[0123]
[0124] The inequality constraint in the optimal control problem can be expressed as in equation (12).
[0125]
[0126] c is a function representing a constraint condition, and is set based on, for example, target parameter information. The state at time 0 is the initial state, and is expressed as in equation (13).
[0127]
[0128] The fact that γ is a mapping from the state x of the detailed system model to the state x′ of the abstract system model can be expressed as in equation (14).
[0129]
[0130] The problem setting calculation unit 222 calculates the output α of the high-level controller so as to minimize the value of the evaluation function g as shown in equation (9) under the constraints of, for example, equations (10) to (14). * , and the value g of the evaluation function g at that time * As shown in equation (6), g * If ≦0, then from the initial state at this time, the output of the high-level controller is changed to α * By executing the skill as above, it can be determined that the goal state can be reached.
[0131] The problem setting calculation unit 222 calculates the minimum value g of the obtained evaluation function. * , and the output α of the high-level controller at that time * the initial state x s , and the target state / known task parameter value β g , and outputs it to the data update unit 223. Alternatively, the problem setting calculation unit 222 outputs the output α *In addition to or instead of this, information indicating that the target state can be reached may be output to the data update unit 223. The data update unit 223 uses this data to update the high-level controller π H Include it in the training data used to learn.
[0132] The method by which the problem setting calculation unit 222 solves the optimal control problem is not limited to a specific method. For example, the problem setting calculation unit 222 may use a known algorithm as a solution search algorithm for optimal control problems or a known algorithm as a round-robin search problem for optimization problems. Alternatively, the problem setting calculation unit 222 may simulate the behavior of the robot 5 and learn behaviors such that the evaluation function value is as small as possible, such as by reinforcement learning.
[0133] For example, when the function f in equation (10) is analytically obtained, the problem setting calculation unit 222 can solve the optimal control problem using any optimal control algorithm such as the Direct Collocation method or Differential Dynamic Programming (DDP).
[0134] On the other hand, when the function f cannot be analytically obtained, for example, when a simulator is used as the function f, the problem setting calculation unit 222 can solve the optimal control problem using a black-box optimization method such as path integral control or a model-free optimal control method. In this case, the problem setting calculation unit 222 determines the control parameter α in accordance with the problem of minimizing the evaluation function g, based on the function c representing the constraint conditions.
[0135] Here, when generating a skill of a grasping operation in the pick-and-place task shown in FIG. 6, the target parameter information and the low-level controller π L Here, "generating a skill" means learning a skill for a task different from the task for which the skill has already been learned. As mentioned above, a different task is a task with different values for unknown task parameters.
[0136] Here, a physical simulator based on the state x, the input u to the robot 5, and the contact force F that is the force for gripping the object to be grasped 6 is used as the system model shown in equation (10). In this case, the equation for determining whether the goal state can be reached is expressed as equation (15).
[0137]
[0138] If the formula (15) is satisfied, it can be determined that the goal state can be reached. In addition, the execution time information of the goal parameter information includes the upper limit "T max ” (T≦T max ) is included. The general constraint condition information of the target parameter information is also included information representing a constraint equation relating to the state x, input u, and contact force F as shown in equation (16).
[0139]
[0140] For example, this constraint is the upper limit of the contact force F, max ” (F≦F max ), the limit of the range of motion (or speed) "x max ” (|x|≦x max ), the upper limit of input u "u max ” (|u|≦u max ) is a comprehensive formula that represents
[0141] Also, the low-level controller π L is assumed to be a servo controller based on PID, for example. Here, the state of the robot 5 is expressed as "x r ”, and the target trajectory of the state of robot 5 is “x rd ", then the input u is expressed as, for example, equation (17).
[0142]
[0143] target trajectory x rd is expressed as, for example, equation (18).
[0144]
[0145] In equations (17) and (18), the control parameters based on the output α of the high-level controller πH are the coefficients of the target trajectory polynomial and the gain of the PID control, and are expressed as in equation (19).
[0146]
[0147] The problem setting calculation unit 222 solves the optimal control problem and obtains the optimal value (α * ) is calculated.
[0148] The data update unit 223 updates the data output from the problem setting calculation unit 222 (X ~ i , g * i , α * i ) to obtain the data set D opt The acquired data set D opt Update.
[0149] As described above, the level set function learning unit 231 opt The level set function learning unit 231 outputs the obtained level set function to the prediction accuracy evaluation function setting unit 232.
[0150] For example, the level set function learning unit 231 uses the acquired data set D opt The evaluation function value shown in is compared with a predetermined threshold value, and the acquired data set D opt In the example of equations (8) and (9), the level set function learning unit 231 determines whether the target state can be reached from the initial state shown in * Whether or not the goal state can be reached is determined based on whether or not the target state is equal to or less than 0.
[0151] Then, the level set function learning unit 231 learns the acquired data set D opt The level set function is learned using a combination of the state shown in the table, the goal state, and the result of determining whether the goal state can be reached as training data.
[0152] Here, the initial state x in the abstract state 0 ' and the target state / known task parameter value βg The optimal value g of the evaluation function g * A function that outputs g * (x 0 ', β g ) The executable state set of the target skill is denoted as χ 0 ' is expressed as in equation (20).
[0153]
[0154] The level set function learning unit 231 uses the acquired data set D opt The initial state x contained in 0 ' and the goal state / known task parameter value β g ' and the function value g * Based on multiple pairs of 0 For example, the level set function learning unit 231 calculates the level set function using a level set estimation method, which is an estimation method using Gaussian process regression based on the idea of Bayesian optimization. Here, this level set function is called g GP It is expressed as:
[0155] Note that this level set function g GP may be defined using a mean value function of a Gaussian process obtained through the level set estimation method, or may be defined as a combination of a mean value function and a variance function. Note that the method by which the level set function learning unit 231 learns the function indicating the feasible state set is not limited to a specific method. For example, the level set function learning unit 231 may obtain the level set function using truncated variance reduction (Truncated Variance Reduction) (TruvA) which is an estimation method using Gaussian process regression similar to the level set estimation method.
[0156] As described above, the level set function may be any model that evaluates the initial state that can be reached with respect to the desired state. H Output value α * is the initial state x 0 ' and the goal state / known task parameter value β g ' and the evaluation function value g *It can be said that the level set function is determined based on the set of π and π. By determining the level set function, it is possible to evaluate the reachable states and known task parameter values, and therefore it is possible to determine the control parameters that can achieve the desired state of the system. Here, the high-level controller π H Output value α * are examples of control parameters.
[0157] Furthermore, a control device for a robot or the like may use a level set function to determine whether a desired state can be reached from an initial state under given known task parameter values, and if the control device determines that a desired state can be reached, the control device may control the controlled object, such as a robot, using control parameters corresponding to the initial state.
[0158] In order to reduce the calculation cost of the level set function, the level set function learning unit 231 may acquire a simplified level set function by polynomial approximation or the like through learning. The level set function in this case is represented by g^. g^ is also referred to as a level set approximation function. The level set function learning unit 231 may learn a level set approximation function g^ that satisfies equation (21).
[0159]
[0160] As described above, the prediction accuracy evaluation function setting unit 232 sets a prediction accuracy evaluation function that indicates an evaluation of the level set function learned by the level set function learning unit 231. The prediction accuracy evaluation function setting unit 232 outputs the obtained prediction accuracy evaluation function to the evaluation unit 233.
[0161] For example, the prediction accuracy evaluation function setting unit 232 sets the search point X ~ Search point X ~ A function indicating an evaluation of the distribution of the search points X in the space of candidates may be learned as a prediction accuracy evaluation function. ~ The space of candidates is the search point X ~ The prediction accuracy evaluation function setting unit 232 determines the search point X ~The space that the domain of ~ Alternatively, the search point X ~ The space of candidates is the search point set X ~ search It may be the initial value of
[0162] For example, as a prediction accuracy evaluation function, the search point X ~ The candidate of X is used as an argument, and the search point X ~ The prediction accuracy evaluation function setting unit 232 may use a function that outputs, as a function value, an evaluation value for the possibility of reaching the target state indicated by the level set function for each candidate of the prediction accuracy evaluation function. ~ A learned search point X within a predetermined distance from the candidate ~ Alternatively, as will be described in the third embodiment, when the variance of the level set function values is determined, the prediction accuracy evaluation function setting unit 232 may set the prediction accuracy evaluation function so that the smaller the variance of the level set function values, the higher the evaluation. However, the method by which the prediction accuracy evaluation function setting unit 232 learns the prediction accuracy evaluation function is not limited to a specific method. Hereinafter, when there is no particular need to distinguish between them, the level set function g GP and the level set function ĝ are collectively referred to as the level set function ĝ.
[0163] As described above, the evaluation unit 233 uses the prediction accuracy evaluation function to determine the high-level controller π H The evaluation unit 233 determines whether or not it is necessary to continue acquiring training data for the search point X. The evaluation unit 233 sets the determination result to a learning continuation flag. ~The minimum value of the prediction accuracy evaluation function in the space of candidates may be calculated. The minimum value of the prediction accuracy evaluation function here refers to the lowest evaluation value. If the minimum value of the prediction accuracy evaluation function is evaluated as being lower than a predetermined threshold, the evaluation unit 233 may determine that it is necessary to continue acquiring training data. On the other hand, if the minimum value of the prediction accuracy evaluation function is evaluated as being higher than or equal to a predetermined threshold, the evaluation unit 233 may determine that it is not necessary to continue acquiring training data.
[0164] Alternatively, the evaluation unit 233 may ~ Alternatively, the evaluation unit 233 may sample prediction accuracy evaluation function values in the space of candidates of π and determine whether or not to continue acquiring training data based on the lowest evaluation value among the obtained prediction accuracy evaluation function values. H The method for determining whether or not it is necessary to continue acquiring training data is not limited to a specific method. For example, the evaluation unit 233 may determine whether or not it is necessary to continue acquiring training data based on a predetermined learning condition in addition to the value of the prediction accuracy evaluation function. The learning condition here may be various conditions. For example, when the number of times training data has been acquired reaches a predetermined number or more, the evaluation unit 233 may determine that it is not necessary to continue acquiring training data even if the evaluation indicated by the prediction accuracy evaluation function has not reached the predetermined evaluation.
[0165] As described above, the high-level controller learning unit 240 is configured to determine whether the evaluation unit 233 is a high-level controller π H If it is determined that it is not necessary to continue acquiring the training data, the acquired data set D opt Using the high-level controller π H Specifically, the high-level controller learning unit 240 learns the acquired data set D opt Among the elements of , the elements that can reach the target state are controlled by the high-level controller π H is the initial state x 0 ' and the target state / known task parameter value β g For the input of ', the output value α * The high-level controller π H Learn about the following.
[0166] The high-level controller learning unit 240 learns the high-level controller π H The model used for learning the above can be a variety of models, for example, a neural network, a Gaussian process regression, or a support vector regression, but is not limited to these.
[0167] (7) Processing Flow Fig. 12 is a diagram showing an example of a skill database update process performed by the learning device 1 according to the first embodiment. The learning device 1 executes the process shown in Fig. 12 for each skill to be generated.
[0168] (Step S101) The search point set initialization unit 211 initializes the search point set X ~ search and the acquired data set D opt For example, the search point set initialization unit 211 initializes the initial state x s and the goal state / known task parameter value β included in the goal state / known task parameter information. g Each of the arbitrary combinations of ~ search As an element of the search point set X ~ search The search point set initialization unit 211 generates the acquired data set D opt After step S101, the process proceeds to step S102.
[0169] (Step S102) The next search point set setting unit 212 sets the search point set X ~ search Specifically, the next search point set setting unit 212 extracts a subset from the search point set X ~ search A subset of the search points X ~ check Then, the next search point set setting unit 212 sets the set search point subset X ~ check Each element of the search point set X ~ searchAs shown in equation (22), the search point subset X ~ check is the initial state x si and the target state / known task parameter value β gi It has elements of combination with.
[0170]
[0171] The next search point set setting unit 212 sets the set subset X ~ check Each element of the search point set X ~ search The process of excluding from can be expressed as in equation (23).
[0172]
[0173] Here, "-" indicates that a subset is removed from the set. After step S102, the process proceeds to step S103.
[0174] (Step S103) The learning device 1 calculates a subset X of the search point set. ~ check Search point X, which is an element of ~ In the loop L11, the number of times the loop is repeated is represented by "i". ~ The target search point X ~ i After step S103, the process proceeds to step S104.
[0175] (Step S104) The system model setting unit 221 calculates the target search point X ~ i For example, the system model setting unit 221 performs various settings for setting the optimal control problem based on the low-level controller π l A system model, constraints on the parameters of the system model, and an evaluation function for the possibility of reaching the goal state are set. After step S104, the process proceeds to step S105.
[0176] (Step S105) The problem setting calculation unit 222 sets an optimal control problem based on the setting by the system model setting unit 221 in step S104. Then, the problem setting calculation unit 222 solves the set optimal control problem and calculates the output α of the high-level controller that minimizes the evaluation function value. * , and the value g of the evaluation function g at that time * After step S105, the process proceeds to step S106.
[0177] (Step S106) The data update unit 223 updates the acquired data set D opt Specifically, the data updating unit 223 updates the subset X of the search point set. ~ check The i-th element of X ~ i and the result of the judgment whether the task was successful or not g * i and the obtained control parameter α * i Combination with (X ~ i , g * i , α * i ) to obtain the data set D opt The data update unit 223 adds the acquired data set D opt The process of updating is expressed as in equation (24).
[0178]
[0179] "{(X ~ i , g * i , α * i )}" is (X ~ i , g * i , α * i After step S106, the process proceeds to step S107.
[0180] (Step S107) The learning device 1 performs the termination process of the loop L11. Specifically, the learning device 1 performs the termination process of the subset X ~ check If it is determined that there are elements for which the loop L11 process has not yet been performed, the learning device 1 continues to perform the loop L11 process on the elements for which the loop L11 process has not yet been performed. In this case, the process returns to step S103. On the other hand, if the subset X of the search point set ~ check If it is determined that the processing of loop L11 has been performed for all elements of, the learning device 1 ends loop L11. In this case, the processing proceeds to step S111.
[0181] (Step S111) The level set function learning unit 231 learns the acquired data set D opt After step S111, the process proceeds to step S112.
[0182] (Step S112) The prediction accuracy evaluation function setting unit 232 calculates the prediction accuracy evaluation function J based on the level set function g. g^ After step S112, the process proceeds to step S110.
[0183] (Step S113) The evaluation unit 233 calculates the prediction accuracy evaluation function J g The evaluation unit 233 determines whether or not it is necessary to continue learning the level set function g based on the prediction accuracy evaluation function J g In addition, it may be configured to determine whether or not the learning of the level set function g^ needs to be continued based on predetermined learning conditions. If the evaluation unit 233 determines that the learning of the level set function g^ needs to be continued (step S113: YES), the process proceeds to step S121. On the other hand, if the evaluation unit 233 determines that the learning of the level set function g^ does not need to be continued (step S113: NO), the process proceeds to step S131.
[0184] (Step S121) The next search point set setting unit 212 calculates the prediction accuracy evaluation function J g Based on this, the search point set X ~search Subset X from ~ check Specifically, the next search point set setting unit 212 extracts the prediction accuracy evaluation function J g Based on this, the search point set X ~ search A subset X of ~ check Then, the next search point set setting unit 212 sets the set subset X ~ check Each element of the search point set X ~ search After step S121, the process returns to step S103.
[0185] (Step S131) The high-level controller learning unit 240 learns the acquired data set D opt Using the high-level controller π H After step S131, the learning device 1 ends the process of FIG.
[0186] As described above, the search point set setting unit 210 sets the search points (x s , β g ) search point X from which training data is acquired for learning the control of the robot 5 ~ The problem setting calculation unit 222 selects the selected search point X ~ Information indicating whether the action indicated by is executable or not, and the selected search point X ~ A high-level controller π controls the robot 5 for the motion indicated by H The data update unit 223 calculates the output value to be output from the selected search point X ~ and the selected search point X ~ Information indicating whether the action indicated by is executable or not, and the selected search point X ~ For the operation shown by H and the output value to be output by the high-level controller π H The evaluation unit 233 acquires training data for learning control of the robot 5 by the robot 5. The evaluation unit 233 determines whether to continue acquiring the training data based on an evaluation of the acquisition status of the training data.
[0187] According to the learning device 1, it is possible to determine whether or not it is necessary to continue learning the control of the robot 5, and it is possible to eliminate unnecessary learning, thereby making learning efficient.
[0188] In addition, the level set function learning unit 231 s , β g ) and input the search point (x s , β g The problem setting calculation unit 222 learns the level set function g^, which outputs an estimate of whether the action indicated by the search point (x s , β g ) is based on the evaluation result of whether the action indicated by the
[0189] The prediction accuracy evaluation function setting unit 232 sets the search point (x s , β g ) and input the search point (x s , β g ) A prediction accuracy evaluation function J that outputs an evaluation value of the estimation accuracy of the level set function g^ for g^ The evaluation unit 233 sets the prediction accuracy evaluation function J g^ Based on this, it is determined whether to continue acquiring training data.
[0190] According to the learning device 1, it is possible to determine whether to continue acquiring training data by using the level set function g. The level set function g is used to select skills when the robot controller 3 controls the robot 5. According to the learning device 1, the amount of work required just to determine whether to continue acquiring training data is relatively small, and in this respect, it is possible to efficiently determine whether to continue acquiring training data.
[0191] The search point set setting unit 210 also calculates the prediction accuracy evaluation function J g^ The evaluation value of the search point (x s , β g ) is selected as a target for acquiring training data for controlling the robot 5. As a result, the learning device 1 selects the high-level controller π HIt is possible to obtain training data that show inputs and outputs that are likely to be inaccurate, and to use the high-level controller π H This allows for efficient learning.
[0192] Also, the search point (x s , β g ) includes known task parameters, which are parameter values of skills in which the behavior of the control target is modularized. This allows the learning device 1 to express differences in the behavior of the robot 5 that can be expressed by parameter values as parameter values of the skills, and to perform control learning so that the same skill can be applied to different behaviors.
[0193] Also, the search point (x s , β g ) is composed of a combination of the initial state of the robot 5 and its operating environment at the start of the skill, the known parameter values of the skill, and the target state of the robot 5 and its operating environment at the end of the skill. This allows the learning device 1 to H It is possible to learn the high-level controller π H and low-level controller π L This makes it possible to perform learning more efficiently than when learning control equivalent to both of the above.
[0194] The robot controller 3 also uses the high-level controller π obtained by learning using the training data acquired by the learning device 1. H According to the robot controller 3, when the robot controller 3 is learning, it is possible to determine whether or not it is necessary to continue learning the control of the robot 5, and it is possible to eliminate unnecessary learning, thereby enabling efficient learning.
[0195] The robot controller 3 also includes a high-level controller π that controls the robot 5 in accordance with the size of the object to be grasped so that the robot 5 can grasp each of the objects to be grasped that have different sizes. HThe robot controller 3 is expected to be able to control the robot 5 with high precision according to the size of the object to be grasped. Second Embodiment When the data acquisition unit 220 acquires data, the high-level controller learning unit 240 learns the high-level controller π H The second embodiment will be described in detail below. The configuration of the control system 100 in the second embodiment is the same as that in the first embodiment. The second embodiment will also be described using the configuration of the control system 100 shown in FIGS. 1 to 10.
[0196] 13 is a diagram showing an example of data input / output in the skill learning unit 15 according to the second embodiment. In the second embodiment, when the data acquisition unit 220 acquires data, the high-level controller learning unit 240 learns the high-level controller, and the high-level controller π obtained by the learning * H to the data acquisition unit 220. Outputting the high-level controller can be performed by outputting the parameter setting values of a predictor that constitutes the high-level controller, such as a neural network or a Gaussian process. In other respects, the input and output of data shown in Figure 13 is the same as the input and output of data in the case of the first embodiment described with reference to Figure 11.
[0197] Figure 14 is a diagram showing an example of a skill database update process performed by the learning device 1 according to the second embodiment. The learning device 1 executes the process shown in Figure 14 for each skill to be generated. Steps S201 to S204 in Figure 14 are the same as steps S101 to S104 in Figure 12. The loop from steps S203 to S207 in Figure 14 is referred to as loop L21.
[0198] (Step S205) As described in step S105, the problem setting calculation unit 222 sets an optimal control problem, solves the set optimal control problem, and calculates a high-level controller π that minimizes the evaluation function value. H The output and the evaluation function value at that time are calculated.
[0199] On the other hand, in step S205, the problem setting calculation unit 222 calculates the high-level controller π H If there is a high-level controller π H In order to avoid deviation from the output value due to the high level controller π H For example, the problem setting calculation unit 222 calculates the output of the obtained high-level controller π H and the high-level controller π H The problem setting calculation unit 222 may then find a solution to the optimal control problem so that the value of the evaluation function is as small as possible. This allows the problem setting calculation unit 222 to minimize the value of the original evaluation function and to minimize the value of the high-level controller π H The output value of the high-level controller π H A solution close to the output value of can be obtained.
[0200] Steps S206 and S207 are the same as steps S106 and S107 in Fig. 12. If the learning device 1 ends the loop L21 in step S207, the process proceeds to step S211.
[0201] (Step S211) The high-level controller learning unit 240 learns the high-level controller π H The determination criterion here is not limited to a specific one. For example, the high-level controller learning unit 240 determines whether or not it is necessary to continue learning the high-level controller π obtained by solving the optimal control problem in step S205. H and the output of the high-level controller π H When the difference between the output obtained using H It may be determined that there is no need to continue learning.
[0202] In step S211, the high-level controller π H If the high-level controller learning unit 240 determines that it is necessary to continue learning the high-level controller π (step S211: YES), the process proceeds to step S221. HIf the high-level controller learning unit 240 determines that it is not necessary to continue learning (step S211: NO), the process proceeds to step S231.
[0203] (Step S221) The high-level controller learning unit 240 learns the acquired data set D opt Using the high-level controller π H The method in which the high-level controller learning unit 240 learns the high-level controller in step S221 is the same as in step S131 in FIG. 12. In step S221, the acquired data set D opt The difference from step S131 is that the image is still being generated. After step S221, the process returns to step S203.
[0204] Steps S231 to S233 are the same as steps S111 to S113 in Fig. 12. If the evaluation unit 233 determines in step S233 that it is necessary to continue learning the level set function g^ (step S233: YES), the process proceeds to step S241. On the other hand, if the evaluation unit 233 determines that it is not necessary to continue learning the level set function g^ (step S233: NO), the process proceeds to step S251.
[0205] Step S241 is the same as step S121 in Fig. 12. After step S241, the process returns to step S203. Step S251 is the same as step S131 in Fig. 12. After step S251, the learning device 1 ends the process in Fig. 14.
[0206] Third Embodiment In the third embodiment, an example will be described in which the learning device 1 learns skills corresponding to differences in tasks that are difficult to express using parameter values. Specifically, in addition to the learning in the first embodiment, the learning device 1 learns metaparameter values for a predictor that constitutes a level set function and a predictor that constitutes a high-level controller. When acquiring training data for a new task and learning skills for executing the task, the learning device 1 learns and sets metaparameter values in advance using training data that has already been acquired so that the prediction accuracy of these predictors is as high as possible. The learning device 1 may also perform learning according to the third embodiment in addition to learning according to the second embodiment. That is, the second and third embodiments can be implemented in combination.
[0207] In the third embodiment, it is assumed that tasks are generated according to some probability distribution, and that correct data of inputs and outputs of the predictor follow some probability distribution determined for each task. j τ can be expressed as τ ∼ τ, where τ represents the probability distribution that the task follows. j represents the task. The fact that the correct data of the input and output of the predictor follows some probability distribution determined for each task is called S j ~D j It can be expressed as follows. j is the task τ j The probability distribution is determined according to S j is the task τ j This represents the correct data for the input and output of the predictor in this case.
[0208] Fig. 15 is a diagram showing an example of the configuration of the skill learning unit 15 according to the third embodiment. In the configuration shown in Fig. 15, the skill learning unit 15 includes a search task setting unit 250 and a meta parameter processing unit 260 in addition to the units shown in Fig. 10. In other respects, the configuration of the control system in the third embodiment is the same as that of the first embodiment. The third embodiment will also be described using the configuration of the control system 100 shown in Figs. 1 to 9.
[0209] The search task setting unit 250 sets a task to be learned by the learning device 1. A task set by the search task setting unit 250 to be learned by the learning device 1 is also referred to as a search task. The search task setting unit 250 assumes a probability distribution T that the generated task will follow, and sets a search task based on the assumed probability distribution T. The method by which the search task setting unit 250 assumes the probability distribution T that the generated task will follow is not limited to a specific method. For example, the probability distribution T may be set in advance, but is not limited to this.
[0210] The meta-parameter processing unit 260 includes a predictor that configures a level set function and a high-level controller π H In the third embodiment, the meta parameter values of the predictors constituting the level set function and the high-level controller π are learned, and the meta parameter values obtained by learning are set in these predictors. H The predictor used in the prediction analysis is a learning model predictor in which parameter values are set according to a probability distribution, such as a Bayesian neural network or a Gaussian process. The meta parameter processing unit 260 learns and sets the probability distribution to which these parameter values follow as meta parameter values. The meta parameter processing unit 260 also evaluates the prediction accuracy of the predictor for which the meta parameters have been set, and determines whether to continue learning the meta parameter values based on the evaluation result.
[0211] Fig. 16 is a diagram showing an example of data input / output in the skill learning unit 15 according to the third embodiment. As explained with reference to Fig. 15, the skill learning unit 15 in the configuration shown in Fig. 16 also includes a search task setting unit 250 and a meta parameter processing unit 260 in addition to the units shown in Fig. 11.
[0212] The search task setting unit 250 sets a search task in response to input task parameter information. The task parameter information includes information regarding the probability distribution T of the task to be generated. For example, the task parameter information may be information indicating the probability distribution T of the task to be generated, and the search task setting unit 250 may set a search task in accordance with this probability distribution T.
[0213] The search task setting unit 250 repeats setting of the search task while the learning continuation flag for the unknown task parameter indicates continuation of learning. The learning continuation flag for the unknown task parameter is a flag that indicates whether to continue learning of the meta parameter value of the predictor. While the learning continuation flag for the unknown task parameter indicates continuation of learning, the search task setting unit 250 sets the next search task each time the learning device 1 finishes learning related to the search task.
[0214] In the third embodiment, the learning continuation flag set by the evaluation unit 233 is also referred to as a learning continuation flag for a known task parameter in order to distinguish it from a learning continuation flag for an unknown task parameter. j " or "j" may be written to indicate that the data is for each task.
[0215] Each time the search task setting unit 250 sets a search task, the learning device 1 j In particular, the search point set initialization unit 211 initializes the search point set X ~ search Furthermore, the system model setting unit 221 performs various settings for setting the optimal control problem according to the search task.
[0216] The meta parameter processing unit 260 processes the entire acquired data set D optall The entire acquired data set D is used to learn the meta parameter values described above and to determine whether to continue learning the meta parameter values. optall is the total acquired data set D acquired by the data update unit 223. opt,j It is a combination of the above.
[0217] For example, the data update unit 223 updates the entire acquired data set D optall The initial value of is set to 0, and the acquired data set D opt,j Each time a data set D is generated, opt,j The entire acquired data set D optall The acquired data set D opt,j The entire acquired data set Doptall The process of combining can be expressed as in equation (25).
[0218]
[0219] The meta parameter values learned by the meta parameter processing unit 260 are set in the predictors that make up the level set set and the predictors that make up the high-level controller. Also, as described above, while the learning continuation flag for the unknown task parameter set by the meta parameter processing unit 260 indicates that learning is continuing, the search task setting unit 250 sets the next search task each time the learning device 1 finishes learning related to a search task.
[0220] 17 is a diagram showing an example of the configuration of the meta parameter processing unit 260. In the configuration shown in FIG. 17, the meta parameter processing unit 260 includes a meta parameter individual processing unit 261 and a learning continuation flag integrating unit 262.
[0221] The meta parameter processing unit 260 includes a meta parameter individual processing unit 261 for each predictor to be trained. In the example of FIG. 16, a level set function and a high-level controller π H These are configured using a predictor and are the targets of learning meta parameter values. In this case, the meta parameter processing unit 260 includes two meta parameter individual processing units 261.
[0222] However, the number of meta parameter individual processing units 261 included in the meta parameter processing unit 260 is not limited to two. For example, H In addition, there may be other objects that are configured using a predictor and are the subject of meta parameter value learning. In this case, the meta parameter processing unit 260 may be provided with an individual meta parameter processing unit 261 for each object that is configured using a predictor and is the subject of meta parameter value learning.
[0223] When distinguishing between the individual meta parameter individual processing units 261, they are referred to as meta parameter individual processing unit 261-1, meta parameter individual processing unit 261-2, ..., meta parameter individual processing unit 261-N, where N is a positive integer indicating the number of meta parameter individual processing units 261 included in the meta parameter processing unit 260.
[0224] The meta parameter individual processing unit 261 learns the meta parameter values of the predictor. If the predictor has multiple meta parameters, the meta parameter individual processing unit 261 learns the values for each meta parameter. For example, if each predictor is configured using a Bayesian neural network and has weight coefficients between nodes and biases for each node as parameters, the probability distributions to which each of these parameters follows correspond to meta parameters. The meta parameter individual processing unit 261 learns the values for each of these meta parameters.
[0225] Furthermore, the meta parameter individual processing unit 261 sets a value of a learning continuation flag for each predictor, which indicates whether or not learning of the meta parameter value needs to be continued for the target predictor. The learning continuation flag for each predictor is also referred to as an individual learning continuation flag. The learning continuation flag integrating unit 262 integrates the values of the individual learning continuation flags to set a value of a learning continuation flag for the unknown task parameter. The learning continuation flag integrating unit 262 corresponds to an example of a learning continuation determination integration means.
[0226] 18 is a diagram showing an example of data input / output in the meta parameter processing unit 260. As described above, a meta parameter individual processing unit 261 is provided for each predictor targeted by the meta parameter processing unit 260. The meta parameter individual processing unit 261 processes all acquired data D optall Upon receiving the meta-learning execution flag or the internal learning evaluation value, the meta parameter individual processing unit 261 outputs the value of the target meta parameter and also sets the value of the individual learning continuation flag.
[0227] The meta-learning execution flag is a flag indicating whether or not to perform learning of meta-parameter values. For example, the entire acquired data set D optallWhen a predetermined number of pieces of data (set elements) for each task are accumulated in the meta parameter processing unit 260, the data update unit 223 may set the value of the meta-learning execution flag to a value indicating that learning of meta parameter values will be performed. Furthermore, when the meta parameter processing unit 260 ends learning of meta parameter values, the data update unit 223 may set the value of the meta-learning execution flag to a value indicating that learning of meta parameter values will not be performed.
[0228] The internal learning evaluation value is a value indicating an evaluation of the prediction accuracy of a predictor. For example, when learning of meta parameter values starts, the meta parameter individual processing unit 261 may calculate a generalization error of the meta parameter. Then, the meta parameter processing unit 260 may calculate an internal learning evaluation value indicating a comprehensive evaluation of all predictors that are the targets of learning of meta parameter values, based on the generalization error of the meta parameter.
[0229] The learning continuation flag integrating unit 262 integrates the values of the individual learning continuation flags to set the value of the learning continuation flag for the unknown task parameter. For example, if the values of one or more individual learning continuation flags indicate that learning continuation is necessary, the learning continuation flag integrating unit 262 sets the value of the learning continuation flag for the unknown task parameter to a value indicating that learning continuation is necessary. On the other hand, if the values of all the individual learning continuation flags indicate that learning continuation is not necessary, the learning continuation flag integrating unit 262 sets the value of the learning continuation flag for the unknown task parameter to a value indicating that learning continuation is not necessary.
[0230] Fig. 19 is a diagram showing a first example of the configuration of the meta parameter individual processing unit 261. In the configuration shown in Fig. 19, the meta parameter individual processing unit 261 includes a training data extraction unit 271, a meta parameter learning unit 272, a generalization error evaluation unit 273, and a learning continuation determination unit 274.
[0231] The training data extraction unit 271 extracts the entire acquired data set D optallThe meta parameter learning unit 272 uses the training data extracted by the training data extraction unit 271 to learn the meta parameter values. The generalization error evaluation unit 273 calculates an evaluation value for the generalization error of the predictor when the meta parameter values learned by the meta parameter learning unit 272 are used. The learning continuation determination unit 274 determines whether to continue learning the meta parameter values based on the evaluation value calculated by the generalization error evaluation unit 273.
[0232] 20 is a diagram showing an example of data input / output in the meta parameter individual processing unit 261 shown in FIG. 19. When the value of the meta-learning execution flag indicates that meta parameter values are to be learned, the training data extraction unit 271 extracts the entire acquired data set D optall The training data extraction unit 271 extracts training data for learning meta parameter values from the meta parameter values. The training data extraction unit 271 repeats the extraction of training data until the value of the meta-learning execution flag reaches a value indicating that continuation of learning is not necessary. The training data extraction unit 271 is an example of a training data extraction means.
[0233] When the value of the meta-learning execution flag indicates that meta-parameter values are to be learned, the meta-parameter learning unit 272 learns meta-parameter values based on training data for learning meta-parameter values, learning parameter information, and predictor information. The training data for learning meta-parameter values includes combinations of input values in the learning model and correct answers for the output values of the learning model for those input values. The meta-parameter learning unit 272 is an example of a meta-parameter learning means.
[0234] The predictor information is information about a predictor having meta parameters to be learned. For example, the predictor information may include information about a function representing the predictor. The learning parameter information is information about meta parameters to be learned. For example, the learning parameter information may include information indicating the number of meta parameters that the predictor to be learned has.
[0235] Here, the predictor to be learned for the meta parameter values is expressed by a function f as shown in equation (26).
[0236]
[0237] x represents the input to the predictor, θ represents the parameter of the predictor, and y represents the output of the predictor. The probability distribution p(y|x, θ) of the output of the predictor is expressed as in Equation (27).
[0238]
[0239] N s is a positive integer indicating the number of parameters of the predictor, and θ = (θ 1 , θ 2 , ..., θ Ns ) The parameter θ i (i=1, 2,..., N s ) follows the probability distribution p(θ|S) as shown in equation (28).
[0240]
[0241] In learning a Bayesian neural network, a conditional probability distribution p(θ|S) is calculated using the data S of the parameter θ. The method by which the learning device 1 calculates the probability distribution p(θ|S) is not limited to a specific method. For example, the learning device 1 may calculate the probability distribution p(θ|S) using the Optimal Gibbs Posterior structure shown in equation (29).
[0242]
[0243] P(θ) represents the prior distribution of the value of the parameter θ. The meta parameter learning unit 272 learns this prior distribution P(θ) as a meta parameter value. β is a parameter called a temperature parameter. The value of the temperature parameter β is set in advance, for example.
[0244] "l(S, f(x, θ))" represents a loss function l based on the difference between the output of the predictor and the correct value of the output based on the correct data S shown as training data. "E" represents the expected value. Specifically, "E θ~P(θ)[exp(-βl(S, f(x, θ)))]" indicates the expected value of "exp(-βl(S, f(x, θ)))" when the parameter θ follows the prior distribution P(θ). The meta parameter learning unit 272 learns the meta parameter values so that the expected value of the loss function shown in equation (30) is as small as possible, for example.
[0245]
[0246] "l(S, f)" in equation (30) θ,P )) represents the loss function l similar to "l(S, f(x, θ))" in equation (29). In equation (30), the function f indicating the predictor is expressed as "f θ,P " indicates the parameter θ and the probability distribution P, which is a meta-parameter.
[0247] As mentioned above, "E" represents the expected value. S~D [l(S,f θ,P ) )] represents the expected value of the loss function l when the correct answer data S follows the probability distribution D. D~Τ [E S~D [l(S,f θ,P ) ]]” is the expected value “E D~Τ [E S~D [l(S,f θ,P ) ]]” represents the expected value.
[0248] For example, the meta parameter learning unit 272 obtains the probability distribution Q(P) of the probability distribution P(θ) as a meta parameter based on equation (31).
[0249]
[0250] "P(P)" represents the prior distribution of the probability distribution P(θ), which is a meta-parameter. λ is a parameter called a temperature parameter. The value of λ is set in advance, for example. N τ is a positive integer representing the number of tasks. "ln" represents the natural logarithm. As mentioned above, "E" represents the expected value. Specifically, "E θ~P(θ) [...]" represents the expected value of the value in brackets ([...]) when the value of the parameter θ follows the probability distribution P(θ). P~P"[...]" represents the expected value of the value in brackets ("[...]]) when the probability distribution P(θ) follows the probability distribution P(P).
[0251] The generalization error evaluation unit 273 calculates an evaluation value of the generalization error of the predictor when the above-mentioned probability distributions P(θ) and Q(P) are used. For example, the generalization error evaluation unit 273 calculates an evaluation value of the generalization error L(Q, T) shown in equation (32).
[0252]
[0253] As mentioned above, "E" represents the expected value. Specifically, "E" on the right side of Equation (32) P~Q [E D~Τ [E S~D [l(S,f θ,P ) ]]] is the expected value "E" shown in equation (30) when the probability distribution P(θ) follows the probability distribution Q(P). D~Τ [E S~D [l(S,f θ,P The generalization error evaluation unit 273 calculates, for example, the value of the right side of equation (33) (the right side of the inequality shown in equation (33)) as the evaluation value of the generalization error L(Q, T).
[0254]
[0255] “C(δ,λ,β)” is the loss function l(S,f θ,f ) is a function determined depending on the type of the generalization error L(Q,T). The right-hand side of equation (33) represents the upper bound of the generalization error L(Q,T). The right-hand side of equation (33) is also written as L^(Q,T).
[0256] The learning continuation determination unit 274 sets the value of the individual learning continuation flag based on the generalization error evaluation value L^(Q, T) calculated by the generalization error evaluation unit 273. The learning continuation determination unit 274 may calculate the value of the individual learning continuation flag I based on equation (34).
[0257]
[0258] The value "0" of the individual learning continuation flag I indicates that there is no need to continue learning the meta parameter values. The value "1" of the individual learning continuation flag I indicates that there is a need to continue learning the meta parameter values. ε is a constant indicating a predetermined threshold value.
[0259] The evaluation value L^(Q, T) of the generalization error indicates a smaller value as the evaluation is higher. Therefore, if the evaluation value L^(Q, T) is equal to or less than the threshold ε, the learning continuation determination unit 274 determines that there is no need to continue learning the meta parameter values. On the other hand, if the evaluation value L^(Q, T) is greater than the threshold ε, the learning continuation determination unit 274 determines that there is a need to continue learning the meta parameter values.
[0260] The learning continuation determination unit 274 may determine whether or not to continue learning of the meta parameter value based on information about the conditions for continuing learning. Fig. 20 shows an example in which the learning continuation determination unit 274 acquires error threshold information and continuation condition information as information about the conditions for continuing learning.
[0261] The error threshold information is a judgment threshold for the generalization error evaluation value L^(Q, T), like the threshold ε. The continuation condition information is information indicating a judgment method other than judgment based on the generalization error evaluation value L^(Q, T). For example, when the number of iterations of learning the meta parameter values reaches a predetermined number, the learning continuation judgment unit 274 may determine that there is no need to continue learning the meta parameter values, even if the generalization error evaluation value L^(Q, T) is greater than the threshold ε.
[0262] However, the method by which the learning continuation determination unit 274 determines whether or not it is necessary to continue learning the meta parameter value is not limited to a specific method. The information on the conditions for continuing learning used by the learning continuation determination unit 274 can be various information depending on the method by which the learning continuation determination unit 274 determines whether or not it is necessary to continue learning the meta parameter value.
[0263] Fig. 21 is a diagram showing a second example of the configuration of the meta parameter individual processing unit 261. In the example of Fig. 21, the meta parameter individual processing unit 261 includes a meta-learning execution determination unit 281 in addition to the units shown in Fig. 19. The meta-learning execution determination unit 281 sets a meta-learning execution flag.
[0264] Fig. 22 is a diagram showing an example of data input / output in the meta parameter individual processing unit 261 shown in Fig. 21. The meta-learning execution determination unit 281 sets the value of the meta-learning execution flag based on the internal learning evaluation value.
[0265] For example, if the evaluation of the prediction accuracy of the predictor indicated by the internal learning evaluation value is lower than a predetermined evaluation, the meta-learning execution determination unit 281 sets the value of the meta-learning execution flag to a value indicating that learning of meta parameter values will be performed. On the other hand, if the evaluation of the prediction accuracy of the predictor indicated by the internal learning evaluation value is higher than or equal to the predetermined evaluation, the meta-learning execution determination unit 281 sets the value of the meta-learning execution flag to a value indicating that learning of meta parameter values will not be performed. The meta-learning execution determination unit 281 corresponds to an example of a meta-learning execution determination means. In this way, the value of the meta-learning execution flag may be set within the learning continuation determination unit 274.
[0266] 23 is a diagram showing an example of a skill database update process performed by the learning device 1 according to the third embodiment. For example, the learning device 1 performs the process shown in FIG. 23 when acquiring training data for multiple skills.
[0267] (Step S301) The data update unit 223 updates the entire acquired data set D optall Specifically, the data update unit 223 initializes the entire acquired data set D optall The value of is set to an empty set. After step S301, the process proceeds to step S302.
[0268] (Step S302) The search task setting unit 250 sets a search task. For example, the search task setting unit 250 sets an unknown task parameter value τ j By selecting j The task τ that is associated with jmay be set in the search task. After step S302, the process proceeds to step S303.
[0269] Steps S303 to S313 in Fig. 23 are the same as steps S101 to S113 in Fig. 12. The loop from steps S305 to S309 in Fig. 23 is denoted as loop L31.
[0270] In step S313, the high-level controller π H If the high-level controller learning unit 240 determines that it is necessary to continue learning the high-level controller π (step S313: YES), the process proceeds to step S321. H If the high-level controller learning unit 240 determines that it is not necessary to continue learning (step S313: NO), the process proceeds to step S331.
[0271] Step S321 in Fig. 23 is the same as step S121 in Fig. 12. After step S321, the process returns to step S305. Step S331 in Fig. 23 is the same as step S131 in Fig. 12. After step S331, the process proceeds to step S332.
[0272] (Step S332) The data update unit 223 updates the entire acquired data set D optall As described above, the data updating unit 223 updates the generated acquired data set D opt,j The entire acquired data set D optall After step S332, the process proceeds to step S333.
[0273] (Step S333) The meta parameter processing unit 260 calculates meta parameter values of the predictor. After step S333, the process proceeds to step S334.
[0274] (Step S334) The meta parameter processing unit 260 determines whether or not it is necessary to continue learning the meta parameter values. If the meta parameter processing unit 260 determines that it is necessary to continue learning (step S334: YES), the process proceeds to step S341. On the other hand, if the meta parameter processing unit 260 determines that it is not necessary to continue learning (step S334: NO), the learning device 1 ends the process of FIG. 23.
[0275] (Step S341) The search task setting unit 250 updates the search task. Specifically, the search task setting unit 250 sets any task that has not yet been set as a search task as the search task. After step S341, the process returns to step S303.
[0276] 24 is a diagram showing an example of a process for calculating meta parameter values of a predictor by the meta parameter processing unit 260. The meta parameter processing unit 260 performs the process of FIG. 24 in step S333 of FIG.
[0277] (Step S401) The meta parameter individual processing unit 261 calculates a meta parameter value for each predictor. The meta parameter individual processing unit 261 also determines, for each predictor, whether or not learning of the meta parameter value needs to be continued. The meta parameter individual processing unit 261 may execute the process of step S401 for each predictor in parallel. Alternatively, the meta parameter individual processing unit 261 may execute the process of step S401 for each predictor sequentially. After the process of step S401 has been completed for all predictors being processed, the process proceeds to step S402.
[0278] (Step S402) The learning continuation flag integrating unit 262 determines whether or not learning of the meta parameter values for all of the multiple predictors is necessary, based on the determination result of whether or not learning of the meta parameter values for each predictor is necessary. After step S402, the meta parameter processing unit 260 ends the process in FIG. 24 .
[0279] 25 is a diagram showing a first example of processing in which the meta parameter individual processing unit 261 calculates meta parameter values for each predictor and determines whether or not learning of the meta parameter values needs to be continued. In step S401 of FIG. 24, the meta parameter individual processing unit 261 performs the processing of FIG. 25 for each predictor.
[0280] (Step S411) The training data extraction unit 271 extracts training data for learning meta parameter values from the entire acquisition data set Doptall. After step S411, the process proceeds to step S412.
[0281] (Step S412) The meta parameter learning unit 272 learns the meta parameter values of the predictor being processed. After step S412, the process proceeds to step S413.
[0282] (Step S413) The generalization error evaluation unit 273 calculates an evaluation value of the generalization error when the meta parameter values obtained by learning are used. After step S413, the process proceeds to step S414.
[0283] (Step S414) The learning continuation determination unit 274 determines whether or not the learning of the parameter value needs to be continued based on the evaluation value of the generalization error. After step S414, the meta parameter individual processing unit 261 ends the process of FIG.
[0284] 26 is a diagram showing a second example of the process in which the meta parameter individual processing unit 261 calculates meta parameter values for each predictor and determines whether or not learning of the meta parameter values needs to be continued. In step S401 of FIG. 24, the meta parameter individual processing unit 261 performs the process of FIG. 26 for each predictor instead of the process of FIG. 25.
[0285] (Step S421) The meta-learning execution determination unit 281 sets the value of the meta-learning execution flag based on the internal learning evaluation value. After step S421, the process proceeds to step S422.
[0286] Steps S422 to S425 in Fig. 26 are the same as steps S411 to S414 in Fig. 25. After step S425, the meta parameter individual processing unit 261 ends the processing in Fig. 26.
[0287] A more specific example of the skill database update process performed by the learning device 1 according to the third embodiment, shown in FIG. 23 , will be described. In step S302, the search task setting unit 250 selects, for example, the shape of an object for which a grasping motion is to be learned as an unknown task parameter. The search task setting unit 250 may sample the unknown task parameter according to a probability distribution T. Alternatively, the search task setting unit 250 may set the unknown task parameter using an algorithm that probabilistically selects the unknown task parameter. The same applies to step S341.
[0288] In step S303, the search point set initialization unit 211 defines a state variable x that represents the position, posture, etc. of the robot 5 and the object to be grasped, and sets the state of the robot 5 and the object to be grasped before the grasp operation is performed as the initial state x si The search point set initialization unit 211 also sets the target state / known task parameter β gi Then, the search point set initialization unit 211 sets the initial state x si and the goal state / known task parameter β gi The pair (x si , β gi ) into the search point set X ~ search,j Set it to the element.
[0289] In step S306, the system model setting unit 221 sets the search point subset X ~ check Search point X, which is an element of ~ i and extract the target state / known task parameters β gi and the set task τ j Based on this, we define the system model (dynamics), the constraints in the system model, and the low-level controller π L Examples of the constraints here include, but are not limited to, the operating area of the robot 5, the upper limit of the input in the specifications of the robot 5, and constraints for collision avoidance.
[0290] Furthermore, the system model setting unit 221 sets the search point X ~ i From the initial state x si and the goal state / known task parameters β gi x included in fi Furthermore, the system model setting unit 221 sets an evaluation function g in the optimal control problem based on these values. The system model setting unit 221 may set the evaluation function g shown in equation (35).
[0291]
[0292] "||・|| 2 " denotes the square norm. ε g is a tolerance parameter indicating the tolerance of the magnitude of the error.
[0293] In step S312, the prediction accuracy evaluation function setting unit 232 sets the prediction accuracy evaluation function J shown in equation (36) for the predictor configured using the Bayesian neural network. g^i may be set.
[0294]
[0295] μ g^j (X ~ ) indicates the predicted mean value. σ g^j 2 (X ~ ) indicates the prediction variance. These values can be obtained for predictions by a Bayesian neural network. γ is a coefficient multiplied by the prediction variance, and can be interpreted as a parameter that sets the confidence region (confidence interval). Alternatively, the prediction accuracy evaluation function setting unit 232 can calculate the level set function g ^ i The function for calculating the entropy of g^i It may be set as follows.
[0296] In step S313, the evaluation unit 233 evaluates the search point set X ~ search,j Each element X of ~ Regarding the above prediction variance σ g^j 2 (X~ ) and calculate σ for all elements. g^j 2 (X ~ ) ≦ ε σ If the following holds, it may be determined that continuation of learning is unnecessary. σ is the prediction variance threshold. σ is also referred to as a dispersion threshold parameter. ~ search,j element of (x si , β gi ) to X ~ Alternatively, the search point set X ~ search,j For all elements of σ g^j 2 (X ~ ) ≦ ε σ is satisfied, or the acquired data set D opt,j If the number of elements reaches a set threshold, it may be determined that continuation of learning is not necessary.
[0297] As described above, the meta parameter learning unit 272 learns the values of meta parameters that indicate a probability distribution in a learning model in which the parameter values follow a probability distribution, based on training data that indicates the input and output of the learning model. The generalization error evaluation unit 273 calculates an evaluation value that indicates an evaluation of the generalization error of the learning model. The learning continuation determination unit 274 determines whether or not to continue learning the values of the meta parameters based on the evaluation value that indicates the evaluation of the generalization error of the learning model. The learning device 1 can determine whether or not to continue learning when learning the meta parameter values of a learning model, thereby enabling efficient learning by eliminating unnecessary learning.
[0298] Furthermore, the training data extraction unit 271 repeats the selection of training data to be used for learning from among the training data for learning meta parameter values until it is determined that continued learning is unnecessary. According to the learning device 1, when learning meta parameter values of a learning model, it is possible to determine whether or not continued learning is necessary, and learning can be performed efficiently in that unnecessary learning can be omitted.
[0299] Furthermore, the meta-learning execution determination unit 281 determines whether to learn meta parameter values based on an evaluation value indicating an evaluation of the generalization error of the learning model. The training data extraction unit 271 selects training data when the meta-learning execution determination unit 281 determines to learn meta parameter values. According to the learning device 1, when learning meta parameter values of a learning model, it is possible to determine whether to continue learning based on an evaluation of the generalization error of the learning model, thereby enabling efficient learning by eliminating unnecessary learning.
[0300] Furthermore, the learning continuation flag integrating unit 262 determines whether or not learning of meta parameter values needs to be continued for all of the multiple learning models based on the determination results of each of the multiple learning continuation determination means corresponding to the multiple learning models. The learning device 1 can determine whether or not learning of meta parameter values needs to be continued for all of the multiple learning models, thereby enabling efficient learning by eliminating unnecessary learning.
[0301] One of the learning models is a high-level controller π that controls the robot 5 to execute a modularized task. H and the parameter values of the skills are included in the input values for the learning model. The meta parameter learning unit 272 learns the meta parameter values using training data for a plurality of skills. According to the learning device 1, the difference in the tasks is handled by learning the meta parameter values, and a plurality of tasks are handled by the high-level controller π H It can be executed in
[0302] The robot controller 3 also uses a high-level controller π H The robot controller 3 corresponds to the difference in tasks by setting the meta parameter values, and performs a plurality of tasks using a high-level controller π H It can be executed in
[0303] The robot controller 3 also includes a high-level controller π that controls the robot 5 in accordance with the shape of the object to be grasped so that the robot 5 can grasp each of the objects to be grasped having different shapes. H The robot controller 3 is expected to be able to control the robot 5 with high precision in accordance with the shape of the object to be grasped.
[0304] 27 is a diagram showing an example of the configuration of a learning device according to the fourth embodiment. In the configuration shown in Fig. 27, a learning device 610 includes a meta parameter learning unit 611, a generalization error evaluation unit 612, and a learning continuation determination unit 613.
[0305] With this configuration, the meta parameter learning unit 611 learns the values of meta parameters that indicate a probability distribution in a learning model in which the parameter values follow a probability distribution, based on training data that indicates the input and output of the learning model. The generalization error evaluation unit 612 calculates an evaluation value that indicates an evaluation of the generalization error of the learning model. The learning continuation determination unit 613 determines whether or not to continue learning the values of the meta parameters, based on the evaluation value that indicates the evaluation of the generalization error of the learning model.
[0306] The meta parameter learning unit 611 corresponds to an example of a meta parameter learning means. The generalization error evaluation unit 612 corresponds to an example of a generalization error evaluation means. The learning continuation determination unit 613 corresponds to an example of a learning continuation determination means. The learning device 610 can determine whether or not to continue learning when learning meta parameter values of a learning model, thereby enabling efficient learning by eliminating unnecessary learning.
[0307] Fifth Embodiment Fig. 28 is a diagram showing an example of the configuration of a control device according to a fifth embodiment. In the configuration shown in Fig. 28, the control device 620 includes a control unit 621. In this configuration, the control unit 621 controls the robot according to the shape of the object to be grasped so that the robot grasps each of objects having different shapes. It is expected that the control device 620 will enable high-precision control of the robot according to the shape of the object to be grasped.
[0308] Sixth Embodiment Fig. 29 is a diagram showing an example of processing in a learning method according to the sixth embodiment. The learning method shown in Fig. 29 includes learning meta parameters (step S611), evaluating a generalization error (step S612), and determining whether to continue learning (step S613).
[0309] In the step S611, the computer learns the values of the meta parameters, which indicate the probability distribution of the parameter values in the learning model, based on the training data indicating the input and output of the previous learning model. In the step S612, the computer calculates an evaluation value indicating the evaluation of the generalization error of the learning model.
[0310] In determining whether to continue learning (step S613), the computer determines whether to continue learning the meta parameter values based on an evaluation value indicating an evaluation of the generalization error of the learning model. According to the learning method shown in Fig. 29, when learning the meta parameter values of the learning model, it is possible to determine whether to continue learning, and learning can be performed efficiently in that unnecessary learning can be avoided.
[0311] Note that a program for executing all or part of the processing performed by the learning device 1, robot controller 3, learning device 610, and control device 620 may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be loaded into a computer system and executed to perform the processing of each part. Note that the term "computer system" here includes hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs (Read Only Memory), and CD-ROMs (Compact Disc Read Only Memory), as well as storage devices such as hard disks built into the computer system. The program may be for implementing part of the aforementioned functions, or may be capable of implementing the aforementioned functions in combination with a program already recorded on the computer system.
[0312] Although an embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention.
[0313] The present invention may be applied to a learning device, a control device, a learning method, and a recording medium.
[0314] 1, 610 Learning device 2 Storage device 3 Robot controller 4 Measuring device 5 Robot 100 Control system 210 Search point set setting unit 211 Search point set initialization unit 212 Next search point set setting unit 221 System model setting unit 222 Problem setting calculation unit 223 Data update unit 230 Prediction accuracy evaluation function learning unit 231 Level set function learning unit 232 Prediction accuracy evaluation function setting unit 233 Evaluation unit 240 High-level controller learning unit 611 Meta parameter learning unit 612 Generalization error evaluation unit 613 Learning continuation determination unit 620 Control device 621 Control unit
Claims
1. a meta-parameter learning means for learning values of meta-parameters indicating a probability distribution in a learning model in which the values of the parameters follow a probability distribution, based on training data indicating an input and an output in the learning model; a generalization error evaluation means for calculating an evaluation value indicating an evaluation of a generalization error of the learning model; a learning continuation determination means for determining whether or not learning of the value of the meta parameter needs to be continued based on the evaluation value; A learning device comprising:
2. a training data extraction means for repeatedly selecting training data to be used for learning from among the training data for learning the values of the meta parameters until it is determined that continuation of learning is not necessary; The learning device according to claim 1 , further comprising:
3. a meta-learning execution determination means for determining whether or not to perform learning of the values of the meta-parameters based on an evaluation value indicating an evaluation of a generalization error of the learning model; Further equipped with the training data extraction means selects the training data when the meta-learning execution determination means determines that learning of the value of the meta parameter is to be performed. The learning device according to claim 2 .
4. a learning continuation judgment integration means for judging whether or not learning of the meta parameter value needs to be continued for the plurality of learning models as a whole based on judgment results of the plurality of learning continuation judgment means corresponding to the plurality of learning models; The learning device according to claim 1 , further comprising:
5. one of the learning models is configured as a control means for controlling a control object to execute a task in which an operation of the control object is modularized, and a parameter value of a skill is included in an input value for the learning model; the meta parameter learning means learns the value of the meta parameter by using training data of a plurality of skills; A learning device according to any one of claims 1 to 4.
6. A control device comprising the control means according to claim 5.
7. A control means for controlling the robot in accordance with the size of the object to be grasped so that the robot can grasp each of the objects to be grasped having different shapes. A control device comprising:
8. The computer learning a value of a meta-parameter indicating a probability distribution in a learning model in which the value of the parameter follows a probability distribution, based on training data indicating an input and an output in the learning model; Calculating an evaluation value indicating an evaluation of a generalization error of the learning model; determining whether or not to continue learning the value of the meta parameter based on the evaluation value; A learning method that includes:
9. On the computer, learning a value of a meta-parameter indicating a probability distribution in a learning model in which the value of the parameter follows a probability distribution, based on training data indicating an input and an output in the learning model; calculating an evaluation value indicating an evaluation of a generalization error of the learning model; determining whether or not to continue learning the value of the meta parameter based on the evaluation value; A program for executing the above.
