Operation control device, robot, operation control system, and operation control method

Abstract

To provide an operation control device and an operation control method that can improve a success rate of work while guaranteeing the safety of a control command from a learning controller.SOLUTION: There is provided an operation control device for controlling a controlled object, comprising: a simulator that uses a control command for the controlled object to simulate movement of the controlled object, and creates state transition information indicating a change in a state of the controlled object; an input device for inputting an operational constraint imposed on operation of the controlled object; a learning operation generator that uses the state transition information to perform machine learning about a relationship between the control command and the state transition information, and creates learning operation information obtained by machine learning; a simulated operation generator that inputs, to the simulator, a changed control command created by changing the control command, and creates simulated operation information using the state transition information corresponding to the control command created by the simulator; and a control command identification device that extracts operation information satisfying the operation constraint from the learning operation information created by the learning operation generator and the simulated operation information generated by the simulated operation generator, and identifies a control command corresponding to the extracted operation information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a motion control device, a robot, a motion control system, and a motion control method that can improve the task success rate and safety of learning control. [Background technology]

[0002] In recent years, labor shortages due to the declining birthrate and aging population, and work in dangerous areas due to natural disasters have become social issues. To solve these issues, it is hoped that robots capable of working in unmaintained environments such as outdoors and human collaborative spaces will be realized. Development of learning control using machine learning is underway as a robust control against variations in environmental conditions (e.g., shape, position, physical properties, etc.) in unmaintained environments.

[0003] However, learning control has a low success rate and performance when the task is performed under conditions that are not learned during training (outside the learning range). Furthermore, safety under such conditions (e.g., collision avoidance, abnormal posture avoidance, etc.) is not guaranteed, which is an obstacle to practical use.

[0004] Patent Document 1 proposes a method of learning by having a simulator simulate control commands from machine learning, and if the results of the simulation are judged to be good, outputting a control command to the control object, and if the results are judged to be bad, stopping the input of the control command to the control object, and providing the results as training data for machine learning.

[0005] Also, Non-Patent Document 1 proposes a method of evaluating the future safety of control commands from a learning controller using a simulator, and if the control commands are not safe, correcting the operation to satisfy safety constraints. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent No. 6457421 [Non-patent literature]

[0007] [Non-Patent Document 1] KPWabersich and MNZeilinger, “Safe exploration of nonlinear dynamical systems: A predictive safety filter for reinforcement learning,” arXiv:1812.05506v2, 2019. Summary of the Invention [Problem to be solved by the invention]

[0008] Although the technique disclosed in Patent Document 1 can ensure safety when a control command to a controlled object is judged to be defective, it fails because the control is interrupted and the work cannot be continued.

[0009] Non-Patent Document 1 describes a method that can continue safe operations when the operation of the controlled object based on a control command is determined to be unsafe, but it does not necessarily generate an operation to complete the task successfully, and there is a high possibility that the task will fail.

[0010] In view of the above, an object of the present invention is to provide a motion control device, a robot, a motion control system, and a motion control method that can improve the success rate of tasks while ensuring the safety of control commands from a learning controller. [Means for solving the problem]

[0011] In order to solve the above problems, the present invention provides a "motion control device for controlling a control object, comprising: a simulator that uses a control command to the control object to simulate the movement of the control object and create state transition information indicating a change in the state of the control object; an input device that inputs motion constraints imposed on the motion of the control object; a learning motion generator that uses the state transition information to perform machine learning to learn the relationship between the control command and the state transition information and create learning motion information obtained by machine learning; a simulated motion generator that inputs a changed control command that changes the control command to the simulator and creates simulated motion information using the state transition information corresponding to the control command created by the simulator; and a control command identifier that extracts motion information that satisfies the motion constraints from the learning motion information created by the learning motion generator and the simulated motion information generated by the simulated motion generator, and identifies a control command corresponding to the extracted motion information."

[0012] The present invention also provides an "action control method for controlling a control object using a computer, the computer comprising a simulator that uses a control command for the control object to simulate the movement of the control object and create state transition information showing a change in the state of the control object, and an input device for inputting action constraints imposed on the action of the control object, the computer uses the state transition information to machine learn the relationship between the control command and the state transition information to create learned action information, a changed control command that changes the control command is input to the simulator, simulated action information is created using the state transition information created by the simulator that corresponds to the changed control command, action information that satisfies the action constraints is extracted from the learned action information and the simulated action information, and a control command that corresponds to the extracted action information is identified."

[0013] The present invention is also described as "a motion control system including a robot and a motion control device that controls the robot, the motion control device comprising: a simulator that simulates the movement of the robot using a control command to the robot and creates state transition information showing changes in the state of the robot; an input device that inputs motion constraints imposed on the motion of the robot; a learning motion generator that uses the state transition information to perform machine learning on the relationship between the control command and the state transition information and creates learned motion information obtained by machine learning; a simulated motion generator that inputs a modified control command that modifies the control command to the simulator and creates simulated motion information using the state transition information created by the simulator, corresponding to the modified control command; and a control command specifier that extracts motion information that satisfies the motion constraints from the learning motion information created by the learning motion generator and the simulated motion information generated by the simulated motion generator and specifies a control command corresponding to the extracted motion information, characterized in that the motion control system controls the robot using the control commands determined by the control command specifier." Effect of the Invention

[0014] According to the present invention, it is possible to obtain an effect that the success rate of a task can be improved while ensuring the safety of a control command from a learning controller. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of an operation control device according to a first embodiment of the present invention. [Diagram 2] FIG. 4 is a diagram showing a processing flow of the operation control device according to the first embodiment of the present invention. [Diagram 3] 4 shows examples of learning motion information and simulated motion information in the first embodiment. [Figure 4] FIG. 11 is a diagram showing a processing flow of an operation control device according to a second embodiment of the present invention. [Diagram 5] 13A and 13B are diagrams showing examples of learning motion information and simulated motion information in the second embodiment. [Figure 6] 4 is a diagram showing a method in which a control command specifier specifies a control command; [Figure 7]FIG. 11 is a diagram showing a processing flow of an operation control device according to a third embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, examples of embodiments of the present invention will be described with reference to the accompanying drawings.

[0017] The present invention is not limited to the embodiments, and various numerical values ​​in the embodiments are merely examples. In this specification and drawings, the same components or components having substantially the same functions are denoted by the same reference numerals, and redundant explanations are omitted. EXAMPLES

[0018] 1 is a diagram for explaining a configuration example of a control device according to a first embodiment of the present invention. The present invention is a motion control device 2 that controls a control target 1, and the motion control device 2 is composed of a simulator 3, a learning motion generator 4, a simulated motion generator 5, an input device 6, and a control command specifier 7.

[0019] The simulator 3 simulates the control target 1 and its surrounding environment in a virtual space, and estimates the transition of the environmental state (the state of the control target 1 and its surrounding environment) in response to a control command to the control target 1.

[0020] Here, the control object 1, the control command, and the surrounding environment are not limited to a specific object. For example, if the control object 1 is a robot, the control command may be the angle, angular velocity, torque amount, and position and speed of each joint constituting the robot, and the surrounding environment may be obstacles or an object to be operated that is grasped using the end effector. Alternatively, if the control object 1 is an automobile, the control command may be the steering angle amount, acceleration / deceleration amount, and the surrounding environment may be roads, signs, and pedestrians. Alternatively, if the control object 1 is an aircraft, the control command may be the control stick or rudder, and the surrounding environment may be the atmosphere or other aircraft.

[0021] The learning action generator 4 is constructed by training in advance appropriate control commands for the environmental state in order to achieve the control objective of the control target 1. During operation, the learning action generator 4 determines a control command based on the environmental state on the simulator 3, and generates the control command and information on the environmental state as learning action information L.

[0022] For example, if the control target 1 is a robot, any control objective can be set, such as time-efficient movement or energy-efficient pick-and-place of the target. Any learning method can be used to generate the learning motion, and for example, Gaussian process regression, deep learning, or reinforcement learning can be adopted.

[0023] In order to obtain candidates for control commands to be output to the control target 1, the simulated operation generator 5 randomly determines a control command for the control target 1 based on a certain probability distribution, and outputs the control command to the control target 1 on the simulator, thereby causing a transition in the environmental state on the simulator. Then, information on the control command and the environmental state is generated as simulated operation information I.

[0024] The probability distribution of the control command is generally given as a normal distribution with arbitrary values ​​for the mean and variance, but any probability distribution can be used. When the probability distribution is a normal distribution, the average can be, for example, the magnitude of the control command output to the controlled object in the previous control cycle. Setting a large value for the variance increases the probability of selecting a control command value that deviates from the average, and it is expected that a greater variety of operations will be generated.

[0025] The input device 6 is an interface that accepts input from a user regarding motion constraints, which are constraints imposed on the control commands and environmental conditions for the controlled object 1 to achieve the control purpose. The number and types of motion constraints can be determined arbitrarily. As motion constraints, for example, constraints that the controlled object, such as a robot or a car, does not collide with an obstacle, and constraints that the control command does not exceed a threshold value can be set.

[0026] The control command identification unit 7 uses the learning motion information L generated by the learning motion generator 4 and the simulated motion information I generated by the simulated motion generator 5 to identify a control command that satisfies the motion constraints given by the input device 6 and can achieve the control goal.

[0027] The following describes a flow of control of the controlled object 1 by the motion control device 2 shown in Fig. 1. Fig. 2 is a diagram for explaining a control method by the motion control device 2 according to the first embodiment of the present invention.

[0028] 2, in step S1, the motion constraints input from the input unit 6 are reflected in the control command specifier 7. In step S2, the state of the control target 1 and the surrounding environment are reflected in the simulator 3. In step S3, the learning motion generator 4 generates learning motion information L based on the environmental information in the simulator 3. At the same time, the simulated motion generator 5 generates simulated motion information I based on the environmental information in the simulator 3.

[0029] Here, in order to specify a control command in step S5 described later, the simulated motion generator 5 performs a plurality of simulations and generates a plurality of pieces of simulated motion information I. Fig. 3 shows an example of a plurality of pieces of simulated motion information I and learning motion information L obtained from a plurality of simulations.

[0030] In Figure 3, the horizontal axis represents time and the vertical axis represents environmental state, with the upper graph showing that multiple pieces of simulated movement information I (I1, I2, I3) were generated in environmental state 1 and learning movement information L1, and the lower graph showing that multiple pieces of simulated movement information I (I1, I2, I3) were generated in environmental state 2 and learning movement information L1.

[0031] 3 shows a part of the information included in the simulated operation information I, but may include all of the environmental conditions necessary for identifying a control command by the control command identifier 7. Here, the white circle for each piece of operation information indicates the value of the environmental condition in each control cycle, and the black circle indicates the value of the environmental condition when the control goal is achieved.

[0032] A method for identifying control commands from multiple pieces of simulated operation information I (I1, I2, I3) will be described later. However, as the number of pieces of simulated operation information I is increased, there is a trade-off in that the quality of the control commands identified in step S5 described later improves while the required computational resources increase.

[0033] In step S3, the learning motion generator 4 and the simulation motion generator 5 can process in parallel, but it is also possible to process one of them first due to a lack of computational resources or other reasons.

[0034] Steps S4 to S7 are processes in the control command specifier 7 to which the learned motion information L and simulated motion information I are input.

[0035] In step S4, it is evaluated whether the learning motion information L satisfies the judgment criterion D1. If the judgment criterion D1 is not satisfied, the process proceeds to step S5, where a control command based on the simulated motion information I is set as a control command to be output to the control object 1. If the judgment criterion D1 is satisfied, the process proceeds to step S6, where a control command based on the learning motion information L is set as a control command to be output to the control object 1.

[0036] The judgment criterion D1 is for judging whether the control target 1 can achieve the control goal while satisfying the motion constraints by the control command based on the learning motion information L. For example, it can be judged that the evaluation criterion is satisfied when the learning motion information L satisfies the motion constraints and the control target 1 controlled based on the learning motion information L can achieve the control goal within a certain time.

[0037] Alternatively, it can be determined that the evaluation criterion is satisfied if the learned motion information L satisfies the motion constraints and a judger that has learned using pairs of the learned motion information L and the success or failure of the control target as training data determines that the control target can be achieved. Here, some of the judgment methods are described, but any method that can judge whether the control target of the control object 1 can be achieved may be adopted as the judgment criterion.

[0038] In step S5, there are several methods for determining a control command based on the simulated motion information I. For example, from the multiple pieces of simulated motion information I (I1, I2, I3) generated in step S3, the simulated motion information I that satisfies the motion constraints and has the shortest time to achieve the control goal is selected (I2 in the example of FIG. 3), and the control command of the simulated motion information I is set as the control command to be output to the controlled object 1.

[0039] If it is determined that only the learning motion information L1 does not satisfy the motion constraints among the four motion information (L1, I1, I2, I3) shown in Figure 3, the control command for the simulated motion information I2, which achieves the control target in the shortest time, is set as the control command to be output to the control target 1.

[0040] In addition, when there is an evaluation index other than the time to achieve the control target, it is possible to adopt a method of selecting the simulated operation information I that has the smallest cost that can be calculated for each piece of operation information (L1, I1, I2, I3), and setting the control command of that simulated operation information I as the control command to be output to the control target 1.

[0041] Here, the cost C can be defined as in equation (1), where N is the number of evaluation indices, Cn is the cost for evaluation index n, and wn is the weight for the cost Cn.

[0042]

number

[0043] The cost Cn is not limited to the time required to achieve the control goal, and it is possible to set a cost related to, for example, energy consumption. Furthermore, if a motion constraint is set as the cost Cn, it is possible to guide the user not to select simulated motion information I that does not satisfy the motion constraint. For example, Cn can be set so that the cost is high when the controlled object 1 collides with an obstacle in the surrounding environment. Furthermore, if the weight wn for the cost Cn is set large, the influence of the cost Cn on the cost C can be increased.

[0044] It is also possible to take into account more complicated motion constraints. For example, by setting a cost Cn related to the stress between the control target 1 and an obstacle, it is possible to take into account motion constraints that allow contact between the control target 1 and the obstacle but restrict the application of a load above a threshold.

[0045] The cost and its weight wn can be set by the user together with the motion constraints via the input device 6. Alternatively, a method of automatically generating them based on the control objective in the learning motion generator 4 is also conceivable.

[0046] After the processes in steps S5 and S6 are completed, the process proceeds to step S7, where the set control command is output to the controlled object 1.

[0047] As a result, when the learning motion generator 4 can generate control commands that satisfy the motion constraints and achieve the control goal, the motion of the learning motion generator 4 is used; when this is not the case, control commands that satisfy the motion constraints based on the simulated motion generator 5 are generated. EXAMPLES

[0048] In the first embodiment, a control method is described in which the operation control device shown in Fig. 1 is applied to a case where the state of the surrounding environment does not change. In contrast, in the second embodiment, a control method is described in which the operation control device shown in Fig. 1 is applied to a case where the state of the surrounding environment changes dynamically.

[0049] Fig. 4 is a diagram for explaining a control method for the second embodiment by the control device according to the present invention. The main difference from the control method in Fig. 2 in the first embodiment is that dynamic changes in the environment are reflected in the simulator 3 at a constant cycle. For this reason, steps S1 to S7 are basically the same as in Fig. 2, and further improvements and additions to be made in the specific processing (mainly improvements associated with repetitive processing) will be shown below each time.

[0050] 4, specifically, when it is determined in step S8 that the control target has not been achieved, the process returns to step S2, and the environmental state at that time is reflected in the simulator 3. This makes it possible to generate a control command based on the updated environmental state even when the environmental state changes dynamically.

[0051] At this time, sensors installed in the control target 1 and the surrounding environment are used to reflect the environmental state in the simulator 3. For example, if the control goal is for the control target 1 to move to a target position while avoiding obstacles, the environmental state of the simulator 3 can be updated by capturing dynamic obstacles (people, other moving objects, objects whose shape changes upon contact, etc.) in the surrounding environment with a camera and estimating the position and speed of the obstacle.

[0052] However, since the environmental state is updated at a constant cycle in the control method shown in Fig. 4, it is difficult to execute the simulation for generating the simulated motion information and the learning motion information in step S3 until the control target is achieved. Therefore, in step S3 of the control method shown in Fig. 4, a method (so-called receding horizon control) in which the simulation is executed only up to a finite time in the future (prediction horizon) in each cycle can be adopted.

[0053] Fig. 5 shows the learned motion information L and simulated motion information L generated in step S3 in the control method of Fig. 4. The vertical and horizontal axes of Fig. 5 are the same as those of Fig. 3, and the information displayed is also the same as that of Fig. 3. In Fig. 5, the time displayed is clearly indicated on the horizontal axis. In the diagram on the left, the prediction period is from t0 to t0+H, whereas in the diagram on the right, the prediction period is from t0+T to t0+T+H.

[0054] Here, in the process of step S3 at the initial time t0, learning motion information L and simulated motion information I based on a simulation up to the prediction horizon H are generated, and in the process of step S3 after a period T, motion information from t0+T to the prediction horizon H is generated. In step S5 of the first embodiment, too, it is possible to generate a control command by utilizing the receding horizon control.

[0055] In steps S4 to S6 processed by the control command specifier 7, a control command is specified based on operation information up to the prediction horizon H shown in Fig. 5. Steps S4 and S6 can be processed in the same manner as in the first embodiment.

[0056] Regarding step S5, as a method for determining a control command based on the simulated motion information I, it is possible to adopt a method similar to that of the first embodiment in which the simulated motion information I that has the smallest calculable cost for each piece of motion information is selected, and the control command for that simulated motion information I is set as the control command to be output to the control object 1. Alternatively, a cost is calculated for each of the multiple pieces of simulated motion information I, and a weighted average for the control commands for each piece of simulated motion information is calculated using the cost, and the value is set as the control command to be output to the control object 1.

[0057] Fig. 6 is a diagram for explaining a case where a cost is calculated for each piece of simulated motion information I, and a weighted average based on the cost is used as a control command. As a method for calculating the weight based on the cost, for example, the inverse of the cost can be used as the weight, but the calculation method is not limited to this. In Fig. 6, the cost for the simulated motion information I13 is the smallest, so the weighted average W1 is calculated to be a value close to the environmental state 1 of the simulated motion information I13.

[0058] Here, the cost C can be defined as in equation (2), where Cn,t is the cost for evaluation index n at time t.

[0059]

number

[0060] As the cost Cn,t, for example, the cost related to the error from the target position of the control target 1 or the cost related to the energy consumption can be set. Also, by setting the cost related to the operation constraints as in the first embodiment, it is possible to generate a control command that satisfies the operation constraints.

[0061] As a result, it is possible to generate effective control commands with limited computational resources while reflecting dynamic changes in the environmental state in the control. EXAMPLES

[0062] In the third embodiment, the control method of the second embodiment shown in FIG. 4 is expanded as shown in FIG. 7 in order to improve the achievement rate of the control target. In the third embodiment, steps S1 to S5, S7, and S8 shown in FIG. 7 are almost the same as the control method of the second embodiment shown in FIG. 4, but the process in the control command specifier 7 of step S6 is different. In FIG. 7 of the third embodiment, the contents of step S6 are further described in detail as S61, S62, and S63. Improvements and ideas in the detailed process of steps S1 to S5, S7, and S8 will be individually explained below.

[0063] In step S4, if the learned motion information does not satisfy the judgment criterion D2, the process proceeds to step S5, and a control command based on the simulated motion information is set as a control command to be output to the control target 1. On the other hand, if the learned motion information satisfies the judgment criterion D2, the process proceeds to step S61.

[0064] The judgment criterion D2 here is a control command based on the learned motion information L, and is used to judge whether the control goal can be achieved, regardless of whether the control object 1 satisfies the motion constraints. For example, it can be said that the evaluation criterion is met when the control object 1 controlled based on the learned motion information L can achieve the control goal within a certain time. Alternatively, it can be said that the evaluation criterion is met when a judger that has learned using pairs of the learned motion information L and the success or failure of the control goal as training data judges that the control goal can be achieved. Here, some of the judgment methods are taken up, but any method that can judge whether the control object 1 can achieve the control goal may be adopted as the judgment criterion.

[0065] The process of step S5 is the same as the process of step S35 in FIG. 4, and the control command to be output to the controlled object 1 is specified using only the simulated operation information.

[0066] In step S61, it is determined whether the learning motion information L satisfies the motion constraints. If it is determined that the motion constraints are not satisfied, the process proceeds to step S62, where a control command is identified based on both the learning motion information L and the simulated motion information I, and set as the control command to be output to the control object 1. If it is determined that the motion constraints are satisfied, the process proceeds to step S63, where a control command based on the learning motion information L is set as the control command to be output to the control object 1.

[0067] In step S62, the procedure for identifying the control command is the same as in step S5. For example, the simulated motion information I with the smallest cost is selected from the multiple pieces of simulated motion information I generated in step S3, and the control command for that simulated motion information I is set as the control command to be output to the control object 1.

[0068] Alternatively, a cost is calculated for each of the multiple pieces of simulated motion information I, a weighted average of the control commands for each piece of simulated motion information is calculated as a weight based on the cost, and the value is set as a control instruction to be output to the control target 1. A difference from step S5 is that step S62 sets the cost Cn,t based on the learning motion information to the cost C in order to improve the achievement rate of the control goal.

[0069] Specifically, the error between the control commands of the control target 1 in the simulated motion information I and the learning motion information L is set as the cost Cn,t in the cost of equation (2). This makes it possible to specify a control command sequence that corrects only the control command at time t that violates the motion constraint in the learning motion information, while referring to the control command sequence in the learning motion information.

[0070] After the processes of steps S5, S62, and S63 are completed, the process proceeds to step S7, where the set control command is output to the controlled object 1. EXAMPLES

[0071] In the fourth embodiment, a description will be given of a control target 1 in FIG. 1 being a robot, and the robot 1 incorporating a motion control device 2 and constituting a motion control system as a whole, thereby acting autonomously.

[0072] In this case, the motion control system includes a robot 1 and a motion control device 2 that controls the robot 1. The motion control device 2 is equipped with a simulator 3 that simulates the movement of the robot 1 using control commands to the robot 1 and creates state transition information indicating changes in the state of the robot 1, an input device 6 that inputs motion constraints imposed on the motion of the robot 1, a learning motion generator 4 that uses the state transition information to perform machine learning to determine the relationship between the control commands and the state transition information and creates learned motion information L obtained by machine learning, a simulated motion generator 5 that inputs a changed control command that has changed the control command to the simulator 3 and creates simulated motion information I using the state transition information created by the simulator 3 corresponding to the changed control command, and a control command identifier 7 that extracts motion information that satisfies the motion constraints from the learning motion information created by the learning motion generator 4 and the simulated motion information I generated by the simulated motion generator 5 and identifies a control command corresponding to the extracted motion information, and controls the robot 1 using the control commands determined by the control command identifier 7. [Explanation of symbols]

[0073] 1: Control target (robot) 2: Motion control device 3: Simulator 4: Learning behavior generator 5: Simulated motion generator 6: Input device 7: Control command specifier

Claims

1. A motion control device for controlling a control target, a simulator that uses a control command for a control object to simulate a movement of the control object and creates state transition information indicating a change in the state of the control object; an input device that inputs motion constraints imposed on the movement of the control object; a learning motion generator that uses the state transition information to learn by machine learning a relationship between the control command and the state transition information and creates learning motion information obtained by machine learning; a simulated motion generator that inputs a changed control command that changes the control command to the simulator and creates simulated motion information using state transition information corresponding to the changed control command created by the simulator; and a control command identifier that extracts motion information that satisfies the motion constraints from the learning motion information created by the learning motion generator and the simulated motion information generated by the simulated motion generator, and identifies a control command corresponding to the extracted motion information.

2. The motion control device according to claim 1 , The control command identifier identifies a control command from the learning motion information if the learning motion information satisfies the motion constraints and can achieve a control goal imposed on the control object, and identifies a control command from the simulated motion information if this is not the case.

3. The motion control device according to claim 1 , The motion control device, characterized in that the simulated motion generator creates a plurality of pieces of simulated motion information, and the control command specifier specifies a control command that satisfies the motion constraint from the plurality of pieces of simulated motion information.

4. The motion control device according to claim 1 , 2. A motion control device comprising: a simulator for updating a state of the controlled object at a constant cycle based on sensor information;

5. The motion control device according to claim 1 , The learning motion generator uses the state transition information to create the learning motion information up to a certain time into the future, and inputs the change control command to the simulator, and the simulated motion generator creates the simulated motion information up to a certain time into the future.

6. The motion control device according to claim 1 , The control command identifier identifies the control command from the simulated motion information when a control goal imposed on the control object cannot be achieved regardless of whether the learning motion information satisfies the motion constraint, identifies the control command from the learning motion information when the motion constraint is satisfied and the control goal can be achieved, and identifies the control command from the simulated motion information and the learning motion information when the motion constraint is not satisfied but the control goal can be achieved.

7. A motion control method for controlling a control target using a computer, comprising: the computer includes a simulator that uses a control command to simulate a movement of the control object and creates state transition information indicating a change in a state of the control object, and an input device that inputs an operation constraint imposed on the operation of the control object; a control command that changes the control command and the state transition information to generate learned motion information; a change control command that changes the control command is input to the simulator; simulated motion information is generated using the state transition information generated by the simulator and corresponding to the change control command; motion information that satisfies the motion constraints is extracted from the learned motion information and the simulated motion information; and a control command that corresponds to the extracted motion information is identified.

8. A robot controlled by the motion control device according to claim 1.

9. A motion control system including a robot and a motion control device that controls the robot, a learning motion generator that uses the state transition information to learn the relationship between the control command and the state transition information by machine learning and creates learning motion information obtained by machine learning; a simulated motion generator that inputs a modified control command that modifies the control command to the simulator and creates simulated motion information using the state transition information created by the simulator, corresponding to the modified control command; and a control command specifier that extracts motion information that satisfies the motion constraints from the learning motion information created by the learning motion generator and the simulated motion information generated by the simulated motion generator and specifies a control command corresponding to the extracted motion information, and controls the robot according to the control command determined by the control command specifier.

Images