Robot, motion generation device, robot control system, and motion generation method

Abstract

In imitation learning, the challenge is to enable the system to accurately replicate the imitated actions. [Solution] The system includes a drive unit 150 that drives the robot 1, a mimicry motion generation unit 110 that generates a target mimicry trajectory 163, which is time-series data of target values ​​related to the operation of the robot 1, by mimicry learning based on state detection information 161 acquired from a state detection unit 130, a state prediction unit 121 that generates a predicted state trajectory 164, which is time-series data related to the predicted operation of the robot, based on the operation characteristics of the drive unit 150, using the current control command 165 to the drive unit 150 and the state detection information 161, a predicted state trajectory evaluation unit 122 that compares the target mimicry trajectory 163 and the predicted state trajectory 164 and evaluates the degree of agreement between them, and a control command optimization unit 123 that corrects the control command 165 based on the degree of agreement and outputs the corrected control command 165 to the drive unit.

Description

Technical Field

[0001] The present invention relates to technologies of robots, motion generation devices, robot control systems, and motion generation methods.

Background Art

[0002] Conventional robot systems require a large amount of programming and high specialized knowledge, which has become an obstacle to the introduction of robots. Therefore, an autonomous learning type robot device has been proposed in which the robot itself determines its actions based on various sensor information attached to the robot device.

[0003] In this autonomous learning type robot device, it is mainly possible to flexibly generate actions in response to various environmental changes by memorizing and imitating teaching actions taught by humans or other robots. Therefore, it is expected that the autonomous learning type robot device can replace robots for complex tasks that humans have conventionally performed.

[0004] Generally, an autonomous learning type robot device is equipped with an imitation learning device. In the imitation learning device, sensor information at the time of action experience is memorized, and parameter adjustment for generating an action is performed. Also, in the imitation learning device, the relationship between input and output is defined in advance. Then, the imitation learning device repeats learning so that an expected output value is output as an imitation action for an input value to the imitation learning device.

[0005] For example, the imitation learning device memorizes the joint angle information of a robot at a certain action experience as time series information. It is assumed that the joint angle information at time (t) is input to the imitation learning device, and the imitation learning device performs time series learning to predict the joint angle information at the next time (t + 1).

[0006] Then, by sequentially inputting the joint angle information of the robot to the imitation learning device that has completed learning, the autonomous learning type robot device can autonomously generate imitation actions according to the environment and changes in its own state.

[0007] Thus, autonomous learning robotic devices can mimic the actions of humans and other robots with less learning time by adjusting the parameters of the mimic learner to imitate the taught actions. However, it is difficult to consider the control characteristics and constraints for controlling the autonomous learning robotic device when the teaching data is generated by humans or other robots. Therefore, even if it is possible to generate actions that mimic the taught actions, the autonomous learning robotic device may not be able to adequately follow the generated actions. This may cause the autonomous learning robotic device to fail when performing its intended task, or the control may become unstable midway through.

[0008] As a means for adjusting the parameters of an imitation learner while taking into account the controllable characteristics and constraints of a robot, for example, a technique such as that disclosed in Patent Document 1 is disclosed. Patent Document 1 discloses a reinforcement learning method, a reinforcement learning program, and a reinforcement learning device in which, according to one embodiment, the state of the object being controlled in reinforcement learning is predicted at each point in time when the state of the object is measured, in a period that is after the time when the current action decision is made and before the time when the next action decision is made, with respect to the constraints on the state of the object, based on the predicted state of the object, the degree of risk for the state of the object at each point in time with respect to the constraints on the state of the object, the search range for the current action on the object is identified according to the calculated degree of risk for the state of the object at each point in time and the degree of influence of the current action on the state of the object at each point in time, and the current action on the object is determined based on the identified search range for the current action on the object (see paragraph 0007). [Prior art documents] [Patent Documents]

[0009] [Patent Document 1] Japanese Patent Publication No. 2021-033767 [Overview of the project] [Problems that the invention aims to solve]

[0010] As described in Patent Document 1, reinforcement learning can be used to consider the controllable characteristics and constraints of a robot. However, reinforcement learning has a high learning cost to acquire the desired motion. In other words, since reinforcement learning is a method of acquiring motion through trial and error, it requires a sophisticated simulation environment to evaluate the motion obtained during the parameter adjustment process, or the construction of a real environment that is sufficiently considered to be an environment desirable for the motion. In addition, unlike imitation learning, reinforcement learning requires starting to learn motion from scratch without any examples of motion generated by humans or other robots. For this reason, reinforcement learning generally tends to take more time and effort to learn compared to imitation learning.

[0011] Furthermore, in reinforcement learning, if control characteristics and constraints are considered during the initial training, it becomes difficult to account for control errors and disturbances that occur when the robot actually operates. The simulation and actual robot environments used for learning movements often differ from the environments in which the robot actually performs tasks. Therefore, it is desirable to be able to correct the movements generated by the imitation learner by considering the control characteristics and constraints of the robot, depending on the actual conditions under which the robot operates.

[0012] In light of this background, the present invention was made, and its objective is to enable accurate tracking of imitated actions in imitation learning. [Means for solving the problem]

[0013] To solve the aforementioned problems, the present invention comprises: a drive unit for driving a robot; a state detection unit for acquiring information about the state of the robot; an imitation motion generation unit that generates a target imitation trajectory, which is time-series data of target values ​​relating to the robot's motion, by imitation learning based on the state detection information acquired from the state detection unit; a state prediction unit that generates a predicted state trajectory, which is time-series data relating to the robot's predicted motion, based on the operating characteristics of the drive unit, using the current control command to the drive unit and the state detection information; a predicted state trajectory evaluation unit that compares the target imitation trajectory generated by the imitation motion generation unit with the predicted state trajectory generated by the state prediction unit and evaluates the degree of agreement between the target imitation trajectory and the predicted state trajectory; and a control command optimization unit that corrects the control command based on the degree of agreement evaluated by the predicted state trajectory evaluation unit and outputs the corrected control command to the drive unit. Other solutions will be described as appropriate in the embodiments. [Effects of the Invention]

[0014] According to the present invention, in imitation learning, it is possible to accurately track imitated actions. [Brief explanation of the drawing]

[0015] [Figure 1] This diagram shows the configuration of the robot according to the first embodiment. [Figure 2] This figure shows an example of the hardware configuration of a motion generation device or robot. [Figure 3] This figure shows an example of a task that a robot can perform. [Figure 4] This flowchart shows the procedure for the motion generation method performed by the imitation motion generation unit in the first embodiment. [Figure 5] This figure shows an example of a target imitation trajectory that is generated. [Figure 6] This flowchart shows the procedure for generating an action performed by the control unit in the first embodiment. [Figure 7A] This is a diagram (part 1) illustrating an example of an evaluation method. [Figure 7B] It is a diagram (part 2) showing an example of an evaluation method. [Figure 8A] It is a diagram (part 1) showing the effects of the first embodiment. [Figure 8B] It is a diagram (part 2) showing the effects of the first embodiment. [Figure 8C] It is a diagram (part 3) showing the effects of the first embodiment. [Figure 8D] It is a diagram (part 4) showing the effects of the first embodiment. [Figure 9A] It is a diagram (part 1) showing an example of an evaluation method in the second embodiment. [Figure 9B] It is a diagram (part 2) showing an example of an evaluation method in the second embodiment. [Figure 10A] It is a diagram (part 1) showing the effects of the second embodiment. [Figure 10B] It is a diagram (part 2) showing the effects of the second embodiment. [Figure 10C] It is a diagram (part 3) showing the effects of the second embodiment. [Figure 10D] It is a diagram (part 4) showing the effects of the second embodiment. [Figure 11] It is a diagram showing an example of the configuration of a robot according to the third embodiment. [Figure 12A] It is a diagram (part 1) showing an example of an evaluation method for the task execution environment and obstacle avoidance of a robot according to the third embodiment. [Figure 12B] It is a diagram (part 2) showing an example of an evaluation method for the task execution environment and obstacle avoidance of a robot according to the third embodiment. [Figure 12C] It is a diagram (part 3) showing an example of an evaluation method for the task execution environment and obstacle avoidance of a robot according to the third embodiment. [Figure 13A] It is a diagram (part 1) showing the effects of the method shown in the third embodiment. [Figure 13B] It is a diagram (part 2) showing the effects of the method shown in the third embodiment. [Figure 13C] It is a diagram (part 3) showing the effects of the method shown in the third embodiment. [Figure 13D]This is Figure (4) illustrating the effects of the method shown in the third embodiment. [Figure 14] This figure shows an example of a candidate target imitation trajectory and an evaluation method for the predicted state trajectory evaluation unit in the fourth embodiment. [Figure 15] This figure shows an example of a robot control system according to this embodiment. [Modes for carrying out the invention]

[0016] Embodiments of the present invention will be described below with reference to the drawings and other figures. The following description provides specific examples of the content of the present invention, and the present invention is not limited to these descriptions. Various changes and modifications are possible by those skilled in the art within the scope of the technical ideas disclosed herein. In addition, in all the figures used to illustrate the present invention, components having the same function are denoted by the same reference numerals, and repeated descriptions may be omitted.

[0017] Furthermore, in this embodiment, "trajectory" refers to a time series of data.

[0018] <First Embodiment> First, a first embodiment of the present invention will be described with reference to Figures 1 to 8D.

[0019] (Configuration of Robot 1) Figure 1 shows the configuration of robot 1 according to the first embodiment.

[0020] Robot 1 includes a motion generation device 100 that generates information for the robot 1 to operate, a state detection unit 130, an environment detection unit 140, and a drive unit 150.

[0021] The state detection unit 130, which acquires information about the state of robot 1, includes an angle sensor 131, a torque sensor 132, and the like. Thus, the state detection unit 130 is composed of sensors that measure the internal state of robot 1.

[0022] Furthermore, the environment detection unit 140, which acquires environmental information around the robot 1, includes a camera 141 and the like for measuring external information around the robot 1. The state detection information 161 acquired by the angle sensor 131 and torque sensor 132 is used to generate the target imitation trajectory 163, which is information for the robot 1's learned imitation movements, and to calculate the predicted state trajectory 164. Note that the state detection unit 130 is not limited to the angle sensor 131 and torque sensor 132. For example, a pressure sensor for measuring pressure information applied to a part of the robot 1, or an IMU for measuring the tilt and posture of the robot 1, may be used as an internal sensor. Note that IMU is an abbreviation for Inertial Measurement Unit.

[0023] Images captured by camera 141 are used to measure environmental information such as the target object of the task that robot 1 is trying to accomplish, and the surrounding environment. Note that the sensors used to measure environmental information around robot 1 are not limited to camera 141. For example, a LiDAR for measuring surrounding point cloud information, or a microphone for measuring sound information generated in the surroundings, may be used as sensors to measure external information around robot 1. LiDAR stands for Light Detection And Ranging.

[0024] The drive unit 150 that drives the robot 1 is composed of actuators 151, etc. The actuators 151 are multiple rotary motors, translational motors, etc., for operating the robot 1. The actuators 151 are controlled by control commands 165 output by the motion generation device 100 and driven by a motor driver (not shown).

[0025] In this embodiment, the robot 1 is assumed to be an arm-type robot composed of multiple joints, or a mobile robot composed of multiple wheels, etc.

[0026] The motion generation device 100 comprises a mimic motion generation unit 110 and a control unit 120. The mimic motion generation unit 110 takes state detection information 161 from angle sensors 131 and torque sensors 132, and environment detection information 162 from camera 141 (information from images captured by camera 141) as input at each time point to generate a target mimic trajectory 163 that the robot 1 should mimic. The state detection information 161 includes angle information, torque information, etc., and the environment detection information 162 includes image information.

[0027] The imitation motion generation unit 110 comprises a memory motion inference unit 111 and an imitation motion linking unit 112. Based on state detection information 161 acquired from the state detection unit 130, the imitation motion generation unit 110 generates a target imitation trajectory 163, which is time-series data of target values ​​300 (see Figure 5) related to a series of movements of the robot 1, through imitation learning. Imitation learning is a machine learning method that generates target values ​​300 for the movements of the robot 1 based on teaching data 166. Teaching data 166 is data collected by a person operating a robot 1 different from the robot 1 controlled by the motion generation device 100. The imitation motion generation unit 110 can also generate the target imitation trajectory 163 using state detection information 161 and environment detection information 162 acquired from the environment detection unit 140. In this embodiment, the target imitation trajectory 163 is generated using state detection information 161 and environment detection information 162, but the target imitation trajectory 163 may also be generated using only state detection information 161.

[0028] The teaching data 166 is generated by the following procedure, for example. For example, a person operates the robot 1 via a manual controller (not shown), and this operation information is used as the teaching data 166. Alternatively, the robot 1 is operated to trace the movements of a person, and the traced trajectory is used as the teaching data 166. When operating the robot 1 to trace the movements of a person, a device for recording the person's movements may be used, or the person may grasp the hand of the robot 1 and move the hand manually to record the person's movements. The teaching data 166 consists of information on the angles of the joints of the robot 1 (angle information), information on the torque of the actuator 151 (torque information), image information captured by the camera 141, etc.

[0029] Furthermore, the teaching data 166 will be appropriately selected (created) according to the nature of the task to be performed by the robot 1.

[0030] The memory-action reasoning unit 111 calculates the target value for the robot 1's next action, taking the state detection information 161 as input. The memory-action reasoning unit 111 uses the teaching data 166 to pre-adjust the learning parameters for imitation learning in order to calculate the target imitation trajectory 163. The teaching data 166 is not used when the memory-action reasoning unit 111 generates the target imitation trajectory 163. In Figure 1, the dashed arrow from the teaching data 166 to the memory-action reasoning unit 111 indicates that the teaching data 166 is used only to adjust the learning parameters for learning to calculate the target imitation trajectory 163, and is not used when generating the target imitation trajectory 163. The imitation learner described above corresponds to the memory-action reasoning unit 111.

[0031] The mimicry action linking unit 112 generates a target mimicry trajectory 163 by linking multiple target values ​​300 calculated by the memory action inference unit 111.

[0032] The control unit 120 takes the information measured by the angle sensor 131 and torque sensor 132 at each time point, along with the target imitation trajectory 163 output by the imitation motion generation unit 110, as input, calculates a control command 165, and outputs it to the actuator 151.

[0033] The control unit 120 includes a state prediction unit 121, a predicted state trajectory evaluation unit 122, and a control command optimization unit 123.

[0034] The state prediction unit 121 uses the current control command 165 to the drive unit 150 and the state detection information 161 to generate a predicted state trajectory 164, which is time-series data relating to a series of predicted movements of the robot 1 based on the operating characteristics of the drive unit 150.

[0035] The predicted state trajectory evaluation unit 122 compares the target imitation trajectory 163 generated by the imitation motion generation unit 110 with the predicted state trajectory 164 generated by the state prediction unit 121, and evaluates the degree of agreement between the target imitation trajectory 163 and the predicted state trajectory 164.

[0036] The control command optimization unit 123 corrects the control command 165 by performing mathematical optimization processing based on the degree of agreement evaluated by the predicted state trajectory evaluation unit 122. Then, the control command optimization unit 123 outputs the corrected control command 165 to the drive unit 150.

[0037] The predicted state trajectory 164 is composed of information on the predicted angle (predicted angle) and information on the predicted torque (predicted angle information and predicted torque information, respectively). The predicted state trajectory 164 is also composed of a conversion formula or conversion map between the current value or torque input to the actuator 151 and the current value input to the actuator 151. However, the predicted state trajectory 164 is not limited to the above content as long as it is possible to compare it with the target imitation trajectory 163.

[0038] The target imitation trajectory 163 is generated by deep learning based on state detection information 161 and environment detection information 162, and represents the target value 300 for the robot 1's movement. In contrast, the predicted state trajectory 164 shows the predicted movement of the robot 1 when the control command 165 is input to the actuator 151 at the current stage. Generally, the target imitation trajectory 163 cannot be directly converted to the control command 165. The target imitation trajectory 163 is expressed in terms of the angles of the robot 1's joints and torque, while the control command 165 is the current value input to the actuator 151. There is generally no formula for converting angles and current values. Furthermore, discrepancies occur between the target imitation trajectory 163 and the control command 165 due to errors in the angle sensor 131 and torque sensor 132. In addition, there are movements that are not a problem when operated by a human, such as sudden changes in direction, but which the robot 1 cannot follow when operating autonomously. In other words, the actual operating characteristics of the robot 1's drive unit 150 may not be satisfied. Therefore, it is necessary to correct the control command 165 based on the target imitation trajectory 163 and the predicted state trajectory 164.

[0039] In the first embodiment, the control unit 120 generates a predicted state trajectory 164 based on the control command 165, etc. The control unit 120 then corrects the control command 165 so that the predicted state trajectory 164 approaches the target imitation trajectory 163. "Approaching the target imitation trajectory 164" means reducing the degree of agreement between the predicted state trajectory 164 and the target imitation trajectory 163. The degree of agreement will be described later. The control unit 120 then generates a predicted state trajectory 164 based on the corrected control command 165 and the state detection information 161. Subsequently, the control unit 120 further corrects the control command 165 based on the generated predicted state trajectory 164 and the target imitation trajectory 163.

[0040] In other words, the control unit 120 generates a provisional control command 165 and evaluates the degree of agreement between the predicted state trajectory 164 generated from the control command 165 and the target imitation trajectory 163. If the degree of agreement is large (not optimal), the control unit 120 corrects the control command 165 to reduce the degree of agreement and satisfy the operating characteristics of the drive unit 150. By repeating this process, the control unit 120 generates a control command 165 that satisfies the operating characteristics of the drive unit 150.

[0041] Incidentally, the reason the control unit 120 does not directly compare the control command 165 with the target imitation trajectory 163 is that the data contents of the control command 165 and the target imitation trajectory 163 are different, making comparison impossible. Therefore, the control unit 120 generates a predicted state trajectory 164 that has a data format that allows comparison with the target imitation trajectory 163.

[0042] (Hardware configuration) Figure 2 shows an example of the hardware configuration of the motion generation device 100 or robot 1.

[0043] The motion generation device 100 is a device for realizing the functions of the simulated motion generation unit 110 and control unit 120 shown in Figure 1, and for example, a computer device is used. The motion generation device 100 includes a CPU 171, ROM 172, RAM 173, display unit 174, input unit 175, communication I / F 176, and system bus 177. Incidentally, "CPU" is an abbreviation for Central Processing Unit. Also, "ROM" is an abbreviation for Read Only Memory. Furthermore, "RAM" is an abbreviation for Random Access Memory. And "I / F" is an abbreviation for Interface.

[0044] The CPU 171 comprehensively controls the operation of the motion generation device 100 and processes the control program implemented in the robot 1 via the system bus 177. The ROM 172 is a non-volatile memory that stores the control program and other data necessary for the CPU 171 to execute its processing. The control program and learning data may also be stored in external memory or a storage medium that can be attached to or removed from the motion generation device 100. These ROM 172, external memory, and removable storage medium are used as examples of storage media that can be read by a computer that stores the control program executed by the motion generation device 100.

[0045] RAM173 is the main memory of the CPU171 and functions as a work area, etc. That is, when processing a control program, the CPU171 loads the control program etc. read from ROM172 into RAM173. Then, the CPU171 executes the loaded control program etc., thereby realizing the functional operation of the imitation operation generation unit 110 and the control unit 120. In this way, the CPU171, ROM172, and RAM173 work together to realize each process executed by the imitation operation generation unit 110 and the control unit 120.

[0046] The display unit 174 consists of a monitor such as a liquid crystal display (LCD). LCD is an abbreviation for Liquid Crystal Display. The display unit 174 can display angle information of the robot 1's joints measured by the angle sensor 131, torque information applied to the robot 1's joints measured by the torque sensor 132, image information captured by the camera 141, etc. In addition, the display unit 174 allows the user to monitor the status of the robot 1 in real time. The input unit 175 consists of a keyboard or a pointing device such as a mouse. Through the input unit 175, the user can instruct the start and end of the control program provided in the motion generation device 100.

[0047] The communication interface 176 is an interface for the motion generation device 100 to communicate with the angle sensor 131, torque sensor 132, camera 141, and actuator 151. The communication interface 176 can be, for example, a LAN interface. LAN stands for Local Area Network. The system bus 177 connects the CPU 171, ROM 172, RAM 173, display unit 174, input unit 175, and communication interface 176 in a communication-enabled manner.

[0048] The functions of each part of the motion generation device 100 shown in Figure 2 can be realized by the CPU 171 executing a control program, as described above. However, at least some of the functions of each part of the motion generation device 100 shown in Figure 2 may be configured to be operated by dedicated hardware. In this case, the dedicated hardware operates based on the control of the CPU 171.

[0049] Furthermore, a GPU or similar device may be used instead of the CPU 171. GPU stands for Graphic Processing Unit. Also, an HDD or SSD may be used instead of the ROM 172. HDD stands for Hard Disk Drive, and SSD stands for Solid State Drive.

[0050] As will be shown later in Figure 15, a motion generation device 100 may be provided outside of the robot 1. In such cases, the hardware configuration of the robot 1 will be the same as that shown in Figure 2.

[0051] (Task performed by Robot 1) Figure 3 shows an example of a task performed by robot 1.

[0052] In this embodiment, it is assumed that robot 1 is grasping peg T1 and performing a task such as inserting peg T1 into hole T2. Camera 141 is installed so as to show the entire process of robot 1 inserting the grasped peg T1 into hole T2. Camera 141 may be fixed to robot 1 or installed outside robot 1, provided that it is configured to communicate with robot 1. However, if camera 141 is installed outside robot 1, it should be installed so as to show the entire process of performing the task, including parts of robot 1 that are closely involved in the task, such as robot 1's hand. The actions of robot 1 holding peg T1 and moving its hand to insert peg T1 into hole T2 are referred to as a "series of actions".

[0053] The tasks to which this embodiment can be applied are not limited to those shown in Figure 3. The tasks to which this embodiment can be applied include any task that a human can perform, such as the robot 1 grasping and moving an object, the robot 1 handling a tool grasped in its hand, or pressing something against something.

[0054] (Method for generating motion using the imitation motion generation unit 110) Figure 4 is a flowchart showing the procedure for generating motions performed by the imitation motion generation unit 110 in the first embodiment. Refer to Figure 1 as appropriate.

[0055] First, in step S101, the imitation action coupling unit 112 is initialized so that the target imitation trajectory 163, which was ultimately generated during the previous calculation, is empty.

[0056] Next, in step S102, the memory operation reasoning unit 111 acquires current angle information from the angle sensor 131, current torque information from the torque sensor 132, and current image information from the camera 141 (acquiring angle, torque, and image from the sensors). The angle in the angle information and the torque in the torque information are, for example, the angle and torque at the joints of robot 1.

[0057] In step S103, the memory and operation reasoning unit 111 predicts the target angle, target torque, and target image for the next step based on the input angle information, torque information, and image information. Of the target angle, target torque, and target image, the target angle and target torque will have a target value of 300 (see Figure 5). The angle and torque predicted by the memory and operation reasoning unit 111 will be the target value of 300 when the robot 1 actually operates. The angle, torque, and image predicted by the memory and operation reasoning unit 111 will be referred to as the target angle, target torque, and target image, respectively. In addition, the information on the target angle, target torque, and target image will be appropriately referred to as the target angle information, target torque information, and target image information, respectively.

[0058] In step S103, the multiple target values ​​300 calculated by the memory operation inference unit 111, namely the target angle, target torque, and target image, are concatenated.

[0059] The prediction is performed using machine learning, such as deep learning. The learning parameters have been pre-adjusted using the training data 166. The target angle and target torque are set to target value 300, and represent the angle and torque at which robot 1 should operate one step ahead. Referring to Figure 5, for the data point of target value 301a, the data point one step ahead is the data point of target value 301b. The target image is a predicted image of the state of the object and its surrounding environment that will change as a result of robot 1's operation. The memory operation reasoning unit 111 outputs the target angle, target torque, and target image. Incidentally, one step ahead means a predetermined time interval.

[0060] Of the target angle, target torque, and target image output in step S103, the target angle and target torque become the target motion (target imitation trajectory 163) that the robot 1 should perform in the future.

[0061] In step S104, the imitation motion linking unit 112 stores the target angle and target torque output by the memory motion inference unit 111 in the target imitation trajectory 163.

[0062] Next, in step S105, the imitation operation linking unit 112 determines whether the number of steps in the target imitation trajectory 163 generated up to that point has been reached, or whether the number of steps has been set in advance. Referring to the target value 300 in Figure 5, the number of steps is the number of data points (diamonds) that make up the target imitation trajectory 163, which is "6".

[0063] If the target imitation trajectory 163 has not yet reached the specified number of steps (S105 → No), in step S106, the memory operation reasoning unit 111 refers to the latest target angle, target torque, and target image. Then, in step S103, the memory operation reasoning unit 111 predicts the target angle, target torque, and target image for the next step based on the latest target angle, target torque, and target image referred to in step S106.

[0064] If the target imitation trajectory 163 has reached a predetermined number of steps (S105 → Yes), in step S107, the imitation operation coupling unit 112 outputs the finally calculated target imitation trajectory 163 to the control unit 120.

[0065] Steps S101 to S107 are "imitation action generation steps".

[0066] (Target imitation trajectory 163) Figure 5 shows an example of a target imitation trajectory 163 that is generated.

[0067] In this embodiment, the joint angle information, torque information, and image information captured by the camera 141 of the robot 1 are input to the imitation motion generation unit 110. The memory motion inference unit 111 generates time-series data of the predicted information (target angle, target torque, target image) for each of these. The time-series data of the target angle is indicated by the target value 301 (diamond), and the time-series data of the target torque is indicated by the target value 302 (diamond). The time-series data of the target image is indicated by the code 303. Incidentally, the code 311 (black circle) indicates the angle in the state detection information 161 acquired from the angle sensor 131, and the code 312 (black circle) indicates the torque in the state detection information 161 acquired from the torque sensor 132. The code 313 indicates the image in the image information acquired from the camera 141.

[0068] The time-series data consists of data points from the time when the generation of the target imitation trajectory 163 begins, up to a predetermined number of steps ahead. As shown in Figure 5, of the three time-series data (target angle, target torque, and target image), the time-series data of the target angle and target torque are output as the target imitation trajectory 163. On the other hand, when the memory operation inference unit 111 generates (learns) the target angle, target torque, and target image in step S103 of Figure 4, all three time-series data are used.

[0069] The three time-series data sets are all composed of data points with a common time step size, as shown in Figure 5. For example, as mentioned above, for the data point of target value 301a, the data point one step ahead is the data point of target value 301b. The number of steps is the number of steps in the target imitation trajectory 163, and as mentioned above, in the example in Figure 5, the number of steps is "6". The time step size is basically the same as the time step size of the training data 166 during learning. However, it is also possible to generate the target imitation trajectory 163 such that the time step size of the target imitation trajectory 163 is shorter than the time step size of the training data 166. By doing so, it becomes possible to request faster target movements from the robot 1. For example, if the time step size of the target imitation trajectory 163 is half the time step size of the training data 166, the movement of the robot 1 can be made half the movement time of the training data 166. In this case, the robot 1 can perform the task in a shorter time, thus improving efficiency. However, it becomes difficult for the robot 1 to follow the movements based on the target imitation trajectory 163. In other words, the robot 1's movements become faster, making it difficult to control it, such as by stopping. Therefore, it is necessary to appropriately adjust the control unit 120, which will be described later.

[0070] (Method for generating operation by the control unit 120) Figure 6 is a flowchart showing the procedure for generating an action performed by the control unit 120 in the first embodiment. Refer to Figure 1 as appropriate.

[0071] First, in step S201, the predicted state trajectory evaluation unit 122 acquires the target imitation trajectory 163 output by the imitation operation coupling unit 112.

[0072] Next, in step S202, the control command optimization unit 123 initializes the control command 165. The control command optimization unit 123 can shorten the time required for subsequent optimization calculations by initializing the control command 165 to the value that was finally determined in the previous calculation. The control command optimization unit 123 may also initialize the control command 165 to "0".

[0073] Next, in step S203, the state prediction unit 121 acquires angle information and torque information measured by the angle sensor 131 and torque sensor 132 (acquires angle information and torque information from the sensors).

[0074] Then, in step S204, the state prediction unit 121 uses the current control command 165, acquired angle information, and torque information as initial values ​​to predict the angle and torque of the robot 1 one step ahead. For the prediction of angle and torque, a prediction model such as dynamic equations that take into account the dynamic characteristics of the robot 1, such as moment of inertia and moment of gravity, is used. By making predictions based on dynamic equations, it becomes possible to determine the control command 165 that the robot 1 can realistically follow. By using dynamic equations, a series of predicted movements of the robot 1 based on the operating characteristics of the drive unit 150 is calculated. The series of predicted movements is such that the robot 1 holds the peg T1 and moves its hand to insert the peg T1 into the hole T2. The angle and torque predicted by the state prediction unit 121 are referred to as predicted angle and predicted torque, respectively. In addition, the information of the predicted angle is referred to as predicted angle information, and the information of the predicted torque is referred to as predicted torque information, as appropriate.

[0075] In step S205, the state prediction unit 121 stores the predicted angle and predicted torque predicted and generated in step S204 in the predicted state trajectory 164. The state prediction unit 121 also stores the conversion formula (or conversion map) between the predicted torque and the current value in the control command 165 in the predicted state trajectory 164.

[0076] Next, in step S206, the state prediction unit 121 determines whether the predicted state trajectory 164 has reached a predetermined number of steps. The number of steps is the same as the number of steps in Figure 4.

[0077] If the predicted state trajectory 164 has not reached the step number (S205 → No), in step S207, the state prediction unit 121 refers to the latest predicted angle and predicted torque (S207). Then, in step S204, the state prediction unit 121 predicts the angle (predicted angle) and torque (predicted torque) for the next step based on the predicted angle, predicted torque, and current control command 165 referenced in step S207, using a prediction model such as the dynamic equations. In this way, the state prediction unit 121 first calculates the predicted state value 202 (see Figures 7A and 7B), which is the predicted state detection information 161, based on the dynamic equations of the drive unit 150, using the control command 165 and state detection information 161 as inputs. In the first embodiment, the predicted state value 202 is the predicted angle and predicted torque. Furthermore, the state prediction unit 121 repeatedly calculates the next predicted state value 202 based on the calculated predicted state value 202, i.e., the predicted angle and predicted torque, and the control command 165 generated by the control command optimization unit 123. This generates multiple predicted state values ​​202, i.e., predicted angle and predicted torque.

[0078] Then, in step S205, the state prediction unit 121 stores the predicted angle and predicted torque, i.e., the predicted state values ​​202, in the predicted state trajectory 164. As a result, the multiple predicted state values ​​202 generated are linked together to create the predicted state trajectory 164.

[0079] The number of steps in the predicted state trajectory 164 is basically assumed to be the same as the number of steps in the target imitation trajectory 163. However, it is also possible to make the number of steps in the target imitation trajectory 163 smaller than the number of steps in the predicted state trajectory 164.

[0080] Furthermore, reducing the number of data points in the target imitation trajectory 163 also has the effect of reducing the computational cost when the memory operation inference unit 111 generates the target imitation trajectory 163. As mentioned above, the memory operation inference unit 111 generates the target imitation trajectory 163 using deep learning, etc., and reducing the number of data points in the target imitation trajectory 163 makes it possible to reduce the computational cost of imitation learning (deep learning).

[0081] In step S205, if the predicted state trajectory 164 has reached the step number (S205 → Yes), in step S208, the predicted state trajectory evaluation unit 122 evaluates the generated predicted state trajectory 164. In the first embodiment, the predicted state trajectory evaluation unit 122 evaluates the predicted state trajectory 164 by calculating the degree of agreement between the target imitation trajectory 163 and the predicted state trajectory 164 as an evaluation value. Details of the evaluation will be explained later in Figures 7A and 7B. In addition, an evaluation value for evaluating the magnitude of the control command 165 may be added to prevent the control command 165 calculated in a later stage from becoming too large.

[0082] As described in step S205, in this embodiment, a predetermined number of target angle, predicted angle, target torque, and predicted torque data (linked predicted angle and predicted torque) are used. This is because the robot 1 cannot respond to sudden changes. In other words, even if a sudden change in the target angle is taught in the target imitation trajectory 163, the actual actuator 151 may not be able to respond to such an angle change. Therefore, by using a predetermined number of predicted target angle, predicted angle, target torque, and predicted torque data, it becomes possible to prepare the actuator 151 to respond to sudden angle changes, etc., from a time point prior to the event.

[0083] Then, in step S209, the control command optimization unit 123 updates (corrects) the control command 165 in a direction that improves the evaluation value of the predicted state trajectory 164. In the first embodiment, the direction in which the degree of agreement between the target imitation trajectory 163 and the predicted state trajectory 164 increases is considered the good direction. Incidentally, the closer the target imitation trajectory 163 and the predicted state trajectory 164 are, the higher the degree of agreement. The update of the control command 165 is performed using optimization calculations that employ general mathematical optimization processes such as gradient descent and random sampling. At this time, the control command optimization unit 123 calculates the optimal control command 165 that minimizes the evaluation value in real time. As a result, the control command optimization unit 123 corrects (updates) the control command 165 by performing mathematical optimization processing that solves an optimization problem that reproduces the target imitation trajectory 163 with desired (predetermined) acceleration / deceleration and force (output of predetermined drive unit 150). In other words, the control command optimization unit 123 corrects (updates) the control command 165 by performing mathematical optimization processing so that the degree of agreement, which is an evaluation value, decreases while satisfying the dynamic equations for the drive unit 150. After that, the control command optimization unit 123 outputs the control command 165 for the robot 1.

[0084] Next, in step S210, the predicted state trajectory evaluation unit 122 determines whether the change in the evaluation value (degree of agreement) has converged and whether the evaluation value has reached the optimal value.

[0085] If the evaluation value has not yet reached the optimal level (S210 → No), in step S211, the state prediction unit 121 refers to the new control command 165 updated in step S209. Then, the processing from step S203 onward is repeated again.

[0086] If the evaluation value has reached the optimal level (S210 → Yes), the control command optimization unit 123 confirms the control command 165 updated in step S209 as the optimal control command 165. Then, in step S212, the control command optimization unit 123 outputs the confirmed optimal control command 165 to the actuator 151.

[0087] Steps S201-S207 and S211 are "state prediction steps". Step S208 is the "predicted trajectory evaluation step". Step S209 is the "correction step". Step S212 is the "output step".

[0088] (Evaluation method) Figures 7A and 7B show an example of an evaluation method.

[0089] The degree of agreement between the target imitation trajectory 163 and the predicted state trajectory 164 is based on the error between the target imitation trajectory 163 and the predicted state trajectory 164, predicted with the same time step size, for each data point included in each trajectory. In Figures 7A and 7B, the data points of the target imitation trajectory 163 are shown with a target value of 300 (diamond), and the data points of the predicted state trajectory 164 are shown with a predicted state value of 202 (circle). The evaluation value is the sum of the squares of the errors for each data point. A smaller evaluation value indicates a better result.

[0090] Referring to Figures 7A and 7B, it is clear that in Figure 7B, the data points of the target imitation trajectory 163 (target value 300), indicated by the target value 300, and the data points of the predicted state trajectory 164 (predicted state value 202), indicated by the predicted state value 202, are closer than in Figure 7A. Therefore, the evaluation value is smaller for the data shown in Figure 7B than for Figure 7A. The predicted state trajectory evaluation unit 122 performs the evaluation in this manner.

[0091] If the number of steps in the target imitation trajectory 163 and the predicted state trajectory 164 are different, the error evaluation is performed only for data points that exist in both the target imitation trajectory 163 and the predicted state trajectory 164 at the same time. In other words, the number of data points to be evaluated may be reduced by decimating some of the data in the target imitation trajectory 163. Alternatively, the number of data points in the target imitation trajectory 163 may be reduced when the imitation motion generation unit 110 generates the target imitation trajectory 163.

[0092] Figures 7A and 7B only show the evaluation of angles, but the same applies to torque, so a detailed explanation is omitted.

[0093] Thus, the target value 300 included in the target imitation trajectory 163 and the predicted state value 202 included in the predicted state trajectory 164 have time information. The predicted state trajectory evaluation unit 122 then evaluates the degree of agreement between the target imitation trajectory 163 and the predicted state trajectory 164 based on the error between the (corresponding) target value 300 and predicted state value 202 at the same time.

[0094] The control command optimization unit 123 performs mathematical optimization processing based on the evaluation (degree of agreement) so that the target value 300 of the target imitation trajectory 163 and the predicted state value 202 of the predicted state trajectory 164 become close, as shown in Figure 7B. The mathematical optimization processing is applied to the correction of the control command 165. In other words, the control command 165 is corrected as a result of the mathematical optimization processing. The smallest evaluation value is obtained when the target value 300 of the target imitation trajectory 163 and the predicted state value 202 of the predicted state trajectory 164 match. However, due to differences in the performance of the actuator 151, the target value 300 of the target imitation trajectory 163 and the predicted state value 202 of the predicted state trajectory 164 generally do not match (although they may match). Therefore, the control command optimization unit 123 corrects the control command 165 of the robot 1 by solving an optimization problem (using mathematical optimization processing) that reproduces the desired acceleration / deceleration and force.

[0095] (Effects of the first embodiment) Figures 8A to 8D show the effects of the first embodiment.

[0096] Figure 8A shows an example of the relationship between the movement of the robot 1 based on the target imitation trajectory 163 and the actual movement when tracking control is performed without considering the control characteristics of the robot 1 to the target imitation trajectory 163 (i.e., the imitation learning described so far). Reference numeral 211 (diamond) indicates the time series of the movement of the robot 1 based on the target imitation trajectory 163 (for example, the time series of the target angle). Reference numeral 212 indicates the time series of the movement of the robot 1 when the robot 1 moves (actual movement) according to the control command 165 based on the predicted state trajectory 164 (for example, the time series of the angle during the actual movement).

[0097] In Figure 8A, robot 1 fails to keep up with the target motion trajectory, and at the moment when the direction of the target motion trajectory changes (reference numeral 213), the actual motion overshoots the target motion.

[0098] Figure 8B shows an example of the task execution result when tracking control is performed without considering the controllability characteristics of the robot 1 relative to the target mimic trajectory, as shown in Figure 8A. As indicated by reference numeral 213 in Figure 8A, the robot 1 overshoots, causing the peg T1 that the robot 1 is gripping to come into contact with the side wall of the hole T2, as shown in Figure 8B.

[0099] Figure 8C shows an example of the relationship between the movement of the robot 1 and the actual movement when tracking control is performed on the target imitation trajectory 163, taking into account the controllable characteristics of the robot 1, using the method of the first embodiment. Note that the controllable characteristics of the robot 1 are taken into consideration by using the dynamical equation in step S204 of Figure 6. Also, in Figure 8C, reference numerals 211 and 212 are the same as in Figure 8A.

[0100] As shown in Figure 8C, according to the method described in the first embodiment, the robot 1 is able to accurately follow the target motion. Therefore, the robot 1 operates as intended without overshooting, even at the timing when the direction of the target motion changes (reference numeral 213 in Figure 8A).

[0101] Figure 8D shows an example of the task execution result when tracking control is performed considering the controllable characteristics of the robot 1 with respect to the target imitation trajectory 163, as shown in Figure 8C. As shown in Figure 8D, according to the method shown in the first embodiment, the peg T1 grasped by the robot 1 can be inserted into the hole T2 without the peg T1 coming into contact with the hole T2.

[0102] In the first embodiment, the motion generation device 100 employs imitation learning, thus reducing the learning cost compared to reinforcement learning. Furthermore, because imitation learning is used, it is possible to provide a robot 1 capable of general-purpose movements without requiring specialized knowledge or complex programming. As a result, the robot 1 can perform delicate and complex human movements.

[0103] In addition, in the first embodiment, the control command 165 is corrected so that the predicted state trajectory 164, which is generated considering the control characteristics of the robot 1, approaches the target imitation trajectory 163. Furthermore, the control command 165 can be generated in real time. In other words, in the first embodiment, the control command 165 is determined by correcting the imitation motion generated by the imitation motion generation unit 110 while considering the control characteristics and constraints of the robot 1. This makes it possible to accurately follow the imitation motion while suppressing the increase in learning cost. In this way, the ability to follow the target imitation trajectory 163 is improved, and the robot 1 can complete the task it is to perform with a high success rate. In short, according to the first embodiment, the robot can accurately follow the imitation motion during imitation learning.

[0104] With previous technologies, it was difficult to consider controllability and constraints when generating teaching data 166, such as when a person operates the robot 1. As a result, when the robot 1 performed a task according to such teaching data 166, various problems occurred. In particular, attempting to follow teaching data 166 that does not properly consider acceleration and deceleration in joints, etc., or force application may lead to a decrease in the success rate of the task or instability of the control. For example, when the robot 1 performs a task, it may accelerate too much and miss its target, or it may try to force the peg T1 into the hole T2 and damage the hole T2.

[0105] According to the first embodiment, by using imitation learning, the control command 165 is determined so that the predicted state trajectory 164, which is generated considering the controllability characteristics of the robot 1, approaches the target imitation trajectory 163, while reproducing the imitation movement with a low learning cost, thereby achieving controllably desirable acceleration, deceleration, and force. In other words, according to the first embodiment, it is possible to improve the controllability of the robot 1 while taking advantage of the low learning cost, which is an advantage of imitation learning.

[0106] In the first embodiment, a predicted state trajectory 164 is generated based on the target imitation trajectory 163 generated by the imitation motion generation unit 110, taking into account the controllable characteristics and constraints of the robot 1. Then, the control command 165 is corrected so that the generated predicted state trajectory 164 is as close as possible to the target imitation trajectory 163. This makes it possible to explicitly consider the controllable characteristics and constraints of the robot 1, and the robot 1 can accurately reproduce the generated imitation motion.

[0107] Furthermore, according to the first embodiment, since the target imitation trajectory 163 and dynamic equations can be selected according to the task to be applied and the robot 1 to be applied, it can handle a variety of tasks that are not dependent on the application, and thus enables general application.

[0108] Furthermore, the imitation motion generation unit 110 (memory motion inference unit 111) generates a target imitation trajectory 163 through imitation learning using the environment detection information 162 acquired from the environment detection unit 140 (camera 141) and the state detection information 161. By using the environment detection information 162 in addition to the state detection information 161, the accuracy of the target imitation trajectory 163 can be improved.

[0109] <Second Embodiment> Next, a second embodiment of the present invention will be described with reference to Figures 9A to 10D.

[0110] (Evaluation method) Figures 9A and 9B show an example of the evaluation method in the second embodiment.

[0111] In the second embodiment, in addition to the degree of agreement between the target imitation trajectory 163 and the predicted state trajectory 164 shown in the first embodiment, three items are evaluated: the amount of change in state detection information 161 and the degree of satisfaction. The amount of change in state detection information 161 is the amount of change in angle and the amount of change in torque in the predicted state trajectory 164. The degree of satisfaction is the degree to which the predicted state trajectory 164 satisfies the pre-set constraints. The amount of change and degree of satisfaction will be described later. A priority level is set by the user for each evaluation value.

[0112] The priority order between each evaluation value (degree of agreement, amount of change, and degree of satisfaction) can be adjusted by weight coefficients that are set in advance for each evaluation value. The control command optimization unit 123 calculates control commands 165 that prioritize satisfying evaluation values ​​for which larger weight coefficients are set. In other words, the control command optimization unit 123 performs optimization calculations for each evaluation value using general mathematical optimization processes such as gradient descent and random sampling.

[0113] Figure 9A shows an example of how to evaluate the change in the predicted angle in the predicted state trajectory 164. Figure 9B shows an example of how to evaluate the degree to which the predicted state trajectory 164 satisfies the pre-set constraints. In Figure 9A, the predicted state value 202 represents the data point based on the predicted angle of the predicted state trajectory 164. In Figure 9B, the symbol 203 represents the angular acceleration calculated from the predicted angle.

[0114] The change in the predicted angle in the predicted state trajectory 164 shown in Figure 9A is the change (symbol 221) between each data point (predicted state value 202) included in the predicted state trajectory 164 and the data point predicted in the preceding and succeeding steps. This change becomes the evaluation value. The evaluation value is the sum of the squares of the changes between each data point and the preceding and succeeding data points. A smaller evaluation value indicates a better result.

[0115] The change in the predicted angle trajectory shown in Figure 9A corresponds to evaluating the magnitude of the angular velocity. Similarly, when evaluating the magnitude of angular acceleration, the change in the time series of angular velocity obtained by differentiating the predicted angle is evaluated around the data point. When considering the magnitude of jerk, the change in the time series of the value obtained by differentiating the angular acceleration is similarly evaluated around the data point. Figure 9A only shows the evaluation of the predicted angle, but the same applies to the predicted torque, so a detailed explanation is omitted.

[0116] As shown in Figure 9A, the predicted state trajectory evaluation unit 122 evaluates the magnitude of any of the velocity, acceleration, and jerk included in the predicted state trajectory 164. In this embodiment, velocity is angular velocity, acceleration is angular acceleration, and jerk is the derivative of angular acceleration. Then, the control command optimization unit 123 corrects the control command 165 through mathematical optimization processing so that the degree of agreement decreases while minimizing the magnitude of any of the velocity, acceleration, and jerk included in the predicted state trajectory 164 as much as possible.

[0117] Figure 9B shows the degree to which the predicted state trajectory 164 satisfies the pre-set constraints.

[0118] The degree to which the predicted state trajectory 164 satisfies the pre-set constraints is evaluated by using, for example, the degree of deviation (indicated by 222) from the set maximum and minimum values ​​for each data point (indicated by 203) based on the predicted state trajectory 164. In the example shown in Figure 9B, for each data point relating to angular acceleration, the degree of deviation from the maximum value (dashed line 231) and minimum value (dashed line 232) is defined as a penalty. The sum of these penalties becomes the evaluation value (degree of satisfaction). The maximum and minimum values ​​are the pre-set constraints. In Figure 9B, only one data point (indicated by 203a) deviates from the maximum value, but if multiple data points deviate, the predicted state trajectory evaluation unit 122 calculates the degree of deviation for each data point and uses the sum of these deviations as the evaluation value.

[0119] In this way, the predicted state trajectory evaluation unit 122 evaluates whether the predicted state values ​​202 included in the predicted state trajectory 164 satisfy the constraints of the drive unit 150 (maximum value and minimum value in the example shown in Figure 9B).

[0120] The maximum and minimum values ​​are set based on the specifications of the actuator 151, etc.

[0121] A smaller evaluation value indicates a better result. Although Figure 9B only shows the evaluation for angular acceleration, the same applies to angular velocity, jerk, and predicted torque, so a detailed explanation is omitted.

[0122] Incidentally, as mentioned above, the user sets priorities between the degree of agreement between the target imitation trajectory 163 and the predicted state trajectory 164, the amount of change in the predicted angle and predicted torque (see Figure 9A), and the degree of satisfaction (see Figure 9B). In this case, it is desirable to set the priorities (weighting coefficients) so that the degree of satisfaction has the highest priority. This is because the degree of satisfaction is related to the specifications of the actuator 151.

[0123] The control command optimization unit 123 calculates the optimal predicted state trajectory 164 that satisfies the constraints in real time by minimizing the evaluation value. As a result, the control command optimization unit 123 corrects the control command 165 so that the degree of agreement decreases while satisfying the constraints of the drive unit 150.

[0124] Note that the process described in Figure 9 is the same process performed in step S208 of Figure 6.

[0125] (effect) Figures 10A to 10D show the effects of the second embodiment.

[0126] Figure 10A shows an example of the relationship between the movement of the robot 1 based on the target imitation trajectory 163 and the actual movement when tracking control is performed considering only the degree of agreement between the predicted state trajectory 164 and the target imitation trajectory 163. Note that in Figures 10A and 10C, reference numerals 211 and 212 are the same as in Figure 8A.

[0127] In other words, Figure 10A shows an example of the relationship between the movement of the robot 1 based on the target imitation trajectory 163 and the actual movement when only the method shown in the first embodiment is used. In Figure 10A, the robot 1 follows the target movement (reference numeral 211) based on the target imitation trajectory 163. However, it reacts hypersensitively to the vibrational movements contained in the human-generated teaching data 166, and the actual movement (reference numeral 212) also becomes a vibrational movement (dotted line in Figure 10A). Such vibrational movements occur because, when the teaching data 166 is created, human tremors, etc., are included in the teaching data 166.

[0128] Figure 10B shows an example of the task execution result when tracking control is performed considering only the degree of agreement between the predicted state trajectory 164 and the target imitation trajectory 163, as in Figure 10A.

[0129] As shown in Figure 10B, if the teaching data 166 includes vibrational motion, when the robot 1 inserts the peg T1 it is gripping into the hole T2, the robot 1 will operate vibrationally as shown by reference numeral 321. As a result, the robot 1 will cause the peg T1 to come into contact with the side wall of the hole T2.

[0130] Figure 10C shows an example of the relationship between the movement of the robot 1 based on the target imitation trajectory 163 and the actual movement when tracking control is performed considering not only the degree of agreement between the predicted state trajectory 164 and the target imitation trajectory 163, but also the amount of change in the predicted angle of the predicted state trajectory 164 and its constraints.

[0131] By applying the method shown in the second embodiment, the robot 1 operates in a manner that maintains the trend of the target imitation trajectory 163 while ignoring excessively oscillating motions included in the target imitation trajectory 163 (dotted line in Figure 10C). This is because the evaluation values ​​are set in such a way that large changes in the predicted angle and predicted torque are suppressed, as shown in Figure 9A.

[0132] Figure 10D shows an example of the task execution results when tracking control is performed considering not only the degree of agreement between the predicted state trajectory 164 and the target imitation trajectory 163, as in Figure 10C, but also the amount of change in the predicted angle of the predicted state trajectory 164 and its constraints.

[0133] As shown in Figure 10D, by applying the method described in the second embodiment, the peg T1 gripped by the robot 1 can be inserted into the hole T2 without the peg T1 coming into contact with the hole T2.

[0134] In the second embodiment, in addition to the method shown in the first embodiment, the control command 165 is determined by considering the amount of change and constraints based on the data points (predicted state values ​​202) of the predicted state trajectory 164. The method shown in the first embodiment is a method of bringing the predicted state trajectory 164, which is generated considering the controllable characteristics of the robot 1, closer to the target imitation trajectory 163. Bringing the predicted state trajectory 164 closer to the target imitation trajectory 163 means that the degree of agreement described above becomes smaller. As a result, the robot 1 can complete the task it is to perform with a high success rate without being affected by vibrational movements or noise unintentionally included in the human-generated teaching data 166. In other words, according to the second embodiment, robustness against vibrations and noise unintentionally included in the teaching data 166 is possible.

[0135] Furthermore, as shown in Figure 9B, an evaluation value (satisfaction level) is calculated based on the degree of deviation from the maximum and minimum values, making it possible to generate a control command 165 that takes into account the performance of the actuator 151.

[0136] <Third Embodiment> Next, a third embodiment of the present invention will be described with reference to Figures 11 to 13D.

[0137] (Configuration of robot 1a) Figure 11 shows an example of the configuration of robot 1a according to the third embodiment.

[0138] The motion generation device 100a includes an imitation motion generation unit 110, a control unit 120, and an obstacle detection unit 181 that detects obstacles C (see Figure 12) present around the robot 1a. The other configurations are the same as in Figure 1. The obstacle detection unit 181 identifies obstacles C from images (image information) measured by the camera 141 and outputs relative position information of obstacles C with respect to the robot 1a. The method for identifying obstacles C is not limited to the method using the camera 141; for example, obstacles C may be identified from point cloud information measured by LiDAR (not shown).

[0139] Figures 12A to 12C show an example of the task execution environment and an evaluation method for obstacle avoidance of robot 1a according to the third embodiment.

[0140] Figure 12A shows the task execution environment of robot 1a, which includes obstacle C, as the target of the third embodiment. In the task execution environment shown in Figure 12A, obstacle C, which was not present when a human generated the teaching data 166, is present between peg T1 and hole T2. In such a case, if robot 1a attempts to insert peg T1 into hole T2 using the shortest distance, there is a risk that robot 1a or the peg T1 that robot 1a is gripping may come into contact with obstacle C along the way, as shown in Figure 12A.

[0141] Figures 12B and 12C show an example of an evaluation function used by the Predicted State Trajectory Evaluation Unit 122 to evaluate the predicted state trajectory 164 (calculation of evaluation value) and assess the possibility of contact between the predicted state trajectory 164 and obstacle C. The evaluation function shown in Figure 12C determines the evaluation value based on the distance between the reference positions R1 to R4 and obstacle C shown in Figure 12B.

[0142] For each of the reference positions R1 to R4 set on the robot 1a or peg T1, which are calculated from each data point included in the predicted state trajectory 164, the predicted state trajectory evaluation unit 122 calculates the distance to the obstacle C. In this process, as shown in Figure 12C, the predicted state trajectory evaluation unit 122 determines that the smaller the distance between the nearest reference position (the reference position R1 to R4 closest to the obstacle C) and the obstacle C, the higher the probability of contact. A larger distance between the nearest reference position and the obstacle C, and a lower probability of contact, is considered a good result.

[0143] Step S209 in Figure 6 is performed by performing an optimization calculation using evaluation values ​​based on the distance between the reference positions R1 to R4 and obstacle C, and evaluation values ​​based on the degree of agreement as shown in the first embodiment. The user sets a priority between the degree of agreement between the target imitation trajectory 163 and the predicted state trajectory 164 and the evaluation values ​​based on the distance between the reference positions R1 to R4 and obstacle C. In this case, it is desirable that the evaluation values ​​based on the distance between the reference positions R1 to R4 and obstacle C take precedence (have a larger weight coefficient) over the degree of agreement between the target imitation trajectory 163 and the predicted state trajectory 164. This is because avoiding contact with obstacle C takes precedence over matching the target imitation trajectory 163 and the predicted state trajectory 164.

[0144] The control command optimization unit 123 corrects the control command 165 by performing optimization calculations for each evaluation value using general mathematical optimization methods such as gradient descent and random sampling. As a result, the control command optimization unit corrects the control command 165 through mathematical optimization processing so that the predicted state trajectory 164 avoids contact with obstacle C while reducing the degree of agreement.

[0145] Note that the processes shown in Figures 12A to 12C are the processes performed in step S208 of Figure 6.

[0146] (effect) Figures 13A to 13D show the effects of the method shown in the third embodiment.

[0147] Figure 13A shows an example of the relationship between the movement of the robot 1a based on the target imitation trajectory 163 and the actual movement when tracking control is performed considering only the degree of agreement between the predicted state trajectory 164 and the target imitation trajectory 163 as an evaluation value. Note that in Figures 13A and 13C, reference numerals 211 and 212 are the same as in Figure 8A.

[0148] In other words, Figure 13A shows an example of the relationship between the movement of the robot 1a based on the target imitation trajectory 163 and the actual movement when only the method shown in the first embodiment is used. As shown in Figure 13A, the robot 1a follows the target movement well without overshooting.

[0149] Figure 13B shows an example of the task execution result when tracking control is performed considering only the degree of agreement between the predicted state trajectory 164 and the target imitation trajectory 163, as in Figure 13A. Since obstacle C did not exist when the human created the teaching data 166, the presence of obstacle C is not taken into account in the target imitation trajectory 163. Therefore, when the robot 1a operates to follow the target imitation trajectory 163, a part of the robot 1a comes into contact with obstacle C, as shown in Figure 13B.

[0150] Figure 13C shows an example of the relationship between the movement of the robot 1a based on the target imitation trajectory 163 and the actual movement when follow control is performed considering not only the degree of agreement shown in the first embodiment as an evaluation value, but also the possibility of contact between the robot 1a and obstacle C. Here, the weighting coefficient of the evaluation value is adjusted to prioritize reducing the possibility of contact with obstacle C over the degree of agreement with the target imitation trajectory 163. In other words, the control command optimization unit 123 performs optimization calculations so that the priority (weighting coefficient) of the evaluation value based on the distance between the reference positions R1 to R4 and obstacle C is greater than the evaluation value based on the degree of agreement shown in the first embodiment.

[0151] When this process is performed, avoiding obstacle C takes priority, even if it means ignoring the target imitation trajectory 163 to some extent. As a result, robot 1a will move to get as close as possible to the target imitation trajectory 163 after avoiding obstacle C.

[0152] Figure 13D shows an example of the task execution result when robot 1a operates as shown in Figure 13C. As shown in Figure 13D, robot 1a operates to avoid obstacle C, and then approaches hole T2 with the peg T1 to be grasped in order to perform its original task.

[0153] In the third embodiment, in addition to the method shown in the first embodiment, the control command 165 is determined so that the robot 1a avoids contact with obstacle C. The method shown in the first embodiment is a method in which the predicted state trajectory 164, which is generated considering the controllable characteristics of the robot 1a, approaches the target imitation trajectory 163. This makes it possible for the robot 1a to complete the task it is to perform without contacting obstacle C that was not present when the teaching data 166 was generated. Therefore, the environmental conditions in which the robot 1a can be applied are relaxed.

[0154] <Fourth Embodiment> Next, a fourth embodiment of the present invention will be described with reference to Figure 14.

[0155] Figure 14 shows an example of a candidate for the target imitation trajectory 163 and an evaluation method of the predicted state trajectory evaluation unit 122 in the fourth embodiment.

[0156] In the fourth embodiment, the imitation motion generation unit 110 outputs a plurality of target imitation trajectories 163.

[0157] In deep learning, multiple target imitation trajectories 163 are generated. This can be achieved by designing the deep learning model so that multiple target imitation trajectories 163 are output. Multiple target imitation trajectories 163 are output, for example, depending on their accuracy. Accuracy is the degree of agreement between the target imitation trajectory 163 and the training data 166. In other words, multiple target imitation trajectories 163 with different degrees of agreement to the training data 166 are intentionally output. These multiple target imitation trajectories 163 are referred to as target imitation trajectories 162a to 162c.

[0158] Specifically, the imitation motion generation unit 110 ranks the multiple target imitation trajectories 162a to 162c that it predicts and outputs in descending order of accuracy. The imitation motion generation unit 110 then outputs a specified number of target imitation trajectories 162a to 162c in the order in which they are judged to have the highest accuracy. In the example shown in Figure 14, three target imitation trajectories 162a to 162c are output, so the imitation motion generation unit 110 outputs the three target imitation trajectories 162a to 162c in descending order of angle.

[0159] The predicted state trajectory evaluation unit 122 then evaluates the degree of agreement between each of the target imitation trajectories 162a to 162c and the predicted state trajectory 164 (calculates an evaluation value). The degree of agreement between the predicted state trajectory 164 and the target imitation trajectories 162a to 162c is based on the error between the data points (predicted state values ​​202) of the predicted target imitation trajectories 162a to 162c and the predicted state trajectory 164 for each of the data points (target values ​​300) included in each of the target imitation trajectories 162a to 162c. The sum of the squares of the errors for each data point is defined as the evaluation value. As a result, a smaller evaluation value indicates a better result.

[0160] Then, the control command optimization unit 123 updates (corrects) the control command 165 by performing optimization calculations using mathematical optimization methods based on these evaluation values. In this way, the control command optimization unit 123 corrects the control command 165 based on the priority set in advance for each of the multiple target imitation trajectories 163, and the degree of agreement between the multiple target imitation trajectories 163 and the predicted state trajectory 164.

[0161] Here, the predicted state trajectory evaluation unit 122 may use the accuracy of the target imitation trajectories 162a to 162c as a weighting coefficient and prioritize evaluating the error with the target imitation trajectories 162a to 162c that have higher accuracy. If the priority of the target imitation trajectories 162a to 162c is the same, the average value of each data point in the target imitation trajectories 162a to 162c becomes the control command 165.

[0162] Note that the process described in Figure 14 is the same process performed in step S208 of Figure 6.

[0163] (effect) In the fourth embodiment, the control command 165 is determined so that the predicted state trajectory 164, which is generated considering the controllable characteristics of the robot 1, matches the multiple target imitation trajectories 162a to 162c. This suppresses excessive tracking of target movements far into the future from the present time, which generally have a low accuracy. As a result, even when an incorrect imitation movement is predicted (the prediction error is large), the task can be performed robustly, contributing to an improved task success rate.

[0164] In both the generation of the target imitation trajectory 163 and the generation of the predicted state trajectory 164, predicted values ​​are used to calculate the predicted values ​​for the next step. Here, the predicted values ​​are the predicted angle and torque (i.e., the predicted state value 202). Therefore, it is expected that the error in each data point will increase as we move from the present, when the generation of the target imitation trajectory 163 and the predicted state trajectory 164 begins, into the future. In other words, it is expected that the error in the data points of the target imitation trajectory 163 and the predicted state trajectory 164 will increase compared to the true data points (true angle and torque). The true angle and torque are the actual angle and torque of robot 1 at that time.

[0165] If the control command 165 is updated using only one target imitation trajectory 163, there is a risk that the control command 165 will be generated that includes the error of the target imitation trajectory 163 being used. In the fourth embodiment, the control command 165 is updated so that the errors of each target imitation trajectory 162a to 162c are averaged out (the errors are mitigated) by using multiple target imitation trajectories 162a to 162c. This makes it possible to obtain the effects described above.

[0166] <Robot control system Z> Figure 15 shows an example of the robot control system Z according to this embodiment.

[0167] In the robot control system Z shown in Figure 15, the motion generation device 100 is provided externally to the robot 1 in a manner that allows communication. The configuration of the motion generation device 100 is the same as in Figure 1, and the configuration of the robot 1 is the same as in Figure 1, except that the motion generation device 100 is not provided internally.

[0168] The angle information acquired by the angle sensor 131 and the torque information acquired by the torque sensor 132, both equipped on robot 1, are input to the memory operation reasoning unit 111 and the state prediction unit 121. The image captured by camera 141 is also input to the memory operation reasoning unit 111. Furthermore, the control command 165 generated by the control command optimization unit 123 is input to the actuator 151 of robot 1 and also to the state prediction unit 121.

[0169] In the first to fourth embodiments, the motion generation device 100 is configured to be mounted inside the robot 1. However, as shown in Figure 15, the motion generation device 100 can be installed outside the robot 1, provided that it is configured to communicate with the angle sensor 131, torque sensor 132, camera 141, and actuator 151.

[0170] As mentioned above, if the motion generation device 100 is installed outside the robot 1 as shown in Figure 15, its hardware configuration will be as shown in Figure 2. Furthermore, the motion generation device 100 shown in Figure 15 may also be equipped with an obstacle detection unit 181 as shown in Figure 1.

[0171] The robot 1 shown in this embodiment can be applied to industrial robot arms, medical robot arms, visual inspection robot systems, and the like.

[0172] The present invention is not limited to the embodiments described above, and includes various modifications. For example, the embodiments described above are described in detail to make the present invention easier to understand, and are not necessarily limited to those having all the configurations described. Furthermore, it is possible to replace parts of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add configurations from other embodiments to the configuration of one embodiment. In addition, it is possible to add, delete, or replace parts of the configuration of each embodiment with other configurations.

[0173] Furthermore, each of the above-mentioned configurations, functions, memory operation inference unit 111 to obstacle detection unit 181, ROM 172, RAM 173, etc., may be implemented in hardware, either partially or entirely, by designing them as integrated circuits, for example. Alternatively, as shown in Figure 2, each of the above-mentioned configurations, functions, etc., may be implemented in software by having a processor such as a CPU 171 interpret and execute programs that realize each function. Information such as programs, tables, and files that realize each function can be stored not only in the HD, but also in memory, a recording device such as an SSD, or a recording medium such as an IC (Integrated Circuit) card, an SD (Secure Digital) card, or a DVD (Digital Versatile Disc). Furthermore, in each embodiment, only those control lines and information lines deemed necessary for explanation are shown, and not all control lines and information lines are necessarily shown in the actual product. In practice, it can be assumed that almost all components are interconnected. [Explanation of Symbols]

[0174] 1,1a Robot 100,100a motion generator 110 Mimicry motion generation unit 111 Memory operation inference unit 112 Mimicry operation linking section 121 State prediction unit 122 Predicted State and Orbit Evaluation Unit 123 Control Command Optimization Unit 130 State detection unit 131 Angle sensor (state detection unit) 132 Torque sensor (state detection unit) 140 Environmental detection unit 141 Camera (Environmental Detection Unit) 150 Drive unit 151 Actuator (Drive Unit) 161 Status Detection Information 162, 162a~162c Environmental detection information 164 Predicted State Orbit 165 Control Command 202 Predicted Status Value 300, 301, 301, 301a, 301b, 302 Target values C Obstacle Z Robot Control System S101 Initialize target imitation trajectory (imitation motion generation step) S102 Acquire angle, torque, and image from the sensor (simulated motion generation step) S103 The learning motion inference unit predicts the target angle, target torque, and target image one step ahead (imitation motion generation step). S104 The imitation motion coupling unit stores the predicted angle and predicted torque in the target imitation trajectory (imitation motion generation step). S105 Determine whether the number of steps in the target imitation trajectory has reached the specified number of steps (imitation motion generation step) S106 Refer to the latest target angle, target torque, and target image (simulated motion generation step) S107 The imitation motion coupling unit outputs the target imitation trajectory (imitation motion generation step) S201 Obtain target imitation trajectory (state prediction step) S202 Initialize control commands (state prediction step) S203 Obtain angle information and torque information from the sensor (state prediction step) S204 The state prediction unit predicts the robot's angle and torque one step ahead (state prediction step). S205 Store predicted angle and predicted torque in the predicted state trajectory (state prediction step) S206 Determine whether the number of steps in the predicted state orbit has reached the specified number of steps (state prediction step) S207 Refer to the latest predicted angle and predicted torque (state prediction step) S208 The predicted state-of-orbit evaluation unit evaluates the predicted state-of-orbit (predicted trajectory evaluation step). S209 The control command optimization unit updates the current control command (correction step). S211 Obtain the latest control command (state prediction step) S212 The control command optimization unit outputs the optimal control command (output step).

Claims

1. The drive unit that drives the robot, A state detection unit that acquires information regarding the state of the robot, Based on the state detection information obtained from the state detection unit, the imitation motion generation unit generates a target imitation trajectory, which is time-series data of target values ​​related to the robot's motion, through imitation learning. A state prediction unit generates a predicted state trajectory, which is time-series data relating to the predicted movement of the robot, based on the operating characteristics of the drive unit, using the control command to the drive unit at the present time and the state detection information. A predicted state trajectory evaluation unit compares the target imitation trajectory generated by the imitation motion generation unit with the predicted state trajectory generated by the state prediction unit and evaluates the degree of agreement between the target imitation trajectory and the predicted state trajectory. A control command optimization unit corrects the control command based on the degree of agreement evaluated by the predicted state trajectory evaluation unit and outputs the corrected control command to the drive unit, Equipped with robot.

2. The aforementioned mimicry operation generation unit comprises a memory operation inference unit and a mimicry operation linking unit, The memory operation reasoning unit, The robot calculates the target value for its next action using the aforementioned state detection information as input. The aforementioned imitation operation linking unit is The memory operation inference unit generates the target imitation trajectory by concatenating the multiple target values ​​it has calculated, The state prediction unit, Based on the dynamic equation of the drive unit, the predicted state value, which is the predicted state detection information, is calculated using the control command and the state detection information as inputs. Furthermore, the next predicted state value is calculated based on the calculated predicted state value and the control command generated by the control command optimization unit. This process is repeated to generate multiple predicted state values, and the predicted state trajectory is generated by concatenating the generated multiple predicted state values. The robot according to feature 1.

3. The robot is equipped with an environment detection unit that acquires environmental information about the surrounding environment, The aforementioned imitation motion generation unit, The target imitation trajectory is generated using the state detection information and the environment detection information obtained from the environment detection unit. The robot according to feature 1.

4. The target value included in the target imitation trajectory and the predicted state value included in the predicted state trajectory have time information, The aforementioned predicted state trajectory evaluation unit is: The degree of agreement between the target imitation trajectory and the predicted state trajectory is evaluated based on the error between the target value and the predicted state value at the same time. The robot according to feature 1.

5. The control command optimization unit, The control command is corrected by performing a mathematical optimization process that solves an optimization problem to reproduce the target imitation trajectory with the desired acceleration, deceleration, and force. The robot according to feature 1.

6. The aforementioned predicted state trajectory evaluation unit is: The magnitude of any of the velocity, acceleration, and jerk included in the predicted state trajectory is evaluated. The control command optimization unit, The mathematical optimization process corrects the control command so that the degree of agreement decreases while minimizing the magnitude of any of the velocity, acceleration, and jerk included in the predicted state trajectory. The robot according to feature 5.

7. The aforementioned predicted state trajectory evaluation unit is: The predicted state values ​​included in the predicted state trajectory are evaluated to determine whether or not the constraints of the drive unit are met. The control command optimization unit, The control command is corrected to approach the target imitation trajectory while satisfying the constraints of the drive unit. The robot according to feature 1.

8. The control command optimization unit, The mathematical optimization process corrects the control command so that the predicted trajectory avoids contact with obstacles while reducing the degree of agreement. The robot according to feature 5.

9. The imitation motion generation unit outputs multiple target imitation trajectories, The control command optimization unit, The control command is corrected based on the priority set in advance for each of the multiple target imitation trajectories, and the degree of agreement between the multiple target imitation trajectories and the predicted state trajectory. The robot according to feature 1.

10. A imitation motion generation unit generates a target imitation trajectory, which is time-series data of target values ​​related to the robot's movement, based on state detection information, which is information obtained from the state detection unit and is also information related to the state of the robot, through imitation learning. A state prediction unit generates a predicted state trajectory, which is time-series data relating to the predicted movement of the robot, based on the operating characteristics of the drive unit, using the control command to the drive unit that drives the robot at the present time and the state detection information. A predicted state trajectory evaluation unit compares the target imitation trajectory generated by the imitation motion generation unit with the predicted state trajectory generated by the state prediction unit and evaluates the degree of agreement between the target imitation trajectory and the predicted state trajectory. A control command optimization unit corrects the control command based on the degree of agreement evaluated by the predicted state trajectory evaluation unit and outputs the corrected control command to the drive unit, Equipped with Motion generator.

11. Robots and, Motion generation device and A robot control system comprising, The aforementioned robot, A drive unit for driving the robot, A state detection unit that acquires information regarding the state of the robot, The motion generation device according to claim 10, Equipped with Robot control system.

12. A motion generation device that generates information for a robot to operate, A imitation motion generation step that generates a target imitation trajectory, which is time-series data of target values ​​related to the robot's movement, based on state detection information, which is information obtained from the state detection unit and is related to the state of the robot, through imitation learning, A state prediction step that generates a predicted state trajectory, which is time-series data relating to the predicted movement of the robot, based on the operating characteristics of the drive unit, using the control command to the drive unit that drives the robot at the present time and the state detection information, A predicted trajectory evaluation step involves comparing the target imitation trajectory generated in the imitation motion generation step with the predicted state trajectory generated in the state prediction step, and evaluating the degree of agreement between the target imitation trajectory and the predicted state trajectory. A correction step is performed to correct the control command based on the degree of agreement evaluated in the predicted trajectory evaluation step, An output step of outputting the corrected control command to the drive unit, A method for generating an action to perform.

Images