Robot control device, robot control system, robot and robot control method

Abstract

To provide a robot control device that enables a robot to learn autonomous operation and autonomous recovery operation at once on the basis of robot sensor information.SOLUTION: A robot control device is provided with an inverse data generating device that generates inverse data that are inverse time-series data, using forward data being measured time-series robot sensor information; a feature amount error calculating part that learns feature amounts of the forward data and of the inverse data and infers a correspondence relation between a forward direction and an inverse direction; a machine learning device that infers motion of a robot in the forward direction and in the inverse direction on the basis of the robot sensor information; and a command part that outputs a command value from the machine learning device to the robot.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technology that is effective when applied to a robot control device, a robot control system, a robot, and a robot control method that autonomously executes recovery operations using machine learning. [Background technology]

[0002] Unlike limited environments such as factories that are prepared for robots, human living environments require advanced expertise and huge development costs to model the surrounding environment, the object being operated, and the robot itself. To solve this problem, the application of machine learning to robots is being promoted, and it is expected that by having robots learn sensor information measured in the real world, recognition and motion generation models can be acquired simultaneously without constructing a detailed environmental model. Machine learning makes it possible to acquire single motions at low cost, such as grasping objects with complex shapes that are difficult to achieve with conventional technology, fitting tasks that require simultaneous control of force and hand position, and movement in diverse environments.

[0003] However, machine learning is not omnipotent, so in order to respond to unexpected situations, it is necessary to comprehensively consider abnormal conditions in advance and implement appropriate exception handling and fault-tolerant design. This still requires huge development costs. In addition, the robot system is designed on the premise that when a sensor abnormality is detected during operation, the production line and robots will be stopped and humans will be present to remove the cause of the error, which poses an issue of reduced operation rate and sustainability of the robot system.

[0004] Therefore, if a robot can use machine learning to learn and execute the actions required for error recovery from its own and the surrounding environment, it is expected that the cost of developing the robot will be reduced and the operating rate will be improved. However, while data-driven robot systems using machine learning can acquire flexible behavior (forward inference) depending on the environment by learning from huge amounts of data, no systematic method has been proposed for realizing recovery actions.

[0005] Background art in this technical field includes, for example, technology such as that described in Patent Document 1. Patent Document 1 discloses a configuration that reduces support by an operator by generating self-supporting actions using a trained model acquired by machine learning or instructing the robot system to perform a recovery action (autonomous recovery action) set in advance in response to an abnormal state that occurs in an autonomous work robot.

[0006] In addition, Non-Patent Document 1 discloses a technology that is a type of machine learning algorithm capable of forward and backward inference, and improves the prediction accuracy of a model by taking into account not only past time-series information but also future time-series information. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Patent Publication No. 2022-63707 [Non-patent literature]

[0008] [Non-Patent Document 1] Mike Schuster and Kuldip K. Paliwal. “Bidirectional recurrent neural networks.” IEEE Transactions On Signal Processing VOL.45,NO.11 (1997): 2673-2681. Summary of the Invention [Problem to be solved by the invention]

[0009] In Patent Document 1, the self-supporting motion is acquired by learning support information such as remote operation by an operator. Therefore, in order to acquire a self-supporting motion generation model, motion teaching costs and model learning costs are incurred.

[0010] In addition, Non-Patent Document 1 requires input of both past and future time series information. However, it is difficult to apply this method to robots because they cannot obtain future information.

[0011] Therefore, an object of the present invention is to provide a robot control device, a robot control system, a robot, and a robot control method that are capable of simultaneously learning or generating autonomous operation and autonomous recovery operation based on robot sensor information. [Means for solving the problem]

[0012] In order to solve the above problems, the present invention is characterized by comprising an inverse data generation device that uses forward data, which is measured robot sensor time series information, to generate inverse data, which is reverse time series data, a feature error calculation unit that learns features of the forward data and the reverse data and infers a correspondence between forward and reverse directions, a machine learning device that infers robot operations in the forward and reverse directions based on the robot sensor information, and a command unit that outputs a command value of the machine learning device to a robot.

[0013] The present invention also provides a robot comprising: a robot control device that controls the robot; the robot comprising: a measurement unit that measures an external environment and transmits the measured value to the robot control device; and a drive unit that drives the robot based on a command value output from the robot control device; and the robot control device comprising: a forward data storage unit that stores forward data, which is robot sensor time-series information measured by the measurement unit; a reverse data generation device that uses the forward data to generate reverse data, which is reverse time-series data, a forward / reverse data storage unit that associates and stores the relationship between the forward data and the reverse data; a feature error calculation unit that learns features of the forward data and the reverse data and infers a correspondence relationship between forward and reverse directions; a machine learning device that infers robot operations in forward and reverse directions based on robot sensor information; and a command unit that outputs a command value of the machine learning device to the robot.

[0014] The present invention also includes the steps of (a) reading a machine learning model defined by a user, (b) learning optimal parameters of the machine learning model according to an objective function using the machine learning model read in the step (a) and pre-accumulated learning data, (c) acquiring robot sensor information, and (d) predicting an operation command value using the machine learning model learned in the step (b) and the robot sensor information acquired in the step (c), and is characterized in that in the step (d), forward inference and backward inference of robot operation are performed. Effect of the Invention

[0015] According to the present invention, it is possible to realize a robot control device, a robot control system, a robot, and a robot control method that are capable of simultaneously learning autonomous operation and autonomous recovery operation based on robot sensor information.

[0016] This will contribute to reducing robot development costs and improving operating rates.

[0017] Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0018] [Figure 1] 1 is a diagram showing a schematic configuration of a robot system according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram illustrating the process of the inverse data generating device of FIG. 1. [Diagram 3] 2 is a diagram illustrating a calculation method of a feature error calculation unit in FIG. 1 and an internal representation obtained thereby. FIG. [Figure 4] 3 is a flowchart showing a robot control method according to the first embodiment of the present invention. [Diagram 5] FIG. 13 is a diagram illustrating an example of an autonomous recovery operation based on sensor information. [Figure 6] FIG. 11 is a diagram illustrating an example of an autonomous recovery operation based on confidence factor information according to the second embodiment of the present invention. [Figure 7] FIG. 11 is a diagram illustrating an example of an autonomous recovery operation based on a linguistic instruction according to the third embodiment of the present invention. [Figure 8] FIG. 13 is a diagram showing an example of determining the prominence of a robot's behavior according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] A robot control device according to an embodiment of the present invention includes an inverse data generation device that generates at least one reverse time series data (reverse data) from at least one measured robot sensor time series information (forward data), a feature error calculation unit that learns features of the forward data and reverse data and enables inference in forward and reverse directions, a machine learning device that is capable of autonomously inferring forward and reverse directions based on the robot sensor time series information by learning at least one or more movements, and a command unit that outputs a command value of the machine learning device to the robot.

[0020] The robot control device according to this embodiment has three features as follows.

[0021] First, this robot control device generates reverse data from learning data (forward data) for carrying out a target task. By learning both forward and reverse data, it becomes possible to generate (infer) forward (same movement as the forward data) and reverse (movement of forward data played in reverse) movements based on sensor information. When an abnormality occurs, it is possible to generate simple autonomous recovery movements such as "returning to a specific past state" by using backward inference. This reduces the cost of collecting the data required for learning. There is no need to learn huge amounts of data, making it possible to keep the costs of data collection and model learning low.

[0022] Next, this robot control device has one model learn multiple actions and autonomously switches between action patterns based on sensor information. This eliminates the need to teach multiple actions in advance and have the model learn them when transitioning to another action learned in the past, and also eliminates the need to separately implement a state transition model for switching between multiple models when an abnormality occurs. In other words, it is possible to reduce the cost of learning the model and eliminate the need to implement a state transition model.

[0023] Finally, this robot control device is equipped with a feature error calculation unit that learns the features of forward and reverse data and enables inference in the forward and reverse directions. This makes it possible to switch the model's inference direction (forward and reverse) and movement pattern based on abnormal conditions or human instructions, and generate autonomous recovery movements.

[0024] The terminology used in the robot control device according to this embodiment is explained below.

[0025] Sensor data refers to the measurement values ​​of sensors that measure the state of each driving part (actuator) of the robot, and the measurement values ​​of sensors that are attached to the outside of the robot and measure the surrounding situation. Examples of the measurement values ​​of sensors that measure the state of each driving part include joint angles, current values, torque values, tactile values, camera images, etc. Examples of sensors that measure the surrounding situation include cameras and motion capture.

[0026] The sequential data is a series of sensor data obtained when a target motion is taught to a robot using an arbitrary motion teaching method. The sequential data is shown in a time series. The sequential data shown in a time series includes data measured at multiple times, and each measurement value at the multiple times is associated with another measurement value.

[0027] Reverse data is data that has been created by reversing (flipping) the chronological order of previously collected forward data. For example, if the time-series data of a certain image is saved as forward data, the data that is created by playing the forward data in reverse becomes reverse data.

[0028] A machine learning model is a mechanism for deriving a result (= output) from input data. In the case of a robot, it is a mechanism for predicting an action command value (result) based on sensor information (input data), and various methodologies can be used for configuring a specific model. The model in the present invention does not depend on a specific machine learning model such as a Recurrent Neural Network, Transformer, or Convolutional Neural Network. Note that in the present invention, the machine learning model, the action generation model, and the model are synonymous.

[0029] The autonomous recovery action includes at least one of the following three actions. The first is to retry the task to achieve the target action (hereinafter, recovery action 1). The second is to transition to another action learned in the past (hereinafter, recovery action 2). The third is to return from the current state to a specific state in the past (hereinafter, recovery action 3). Specifically, while forward inference predicts the action command value to perform the target action, backward inference predicts the action command to return from an arbitrary state to the initial state. In this way, if an abnormal state occurs during forward inference, it is possible to return to an arbitrary state using backward inference. While recovery actions 1 and 2 are autonomously performed by the robot based on sensor information, recovery action 3 can be an automatic return to a preset state or a state transition based on a human instruction.

[0030] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the drawings, the same components are designated by the same reference numerals, and detailed description of overlapping parts will be omitted. EXAMPLES

[0031] (Configuration of the robot control device) First Embodiment A robot control device and a robot control method according to a first embodiment of the present invention will be described with reference to FIGS.

[0032] FIG. 1 is a diagram showing an example of the configuration of a robot system 1 including an inverse data generating device 30 and a feature error calculation unit 45 in this embodiment. In this embodiment, a desired motion is taught to the robot device 10 using any motion teaching method such as teleoperation, and the robot sensor time series information measured at that time is stored in the forward data storage unit 21, and the machine learning device 40 learns using the inverse data generated by the inverse data generating device 30 and the forward data stored in the forward data storage unit 21. Motion teaching is a task of teaching a robot a target motion, and the sensor data of the robot is recorded at a constant sampling rate. Representative methods of motion teaching include programming description in which a motion is described and executed in advance using a programming language, direct teaching in which a person picks up a robot in a torque-free state or a self-weight guarantee mode and moves it, and teleoperation using a 3D mouse, VR (Virtual Reality), a haptic device, a remote control device of the same or different structure, etc. In the following, the processing contents of each device and a method of generating autonomous motion are described. (Robot configuration) The robot device 10 may be configured, for example, by a multi-joint robot. A measurement unit 11 of the robot device 10 measures sensor values ​​such as sensor information mounted on the robot, joint angle and torque information, RGB color images, depth images (distance images), and monochrome images. A drive unit 12 of the robot device 10 generates motion by changing the angles of each joint. The manipulator may be driven by an electric motor, or may be driven by an actuator that operates with fluid pressure such as hydraulic pressure or pneumatic pressure. The drive unit 12 is driven according to motion instruction information output from an information processing device. In addition, the drive unit 12 is not limited to a multi-joint robot, and may be a movable machine capable of numerical control (NC). (storage device) The robot system 1 is equipped with a storage device 20 having a forward data accumulation unit 21 that stores robot sensor time series information (forward data) and a forward / reverse data accumulation unit 22 that stores the forward data and reverse time series data (reverse data) in association with each other. (Inverse data generation method) In this embodiment, the inverse data generating device 30 changes the inverse data generating processing method depending on whether the data is reversible or irreversible among the sensor information (camera and joint angle, torque information, contact information, etc.) collected in advance. Fig. 2 shows an example of inverse data generated from forward data, and the relationship between the forward data and the reverse data is linked and stored in the forward / reverse data storage unit 22 in order to allow the feature error calculation unit 45 to learn the correspondence between the forward data and the reverse data.

[0033] Here, reversible data refers to information that does not have time characteristics, such as images and joint angles, and whose time series order can be rearranged. A method for verifying reversible data is to temporarily store inverse data A generated by rearranging the time series of forward data A collected in advance, for example images and joint angles measured when a joint angle is applied to a robot, and rearrange the time series to generate forward data B. At this time, information in which there is no significant difference in the values ​​of the original forward data A and forward data B generated by the inverse data is considered to be reversible data. Note that forward data and inverse data do not have exactly the same values ​​due to subtle noise that occurs in the real world.

[0034] On the other hand, irreversible data is information that has hysteresis characteristics, such as torque information and tactile information, and cannot simply be rearranged in chronological order. The method of verifying irreversible data is to compare the values ​​of the original forward data A and the forward data B generated by the reverse data, just like reversible data, and to classify information with different characteristics as irreversible data. For example, torque information depends on the environment, such as the direction of gravity, the direction of movement, and speed, as well as past inputs.

[0035] In the present invention, the lossless data processing unit 31 generates inverse data by rearranging the time series order of the lossless data. The irreversible data processing unit 32 generates inverse data by rearranging the time series order after applying linearization, smoothing, etc. to the irreversible data. By performing processes such as smoothing, the irreversible data is converted into lossless data in a pseudo manner.

[0036] 1 can be realized by a hardware configuration including, for example, a central processing unit (CPU) and a recording device such as a memory, a hard disk, or a solid state drive (SSD). In this case, the functions of the inverse data generation device 30 and the machine learning device 40 are realized by interpreting and executing a program installed in the CPU, and the function of the storage device 20 is realized by a recording device such as a memory.

[0037] In addition, the storage device 20 may be placed in a location separate from the inverse data generation device 30 and the machine learning device 40, and configured to be connected to the inverse data generation device 30, the machine learning device 40, and the robot device 10 via a computer network such as the Internet.

[0038] Although FIG. 1 shows a configuration example in which the feature error calculation unit 45 and the command unit 47 are included in the machine learning device 40, they may be arranged independently of the machine learning device 40. (Model learning method 1) In this embodiment, the machine learning device 40 calculates (learns) optimal parameters of the model 41 in a learning unit 44 according to an objective function using a model definition unit 43 in which the structure of a model specified in advance by a user is defined, and learning data accumulated in the forward / reverse data accumulation unit 22. The optimal parameters obtained by learning are stored in a weight storage unit 42 of the machine learning device 40.

[0039] The objective function used in the error calculation unit 46 of the learning unit 44 can be set by the user according to the purpose, and may be a mean square error, etc. In this embodiment, parameters of a model constituting the machine learning device 40 are optimized so as to predict an operation command value of the robot device 10 at time t+1 from various sensor information at time t.

[0040] Input / output times and time spans can be set arbitrarily. Specifically, it is possible to predict the operation command value at time t+10 from sensor information at time t, and to predict the operation command value at a specific time or the operation command value at multiple times (t+1, t+2, t+3) from sensor information at multiple times (t-2, t-1, t).

[0041] The command unit 47 inputs the predicted operation command value to the drive unit 12 of the robot device 10, thereby making it possible to generate an operation based on the predicted command value. (Model learning method 2) In this embodiment, the feature error calculation unit 45 performs correspondence learning of the internal representations of the forward data and the reverse data. Fig. 3 shows the calculation method of the feature error calculation unit 45 and the internal representation obtained thereby. The forward data / reverse data pairs stored in the forward / reverse data storage unit 22 are taken out and input to the model 41 to perform learning.

[0042] In general machine learning, forward data and reverse data are learned as data with completely different characteristics, so features such as those shown in internal state 451 are formed self-organically inside the model. Therefore, although it is possible to generate the movements of the forward data and reverse data from the starting point to the end point, it is difficult to generate the reverse movement using the features of the reverse data during the forward movement generation.

[0043] Therefore, the inventors have devised a new error function that associates the internal representations of forward data and reverse data. Specifically, the feature quantities (f0, f1, ... f T ) and the inverse data features (b0, b1, … b T ) is set as a new error function so that the distance between the two is small.

[0044] By training the model to reduce the distance between the features of the forward data and the reverse data using a mean square error or the like, features such as those shown in internal state 452 are formed in a self-organizing manner inside the model. Since the features of the forward data and the reverse data are close, it becomes easy to generate a reverse action by utilizing the features of the reverse data during the generation of a forward action. In the present invention, autonomous recovery actions are generated by utilizing these features.

[0045] The error function set by the feature error calculation unit 45 is added to the error function defined in the above model learning method 1, and the model 41 is optimized (learned) as one error function. (Movement generation flow) The motion generation flow S10 of the robot system 1 will be described below with reference to Fig. 4. In the process of Fig. 4, a machine learning device 40 is used to predict a command value by inputting sensor information measured by the measurement unit 11 to a model 41, and the command value is transmitted to the drive unit 12 via a command unit 47. The robot device 10 generates sequential motions to achieve a target motion. The entire flow will be described below.

[0046] When the process starts, first, in step S11, a machine learning model defined by the user is read from the model definition unit 43.

[0047] Next, in step S12, various parameters constituting the machine learning model acquired by the action learning are read from the weight storage unit 42.

[0048] Next, in step S13, various pieces of sensor information measured by the measurement unit 11 are acquired.

[0049] Next, in step S14, an operation command value is predicted using the various sensor information acquired in step S13 and the machine learning device 40. The predicted operation command value is transmitted to the drive unit 12 via the command unit 47, thereby driving the actuator.

[0050] Finally, in step S15, it is determined whether the target motion has been achieved, and if the target motion is in the middle of being performed (No), the process returns to step S13. If it is determined that the target motion has been achieved (Yes), the process ends.

[0051] By repeating the above steps, the robot device 10 can generate successive movements based on the sensor information measured by the measurement unit 11 until a target movement is achieved. (Autonomous recovery action generation) In this embodiment, the robot learns multiple operation patterns, and autonomously switches operations when an abnormality occurs. Figure 5 shows an example of autonomous recovery operation based on sensor information.

[0052] By having the robot learn in advance the three-step door-opening action for a push door and a sliding door ((1) reach, (2) grasp, (3) push or (4) pull), the characteristics of two types (push door / sliding door) are self-organized inside the model 41 of the machine learning device 40, as shown in the internal state 401.

[0053] The features of the two types of movements (push door / sliding door) that are originally different are formed as shown in internal state 451 in Figure 3, but by using the feature error calculation unit 45 of the present invention, the internal states of the common tasks ((1) reach, (2) grasp) draw similar trajectories.

[0054] On the other hand, the actions of (3) pushing and (4) pulling are different, so the internal state branches. Here, the robot is made to make forward inference in an environment where it is not visually clear whether it is a push door or a sliding door. Forward inference is to predict the action yt+1 to be taken at the next time t+1 based on the sensor information xt at a certain time t. It is equivalent to predicting the near future state and its command value from the current state. The robot can reach and grasp the doorknob, but since the doorknob does not open even if it is pushed after that, the load information measured by the torque sensor increases and an abnormal state is detected. At this time, as shown by the thick line in the internal state 401, the robot returns to the branching point by backward inference and is able to autonomously perform a recovery action by performing the action of (4) pulling. Backward inference is to predict the action yt-1 to be taken at the previous time t-1 based on the sensor information xt at a certain time t. It is equivalent to predicting the state one time ago and its command value from the current state.

[0055] Switching from forward inference to backward inference, and switching from (3) pushing action to (4) pulling action, are performed automatically within the model. For example, the Recurrent Neural Network automatically performs behavioral transitions that minimize errors (states where abnormal states are unlikely to occur) due to the model's unique pull-in effect, so by using this function, inference and action switching can be achieved.

[0056] In a typical robot system, when an abnormality occurs, the robot must return to the initial position and restart the task, which reduces work efficiency. On the other hand, in the present invention, when an abnormality is detected, it is possible to generate an immediate recovery action without returning to the initial position, which is expected to improve the operating rate of the robot.

[0057] The above embodiment makes it possible to execute autonomous operations based on past learning experiences and autonomous recovery operations in the event of an abnormality. The past learning experiences refer to features that are formed in a self-organized manner inside the model when a machine learning model is trained using forward data, reverse data, or both data collected in advance. For example, when an abstract representation of the training data is trained inside the model and a robot operation is trained, motion patterns such as "push, pick up, pull" are stored as features. When generating robot operations, the motion patterns obtained during training are fine-tuned based on current sensor information to generate motions appropriate for the situation. Since the internal state of the model is high-dimensional information, the features (motion patterns) of the internal state can be visualized by using a dimension reduction method such as principal component analysis. Furthermore, in the above embodiment, while a model is generally trained using only forward data, by adding reverse data training, the following three effects can be obtained.

[0058] (1) By learning from forward and reverse data, it is possible to emphasize the chronological relationships of the data (the order and chronological relationship in the time series), enabling robust and highly accurate motion generation (inference) even when disturbances or changes in lighting occur.

[0059] (2) When learning forward data in which some sensor data is missing (outliers) or where the object in view is occluded (occlusion), model learning becomes unstable because the time series data changes suddenly or the target object is occluded. By training a model using forward and inverse data, such abnormal states can be compensated for, resulting in stable learning.

[0060] (3) Stable inference is possible even if occlusion occurs while the robot is working autonomously or if some of the sensor data is noisy or missing. EXAMPLES

[0061] Second Embodiment A robot control device and a robot control method according to a second embodiment of the present invention will be described with reference to FIG. (System Configuration) 6 is a diagram showing an example of the configuration of a machine learning device 40 including an index estimation unit 48 that outputs the degree of confidence in the work of the robot in this embodiment and estimates an action index based on the degree of confidence. If the robot is taught two types of actions (push door / sliding door), the machine learning device 40 is made to learn using the action index in addition to the robot information. This makes it possible to learn (store) an action for each index, and when generating an action, it is possible to make the robot execute a desired action pattern by inputting the index of the action to be executed into the model 41.

[0062] For example, by making the machine learning device 40 learn by setting the door-opening action index for a push-door type to 0 and the door-opening action index for a sliding-door type to 1, a push-door type door-opening action is generated by inputting 0 into the action index during inference. The action index can be a numerical value defined by a person in advance, or linguistic information, and can also be determined by learning. (Autonomous recovery action generation based on confidence level) The confidence level in this embodiment is an index that indicates the accuracy of a predicted value, and is realized by various implementation methods. As an example, the average and variance of the joint angle trajectory can be considered. If the information transmitted to the driving unit 12 via the command unit 47 of the machine learning device 40 is the joint angle of the robot, the machine learning device 40 predicts the average of the joint angle as a control command, and predicts the variance of the joint angle as the confidence level. In a situation close to the learning environment, the robot can predict an operation command value with a small prediction variance. On the other hand, when the confidence level is low, the prediction variance of the robot becomes large. This function can be realized by using a probability model that can calculate the average and variance.

[0063] When the confidence level (prediction variance) predicted by the model falls below (exceeds) a specific threshold, the index estimation unit 48 can execute an autonomous recovery operation by switching to an appropriate operation index. As an example of application to an actual machine, for example, an operator mistakenly inputs a push door index into the model, and the robot executes an operation based on sensor information. However, since an actual door is a sliding door, the door does not open after the doorknob is grasped. At this time, the confidence level output by the model becomes small (the prediction variance becomes large), so when the variance value exceeds a predetermined threshold, the index estimation unit 48 switches to a sliding door operation, and the robot can instantly switch operations. Since the operation can be switched immediately without returning to the initial position, this embodiment can also be expected to improve the robot operating rate. EXAMPLES

[0064] Third Embodiment A robot control device and a robot control method according to a third embodiment of the present invention will be described with reference to FIG. (System Configuration) 7 is a diagram showing an example of the configuration of a machine learning device 40 including an instruction understanding unit 49 that understands abstract instructions such as voice and text in this embodiment. In this embodiment, a backward inference is executed for a step (time) instructed by an operator for a robot in forward inference. The instruction understanding unit 49 understands the number of steps to be executed from the operator's abstract instruction, and inputs the value to the machine learning device 40.

[0065] This makes it possible, if an abnormal state occurs, to move the machine learning device 40 to a specified position and issue new operation instructions or instructions to retry a task from there.

[0066] From an engineering perspective, it is possible to move to a given position by storing a past trajectory and replaying the trajectory in reverse. However, since the internal state of the machine learning device 40 cannot be changed (updated) by replaying the trajectory in reverse, it is possible to simultaneously replay the trajectory in reverse and update the internal state by performing the backward inference disclosed in the present invention. Manifestation An example of a method for confirming whether or not the present invention is implemented will be described below. However, the method for confirming the manifestation is not limited to this.

[0067] Internal state 451 and internal state 452 in FIG. 3 are an example of the visualization result of dimensional compression of high-dimensional features (internal states) formed inside a machine learning model using principal component analysis or the like. As a result of learning forward data and reverse data related to a certain specific operation, internal state 451 is an internal state when the present invention is not used, and different internal states are learned for the forward data and reverse data. On the other hand, as shown in internal state 452, by using the present invention, the internal states of the forward data and reverse data match. This is because the feature error calculation unit 45 learns (matches features) so that the internal states of the forward data and reverse data match.

[0068] Figure 8 shows an example of robot behavior in response to environmental changes. A robot that has learned to grasp objects on a conveyor belt where objects flow from left to right generates autonomous motion to grasp the target object under normal circumstances, as shown in the left figure. On the other hand, as shown in the right figure, when the direction of the conveyor belt is reversed (from right to left), the robot generates a motion to return to its initial position. This is because the state (direction of movement) of the conveyor belt visually corresponds to reverse data, so the robot generates a motion of reverse inference. As the robot's behavior changes in response to environmental changes in this way, it is possible to confirm the salience.

[0069] The present invention is not limited to the above-described embodiments, and includes various modified examples. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the configurations described. It is also possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to add, delete, or replace a part of the configuration of each embodiment with another configuration. [Explanation of symbols]

[0070] 1. Robot system 10. Robot device 11…Measuring section 12...Drive unit 20…Storage device 21... Forward data storage section 22...Forward and reverse data storage unit 30...Inverse data generation device 31...Reversible data processing unit 32...Irreversible data processing section 40...Machine learning device 41…Model 42...Weight storage section 43...Model definition section 44…Learning Department 45…Feature error calculation unit 46...Error calculation section 47…Command Department 48…Index Estimation Section 49…Instruction understanding section 401,451,452...Internal state.

Claims

1. an inverse data generating device that generates inverse data, which is reverse time-series data, using forward data, which is measured robot sensor time-series information; a feature error calculation unit that learns features of the forward data and the reverse data and infers a correspondence relationship between the forward direction and the reverse direction; a machine learning device that infers forward and reverse robot motions based on robot sensor information; A command unit that outputs a command value of the machine learning device to a robot; A robot control device comprising:

2. The robot control device according to claim 1 , The machine learning device is a robot control device that autonomously infers the robot movements in the forward and reverse directions without performing complex work procedures and rearrangements by learning multiple operational experiences.

3. The robot control device according to claim 1 , The inverse data generation device classifies the robot sensor time series information into reversible data and irreversible data based on the characteristics of the forward data including hysteresis, and generates inverse data according to the characteristics of each data.

4. The robot control device according to claim 1 , A robot control device characterized by comprising a forward / reverse data storage unit that generates reverse data from forward data collected in advance and stores the forward data and reverse data in association with each other.

5. The robot control device according to claim 1 , The robot control device is characterized in that the feature error calculation unit adds a constraint to an error function so that the internal state of the model of the machine learning device matches between forward data and inverse data.

6. The robot control device according to claim 1 , A robot control device comprising: an index estimation unit that outputs a degree of confidence in a robot's work and estimates a motion index based on the degree of confidence.

7. The robot control device according to claim 1 , A robot control device comprising an instruction understanding unit that understands abstract instructions including voice and text.

8. The robot control device according to claim 4, The machine learning device includes: a model definition unit in which a structure of a model specified in advance by a user is defined; A learning unit that learns optimal parameters of the model, A robot control device, characterized in that the learning unit learns optimal parameters of a model according to an objective function using the model definition unit and learning data stored in the forward and reverse data storage unit.

9. The robot control device according to claim 4, The feature error calculation unit extracts pairs of forward data and reverse data stored in the forward / reverse data storage unit, and inputs them into a model to learn correspondence between internal representations of the forward data and the reverse data.

10. Robots and a robot control device for controlling the robot, The robot comprises: A measurement unit that measures an external environment and transmits the measured information to the robot control device; a drive unit that drives the robot based on a command value output from the robot control device, The robot control device includes: a sequential data storage unit that stores sequential data, which is robot sensor time-series information measured by the measurement unit; an inverse data generating device that generates inverse data, which is inverse time series data of the forward data, using the forward data; a forward / reverse data storage unit that stores the forward data and the reverse data in association with each other; a feature error calculation unit that learns features of the forward data and the reverse data and infers a correspondence relationship between the forward direction and the reverse direction; a machine learning device that infers forward and reverse robot motions based on robot sensor information; A robot control system comprising: a command unit that outputs a command value of the machine learning device to the robot.

11. A robot control device; A measurement unit that measures an external environment and transmits the measured information to the robot control device; a drive unit that drives the robot based on a command value output from the robot control device, The robot control device includes: a sequential data storage unit that stores sequential data, which is robot sensor time-series information measured by the measurement unit; an inverse data generating device that generates inverse data, which is inverse time series data of the forward data, using the forward data; a forward / reverse data storage unit that stores the forward data and the reverse data in association with each other; a feature error calculation unit that learns features of the forward data and the reverse data and infers a correspondence relationship between the forward direction and the reverse direction; a machine learning device that infers forward and reverse robot motions based on robot sensor information; A robot comprising: a command unit that outputs a command value of the machine learning device to the robot.

12. (a) loading a user-defined machine learning model; (b) learning optimal parameters of the machine learning model according to an objective function using the machine learning model read in the (a) step and pre-accumulated learning data; (c) acquiring robot sensor information; (d) predicting an operation command value using the machine learning model learned in the (b) step and the robot sensor information acquired in the (c) step; having A robot control method, characterized in that in the step (d), forward inference and backward inference of the robot's operation are performed.

13. The robot control method according to claim 12, A robot control method, characterized in that in the step (b), a plurality of motion experiences are learned and forward inference and backward inference are performed.

14. The robot control method according to claim 12, A robot control method, characterized in that the learning data in step (b) includes inverse data, which is inverse time series data generated by classifying the robot sensor time series information into reversible data and irreversible data based on the characteristics of forward data, which is measured robot sensor time series information, including hysteresis, and generating inverse data in accordance with the characteristics of each.

15. The robot control method according to claim 12, a robot control method, characterized in that the learning data in step (b) is learning data that is generated by generating inverse data, which is reverse time-series data, from forward data, which is robot sensor time-series information collected in advance, and storing the learning data by linking the relationship between the forward data and the inverse data.

16. The robot control method according to claim 12, A robot control method characterized in that in the step (d), a constraint is added to an error function so that the internal state of the machine learning model is consistent between forward data and inverse data.

Images