Robot control device, robot control system, and robot control method

The robot control device and method automatically calculate motion transition probabilities and generate sequences by comparing pre- and post-motion information, addressing the inefficiencies of manual learning in existing systems, thereby improving robot control efficiency and accuracy.

JP2024157873APending Publication Date: 2024-11-08HITACHI LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023072499
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-26
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

Existing robot control systems require time-consuming state transition model updates due to numerous state and command candidates, necessitating manual measurement and learning for each transition estimation.

Method used

A robot control device and method that calculates motion transition probabilities and generates a series of motions by comparing pre- and post-motion information using similarity calculations, eliminating the need for manual specification or prior learning of transition probabilities.

Benefits of technology

Enables automatic estimation of motion transitions and generation of executable motion sequences without the need for real-time measurement and learning, enhancing efficiency and accuracy in robot control.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024157873000001_ABST
    Figure 2024157873000001_ABST
Patent Text Reader

Abstract

To provide a robot control device capable of calculating the propriety of transition between operations and automatically determining the propriety of transition between a series of operations without programming and learning.SOLUTION: A robot control device for controlling an operation of a robot, includes: a storage unit that stores a plurality pieces of operation information in which the operation of the robot, pre-operation information which includes at least one of a pre-operation surrounding work environment of the robot and a pre-operation state of the robot, and post-operation information which includes at least one of a post-operation surrounding work environment and a post-operation state of the robot are associated with each other; a series-of-operations generation unit that generates a series of operations of the robot by using the degree of similarity between post-operation information pertaining to a certain operation and pre-operation information pertaining to another operation, a current operation state which includes at least one of the current surrounding work environment of the robot and the current state of the robot, and a target given to the robot; and a robot control unit that controls the robot on the basis of the series of operations of the robot which have been generated by the series-of-operations generation unit.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a robot control device, a robot control system, and a robot control method for controlling the operation of a robot. [Background technology]

[0002] In recent years, there has been an increasing demand for the automation of tasks using robots in the fields of manufacturing, logistics, services, construction, facility maintenance, etc. For example, when this type of robot needs to perform multiple tasks to achieve a goal, it is necessary to not only determine the actions required to perform each task, but also to estimate the state transitions resulting from each action, and then execute a series of actions in the order in which the transitions are possible.

[0003] Here, as a conventional technique for estimating an operating state transition during robot control, a learning device of Patent Document 1 is known. For example, claim 1 of Patent Document 1 discloses a learning device including: "a creation unit for creating an aggregated state transition model including a plurality of state transition models for predicting a next state of a controlled object based on a measured state of the controlled object and a command for the controlled object, and an aggregation unit for aggregating prediction results by the plurality of state transition models; a command generation unit for executing each process for each control period, which inputs a measured state of the controlled object, generates a plurality of candidates for a command or command sequence for the controlled object, obtains a plurality of states or state sequences of the controlled object predicted using the aggregated state transition model from the state of the controlled object and the plurality of candidates for the command or command sequence for the controlled object, calculates a reward corresponding to each of the plurality of states or state sequences of the controlled object, and generates and outputs a command that maximizes the reward based on the calculated reward; and a learning unit for updating the aggregated state transition model so that an error between a next state of the controlled object predicted in response to an output command and a measured state of the controlled object corresponding to the next state is reduced." [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2022-41294 A Summary of the Invention [Problem to be solved by the invention]

[0005] In the learning device of Patent Document 1, it was necessary to update the aggregate state transition model so as to reduce the error between the next state of the controlled object predicted in response to the output command and the measured state of the controlled object corresponding to the next state.

[0006] However, when there are many candidates for states and commands, it is necessary to update the state transition model each time using the next state measured under each state and command, which can take a long time to learn. For this reason, it is desirable to be able to automatically estimate whether a transition is possible without measuring and learning the state each time.

[0007] Therefore, an object of the present invention is to provide a robot control device and a robot control method that can calculate whether a transition between actions is possible and generate a series of actions that can be transitioned. [Means for solving the problem]

[0008] The robot control device according to the present invention is a robot control device that controls the movement of a robot, and includes a memory unit that stores multiple pieces of movement information that associate the movement of the robot, pre-movement information including at least one of the robot's working environment and the robot's state before the movement, and post-movement information including at least one of the working environment and the robot's state after the movement, a series of movements generation unit that generates a series of movements of the robot using the similarity between the post-movement information of a certain movement and the pre-movement information of another movement, a current movement state including at least one of the robot's current working environment and the robot's current state, and a goal given to the robot, and a robot control unit that controls the robot based on the series of movements of the robot generated by the series of movements generation unit.

[0009] In addition, a robot control method according to the present invention is a robot movement control method for controlling the movement of a robot, which uses a series of movements generation unit to generate a series of movements of the robot from a plurality of movement information correlating the movements of the robot stored in a memory unit, pre-movement information including at least one of the work surrounding environment and the state of the robot before the movement, and post-movement information including at least one of the work surrounding environment and the state of the robot after the movement, using a similarity between post-movement information of a certain movement and pre-movement information of another movement, a current movement state including at least one of the current operating environment of the robot and the current state of the robot, and a goal given to the robot, and uses a robot control unit to control the robot based on the series of movements of the robot generated by the series of movements generation unit. Effect of the Invention

[0010] According to the present invention, it is possible to provide a robot control device and a robot control method capable of calculating whether a transition between actions is possible and generating a series of actions into which a transition is possible. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a schematic diagram showing a usage environment of a robot control device according to a first embodiment. [Diagram 2] FIG. 2 is a block diagram showing a hardware configuration of the robot control device according to the first embodiment. [Diagram 3] FIG. 2 is a functional block diagram of the robot control device according to the first embodiment. [Figure 4] 11A and 11B are diagrams showing examples of a method for calculating a similarity between actions. [Diagram 5] FIG. 11 is a functional block diagram of a robot control device according to a second embodiment. [Figure 6] FIG. 11 is a diagram showing an example of a method for generating a series of actions. [Figure 7] FIG. 11 is a diagram showing an example of hierarchical operation information. [Figure 8] FIG. 13 is a diagram showing an example of a display screen that notifies a user of a transition failure and accepts a control input setting. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] First, before describing a robot control device and a robot control method according to an embodiment of the present invention, the premise of the embodiment of the present invention and the terminology used in the embodiment of the present invention will be described. In the present invention, a robot is controlled so as to complete a target task by executing a series of operations. In the present invention, a "state" refers to information about the robot itself at a certain point in time. The "state" includes, for example, information about the robot position and orientation measured by a measurement sensor described later. In addition, the "work environment" refers to information about the environment surrounding the robot at a certain point in time where the robot works. The "work environment" includes, for example, information about the work environment measured by a measurement sensor described later. In addition, the "state" and the "work environment" may be differential information in time and space of the work environment information and the robot position and orientation information, or may be information obtained by processing primary information such as the recognition result of items around the robot. In addition, an "operation" in the present invention refers to a transition of one or more states, and a "series of operations" refers to a sequence of operations consisting of a plurality of operations. A robot is controlled by giving a command to the robot to execute a predetermined operation.

[0013] Normally, to execute a series of actions, the next action must be executed following the previous action, but it is difficult to know in advance whether a certain action can be executed after another action. Therefore, in the present invention, the similarity between the post-action information of a certain action and the pre-action information of another action is calculated using action information that associates the action with post-action information on the state after the action and pre-action information on the state before the action, and a series of actions that can be executed is generated based on the calculated similarity.

[0014] Because the robot control device and robot control method according to the present invention are equipped with such a configuration, the similarity between actions can be automatically calculated, eliminating the need to manually specify or learn the transition probability between actions in advance.

[0015] Hereinafter, a robot control device and a robot control method according to an embodiment of the present invention will be described with reference to the drawings. In the drawings used in this specification, the same or corresponding components are designated by the same reference numerals, and repeated description of these components may be omitted. EXAMPLES

[0016] A robot control system including a robot control device 1 according to a first embodiment of the present invention will be described below with reference to FIGS.

[0017] FIG. 1 is a schematic diagram showing an environment in which a robot control device 1 is used. In the figure, a robot 2 is a multi-joint arm robot having a multi-joint arm 21 and a hand 22, and is controlled by a robot control device 1 connected to the robot control device 1 so as to be communicable by wire or wirelessly. The robot 2 may be an orthogonal type robot. The robot 2 may be a device that incorporates the robot control device 1. The robot 2 may be a mobile robot. The measurement sensor 3 is a sensor such as a camera, a distance sensor, or a microphone that measures the working environment of the robot 2. The measurement sensor 3 is connected to the robot control device 1 so as to be communicable by wire or wirelessly. The measurement sensor 3 may be a device that is incorporated in the robot 2. The items 4 are various items such as a cupboard 4a, a kettle 4b, a faucet 4c, and a stove 4d that are handled by the robot 2. The instruction input device 5 is used when a user instructs the robot 2 to perform an operation. The instruction input device 5 is, for example, a human interface device such as a keyboard, a mouse, or a liquid crystal display, or a smartphone, a tablet, or the like. The instruction input device 5 is connected to the robot control device 1 so as to be communicable with each other via wire or wirelessly.

[0018] 1, the robot 2 is installed at a location where it can grasp or operate any object 4 with the hand 22 by moving the articulated arm 21, and the measurement sensor 3 is installed at a location where it can capture an image of the entire working environment of the robot 2. The measurement sensor 3 may be directly attached to the robot 2, or may be a joint angle sensor, acceleration sensor, tactile sensor, etc. built into the robot 2.

[0019] 2 is a block diagram showing a hardware configuration of the robot control device 1. As shown in the figure, the robot control device 1 is a computer including a processor 11 such as a CPU, a storage device 12 such as a semiconductor memory, an input unit 13 that receives information from an instruction input device 5, an output unit 14 that transmits information to the instruction input device 5, a communication interface 15 that communicates with the robot 2 and the measurement sensor 3, and a bus 16 that connects them. Note that the processor 11 executes a predetermined robot control processing program 12a to realize each of the functional units described below, but the following will explain the details of the present invention while omitting explanations of well-known techniques of this type.

[0020] 3 is a functional block diagram of the robot controller 1. As shown in this figure, the robot controller 1 stores pre-action information I relating to a state before an action of the robot 2 when the robot 2 executes an action. b and post-operation information I aThe robot control unit 11c includes a storage unit 12b that stores the motion information m that associates the motions, a similarity calculation unit 11a that calculates the similarity between the motions based on the motion information m received from the storage unit 12b, a sequence of motion generation unit 11b that generates a sequence of motions that can be executed using the calculated similarity, a current motion state including at least one of the current motion surrounding environment of the robot and the current state of the robot, and a target given to the robot, and a robot control unit 11c that requests a command to execute the generated sequence of motions. The command obtained by the robot control unit 11c is transmitted to the robot 2 via the communication interface 15 to control the robot 2. Note that, in the first embodiment, the similarity calculation unit 11a that calculates the similarity between the motions based on the motion information m is provided, but the storage unit 12b may store the similarity between the motions calculated in advance based on the motion information m.

[0021] Pre-operation information I b The post-operation information I is information on at least one of the work environment before operation and the state of the robot 2. If both are provided, the robot control described below can be performed with higher accuracy. a is information about at least one of the working environment after operation and the state of the robot 2. If both are provided, the robot control described below can be performed with higher accuracy.

[0022] For example, when the measurement sensor 3 is a camera, the storage unit 12b stores an image P, such as an RGB image, a gray image, or an infrared image, captured by the camera as pre-operation information I. b and Post-operation information I a In addition, for example, when the measurement sensor 3 is a distance sensor, the storage unit 12b stores the depth image, point cloud data, etc. measured by the distance sensor as pre-operation information I b and Post-operation information I a The distance sensor may be, for example, a LiDAR, a laser displacement meter, a stereo camera, an infrared distance meter, a laser distance meter, or a TOF distance sensor. For example, when the measurement sensor 3 is a microphone, the storage unit 12b stores voice data recorded by the microphone as pre-operation information I. b and Post-operation information Ia In addition, for example, if the measurement sensors are joint angle sensors, acceleration sensors, and tactile sensors built into the robot 2, the storage unit 12b stores the joint angle data, acceleration data, and tactile data measured by each sensor as pre-motion information I b and Post-operation information I a Obtain as.

[0023] The pre-operation information I stored in the storage unit 12b b and Post-operation information I a does not have to be the information measured by the measurement sensor 3 itself, but may be information obtained by processing the information. For example, it may be differential information between measurement information such as the velocity or acceleration of a joint angle, or optical flow calculated from time-series images. In addition, for example, an image that has been subjected to homography transformation to match the viewpoint as seen from a predetermined coordinate system may be used, or information normalized based on the average and standard deviation of measurement information obtained in advance may be used.

[0024] Also, pre-operation information I b and Post-operation information I a may be, for example, a feature vector, an attention area, and an article recognition result extracted from the measurement information. Examples of the method of extracting the feature vector, calculating the attention area, and recognizing the article include extraction of feature points and feature vectors using methods such as SIFT and ORB, calculation of the attention area using a saliency map, deep learning using methods such as R-CNN, YOLO, SSD, and Vision Transformer, segmentation using methods such as Mask R-CNN and GAN, and grip point recognition using methods such as Dex-Net.

[0025] In addition, pre-operation information I b and Post-operation information I amay be a set of measurement information obtained by measuring multiple times, or may be a combination of multiple types of measurement information. It may also be a probability distribution of measurement information calculated based on these sets of measurement information. Examples of methods for calculating a probability distribution from a set of measurement information include maximum likelihood estimation and posterior distribution estimation using a Gaussian distribution or a mixture of Gaussian distributions, or probability distribution estimation in a feature space using a deep learning method such as VAE.

[0026] The storage unit 12b may also store control information related to a control law in the robot control unit 11c described later in association with the control information. Examples of the control information include the torque train of the robot 2 during operation, the target value of the joint angle of the robot 2, and the placement position of the article 4. In addition, for example, when a learning method such as a neural network is used as the control law, the weight of the neural network learned in advance may be stored as the control information. In addition, when multiple control laws exist, the name of the control law or an index for identifying the control law may be stored as the control information.

[0027] The storage unit 12b may also store performance information indicating the content performed by an action in a linked manner. An example of the performance information is detailed linguistic information regarding the effect or state achieved by the action. For example, the action information m of turning on the stove can be linked to a plurality of linguistic information such as boiling water and grilling ingredients as performance information. Note that the pre-action information I corresponding to such linguistic information may be stored in a form similar to the above. b and Post-operation information I a may be classified and then the motion information m may be stored.

[0028] Even if performance information is not linked and stored in the memory unit 12b, it is possible to infer it from linguistic information, etc. using machine learning, and to grasp the current state of the robot 2 and the working environment around the robot 2.

[0029] Also, a state input unit that receives a current state is newly provided, and the action sequence generating unit 11b can generate an action sequence based on the current state and the similarity degree received from the input unit.

[0030] Furthermore, in the robot control device 1, the current state is acquired through the communication interface in the form of measurement information obtained from the above-mentioned measurement sensor 3 and sensors built into the robot.

[0031] Feedback control such as PID control is an example of a method for calculating a robot's command value from the motion and current state. Alternatively, the robot's command value may be calculated using a control rule that has been learned in advance. A reinforcement learning method such as Q-learning is an example of a method for learning a control rule. Alternatively, a deep reinforcement learning method such as DQN or A3C may be used.

[0032] By grasping the current operating state of the robot 2 and the working environment around the robot 2, it becomes possible to flexibly generate a series of operations according to the current state.

[0033] In addition, a new goal input unit that receives a goal may be provided, and the sequence of actions generating unit 11b may generate a sequence of actions to reach a specified goal based on the similarity calculated by the similarity calculation unit 11a and the target actions and states received from the goal input unit.

[0034] When the goal received from the goal input unit is information such as an action name specifying a target action, or linguistic information indicating a target state, a group of actions having performance information matching the specified action and the specified linguistic information is set as the target action, and a series of actions leading to the target action is calculated.

[0035] In addition, when the goal is a goal state, the action that can reach the goal state is set as the goal action, and a series of actions that reach the goal state are generated. For example, the post-action information I a A method is to set the target action as the action with the highest similarity to the target state. a If is the probability distribution of the measured information, then the post-movement information I a By using the probability of generating the goal state in as the probability of reaching the goal state, it is possible to calculate an action that can reach the goal state.

[0036] Examples of methods for calculating a sequence of actions that reach a target action from similarity and a target action include the A-star (A*) algorithm and the RRT-star (RRT*) algorithm. Examples of methods for calculating a sequence of actions that reach a target action from transition probability and a target action include estimation by dynamic programming such as LMDP (Linearly-solvable Markov Decision Process), active inference, and belief propagation.

[0037] <Details of Similarity Calculation Unit 11a> The similarity calculation unit 11a first calculates post-action information I a and pre-action information I included in the action information of another action b For example, in the example of FIG. 4 in which the actions a1 "take out the kettle 4b from the cupboard 4a," a2 "operate the faucet 4c to pour water into the kettle 4b," and a3 "turn on the stove 4d" are available as executable actions of the robot 2, first, starting from the action a1, the post-action information I 1a and pre-action information I included in the action information m2 of action a2 b2 and the post-action information I 1a and pre-action information I included in the action information m3 of action a3 b3 Similarly, the similarity between the pre- and post-motion information when the starting point is the motion a2, and the similarity between the pre- and post-motion information when the starting point is the motion a3 are also calculated.

[0038] In this example, the post-action information I 1a Image P registered in 1a Both of the images show the state in which an empty kettle 4b is placed on the pot placement section of the stove 4d. 2b Image P registered in 2b Both of the images show the state in which an empty kettle 4b is placed on the pot placement section of the stove 4d. On the other hand, the pre-action information I 3b Image P registered in 3bBoth of the images are taken when the full kettle 4b is placed on the pot placement section of the stove 4d. 1a and image P before action a2 2b The image P after the action a1 is very similar. 1a and image P before action a3 3b In the example of FIG. 4, the post-action information I of the action a1 is different. 1a and pre-operation information I for operation a2 2b The similarity of the action a1 is determined to be "high", and the post-action information I 1a and pre-operation information for operation a3 I 3b The similarity between the motion information m is determined to be "low". If there is a possibility that the robot 2 repeats the same motion, the similarity between the same motion information m may be calculated. Also, a subset of the motion information m to calculate the similarity may be specified in advance among the motion information m stored in the storage unit 12b.

[0039] As a specific example of a similarity calculation method, for example, b and Post-operation information I a When a single piece of measurement information or multiple pieces of measurement information are used, examples of similarity include the cosine similarity between the pieces of measurement information, its average value, and its maximum and minimum value. Alternatively, the degree of dissimilarity, that is, the dissimilarity, may be calculated by first finding the Euclidean distance between the pieces of measurement information, and its average value, maximum and minimum value, and then the similarity may be calculated from the dissimilarity. Examples of methods for calculating similarity from dissimilarity include the inverse of dissimilarity and the value of the Poisson distribution with dissimilarity as a variable.

[0040] As another example of a similarity calculation method, for example, b and Post-operation information I a If the measured information consists of a probability distribution, the dissimilarity can be calculated by calculating the Kullback-Leibler (KL) distance and the Wasserstein distance, which are the distances between the probability distributions, and the similarity can be calculated from the dissimilarity as before. a Pre-operation information of measurement data obtained by sampling one or more from I bThe generation probability and its average value, maximum and minimum values ​​can also be used as similarity measures.

[0041] As described above, the similarity may be calculated by the similarity calculation method based on processed information of measurement information, such as difference information or an object recognition result, instead of measurement information.

[0042] <Details of the sequence of actions generating unit 11b> Next, a process performed by the action sequence generator 11b when generating an action sequence using the similarity will be described in detail.

[0043] The sequence of actions generating unit 11b generates a sequence of actions based on the similarity calculated by the similarity calculating unit 11a. An example of a method for generating a sequence of actions will be described. The sequence of actions generating unit 11b generates a sequence of actions for connecting the current state and the goal of the robot using the similarity, with the current state including either the robot's work environment or the robot's state received from the state input unit as a starting point and the robot's goal received from the goal input unit as an end point. The sequence of actions generating unit 11b may use the state and goal of the robot that the robot 1 infers from the work environment or the like.

[0044] <Details of the robot control unit 11c> Finally, the details of the process performed by the robot control unit 11c will be described. The robot control unit 11c calculates commands for controlling the robot based on the series of movements obtained by the series of movements generation unit 11b. The robot control unit 11c calculates commands corresponding to the series of movements by sequentially obtaining command values ​​for the robot required to execute each movement in the series of movements using a control rule. As an example of the control rule, for example, the pre-movement information I b Information after operation from I a Examples of trajectory planning methods include the A* algorithm and RRT, which plan a trajectory to reach the target.

[0045] When calculating commands from the control rules, the above-mentioned control information may be used. For example, a method may be used in which the torque sequence of the robot stored as the control information is output as is. In addition, a command value may be calculated using PID control or the like based on the sequence of target values ​​of the robot's joint angles stored as the control information.

[0046] In addition, a machine learning method such as a neural network may be used as the control rule. The weights of the neural network stored as the control information are used to calculate the pre-operation information I b and post-operation information I a The command values ​​can be found by directly estimating the corresponding command values ​​from

[0047] <Effects of this embodiment> As described above, according to the robot control device of this embodiment, the post-operation information I a and other pre-operation information I b Therefore, the robot control device of this embodiment can automatically calculate the transition probability between actions and generate a series of actions without manually specifying or learning the transition probability between actions in advance. EXAMPLES

[0048] Next, a robot control device 1 according to a second embodiment of the present invention will be described. Note that a description of commonalities with the first embodiment will be omitted to avoid duplication. Fig. 5 is a functional block diagram of the robot control device 1 according to the second embodiment. A probability calculation unit 11d that calculates a transition probability between actions based on the similarity calculated by the similarity calculation unit 11a is added to the robot control device 1.

[0049] <Details of the Probability Calculation Unit 11d> The details of the process performed by the probability calculation unit 11d will be described with reference to FIG. 4. In the following, the measurement sensor 3 is a camera, and the action information m (pre-action information I b , Post-operation information I a) is assumed to have images P captured by a camera registered together with information showing the respective probability distributions. For example, the stored information for the action a1 “take out” is a past image P captured before the execution of the action. 1b However, pre-operation information I b The concentric circles in the stored information in FIG. 4 are images that express the probability of each image depending on where the image is located within the circle, with the color becoming darker (representing a higher probability distribution) toward the center and lighter (representing a lower probability distribution) toward the periphery. Therefore, in this example, the image on the right, P 1b Image P on the left 1b It has been shown that the probability is high.

[0050] The probability calculation unit 11a calculates the transition probability between the actions based on the similarity. As in the above, the transition probability between the same action information m may be calculated.

[0051] As a method for calculating the transition probability based on the similarity, there is a method in which the transition probability is calculated by normalizing the similarity vector obtained by calculating the similarity between a certain action and all actions that can be transitioned from that action. In addition, the transition probability may be calculated from the value of the softmax distribution with the obtained similarity vector as a variable. In addition, the post-action information I a Pre-operation information I of measurement data obtained by sampling one or more b For example, the transition probability can be calculated by using the generation probability in i Pre-operation information I b Let p be the probability that image P is generated. i (P|I b ), operation information m i Post-operation information I a Let p be the probability that image P is generated. i (P|I a ), operation information m i The initial probability of p(m i ), and the motion information m i The posterior probability of p(m i |P), then the action information mi From operation information m j Transition probability p(m j |m i ) can be calculated using the following formula 1.

[0052]

number

[0053] When a prior distribution of transition probabilities exists, the transition probabilities may be calculated by finding posterior probabilities using the found transition probabilities. In addition, transitions between actions corresponding to similarities equal to or less than a predetermined threshold may be determined as impossible and the transition probabilities may be set to 0. A specific action may be set as a terminal condition, and all transition probabilities from that action to other actions may be set to 0.

[0054] To explain the process of this step based on the example of FIG. 4, in step S1, the post-action information I 1a and pre-operation information I for operation a2 2b The similarity of the action a1 is determined to be "high", and the post-action information I 1a and pre-operation information for operation a3 I 3b is determined to be "low," the probability calculation unit 11a determines that the transition probability from the action a1 to the action a2 is high and that the transition probability from the action a1 to the action a3 is low.

[0055] In addition, since the similarities between other actions are also calculated in step S1, the probability calculation unit 11a can ultimately determine that the transition probabilities from action a1 to action a2 and from action a2 to action a3 are high, and the transition probabilities from action a1 to action a1, from action a1 to action a3, from action a2 to action a1, from action a2 to action a2, from action a3 to action a1, from action a3 to action a2, and from action a3 to action a3 are low.

[0056] <Details of the sequence of actions generating unit 11b> Next, the details of the process performed by the sequence of actions generating unit 11b when generating a sequence of actions using transition probabilities will be described. Fig. 6 is a diagram showing an example of a method for generating a sequence of actions according to this embodiment. In Fig. 6, similar to Fig. 4, a "large" transition probability is indicated by a solid arrow, and a "small" transition probability is indicated by a dotted arrow.

[0057] The sequence of actions generating unit 11b generates a sequence of actions based on the transition probability calculated by the probability calculating unit 11d. An example of a method for generating a sequence of actions is to sample the next action according to the transition probability from a pre-specified starting action, and repeat this to generate a sequence of actions of a certain length. The next action may be uniquely selected from the maximum transition probability. Also, a probability distribution of the starting action may be set and the starting action may be obtained from the sample. In the example of FIG. 6, since the transition probability from action a1 to action a2 and from action a2 to action a3 are high, the sequence of actions generating unit 11b generates a combination of actions executed in the order of action a1, action a2, and action a3, surrounded by a thick dashed line, as a sequence of actions.

[0058] When generating a sequence of actions of any length, the length T of the sequence of actions may also be sampled from a pre-specified probability distribution, and a sequence of actions may be generated with a length equal to the sampled length T. In addition, when a sequence of actions that is a terminal condition exists, the sequence of actions may be generated by continuing to generate actions until the terminal condition is reached.

[0059] Examples of a method for calculating a series of actions that reach a target action from the transition probability and the target action include estimation using dynamic programming such as LMDP (Linearly-solvable Markov Decision Process), active inference, and belief propagation. A series of actions may be generated based on the hierarchical structure of action information m. FIG. 7 is a diagram showing an example of a method for generating a series of actions from hierarchical action information m in this embodiment. In the figure, the action information m1 of action a1 "take out" is hierarchically structured, and the information is further divided into action a1, action a2, action b3, action c4, action d5, action e6, action f7, action f8, action f9, action f10, action f11, action f12, action f13, action f14, action f15, action f16, action f17, action f18, action f19, action f20, action f21, action f22, action f23, action f24, action f25, action f26, action f30, action f31, action f40, action f51, action f42, action f52, action f60, action f71, action f82, action f93, action f94, action f95, action f116, action f126, action f130, action f140, action f150, action f160, action f251, action f261, action f27, action f32, action f44, action f55, action f46, action f56, action f61, action f72, action f83, action f94, action f95, action f151, action f162, action f173, action f184, action f195, action f252, action f265, action f31, action f45, action f56, action f46, action f57, action f68, action f71, action f85, action f 11 Action information for "Open the cupboard" 11 , operation a 12 Action information for "Put the kettle on the stove" 12 , operation a13 Action information for "Close the cupboard" 13 In this example, when the motion information m1 of the motion a1 “take out” is selected as one motion in the sequence of motions, it is possible to generate a sequence of motions from the hierarchical motion information m by replacing it with a sequence of motions generated based on the motion transition probability in the lower layer. EXAMPLES

[0060] Next, a robot control device 1 according to a third embodiment of the present invention will be described. Note that commonalities with the above-mentioned embodiments will be omitted to avoid duplication.

[0061] In the above embodiment, for example, if a motion transition fails when controlling the robot 2 based on the generated sequence of motions, it may become impossible to execute the subsequent sequence of motions.

[0062] Therefore, in this embodiment, a new action transition determination unit is provided that outputs an action transition determination result, which is the result of determining whether the action transition has been successful or not, and when it is determined that the action transition has failed, a series of actions is regenerated.

[0063] A method for determining whether an action transition has failed is, for example, to compare the state measured when a certain action in a series of actions is completed with the pre-action information I of the next action in the series of actions. b It can be determined whether the similarity between and is lower than a predetermined threshold. b If is the probability distribution of the measured information, then the pre-motion information I b Alternatively, the determination may be made based on whether the occurrence probability of the measured state in is lower than a predetermined threshold value.

[0064] As an example of a method for regenerating a series of actions, for example, a method is given in which the action before the transition in the transition determined to be a failure is used as a starting point and the remaining series of actions is regenerated using the transition probability. In addition, for example, a constraint may be set so that the same series of actions as when the transition failed is not generated by regenerating the series of actions after setting the transition probability of the action after the transition to 0. EXAMPLES

[0065] Next, a robot control device 1 according to a fourth embodiment of the present invention will be described. Note that a duplicated description of points common to the above-mentioned embodiments will be omitted.

[0066] In the above embodiment, for example, the pre-operation information I b and Post-operation information I a If there is a bias in the measurement information used as the transition probability, or if the work environment changes little by little, a difference may occur between the transition probability calculated by the probability calculation unit 11a and the actual transition probability.

[0067] Therefore, in this embodiment, a similarity update unit that updates the similarity or a transition probability update unit that updates the transition probability is newly provided, and the similarity or transition probability is updated based on the generated series of actions and the state measured when the series of actions is executed.

[0068] As an example of a method for updating the similarity, we use the pre-action information I b and Post-operation information I a Updated pre-operation information I b and Post-operation information I a For example, the pre-action information I b and Post-operation information I a is the probability distribution of the measured information, calculate the posterior probability distribution using the measured state, and use the obtained posterior probability distribution to calculate the pre-action information I b and Post-operation information I a may be updated.

[0069] Also, for example, when the action transition determination result can be acquired, the similarity may be directly updated based on the action transition determination result. One example of a method for directly updating the similarity is a method for lowering the similarity by using a predetermined value when it is determined that the transition has failed. Also, when a certain number of transition failures occur, the similarity between the target actions may be set to 0.

[0070] As an example of how to update the transition probability, we use the pre-action information I b and Post-operation information I a Updated pre-operation information I b and Post-operation information I a For example, the transition probability can be recalculated using the pre-action information I b and Post-operation information I a is the probability distribution of the measured information, calculate the posterior probability distribution using the measured state, and use the obtained posterior probability distribution to calculate the pre-action information I b and Post-operation information I a may be updated.

[0071] Also, for example, when the action transition determination result can be acquired, the transition probability may be directly updated based on the action transition determination result. One example of a method for directly updating the transition probability is a method of lowering the transition probability by using a predetermined constant value when it is determined that the transition has failed. Also, when a certain number of transition failures occur, the transition probability between the target actions may be set to 0. EXAMPLES

[0072] Next, a robot control device 1 according to a fifth embodiment of the present invention will be described. Note that a duplicated description of points common to the above-mentioned embodiments will be omitted.

[0073] In this embodiment, the operation transition judgment result generated by the operation success / failure judgment unit of Example 3 is displayed on the display unit 51 of the instruction input device 5, and the input of a control instruction to continue the operation when it is judged that the operation transition has failed is accepted by the control instruction input unit 52, thereby making it possible to continue the operation of the robot 2 even if it is judged that the operation transition has failed.

[0074] FIG. 8 is an example of a display screen for notifying the user of a transition failure and accepting a control instruction from the user. In this example, a window corresponding to the display unit 51 is used to display information (in this example, a cross mark) indicating a failure of the transition from the action a2 "pour" to the action a3 "ignite", together with an image P showing the current work environment. Then, in the window corresponding to the control instruction input unit 52, when it is determined that the transition of the action has failed, a candidate for the next action to be executed instead (for example, manual control using a controller, etc.) is displayed, and the selection of the alternative action is accepted from the user. After accepting the input of the alternative action by checking a desired check box in the window of the control instruction input unit 52, when the series of actions resume button 53 is pressed, the series of actions starting from the selected alternative action are regenerated, so that the robot control can be continued. Note that when the control completion button 54 is pressed, the control of the robot 2 may be completed without resuming the series of actions.

[0075] Furthermore, for example, the control instruction may be an instruction for controlling the robot 2. By controlling the robot 2 based on the received instruction and then receiving an input of an alternative operation, even if recovery cannot be achieved by any of the operations, it is possible to continue controlling the robot based on the received instruction.

[0076] When the action transition determination result is a failure, robot control may be continued by regenerating the series of actions as described above, and control instructions may be accepted only if failure occurs a certain number of times or more. EXAMPLES

[0077] Sixth embodiment Next, a robot control device 1 according to a sixth embodiment of the present invention will be described. Note that a duplicated description of points common to the above-mentioned embodiments will be omitted.

[0078] In this embodiment, when there are multiple transition probabilities with the same value, the one with the lowest energy consumption (battery consumption) is selected. This makes it possible to suppress the energy consumption of the robot 2. Also, when there are multiple transition probabilities with the same value, the one with the quickest operation completion time is selected. This makes it possible to shorten the working time of the robot 2.

[0079] Finally, a robot control method used in the robot control system will be described.

[0080] First, a robot control method will be described in the case where similarities between actions already exist, for example, the similarities are stored in the storage unit 12b.

[0081] < <step1>> Using the series of actions generation unit 11b, a series of actions of the robot is generated using a plurality of pieces of action information that associates the robot actions stored in the memory unit 12b with pre-action information including at least one of the work surroundings and the state of the robot before the action and post-action information including at least one of the work surroundings and the state of the robot after the action, the similarity between the post-action information of a certain action and the pre-action information of another action, a current action state including at least one of the robot's current operating environment and the robot's current state, and a goal given to the robot.

[0082] < <step2>> A robot control unit is used to control the robot based on the series of robot movements generated by the series of movement generation unit.

[0083] Next, a robot control method in which the similarity calculation unit 11a calculates the similarity between actions will be described.

[0084] < <step1>> Using the similarity calculation unit 11a, the similarity between the post-action information of one action and the pre-action information of another action is calculated from a plurality of action information that associates the robot action stored in the memory unit 12b with pre-action information including at least one of the work surrounding environment and the state of the robot before the action, and post-action information including at least one of the work surrounding environment and the state of the robot after the action.

[0085] < <step2>> Using the series of actions generation unit 11b, a series of actions of the robot is generated using the robot actions stored in the memory unit 12b, pre-action information including at least one of the work surrounding environment and the robot's state before the action, post-action information including at least one of the work surrounding environment and the robot's state after the action, the similarity calculated by the similarity calculation unit 11a, the current action state including at least one of the robot's current action environment and the robot's current state, and a goal given to the robot.

[0086] < <step3>> A robot control unit is used to control the robot based on the series of robot movements generated by the series of movement generation unit.

[0087] The present disclosure is not limited to the above-mentioned embodiments, and includes various modified examples. The above-mentioned embodiments have been described in detail to easily explain the present disclosure, and are not necessarily limited to those having all the configurations described. It is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. In addition, it is also possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.

[0088] The above configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware such as an integrated circuit. The above configurations, functions, etc. may be realized by software by a processor interpreting and executing a program that realizes each function. Information such as the program, table, file, etc. that realizes each function can be stored in a recording device such as a memory, a hard disk, or an SSD (Solid State Drive), or a recording medium such as a flash memory card or a DVD (Digital Versatile Disk).

[0089] In each embodiment, the control lines and information lines are those that are considered necessary for the explanation, and not all of the control lines and information lines in the product are necessarily shown. In reality, it may be considered that almost all components are connected to each other. [Explanation of symbols]

[0090] 1 Robot control device 11 Processors 11a Similarity calculation part 11b Series of motion generation section 11c Robot control unit 11d Probability calculation section 12 Storage Devices 12a Robot control processing program 12b Storage section 13 Input section 14 Output section 15 Communication Interface 16 Bus 2. Robot 21 Articulated Arm 22 Hands 3. Measurement Sensor 4 Goods 4a Cupboard 4b Kettle 4c faucet 4d Stove 5. Input device 51 Display section 52 Control instruction input section

Claims

1. A robot control device for controlling an operation of a robot, a storage unit that stores a plurality of pieces of operation information that associate a robot operation with pre-operation information including at least one of the robot's work surrounding environment and the robot's state before the operation, and post-operation information including at least one of the robot's work surrounding environment and the robot's state after the operation; a sequence of actions generating unit that generates a sequence of actions for the robot using the plurality of pieces of action information, a similarity between post-action information of a certain action and pre-action information of another action, a current action state including at least one of a current surrounding working environment of the robot and a current state of the robot, and a goal given to the robot; a robot control unit that controls the robot based on the series of actions of the robot generated by the series of actions generation unit.

2. The robot control device according to claim 1 , The robot control device is characterized in that the sequence of motion generation unit connects the motion information with high similarity together to generate a sequence of motions for the robot.

3. The robot control device according to claim 1 or 2, The robot control device is characterized in that the series of motion generation unit generates a series of motions of the robot by connecting the motion information with each other, with the current motion state as a starting point and the goal as an end point.

4. The robot control device according to claim 1 , The robot control device according to the present invention, wherein the memory unit stores the similarity between the pieces of operation information.

5. The robot control device according to claim 1 , A robot control device comprising: a similarity calculation unit that uses the motion information to calculate a similarity between motion information of a certain motion and pre-motion information of another motion, and inputs the similarity to the storage unit.

6. The robot control device according to claim 1 , an action transition determination unit that determines an action transition determination result, which is a determination result of whether or not the action transition has been successful; a similarity update unit that updates the similarity, the similarity update unit updates the similarity based on the series of actions, a current state, and the action transition determination result; The robot control device, wherein the sequence of actions generation unit regenerates the sequence of actions when the action transition determination unit determines that the action transition has failed.

7. The robot control device according to claim 1 , The state of the work environment includes any one of image, sound, and distance information acquired from the work environment, and processed information thereof; A robot control device, characterized in that the state of the robot includes any one of a joint angle, an acceleration, and a tactile sense of the robot.

8. The robot control device according to claim 1 , a probability calculation unit that calculates a probability of transition from a certain motion to another motion by using a similarity between post-motion information of the certain motion and pre-motion information of the other motion; A robot control device characterized in that the sequence of actions generation unit generates a sequence of actions for the robot using the probability.

9. The robot control device according to claim 8, the probability calculation unit calculates a probability of transition from the one motion to the other motion based on a similarity in distribution of the work surrounding environment and the state of the robot between the post-motion information and the pre-motion information; A robot control device characterized in that the sequence of actions generation unit generates a sequence of actions for the robot using the probability.

10. The robot control device according to claim 9, a motion transition determination unit that determines a motion transition determination result that is a determination result of whether or not a motion transition has been successful; The robot control device, wherein the sequence of actions generation unit regenerates the sequence of actions when the action transition determination unit determines that the action transition has failed.

11. The robot control device according to claim 10, A transition probability update unit that updates the probabilities; A robot control device characterized in that the transition probability update unit updates the probability based on the series of actions, a current state, and the action transition determination result.

12. The robot control device according to claim 1 , A robot control device, characterized in that the pre-operation information and the post-operation information are distributions of the work surrounding environment and the state of the robot.

13. The robot control device according to claim 1 , A robot control device comprising a state input unit that receives a current state including either one of the work surrounding environment and the state of the robot.

14. The robot control device according to claim 1 , A robot control device comprising a goal input unit for receiving the goal.

15. A robot motion control method for controlling a motion of a robot, comprising: a series of actions of the robot is generated using a series of actions generation unit, based on a similarity between the post-action information of a certain action and the pre-action information of another action from a plurality of pieces of action information that associates the robot actions stored in the storage unit with pre-action information including at least one of the work surrounding environment and the state of the robot before the action, and post-action information including at least one of the work surrounding environment and the state of the robot after the action, based on a current action state including at least one of the current action environment and the current state of the robot, and a goal given to the robot; A robot control method comprising: controlling the robot using a robot control unit based on a series of robot actions generated by the series of action generation unit.

Citation Information

Patent Citations

  • Learning device, learning method, learning program, control device, control method, and control program

    JP2022041294A