Robot control device, robot control method, and program

Abstract

The present invention relates to a robot control device for controlling a robot, the robot control device comprising: an arrival point estimator that estimates a target position of the robot on the basis of observation information relating to the state of work performed by the robot; and a plurality of constraint expressions estimator that estimates a plurality of constraints to be adhered to during the work by the robot and the importance of each of the plurality of constraints on the basis of the observation information and verbal instructions for the robot.

Description

Robot control device, robot control method, and program

[0001] The present invention relates to a robot control device, a robot control method, and a program for controlling a robot in consideration of safety within a working range.

[0002] In recent years, a plurality of robot control technologies that utilize the knowledge possessed by a base model such as a large language model (hereinafter referred to as "LLM") have emerged. As a result, by inputting an instruction given by a person and the surrounding situation such as an image, a trajectory of an operation to be executed by the robot can be generated, and various tasks can be performed.

[0003] For example, Non-Patent Document 1 discloses a technique for generating an operation of a robot expressed in text in an End-to-End manner by inputting a human instruction and a current image by training a visual language model (hereinafter referred to as "VLM") with the working data of the robot and large-scale Web data.

[0004] Non-Patent Document 2 discloses a technique for generating a robot trajectory using an LLM with Zero-shot (without additional learning).

[0005] Non-Patent Document 3 discloses a technique for preparing diffusion models with different roles for estimating the tip position of a robot arm and generating joint angles, and generating a trajectory that conforms to both while observing the constraints of the movable range of the arm.

[0006] RT-2- Vision-Language-Action Mode...r Web Knowledge to Robotic ControlLanguage Models as Zero-Shot Trajectory GeneratorsHDP_Hierarchial Diffusion Policy for Kinematics -wave Multi-Task Robotic ManipulationCLIP_Learning Transferable Visual Models From Natural Language SupervisionTREBI_Safe Offline Reinforcement Learning with Real-Time Budget Constraints

[0007] However, conventional technologies cannot generate actions that simultaneously take into account multiple constraints that must be followed during the operation.

[0008] For example, according to the technology disclosed in Non-Patent Document 1, it is necessary to prepare data in advance that simultaneously adheres to multiple constraints, but it is practically difficult to prepare training robot work data that covers all situations in advance.

[0009] Furthermore, while the technology disclosed in Non-Patent Document 1, as well as the technology disclosed in Non-Patent Document 2, also imposes conditions on the generated trajectory, it is not possible to reliably ensure that the constraints are followed.

[0010] To safely perform tasks in a real-world environment, it is necessary to estimate multiple constraints and their importance, determined by the robot itself, its surrounding environment, and the nature of the work, and then generate a trajectory that takes these constraints into account.

[0011] This invention has been made in view of these circumstances, and aims to provide a robot control device, a robot control method, and a program that can generate optimal movements while adhering to multiple constraints according to the work context.

[0012] To achieve the above objectives, the present invention employs the following measures.

[0013] A first aspect of the present invention is a robot control device for controlling a robot, comprising: a destination estimator that estimates the target position of the robot based on observational information relating to the robot's work status; and a multiple constraint representation estimator that estimates a plurality of constraints to be observed when the robot performs work, and the importance of each of the plurality of constraints, based on the observational information and verbal instructions to the robot.

[0014] A second aspect of the present invention is a robot control device of the first aspect, further comprising a trajectory generator that generates a robot trajectory that adheres to a plurality of constraints based on a set of a plurality of constraints and their importance, and a target position.

[0015] A third aspect of the present invention is a robot control method performed by a robot control device for controlling a robot, wherein the processor of the robot control device performs the steps of: estimating the target position of the robot based on observational information relating to the robot's work status; estimating a plurality of constraints to be observed when the robot performs work, and the importance of each of the plurality of constraints, based on the observational information and verbal instructions to the robot; and generating a robot trajectory that adheres to each of the plurality of constraints, based on the set of each of the plurality of constraints and its importance, and the target position.

[0016] A fourth aspect of the present invention is a program applied to a robot control device for controlling a robot, which enables the processor to implement a function for estimating the target position of the robot based on observational information regarding the robot's work status, a function for estimating a plurality of constraints to be observed when the robot performs work and the importance of each of the plurality of constraints based on the observational information and verbal instructions to the robot, and a function for generating a robot trajectory that adheres to each of the plurality of constraints based on the set of each of the plurality of constraints and its importance and the target position.

[0017] According to the robot control device, robot control method, and program of the present invention, it is possible to generate optimal movements while simultaneously considering constraints according to the work context.

[0018] Figure 1 is a diagram illustrating the first requirement for generating optimal operation that satisfies multiple constraints. Figure 2 is a block diagram showing an example of the functional configuration of a robot control device to which the robot control method according to an embodiment of the present invention is applied. Figure 3 is a conceptual diagram showing the processing flow by the robot control device according to an embodiment of the present invention. Figure 4 is a simplified diagram showing the hardware configuration of a typical computer.

[0019] Embodiments of the present invention will be described below with reference to the drawings. The drawings are schematic or conceptual. In this specification and in each drawing, elements similar to those described in previously shown drawings are denoted by the same reference numerals, and detailed or redundant explanations are omitted as appropriate.

[0020] As mentioned above, the objective of the robot control device, robot control method, and program of the present invention is to generate optimal movements that adhere to multiple constraints according to the work context. Here, constraints are things that must be observed when the robot performs work. Furthermore, optimal movements refer to movements that efficiently carry out the work. For example, even if a movement can reliably perform the work, if it moves very slowly and cannot complete the work within the expected time, it cannot be called an optimal movement.

[0021] To achieve this objective, the following three requirements are necessary.

[0022] The first requirement is the ability to estimate complex constraints from the current work situation and convert them into a format that can be handled by the model.

[0023] Figure 1 illustrates the first requirement for generating optimal behavior that satisfies multiple constraints.

[0024] Examples of complex constraints include, for instance, as illustrated in Figure 1, when instructing the robot 20 to move a glass 30 filled with water, there are speed constraints (Constraint 1) that require the glass 30 to move at a constant speed, and state constraints (Constraint 2) that require the movement trajectory to be generated within the robot 20's range of motion.

[0025] The second requirement is the ability to handle unfamiliar combinations of constraints that are not included in the training data.

[0026] The third requirement is the ability to flexibly respond to constraints of varying importance. For example, taking the velocity constraint and state constraint explained in the first requirement as an example, the state constraint of generating a trajectory within the robot's range of motion is a constraint that must be strictly adhered to, while the velocity constraint of moving the cup at as constant a speed as possible is a constraint that can be compromised.

[0027] To meet these requirements, a robot control device to which the robot control method according to the embodiment of the present invention is applied interprets the work situation, for example, using LLM / VLM, estimates multiple constraints to be observed and their importance, and then generates a trajectory under multiple constraints based on a diffusion model (expressed as Box constraints and importance), thereby expressing general constraints as "multiple combinations of basic constraints" and their "importance," and ensuring that they are observed.

[0028] Figure 2 is a block diagram showing an example of the functional configuration of a robot control device to which a robot control method according to an embodiment of the present invention is applied.

[0029] The robot control device 10 includes a destination estimator 12, a multiple constraint representation estimator 14, and a multiple constraint trajectory generator 16.

[0030] The destination estimator 12 has an estimation model that estimates the goal position from work status information such as images. When observation information A such as images and descriptions related to the current posture and work status of the robot 20 is input to this estimation model, it recognizes the task of the work performed by the robot 20, estimates the destination C which is the target position of the robot 20, and outputs the estimated destination C to the multiple-constrained trajectory generator 16.

[0031] Using this estimation model, the destination estimator 12 can estimate the destination C of the action that the robot 20 should perform, taking environmental information (images, point clouds, etc.) as observation information A as input. Existing methods that generate actions using point clouds or video information as input can be used in this estimation model.

[0032] When the multiple constraint representation estimator 14 receives the aforementioned observation information A and a linguistic instruction B for the robot 20, such as "Put the cup on the other side of the table," it recognizes the constraints based on these inputs, estimates a finite number of Box constraints D that must be followed from the current situation, and the importance E of each Box constraint, and outputs the pair of each Box constraint D and its importance E to the multiple constraint trajectory generator 16.

[0033] A Box constraint is a constraint that ensures a variable falls within a certain range. Examples include a speed limit that keeps the velocity v within a certain range (a < v < b) (where a is the lower limit and b is the upper limit), and a state constraint that generates the robot 20's trajectory so that it passes only within a certain region or avoids it.

[0034] More specifically, the multiple constraint representation estimator 14 can estimate multiple constraints D and their importance E that the robot 20 must adhere to in the current task, taking as input video or images related to the work content as observation information A, or verbal instructions B from a person. Here, constraints can be selected from a predetermined set of constraints, as in the Box constraint mentioned above, or constraints to be adhered to can be selected according to the work content. Furthermore, their importance (expressed as discrete or continuous values) can be estimated. Existing methods such as VLM and LLM can be used for estimation. In addition, methods that can consider the relationships between constraints may be added, such as a graph neural network (hereinafter referred to as "GNN") which learns relationships by treating one constraint as a point on a graph, for example, "move a glass of water" → "maintain a constant speed so as not to spill the glass" (constraint 1), "generate motion within the range of motion of the robot arm" (constraint 2).

[0035] For example, the multiple constraint representation estimator 14 can represent images and text as vectors in a common latent space by using a VLM that can handle "images" and "text" in a common latent space, such as CLIP and GPT-4o, and estimate importance based on the similarity of the vectors.

[0036] To do this, first prepare (a) an image of the current work situation (such as an image from the robot's first-person perspective), (b) text describing the task instructions, and (c) a manual explanation of several pre-specified Box constraints.

[0037] Next, the multiple constraint representation estimator 14 uses CLIP to convert (a), (b), and (c) into vector representations in a common latent space (embedding).

[0038] The multiple constraint representation estimator 14 then calculates the similarity (cosine similarity, etc.) of the vectors (a) + (b) and (c) in the latent space, considers Box constraints with high similarity, or Box constraints with similarity above a predetermined threshold, as constraints with high importance, and outputs them as continuous value importance after normalization.

[0039] The multiple constraint representation estimator 14 can also use an LLM / VLM such as GPT-4o to have the LLM respond with a Box constraint that should be selected from the current situation, and by specifying a task to estimate importance in the prompt statement entered into the LLM, it can also respond with an importance value.

[0040] This can be done by providing the multiple constraint representation estimator 14 with "task instruction text" and "combinations of Box constraints," prompting it to estimate the Box constraints that must be followed, and then prompting it to provide a response. This specification can be a continuous value, or a discrete importance level such as "Box constraints that must be followed at all costs," "Box constraints that should be followed to a moderate degree," or "Box constraints with low priority."

[0041] The multi-constraint trajectory generator 16 has a generation model that generates a trajectory F that satisfies the constraints based on the arrival point C, the plurality of Box constraints D, and the importance E of each box constraint. This generation model is, for example, a model using the method of Safe offline RL based on a diffusion model, and can only handle quantitative expressions. Therefore, the arrival point estimator 12 converts the ambiguous information of the observation information A into quantitative information of the arrival point C and outputs it to the multi-constraint trajectory generator 16. Further, the multi-constraint expression estimator 14 converts the ambiguous information of the observation information A and the language instruction B into quantitative information of the Box constraint D and the importance E and outputs it to the multi-constraint trajectory generator 16.

[0042] Thus, the generation model uses these quantitative information to generate a trajectory F of the robot 20 that satisfies a plurality of constraints based on the set of the plurality of Box constraints D and its importance E and the arrival point C.

[0043] The multi-constraint trajectory generator 16 can further generate an operation trajectory that satisfies a plurality of constraints by expanding an existing trajectory generation method with constraints. This can be implemented, for example, by the generation model using (1) a method of making the generation model satisfy a plurality of soft constraints (extension of TREBI) and adjusting weights according to the importance of the constraints to generate a trajectory. Also, (2) for constraints that must be satisfied absolutely, it can be implemented by using a method of making the generation model satisfy hard constraints (TLDM) and generating a trajectory in the same way otherwise. Alternatively, (3) a plurality of adaptable hard constraints are prepared in advance, the hard constraints estimated to be required by the multi-constraint expression estimator 14 are selected and adapted, and it can also be implemented by combining a plurality of methods (TLDM) of making the generation model satisfy hard constraints to generate a trajectory.

[0044] A method using the extension of (1) TREBI (Trajectory-based REal-time Budget Inference) will be specifically described below.

[0045] As a problem setting, assume a Constrained MDP (Constrained MDP) in which the budget b_j (j = 1,..., N_c) for satisfying N_c multiple constraints as shown in the following formula is variable.

[0046]

[0047] In Equation (1), R(τ) and C_j(τ) are the cumulative reward and the cumulative sum of the cost functions of the j-th constraint. Equation (2) represents a plurality of safety constraints expressed as the cumulative sum of N_c cost functions (the constraint is satisfied when the cumulative sum C_j of the cost function corresponding to the j-th constraint is less than or equal to the budget b_j), and Equation (3) represents a constraint derived from Offline RL.

[0048] Next, similar to the trajectory method that satisfies a single constraint of TREBI, the generated distribution is decomposed into a product as shown in the following (Equation 4) and (Equation 5), and the constraint is incorporated as guidance for the diffusion model.

[0049]

[0050] Next, TREBI, which conventionally only corresponded to a single constraint, is extended to multiple constraints as shown in the following (Equation 6) to (Equation 8).

[0051]

[0052] Equation (7) represents the evaluation of the trajectory, and Equation (8) represents the cost value corresponding to the j-th constraint. Also, α is a scale parameter. Thus, the evaluation and the constraint are separated.

[0053] When expressing the weight (for example, priority) of each constraint using the above (Equation 6) to (Equation 8), it is expressed as the following (Equation 9).

[0054]

[0055] Here, ω ｊ is the reward (ω ０ = 1) and the weighting of the constraint (ω ｊ ).

[0056] The multi-constraint trajectory generator 16 can generate a trajectory with the diffusion model using Equation (9).

[0057] Next, an operation example of the robot control device 10 to which the robot control method according to the embodiment of the present invention configured as described above is applied will be described using FIG. 3.

[0058] Figure 3 is a conceptual diagram showing the processing flow by a robot control device according to an embodiment of the present invention.

[0059] When the robot control device 10 controls the robot 20, observational information A, such as environmental information (images, point clouds, etc.), is input to the destination estimator 12. The destination estimator 12 uses existing task recognition methods, such as PerAct, to estimate the destination C of the action that the robot 20 should perform based on these inputs. The estimated destination C is output to the multiple-constrained trajectory generator 16.

[0060] The multiple constraint representation estimator 14 receives observation information A and language instruction B as input. The multiple constraint representation estimator 14 uses LLM or the like to recognize constraints based on these inputs and estimates a finite number of Box constraints D that must be followed from the current situation, and the importance E of each Box constraint. The estimated Box constraints D and importance E are output to the multiple constraint trajectory generator 16.

[0061] In the multiple-constrained trajectory generator 16, a diffusion model is applied, and a trajectory F that adheres to the constraints is generated as input to the controlled robot 20, according to the destination C, multiple Box constraints D, and their importance E. By receiving the generated trajectory F as input, the controlled robot 20 can generate optimal robot movements while adhering to multiple constraints according to the work context.

[0062] Such a robot control device 10 can be implemented using a computer such as a PC.

[0063] Figure 4 is a simplified diagram showing the hardware configuration of a typical computer.

[0064] Computer 100 receives some kind of input from the outside, processes it, and outputs the result to the outside. Input is handled by input device 120, and output is handled by output device 140. CPU (Central Processing Unit) 110 controls the entire flow of data and processing, and performs calculations and other processing. In Figure 4, solid arrows represent the flow of data, and dashed arrows represent the flow of control.

[0065] For computer 100 to perform processing, it needs a program that describes the processing procedure. The program contains a series of instructions and the data used by those instructions, all of which are temporarily stored in memory 130. CPU 110 retrieves the instructions from memory 130 one by one, interprets them, and operates according to those instructions.

[0066] Therefore, the destination estimator 12, the multiple constraint representation estimator 14, and the multiple constraint trajectory generator 16 of the robot control device 10 according to this embodiment are realized by the CPU 10 operating according to a program stored in the memory 130.

[0067] Furthermore, although not shown in Figure 4, the computer 100 can incorporate a storage device such as an SSD (Solid State Drive) or HDD (Hard Disk Drive). Such a storage device can store the estimation model used by the destination estimator 12, the generative model used by the multiple constraint representation estimator 14, and the diffusion model used by the multiple constraint trajectory generator 16.

[0068] As described above, according to the robot control device 10 to which the robot control method according to the embodiment of the present invention is applied, the robot control device has a hierarchical configuration consisting of three components: a destination estimator 12, a multiple constraint representation estimator 14, and a multiple constraint trajectory generator 16. By generating a trajectory F that adheres to multiple constraints according to their importance based on the work content and surrounding environment, it is possible to generate optimal robot movements that adhere to multiple constraints according to the work context.

[0069] In particular, the multiple constraint representation estimator 14 makes it possible to interpret combinations of quantitative constraints that must be followed without requiring prior preparation such as manual annotation work or the preparation of data corresponding to multiple constraints.

[0070] Furthermore, the multiple-constrained trajectory generator 16 enables reliable task execution in situations where safety and reliability are paramount, such as collaborative work between humans and robots, physical work involving humans, and manipulation of hazardous or fragile objects.

[0071] These features allow us to perform tasks with maximum efficiency while flexibly adhering to constraints according to the situation, rather than simply adhering to estimated constraints and thus reducing work efficiency.

[0072] These effects can be achieved regardless of the robot's shape (arm, humanoid, or quadruped, etc.) or the type of task (manipulation or movement task, etc.) by preparing data on the target task and the robot's trajectory.

[0073] Furthermore, after operating the controlled robot along the generated trajectory, if it is determined that "the water is about to spill even though it was moved at speed V," the speed constraint can be made stricter. This feedback of observational information A can be provided to the multiple constraint expression estimator 14, allowing the constraints and their importance to be changed in real time.

[0074] As described above, the present invention provides a robot control device, a robot control method, and a program that can generate optimal movements while adhering to multiple constraints according to the work context.

[0075] The present invention is not limited to the embodiments described above, and in the implementation stage, the components can be modified and implemented without departing from the spirit of the invention. Furthermore, various inventions can be formed by appropriately combining the multiple components disclosed in the above embodiments. For example, some components may be deleted from all the components shown in the embodiments. Moreover, components from different embodiments may be appropriately combined.

[0076] 10 Robot control device 12 Destination estimator 14 Multiple constraint representation estimator 16 Multiple constraint trajectory generator 20 Robot 30 Cup 100 Computer 110 CPU 120 Input device 130 Memory 140 Output device A Observation information B Language instruction C Destination D Constraints E Importance F Trajectory satisfying the constraints

Claims

1. A robot control device for controlling a robot, comprising: a destination estimator that estimates the target position of the robot based on observational information relating to the robot's work status; and a multiple constraint representation estimator that estimates a plurality of constraints to be observed when the robot performs work, and the importance of each of the plurality of constraints, based on the observational information and verbal instructions to the robot.

2. The robot control device according to claim 1, further comprising a trajectory generator that generates a robot trajectory that adheres to each of the plurality of constraints based on the set of each of the plurality of constraints and their importance, and the target position.

3. A robot control method performed by a robot control device for controlling a robot, wherein the processor of the robot control device performs the steps of: estimating the target position of the robot based on observational information relating to the working status of the robot; estimating a plurality of constraints to be observed when the robot performs work, and the importance of each of the plurality of constraints, based on the observational information and linguistic instructions to the robot; and generating a trajectory of the robot that adheres to each of the plurality of constraints, based on the set of each of the plurality of constraints and its importance, and the target position.

4. A program to enable the processor to implement the following functions: a function to estimate the target position of the robot based on observational information regarding the robot's work status; a function to estimate a plurality of constraints to be observed during the robot's work and the importance of each of the plurality of constraints based on the observational information and verbal instructions to the robot; and a program to generate a trajectory of the robot that adheres to the plurality of constraints based on the set of each of the plurality of constraints and its importance and the target position.

Images