Robot control device

Abstract

Robot control device that controls a robot, wherein the robot control device comprises: A machine learning model that receives input from any given element of situation information selected from elements of situation information and outputs a value that is one of a binary value, a continuous value, and a multidimensional vector value indicating the success or failure or degree of success of an operation of a provided operation control in a situation specified by the arbitrary given element of received situation information, wherein the elements of situation information are information about the robot or an object in the vicinity of the robot and: (i) physical properties that include a physical structure and a shape; (ii) a current state of at least one of a position, a location, and a temperature, which is a time-varying property;(iii) a relative relationship between the physical properties or between the present states; and (iv) a combination of the physical properties, a combination of the present states, or a combination of the physical properties and the present state; ; a process control selection device that obtains the output value for at least one process control by providing the arbitrary given element of situation information to the machine learning model, if the arbitrary given element of situation information is received, and identifies a process control to be executed based on the output value; and a process control execution device that executes the process control identified by the process control selection device, wherein The process control, which is to be evaluated by the machine learning model, is not a control for a robot to perform a single and primitive movement determined by a single direction vector or a target position, but rather a control for a robot to achieve a process goal by detecting nearby objects using a sensor with interaction with other objects.

Description

Technical field

[0001] The present disclosure relates to the control of a robot, including a sensor of a type that, using the sensor, detects a nearby object through interaction with other objects (including objects such as a wall or floor, which may be referred to as the environment). The present disclosure also relates to the structure of a control program for managing, evaluating, verifying, and improving process control to be performed by a robot. Background of the invention

[0002] Robots (e.g., an autonomously controlled household robot and a care robot) that perform a process or activity requiring interaction with another object in a free (open) environment, where the environment and task are not defined, must operate flexibly according to the environment or, in other words, the situation (hereinafter also referred to as "environment / situation") in which the robots are deployed. Furthermore, these robots must perform various processes according to the tasks.

[0003] It appears that the aforementioned robots and robotic arms are similar in factory work areas; however, the robotic arms only move along a single path (route) while handling a single object (i.e., an environment, a situation, and a task are defined for each robotic arm). Accordingly, the requirements that the robotic arms must meet differ from those of the aforementioned robots. Furthermore, putting the aforementioned robots into active use requires evaluation, verification, and improvement.

[0004] Examples of the operations to be performed by the aforementioned robots are as follows. The robots (i) move objects placed in different locations to other different locations, (ii) hold an object by one side of the object, move while holding the object, and place the object in another location, (iii) change direction after holding an object, (iv) pull an attached rod through an opening in an object, (v) insert a held object into a gap between objects, and (vi) stack objects.

[0005] Furthermore, examples of an environment / situation in which the aforementioned robots perform an operation include: (i) different places and environments where an object is placed, (ii) the presence of types of objects nearby, and (iii) physical interaction between objects.

[0006] As an example of taking an object nearby into account, a system has been proposed that checks for the presence of an object nearby (a space occupied by the object) and checks whether a robot physically affects the object or not (see, for example, patent literature (PTL) 1).

[0007] Additionally, a system was proposed that inputs sensor information into a machine learning model and controls a target device according to an output of the machine learning model in order to implement flexible control of the target device (see, for example, PTL 2).

[0008] Another system that is proposed allows an operator to demonstrate a work movement to a robot, so that the robot, which has a different mechanism than the operator, can perform a movement that the operator intends to achieve, even in an environment different from the environment in which the robot was trained (see PTL 3).

[0009] In one work (see for example PTL 4) a device for machine learning is presented, which describes both a robot system and a method for machine learning for the transfer of workpieces by a robot.

[0010] Previous work has already described systems (see PTL 5) that allow users to make adjustments according to working conditions while performing a task, while the system executes a predetermined task using a learning module.

[0011] Patent literature PTL 6 presents methods and devices for deep machine learning related to predicting the gripping outcome of a robot's end effector in a possible gripping pose. It includes additional implementations focused on training and utilizing both a geometry network and a network for predicting the gripping outcome.

[0012] Developments in the field of machine learning models (see, for example, PTL 7) that can be trained demonstrate their applicability to robot-assisted grasping and manipulation tasks. These models can be trained using simulated data from simulated robots performing various manipulations on different simulated objects. Alternatively, the model can also be trained using real-world training examples based on data from physical robots performing manipulations on various objects.

[0013] A known control device includes a processor configured to execute computer-executable instructions for controlling a robot. The processor calculates an image processing parameter based on an image captured by a camera, using machine learning. It recognizes the target object using the calculated image processing parameter and controls the robot based on the target object recognition result (see PTL 8). Citation list for patent literature PTL 1: JP 2005 - 081 445 A PTL 2: JP 2017 - 064 910 A PTL 3: JP 2018 - 051 652 A PTL 4: DE 10 2018 215 057 A1 PTL 5: EP 3 401 847 A1 PTL 6: WO 2018 / 236 753 A1 PTL 7: US 2019 / 0 084 151 A1 PTL 8: US 2018 / 0 222 057 A1 Invention Summary Technical Problem Statement

[0014] The conventional system disclosed in PTL 1 described above determines whether a nearby object occupies a space or not and performs an operation in an area where no object is present. Accordingly, the system disclosed in PTL 1 does not, for example, consider (i) an operation requiring physical interaction, such as pushing an object aside to place another object in its place, and (ii) a change in an operation according to the nature of an object. Consequently, the system disclosed in PTL 1 is incapable of responding to a control that involves such operations.

[0015] The system disclosed in PTL 2 can implement flexible control of a target device in a wide range of situations under a single control using a machine learning model; however, it is difficult for the system to manage, evaluate, verify, and modify process controls for each individual situation.

[0016] In publication PTL 3, the robot learns by having the operator demonstrate the movement. In PTL 5, the system learns during the execution of the task. PTL 7 describes how the system learns to grasp using simulated and real physical data. The patent presented here, however, is for a robot control device that is not a controller for a robot to perform a single, primitive movement, but rather a controller for a robot to achieve a specific goal.

[0017] PTL 4 and PTL 6 also exhibit fundamental differences from the process control management system presented here. Furthermore, the present invention differs from approaches that describe a machine learning device for transferring workpieces with a robot (see, for example, PTL 4), as well as the system that can make gripping outcome predictions via machine learning (see PTL 6). It is evident that the system presented in this document differs from those mentioned above. It enables interaction with other objects, encompassing both physical contact and non-contact interaction.

[0018] The control device disclosed in PTL 8 is designed to recognize the target object from an image, perform calculations using machine learning, and control the robot based on the target object recognition result. In contrast, in the present patent, the training data is generated by a training data generation device from the following sources: (i) a design specification value, (ii) a result of an operation control process performed by an actual robot, (iii) a rule of thumb developed by a person, and (iv) a result of the operation control process performed in a physics simulation facility.

[0019] In the event that a modification is made to an operation control due to an error in a specific situation, conventional techniques do not consider the effect of the modification on the operation control in a different situation. The modification can affect the operation control performed in that other situation. The modification can change the operation, and thus the operation can fail or be unsuccessful, even if the operation was originally successful in other situations (in any situation where an operation capable of achieving a task was carried out). For this reason, evaluation and verification must be performed again in each situation to determine whether an operation is successful or not.

[0020] The present disclosure aims to provide a robot control device that can individually manage and modify the control of operations to be carried out according to the respective situations, while implementing flexible control of a robot according to a wide range of environments / situations. Solution to the task

[0021] To provide such a robot control device, the present invention, according to a first embodiment, comprises a robot control device that controls a robot. The robot control device is characterized in that it comprises: a machine learning model that receives input from any given element of situation information selected from elements of situation information, and outputs a value that is one of a binary value, a continuous value, and a multidimensional vector value, indicating the success or failure, or degree of success, of an operation of a provided operation control in a situation specified by any given element of received situation information.wherein the elements of situation information are information about the robot or an object in the vicinity of the robot and: (i) physical properties, including a physical structure and a shape; (ii) a current state of at least one of a position, a location (orientation), and a temperature, which is a time-varying property; (iii) a relative relationship between the physical properties or between the current states; and (iv) a combination of the physical properties, a combination of the current states, or a combination of the physical properties and the current state; an operation control selection device that obtains the output value for at least one operation control by providing the arbitrary given element of situation information to the machine learning model,if any given element of situation information is received, and based on the output value, an operation control to be executed is identified; and an operation control execution device that executes the operation control identified by the operation control selection device. The operation control is a control routine for the robot to achieve an operation goal by having the robot detect an object in the vicinity of the robot using a sensor and an interaction with another object.

[0022] The present invention according to the first embodiment may comprise: a machine learning model that receives input from any given element of situation information selected from elements of information that are information about a nearby object or a target robot itself, and comprises: (i) physical properties that include a physical structure and a shape, (ii) a present state of a time-changing property (a position, a location (orientation), or a temperature), (iii) a relative relationship between the foregoing, and (iv) a combination of the foregoing, and that outputs a value indicating a success or failure or degree of success (one of a binary value, a continuous value, and a multidimensional vector value) of an operation of a provided operation control in a received situation;A process control selection device that obtains an output value for each of the process controls corresponding to a process goal by providing an element of situation information to the machine learning model, if the process goal and the element of situation information are received, and selects a process control to be executed from among the process controls based on the output values, corresponding to the process goal; and a process control execution device that executes the process control selected by the process control selection device. The process control can be a control routine for achieving a process goal by detecting a nearby object using a sensor that interacts with another object.

[0023] The present invention according to the first embodiment can be characterized in that it comprises: machine learning models provided according to the number of process controllers corresponding to a process target for each of process targets, each of which receives input from any given element of situation information selected from elements of information (elements of situation information) that are information about a nearby object or a target robot itself, and which comprise: (i) physical properties including a physical structure and shape, (ii) a current state of a time-changing property (a position, orientation, or temperature), (iii) a relative relationship between the foregoing, and (iv) a combination of the foregoing, and outputs output values.which each indicate a success or failure or degree of success (one of a binary value, a continuous value, or a multidimensional vector value) of an operation of a provided operation control in a received situation; an operation control selection device that obtains output values ​​for the operation controls corresponding to the operation goals by providing each of the machine learning models provided according to the number of operation controls corresponding to the operation goal with an element of situation information, if the operation goal and the elements of situation information are received, and selects an operation control to be executed from among the operation controls corresponding to the operation goals based on the output values; and an operation control execution device,which executes the operation control selected by the operation control selection device. The operation control can be a control routine for achieving an operation goal by detecting a nearby object using a sensor that interacts with another object.

[0024] A variant of the present invention according to the first embodiment can be characterized in that, even if the machine learning model is either created individually for each provided process control or has a single model configuration, learning is carried out using a model that is capable of learning about a target process control using a method in which the output value for a process control other than the process control to be learned is not affected.

[0025] The present invention according to a second embodiment is characterized in that the arbitrary given element of information comprises at least one element of an absolute position, a relative position, a location (orientation), a shape, or a type of the object. That is to say, the arbitrary given element of situation information is characterized in that it comprises at least one element of a position (an absolute position or a relative position), a location (orientation), a shape, and a type of an object.

[0026] The present invention according to a third embodiment is further characterized in that it comprises a training data generation device which generates a data pair as training data for the machine learning model, in which situation data and success or failure data are paired, wherein the data pair is generated from at least one of (i) a design specification value, (ii) a result of an operation of process control performed by an actual robot, (iii) a rule of thumb developed by a person, and (iv) a result of the operation of process control performed in a physics simulation device. That is to say, the present invention according to the third embodiment is further characterized in that it comprises a training data generation device which generates a data pair as training data for the machine learning model, in which situation data and success or failure data are paired.The data pair consists of at least one design specification value, a result of a process control operation performed by an actual device, a rule of thumb developed by a person, and a result of a process control operation performed in a physics simulation facility.

[0027] The present invention according to a fourth embodiment is characterized in that, if the training data generation device generates the data pair using the physics simulation device, the training data generation device (i) provides the physics simulation device with a condition, in addition to a process control, that defines a state in which a process control operation is successful, (ii) executes the process control operation while the elements of situation information, each specifying an environment, are changed variously in the physics simulation device, (iii) tests whether the process control operation is successful or not, and (iv) generates the data pair based on a result of the test.That is, the training data generation device is characterized in that, if the training data generation device performs the generation using the physics simulation device, the training data generation device, in addition to a process control, provides a condition that defines a state in which a process control operation is successful, modifies environments (the elements of situation information) in the physics simulation device in various ways, executes the provided process control, tests whether the process control operation is successful or not, and generates the data pair based on a result of the test.

[0028] The present invention according to a fifth embodiment is characterized in that, if the training data generation device tests whether the process control operation is successful or not using the physics simulation device, the training data generation device identifies at least one of the elements of situation information that indicate an environment in which the provided process control operation is successful, and from the at least one element of situation information identified as a start element to be sampled, samples the elements of situation information that each indicate an environment in which the process control operation is tested.That is, the present invention according to the fifth embodiment is characterized in that it provides at least one of environments (elements of situation information) in which the provided process control operation is successful, and samples environments to be tested (the elements of situation information), from which at least one of the environments is used as a starting point for sampling if a success or failure of the process control operation is tested in the physics simulation device. Advantageous effects of the invention

[0029] The present invention provides a robot control device that can individually manage and modify the control of operations to be performed according to the respective situations, while implementing flexible control of a robot according to a wide range of environments / situations.

[0030] More precisely, according to the first embodiment of the present invention, it can flexibly determine situations in which a process control operation is successful and can execute the operation appropriately. In other words, even if a process control operation is conceived by a person and is of a high level and extremely complex due to a diverse and enormous amount of information, or is difficult for a person to describe, the use of a machine learning model makes it easy to define a boundary for each of the ranges of situations in which the process control operation is successful.

[0031] According to the variant of the present invention according to the first embodiment, a machine learning model that determines a situation can add a determination about situations with respect to a new process control, and can modify a determination about situations with respect to an existing process control, without affecting a determination to be made about a situation with respect to another process control.

[0032] The present invention according to the second embodiment produces an effect that is equivalent to the effect produced by the present invention according to the first embodiment.

[0033] The present invention according to the third embodiment produces an effect equivalent to the effect produced by the present invention according to the first embodiment.

[0034] According to the present invention in its fourth embodiment, it is possible to automatically create and modify (train and retrain) a machine learning model that determines situations.

[0035] According to the fifth embodiment of the present invention, if a situation-determining machine learning model is automatically created and modified (trained and retrained), it is possible to effectively perform a test on each of the areas of situations in which an operation of a process control is successful by using provided situation information as a starting element for sampling, if the situation to be tested is sampled. Brief description of the drawing Fig. Figure 1 shows a representation illustrating the basic concept of a robot control device according to an exemplary embodiment. Fig. Figure 2 shows a representation illustrating details of the basic concept of the robot control device according to the exemplary embodiment. Fig. Figure 3 shows a block diagram illustrating a configuration of the robot control device according to the exemplary embodiment. Fig. Figure 4 shows a representation illustrating a preliminary processing operation carried out by the robot control device according to the exemplary embodiment. Fig. Figure 5 shows a representation illustrating processes (process controls) that are carried out by the robot control device according to the exemplary embodiment. Fig. Figure 6 shows a representation illustrating a processing operation that is carried out during maintenance, in other words, during an evaluation and verification of the robot control device according to the exemplary embodiment. Fig. Figure 7 shows a representation illustrating a specific example of a machine learning model included in the robot control device according to the exemplary embodiment. Fig. Figure 8 shows a representation illustrating processes (process controls) that are carried out by a robot control device according to variant 1 of the embodiment. Fig. Figure 9 shows a representation illustrating processes (process controls) that are carried out by a robot control device according to variant 2 of the embodiment. Fig. Figure 10 shows a representation illustrating the machine learning model according to the exemplary embodiment. Description of the exemplary embodiments

[0036] Examples of embodiments according to the present disclosure are described in detail below.

[0037] Fig. Figure 1 shows a representation illustrating the basic concept of a robot control device according to an exemplary embodiment. Fig. Figure 1 illustrates a range of environments, in other words, situations (hereinafter also referred to as "environment(s) / situation(s)"), when a robot arm handles target objects. In this illustration, the term "NG" denotes a range of environments / situations in which a robot arm is not in operation.

[0038] As shown in the illustration, the robot control device, according to the exemplary embodiment, uses a concept of areas of environments / situations in which a process control will not fail and can be achieved. That is, the areas described above are automatically calculated for each process goal and process control. In addition, these process controls are managed individually.

[0039] Here, the term "environment / situation" refers to types of target objects and objects within the environment, the way these objects are positioned, the states of these objects, the environments in which these objects are positioned, and so on. Since one meaning of "environment / situation" encompasses types of different objects, positional relationships, situations, etc., the term conceptually denotes a multidimensional space. Furthermore, depending on the process, the term "environment / situation" also indicates the shape and position (orientation) of a robot. If the case of multiple robots working together is considered, the term "environment / situation" also indicates a positional relationship and a positional relationship (orientation relationship) of each robot.

[0040] Although the embodiment is described using a robot arm as an example, it should be noted that the robot control device according to the embodiment is applicable to all types of robots, regardless of their shape and structure. Although information indicating the degree to which process control can be achieved is represented by binary values, which are in Fig. 1. While "OK" and "NG" are binary, the values ​​representing this information are not limited to these binary values. The information can be represented by analogous values ​​such as probabilities within the range of 0 to 1.

[0041] Fig. Figure 2 shows a diagram illustrating details of the basic concept of the robot control device according to the exemplary embodiment. As shown in the diagram, one or more process controllers correspond to each of the process goals in the robot control device according to the exemplary embodiment. Each of the process controllers is managed individually and is independent.

[0042] part (a) of Fig. Figure 2 shows an example of a conceptual space representing environments / situations for a process goal A (e.g., relocating an object). In this example, process control A1 can be achieved in area A1 of environments / situations, where process control is possible in each of these areas. Process control A2 can be executed in area A2 of environments / situations, where process control is possible in each of these areas. Process control A3 can be executed in area A3 of environments / situations, where process control is possible in each of these areas.

[0043] Part (b) in Fig. Figure 2 shows an example of a conceptual space illustrating environments / situations for a process goal B (e.g., a change in the position (orientation) of an object). In this example, process control B1 can be executed in area B1 of environments / situations, in each of which process control is achievable. Process control B2 can be executed in area B2 of environments / situations, in each of which process control is achievable. Process control B3 can be executed in area B3 of environments / situations, in each of which process control is achievable.

[0044] In the robot control device according to the exemplary embodiment, one or more process controls correspond to one or more process goals, taking into account areas in which, as in Fig. Figure 2 shows that a process control does not fail in each case. Alternatively, a single process control can correspond to a multitude of process controls, since there is a case where a single process control can achieve a multitude of different process goals for various reasons.

[0045] A single process controller specifies one or more control routines (control programs) for a robot to achieve a process goal by detecting nearby objects using a sensor that interacts with other objects. Furthermore, the process controller, which is to be evaluated by a machine learning model (ML model) included in the robot control device according to the exemplary embodiment, is not a controller for a robot to perform a single and primitive movement determined by a single direction vector or a single target position, but rather a controller for a robot to achieve a process goal by detecting nearby objects using a sensor that interacts with other objects (hereinafter referred to as "process goal achievement processing requiring detection and interaction").Accordingly, it is possible (i) to implement a robot control system that executes safely, and (ii) to manage ranges of situations in which each operation of a process control system achieves a process goal (ranges of situations in which each operation of a process control system is successful). This creates the effect of enabling advanced process control management.

[0046] It is important to note that "interaction with other objects" performed by a robot includes not only cases where the robot is in physical contact with other objects, but also cases where the robot has a non-contact relationship with other objects, instead of or in addition to the case where the robot is in physical contact with other objects. The non-contact relationship can include, for example, a spatial relationship (a predetermined relationship between positions and / or distances), a presence relationship (a robot's detection of an object's presence), and a communication relationship (communication between a robot and an object wirelessly, etc.).

[0047] Furthermore, "process goal achievement processing that requires sensing and interaction" is, for example, a series of primitive movements that achieve a process goal using a result obtained from a sensor as feedback information, and can usually be expressed as a sequence of movements determined depending on the result of a sensing.

[0048] It should be noted that the movements include an arm movement, an arm action or arm operation, a hand movement, a hand action or hand operation, a hand part, a hand unit action, a position (orientation), a position control (orientation control), a body part movement, a leg part movement, etc.

[0049] The following can be applied as a concept of a relationship between an operation goal and an operation control: (i) an operation control is logic (in other words, a procedure or algorithm) designed to achieve an operation goal, and (ii) the operation goal is an effect produced by an operation control alone (in other words, an effect expected as a result of executing a single operation control, a purpose of using the operation control alone, and / or a goal that the operation control is intended to achieve). Optionally, an operation goal need not be an effect produced or expected by a sequence of operation controls in which a multitude of operation controls are composed, and / or by a set of operation controls.In this case, the multitude of process controls can include a multitude of process controls with different control details, a multitude of process controls with the same control detail, or a mixture of the multitude of process controls with different control details and the multitude of process controls with the same control detail.

[0050] Furthermore, the process control and all processing performed by the robot control device utilize the concepts of logic, instruction value, and static parameter. Logic is a detail of a process, algorithm, procedure, and / or processing routine, and typically does not need to change while the robot control device is operating or performing a process. An instruction value is a value and / or parameter provided to logic. While the robot control device is operating or performing a process, the instruction value takes on various values ​​and can be a value provided to logic as an instruction, a statement, and / or a value provided to logic.Except where the instruction detail is predetermined when the robot control device is provided, an instruction value is usually determined according to the situation of the robot control device, if it is in operation or performing an operation. Types of instructions and sequences of instructions are determined; in other words, a value is determined that the instruction value will assume. A static parameter is a parameter provided to logic and differs from an instruction value in that the static parameter usually assumes the same value without any change in value if the robot control device is in operation or performing an operation.Although a procedure (in other words, a logic and an algorithm) is not modified, the static parameter is a finely modifiable value and / or a finely modifiable parameter if an effect of a processing operation is modified or finely modified, usually not while the robot control device is in operation or in an operation, but rather during a time when the robot control device is not in operation or in an operation, such as when the robot control device is undergoing maintenance. In this case, an operation control corresponds to a logic, or a combination of a logic and a static parameter, and does not include an instruction value. Furthermore, an instruction value corresponds to an operation goal or (not in . Fig. 3) information associated with an operation target. Furthermore, operation controls that have the same logic or the same combination of logic and a static parameter are treated as a single operation control. Accordingly, for example, a difference in instruction values ​​does not result in the provision of multiple operation controls. In this case, the entire processing of this disclosure is conceptually performed as follows: (i) receiving an input of an operation target or receiving inputs of an operation target and an instruction value; (ii) obtaining situation information from a sensing device; and (iii) selecting an operation control (a logic, or a combination of logic and a static parameter) based on the operation target (and instruction value) and the situation information.Alternatively, the entire processing performed by the robot control device according to the present disclosure need not include (or may exclude) the following operations: (i) obtaining situation information from a sensing device; and (ii) selecting an instruction value based on the situation information.

[0051] Furthermore, an operation goal is an effect brought about by a series of operation controls, the sequence of which is defined. A series of operation controls, whose sequence is defined, can be treated as a single operation control. If, in addition, the sequence of one series of operation controls differs from the sequence of another series of operation controls, it can be assumed that these series of operation controls have different operation goals to be achieved.

[0052] Fig. Figure 3 shows a block diagram illustrating a configuration of a robot control device 10 according to the exemplary embodiment. A robot control device 10 is a device for controlling a robot and comprises: a process target input device 12; a detection device 14; a process control selection device 16; a process control execution device 18; a machine learning (ML) model 20 describing ranges of situations in which process control corresponding to a process target can be achieved; a process control set 22; and an input device 24 for various information elements that inputs different information, including, for example, a process target, a design specification value, an actual device process value, a rule of thumb, a process control, a success determination standard, and a base situation for a process.A data pair generation device 26, which generates a data pair in which situation data and success or failure data are paired, indicating the success or failure of an operation of an operation control; a physics simulation device 28; and a situation range verification device 30, which verifies ranges of situations in which operation control is achievable. A robot control device 10 comprises a model (ML model) 20 in which ranges of situations in which operation control is achievable are described, and obtains data samples for model learning from a design specification value, actual device operation values, rules of thumb, a physics simulation device (physics simulation device 28), etc. This processing is performed automatically, regardless of the level of detail of an operation control.

[0053] An input device 12 for a process target is a processing device that receives at least one process target and outputs at least one process target to a process control selection device 16. An input device 12 for a process target is, for example, a communication interface that receives information from an external source.

[0054] The acquisition device 14 is a processing device that obtains any given element of situation information from elements of situation information and outputs the obtained element of situation information to the process control selection device 16. The acquisition device 14 is, for example, a camera or an accelerometer.The elements of information are information about a robot or an object near the robot, and include: (i) physical properties, which include a physical structure and a shape; (ii) a present state of at least one of a position, orientation, and / or temperature, which is a time-varying property; (iii) a relative relationship between the physical properties or between the present states; and (iv) a combination of the physical properties, a combination of the present states, or a combination of the physical properties and the present state.

[0055] In addition to the aforementioned elements of situational information, situational information may address or include information about states relating to the following: information possessed by an object; an internal state of an object; a mentality, feeling, way of thinking, detail of a thought, and habit of a nearby person or animal; any procedure or process possessed by a nearby person, animal, or device, etc., which may be obtained from light (illumination and shadow), air, sound, speech, appearance, and state of an object, person, or animal through processing using a camera and through direct questioning of an object or person.

[0056] An ML model 20 is a model that receives input situation information and outputs a value that is a binary value, a continuous value, or a multidimensional vector value, indicating the success or failure, or degree of success, of an operation performed by a provided operation control in a situation specified by the received situation information. For example, an ML model 20 is related to a deep neural network (so-called "deep neural network," DNN) or deep learning, which will be discussed later with reference to Fig. 7. It should be noted that a binary value, a continuous value, or a multidimensional vector value indicating success or failure or a degree of success of the operation provided by the operation control is not limited to being a direct numeric expression. It may also be, for example, (i) an indirect numeric expression that indirectly expresses success or failure or a degree of success of the operation using a numeric value that is a combination of the numeric value and an element, (ii) a numeric expression that indicates success or failure or a degree of success of the operation in a way that a person cannot easily understand the details, or (iii) a text format instead of a numeric expression.Additionally, a binary value, a continuous value, or a multidimensional vector value indicating success, failure, or a degree of success of the provided process control process can be, for example, a decision tree comprising a random forest, a Bayesian interference procedure comprising a Bayesian network, or a statistical model based on probability theory. Beyond the above, a binary value, a continuous value, or a multidimensional vector value indicating success, failure, or a degree of success of the provided process control process can be any type of explainable model whose modeled detail is not complete but is understandable to a certain degree to a person.

[0057] Furthermore, the ML model 20 can output a value indicating whether the received situation information matches a situation in which operation control can be properly achieved, or a situation in which an operation control process is successful.

[0058] Additionally, according to Fig. 10 an ML model 20 output a value indicating whether a process control can be executed as intended in a situation specified by the received situation information or not. Fig. Figure 10 shows a representation illustrating an ML model 20 according to the embodiment.

[0059] Furthermore, a value indicating the success, failure, or degree of success of an operation, as output by a machine learning model 20, need not necessarily be a value indicating the degree of success of the operation in a situation where operation control is marginally successful, but can be a degree of success of the operation in a situation where operation control is certain or sufficiently successful. In other words, the degree of success of an operation can indicate the degree of success of an operation in the received situation where it is accurately and reliably estimated that operation control is achievable.Even if a situation is sufficiently captured or observed, some degree of uncertainty or an uncaptured or unobserved internal state remains, and indeed, this uncertainty or uncaptured or unobserved internal state cannot be completely eliminated and is certainly present to some extent. However, if the uncertainty or the uncaptured or unobserved internal state is insignificant, a machine learning model can output a value indicating the degree of estimation that, based on the obtained situation information, a state can be determined as one in which an operation of process control is sufficiently successful, taking into account the uncertainty or the uncaptured or unobserved internal state.

[0060] A process control set 22 is a set of one or more process controls achieved by a robot, or in other words, a set of one or more process controls that are candidates for a process control that can be identified (i.e., selected) by the process control selection device 16. For example, a process control set 22 is a set of control routines stored in memory.

[0061] A process control selection device 16 is a processing device that (i) obtains an output value by providing situation information to a machine learning model 20 for at least one process control, if a process target is received from the process target input device 12 and the situation information is received from the capture device 14; (ii) identifies (i.e., selects) a process control to be executed from the process control set 22 based on the obtained output value; and (iii) communicates the identified process control to the process control execution device 18.

[0062] A process control execution device 18 is a processing device that executes a process control identified by a process control selection device 16. A process control execution device 18 can be a structural element (see Fig. 3, Fig. 8 and Fig. 9), which receives situation information 14a obtained from a detection device 14 by means of a process control selection device 16, and executes a process control based on a value specified by situation information 14a. Optionally, in addition to situation information 14a obtained from a detection device 14, a process control execution device 18 can be a structural element that independently receives, for each process control, information other than situation information 14a, such as a value expressed in a form different from a value specified by situation information 14a, sensor values ​​obtained from various sensors, and information obtained from an external device, and executes a process control based on situation information 14a and the aforementioned values ​​or on the aforementioned values ​​alone.

[0063] An input device 24 for various information elements is a processing device that receives various types of information, such as an operation goal, a design specification value, an actual device operation value, a rule of thumb, an operation control, a success determination standard, and an operation base situation, and outputs various types of information to a data pair generation device 26. The input device 24 for various information elements is, for example, a communication interface that receives information from outside.

[0064] A data pair generation device 26 is an example of a training data generation device that, as training data for a machine learning model 20, generates a data pair in which situation data and success or failure data are paired, indicating the success or failure of an operation of an operation control. The data pair is generated from (i) a design specification value, (ii) an outcome of an operation of an operation control performed by an actual robot, (iii) a rule of thumb developed by a person, and / or (iv) an outcome of an operation of an operation control performed in a physics simulation facility (physics simulation facility 28).If data pairs are generated using the physics simulation device (physics simulation device 28) in which situation data and success or failure data are paired, (i) the data pair generation device 26, in addition to an operation control, provides a condition to the physics simulation device that defines a state in which an operation of the operation control is successful, (ii) the data pair generation device 26 executes the operation control while elements of information, each specifying an environment, are modified variously in the physics simulation device, (iii) the data pair generation device 26 tests whether the operation of the operation control is successful or not, and (iv) based on a result of the testing, the data pair generation device 26 generates a data pair in which situation data and success or failure data are paired.

[0065] A physics simulation device 28 is a processing device that executes a provided process control by means of a simulation as a substitute for a robot and returns a result of the execution.

[0066] A situation domain check unit 30 is a processing unit that checks a (preliminary processing) machine learning model 20 provided for operation controls to determine, for a given operation target and situation, which operation controls are associated with the operation target and situation, or to determine that no operation control is associated with the operation target and situation. If necessary, a new operation control is created and associated according to a preliminary processing procedure.

[0067] It should be noted that each of the structural elements described above can be implemented as software, using, for example, memory for storing a program and data, and a processing unit for executing the program, or as hardware, such as an electronic circuit, exemplified, for instance, by an integrated semiconductor circuit. All or some of the structural elements included in the robot control device 10 as described above can be contained within a robot, or they can be provided in a device outside the robot that can communicate with the robot.

[0068] Furthermore, if a machine learning model is used to implement process control (i.e., an implementation of process control or a concretization of a process to enable the actual implementation of process control in a real-world situation), the machine learning model for implementing process control (hereinafter also referred to as a process control machine learning model) and an ML model 20 can be integrated into a single machine learning model. For example, if a type of input for a process control machine learning model and a type of input for an ML model 20 are the same, the aforementioned single machine learning model receives such a type of input as a common input and outputs each of the values ​​indicating a process control value and a success or failure of the process control that is normally output by the ML model 20.This means that the ML model 20 does not have to be limited to a specific physical configuration. For example, the ML model 20 can be an individual machine learning model created for each operation goal, an individual machine learning model created for each operation control, a single machine learning model corresponding to a variety of operation goals, and a single machine learning model corresponding to a variety of operation controls.

[0069] In addition to the ML model 20, each of the functional structural elements included in the robot control device 10, such as the sensing device 14, the process control selection device 16, the process control set 22, and the process control execution device 18, can be created by a machine learning model. Furthermore, a multitude of functional structural elements can be created by a single machine learning model. Accordingly, some or all of the functional structural elements included in the robot control device 10 can be created by a single machine learning model or by a multitude of machine learning models.

[0070] If a machine learning model is an individual machine learning model created for each process control, the input and output of each machine learning model need not be the same.

[0071] If a machine learning model is not an individual machine learning model created for each process control, but has a single model configuration, the following two cases optionally apply with regard to the output of a model if a process is added and a range of situations is modified.

[0072] If a model: (i) is a model that has a fixed configuration and changes only one output value when a task is added and a range of situations is modified; and (ii) is a model capable of learning about a target task controller using a method in which an output value is unaffected or only slightly affected with respect to a task controller other than the task controller being learned, then, for example, an output of the model is a multidimensional vector value. The model is trained to output a value of a new pattern when a task controller is added.

[0073] If a model: (i) is a model that modifies and / or extends the model configuration and changes an output value if an operation control is added and a range of situations is modified; and (ii) is a model capable of learning about a target operation control using a procedure in which an output value is not affected or is only slightly affected with respect to an operation control other than the operation control being learned from, a new operation control is added by modifying the structure of an output value calculation. For example, an output calculation device for a new operation control is added to the model configuration.

[0074] For the sake of simplicity, the following describes an individual machine learning model that is created for each process control.

[0075] Fig. Figure 4 shows a representation illustrating a preliminary processing operation performed by a robot control device 10 according to the present embodiment. Here, the block diagram shown in Figure 4 differs from the representation shown in Figure 4. Fig. Figure 3 illustrates only blocks that relate to preliminary processing.

[0076] For training the ML model 20, which is to be used for an operation controller, the data pair generation device 26 generates a large number of data pairs, each pairing situation data with success or failure data. These data pairs are the training data for the ML model 20. The generation of data pairs, each indicating a success or failure of an operation of an operation controller in a given situation, produces the following: (i) data pairs generated for different situations, based on a design specification value, an actual device operation value, a rule of thumb according to an operation controller developer, etc.; (ii) Data pairs generated on the basis of a standard for determining whether a process control and a process are successful or not; and (iii) Data pairs generated on the basis of results of an execution of a process control in different situations using a physics simulation device 28 in which a basic (representative) situation for a process control operation is input.

[0077] Fig. Figure 5 shows a diagram illustrating processes (process controls) carried out by a robot control device 10 according to the exemplary embodiment. Here, the processes are described in Fig. The block diagram shown in section 3 illustrates only blocks relating to processes (process controls) performed by the robot control device 10.

[0078] A process control selection device 16 (i) inputs situation data indicating the current situation, obtained from a capture device 14, into abstract models (ML models 20, provided according to the number of process controls corresponding to a process target), each corresponding to a process target entered by the process target input device 12, and each describing the range of situations, and (ii) checks an output from each ML model 20 (each ML model 20 outputs a derived value indicating whether or not a process control corresponding to the process target can be obtained).

[0079] From the output values ​​of the ML models 20, the operation control selection unit 16 determines an operation control that is achievable in the current situation (note that operation controls are achievable if areas overlap) and selects an operation control from an operation control set 22. Subsequently, the operation control execution unit 18 executes the selected operation control. In summary, the operation control selection unit 16 inputs the current situation to ML models 20, infers a success or failure of an operation from each operation control, and accepts and executes an operation control for which no failure is determined.

[0080] Optionally, the operation control selection device 16 (i) may temporarily select operation control candidates that match the operation target based on an operation goal or based on an operation goal and the instruction values ​​described above, (ii) determine a successful candidate using a machine learning model, and (iii) select an operation control based on any of the determination result, the operation goal, or the instruction value.

[0081] Fig. Figure 6 shows a representation illustrating processing carried out during maintenance, in other words, during evaluation and verification of a robot control device 10 according to the exemplary embodiment. Here, the following are from the Fig. The block diagram shown in Figure 3 illustrates maintenance, in other words, evaluation and verification.

[0082] A situation domain checker 30 checks the (preliminary processing-trained) machine learning model 20 for operation controls to determine, for a given operation target and situation, which operation controls are associated with the operation target and situation, or to determine that no operation control is associated with the operation target and situation. If necessary, a new operation control is created and associated according to a preliminary processing procedure.

[0083] Furthermore, if an improvement to an operation control to be performed in a situation is required for an operation goal, the situation-domain review device 30 determines (verifies) an operation control associated with the situation and modifies the operation control. Because operation controls are completely isolated according to their respective domains of situations in which each of the operation controls is achievable (i.e., each operation control is independent), the modification of one operation control does not affect other operation controls.

[0084] According to the above description, a verification of the success or failure of a process control in a situation (verification 1) can be performed by the situation domain verification unit 30, which checks an output of the ML model 20. Furthermore, an improvement of a control to be executed in a situation can be performed by the situation domain verification unit 30 (verification 2), which searches the ML models 20 and modifies a process control associated with the situation.

[0085] Fig. Figure 7 shows a diagram illustrating a specific example of the ML model 20, which is included in a robot control device 10 according to the exemplary embodiment. In the exemplary embodiment, the ML model 20 uses a deep neural network (DNN), specifically a three-dimensional convolutional neural network. This DNN takes as input the presence or absence of objects at a given position in space, types of objects, etc., to output a success or failure (OK or NG) of an operation of an operation control. Optionally, the DNN can be any model related to deep learning.

[0086] According to the above description, the robot control device 10 according to the exemplary embodiment fulfills the following four requirements 1 to 4 for an implementation of the control of a robot.

[0087] Requirement 1 requires the implementation of a flexible control system for a robot corresponding to a wide range of environments / situations.

[0088] A robot control device 10 according to the embodiment can fulfill requirement 1, since a variety of process controls can be combined and there is no restriction to one type of control method.

[0089] Requirement 2 requires the implementation of a robot control system that takes into account a physical event (e.g., an interaction with an object by a target object, such as a shock given to an object by the target object, friction caused by the target object, movement of the target object along an object, and a change in equilibrium due to the target object).

[0090] A robot control device 10 according to the embodiment can fulfill requirement 2, since a physics simulation device is used to determine the physical events described above.

[0091] Requirement 3 requires, for each control routine (hereinafter also referred to as process control), a clarification of a range of environments / situations in which the control routine is successful for executing a process (i.e., a control routine is executed in such a way that a goal is achieved).

[0092] A robot control device 10 according to the exemplary embodiment can fulfill requirement 3 because ranges of situations in which process control is achievable are modeled according to the robot control device 10. This makes it possible to determine a control routine that is achievable, and therefore, assurance of process success, quality, and process integrity can be provided. This enables advanced process management.

[0093] Requirement 4 requires individual adjustment of each process control.

[0094] The robot control device 10 according to the embodiment can fulfill requirement 4, since process controls are handled individually for each area of ​​situations in which process control is achievable.

[0095] It should be noted that a robot controlled by the robot control device 10 according to the exemplary embodiment differs from a robot in, for example, a factory, and is used in an unknown location with correspondingly wide ranges of environments, or in other words, different situations. For this reason, the robot is sometimes deployed in a situation where a control routine for an operation cannot be executed as intended, and this can cause a malfunction of the robot. This malfunction can lead to damage to an object and injury to a person near the robot, and such damage and injury must be prevented.Furthermore, an unforeseeable, unintentional action performed by a person (an adult or a child) near the robot may result in injury to that person, as such a condition could not have been predicted by the robot.

[0096] In such a case, the following can be applied to the robot control device 10. A robot control device 10 according to the exemplary embodiment can be a safety device, a safety mechanism, or an interlock for a control routine for an operation of the robot. The ML model 20, which is included in the robot control device 10 according to the exemplary embodiment, outputs a value that, in the current environment / situation, indicates (i) a success of an operation of an operation control (i.e., the operation control can be executed as intended) or (ii) a failure of the operation of the operation control (i.e., the operation control cannot be executed as intended). Thus, the robot control device 10 according to the exemplary embodiment ensures that a control routine is not executed in an environment / situation in which the control routine cannot be executed as intended (see Fig. 10).

[0097] It should be noted that in the embodiment described above, a process goal and situation information are sent to the process control selection device 16 according to Fig. 5 can be entered, however, an operation goal and situation information or an operation goal, situation information and an (unshown) instruction value can be entered to the ML model 20. Fig. Figure 8 shows a diagram illustrating operations (operation controls) performed by a robot control device 10a according to variant 1 of the embodiment. In variant 1, the ML model 20 is a single machine learning model corresponding to a plurality of operation controls. When the ML model 20 receives operation controls from the operation control selection device 16, it returns output values ​​to the operation control selection device 16, each indicating a success or failure of an operation from each received operation control that achieves the input operation goal 12a in a situation specified by the input situation information 14a. Based on the output values ​​from the ML model 20 for each operation control, the operation control selection device 16 selects one operation control, which is operation control 22a.

[0098] In the embodiment described above, the process control selection device 16 and the ML model 20 are different processing devices according to Fig. 3, however, the process control selection device 16 and the ML model 20 can be integrated into a single processing device. Fig. Figure 9 shows a diagram illustrating processes (process controls) performed by a robot control device 10b according to variant 2 of the embodiment. In variant 2, a machine learning (ML) model 20a has an effect that combines the effects of the process control selection device 16 and the ML model 20 according to the embodiment. That is, the ML model 20a evaluates whether each process control achieves the input process goal 12a in a situation specified by the input situation information 14a, and selects a process control, which is process control 22a, based on the result of the evaluation. The ML model 20a described above can, for example, be implemented by a deep neural network (DNN) with a middle layer corresponding to the process control selection device 16.

[0099] Although a large number of process goals and a large number of process controls are used in the embodiment described above according to Fig. Furthermore, if two are available that are capable of achieving the respective operation goals, the configuration is not limited to the above. There can be a single operation goal for the entire configuration, and the number of operation controls capable of achieving that one operation goal can be one. If there is only one operation goal, the operation goal input device 12 does not need to be provided. However, if there are multiple operation controls that correspond to the single operation goal, it is important that one operation control is selected using a machine learning model 20, even if there is only one operation goal.

[0100] Even if there is only one process control corresponding to a process goal, it is also important to determine, using ML Model 20, whether or not the process control is achieved. If it is determined that process control cannot be achieved, the ongoing processing routine can be terminated, or the processing can be modified by transmitting situation information to a host computer device indicating the current situation. This corresponds to an operation as described above, performed by a safety device or safety mechanism.

[0101] In the generation and acquisition of situational information, which differs from the acquisition of situational information using the sensing device 14, situational information can be obtained by (i) calculating future situational information, (ii) receiving situational information acquired from a sensing device of another robot, (iii) referring to past situational information stored after a certain period of time, and (iv) directly receiving communication such as a response to a request from an object or person. Accordingly, it is possible to acquire broad ranges of situational information, including various perspectives.

[0102] Furthermore, a machine learning model, a process controller, and / or a new action unit can transmit a signal regarding a process controller to a sensing device 14, and the sensing device 14 can perform more advanced sensing based on the signal, taking into account a property of the process controller. For example, if the success of a process controller operation changes due to a subtle difference in the positional relationship between nearby objects, a sensing device 14 can perform sensing focusing on the subtle difference in the positional relationship. This makes it possible to obtain situational information that is more effective in determining whether a process controller operation is successful or not.

[0103] According to the above description, it is possible to output values ​​more precisely and accurately, according to the situation, that indicate a success or failure or degree of success of a process of process control. Industrial applicability

[0104] The present disclosure can be used as a robot control device, in particular as a robot control device that can individually manage and modify the control of operations to be carried out according to the respective situations, while implementing flexible control of a robot according to a wide range of environments / situations.

[0105] A robot control device that controls a robot, and: a machine learning model (ML model) (20) used to describe ranges of environments, or in other words, situations in which control is attainable for a control routine (operation control) that performs an operation to achieve an operation goal; an operation control selection device (16) that selects an appropriate operation control for the current situation based on an output value from the ML model (20); and an operation control execution device (18) that executes the selected operation control. The operation control is a control routine for a robot to achieve an operation goal by the robot detecting an object in the vicinity of the robot that interacts with another object. Reference symbol list 10, 10a, 10b Robot control device 12. Process target input setup 12a Process objective 14 Recording device 14a Situational Information 16 Process control selection device 18 Process control execution device 20, 20a Machine learning model (ML model) 22 Process control set 24 Input devices for various information elements 26 Data pair generation device 28 Physics simulation facility 30 Situational Area Review Device

Claims

[1] A robot control device that controls a robot, the robot control device comprising: a machine learning model that receives input from any given piece of situation information selected from pieces of situation information and outputs an output value that is one of a binary value, a continuous value, and a multidimensional vector value indicating a success or failure or a degree of success of an operation of a provided operation control in a situation indicated by any given piece of received situation information, wherein the pieces of situation information are information about the robot or an object in a vicinity of the robot and include: (i) physical properties including a physical structure and a shape; (ii) a current state of at least one of a position, an attitude, and a temperature, which is a time-varying property;(iii) a relative relationship between the physical properties or between the current states; and (iv) comprise a combination of the physical properties, a combination of the current states, or a combination of the physical properties and the current state; an operation control selection device that obtains the output value for at least one operation control by providing the any given piece of situation information to the machine learning model if the any given piece of situation information is received, and identifies an operation control to be executed based on the output value; and an operation control execution means which executes the operation control identified by the operation control selection means, wherein the operation control to be evaluated by the machine learning model is not a control for a robot to perform a single and primitive movement determined by a single movement direction vector or a target position, but a control for a robot to achieve an operation goal by detecting nearby objects using a sensor with an interaction with other objects. [2] The robot control device according to claim 1, wherein the any given piece of situation information includes at least one of an absolute position, a relative position, a posture, a shape, or a kind of the object. [3] Robot control device according to claim 1 or 2, further comprising: a training data generating device that generates, as training data for the machine learning model, a data pair in which situation data and success or failure data are paired, the data pair being generated from at least one of (i) a design specification value, (ii) a result of an operation of operation control performed by an actual robot, (iii) a rule of thumb developed by a person, and (iv) a result of the operation of operation control performed in a physics simulation device. [4] The robot control device according to claim 3, wherein if the data pair is generated using the physics simulation means, the training data generation means (i) provides the physics simulation means, in addition to an operation control, with a condition defining a state in which an operation of the operation control is successful, (ii) executes the operation control while variously changing, in the physics simulation means, the items of situation information each indicating an environment, (iii) tests whether the operation of the operation control is successful or not, and (iv) generates the data pair based on a result of the testing. [5] The robot control device according to claim 4, wherein, if testing using the physics simulation means determines whether the operation of the operation control is successful or not, the training data generation means identifies at least one of the pieces of situation information each indicating an environment in which the operation of the provided operation control is successful, and samples, from the at least one piece of situation information identified as a start item to be sampled, the pieces of situation information each indicating an environment in which the operation of the operation control is tested.

Images