Constraint condition learning device, constraint condition learning method, and program

Abstract

A limiting condition learning device 1X mainly comprises a conversion means 15X and a limiting condition inference means 16X. The conversion means 15X converts first time-series data pertaining to the state and input of a task-executing robot system over a task execution period into second time-series data represented by a proposition. The limiting condition inference means 16X infers a limiting condition pertaining to the task as a logical expression on the basis of the second time-series data and information indicating whether or not the task has been executed successfully.

Description

[Technical field]

[0001] The present disclosure relates to a constraint condition learning device, a constraint condition learning method, and a storage medium that perform processing related to learning constraint conditions used in motion planning of a robot. [Background technology]

[0002] A system has been proposed that, when a constraint condition is given, formulates a robot motion plan that satisfies the constraint condition. For example, Patent Document 1 discloses a system that generates motion control logic for an entire autonomous system that satisfies a list of set constraint types and control logic that calculates a control command to be output to a drive device. Patent Document 1 also discloses a point that relaxes at least a part of the set constraint types when it is determined that the motion control logic and the control logic are not feasible. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication WO2014 / 141351 Summary of the Invention [Problem to be solved by the invention]

[0004] When formulating a motion plan for a robot to execute a task, it is necessary to store constraints in advance when planning the motion. However, there are problems such as the difficulty and time required to manually set all of these constraints.

[0005] In view of the above-mentioned problems, one of the objects of the present disclosure is to provide a constraint condition learning device, a constraint condition learning method, and a storage medium that are capable of preferably learning constraint conditions necessary for robot motion planning. [Means for solving the problem]

[0006] One aspect of the constraint learning device is A conversion means for converting first time series data relating to a state and an input during an execution period of a task of a robot system that executes a task into second time series data expressed by a proposition; The task is executed based on the second time series data and information on whether the task was successful. necessary to succeed A constraint condition estimation means for estimating a constraint condition as a logical expression; The constraint learning device has the following structure.

[0007] One aspect of the constraint learning method includes: The computer converting first time series data relating to a state and an input of a robot system executing a task during the execution of the task into second time series data expressed by a proposition; The task is executed based on the second time series data and information on whether the task was successful. necessary to succeed Estimate the constraints as logical expressions, This is a constraint learning method.

[0008] One aspect of the program is converting first time series data relating to a state and an input of a robot system executing a task during the execution of the task into second time series data expressed by a proposition; The task is executed based on the second time series data and information on whether the task was successful. necessary to succeed This is a program that causes a computer to execute a process of estimating constraint conditions as logical expressions. Effect of the Invention

[0009] Constraints related to the task can be learned advantageously. [Brief description of the drawings]

[0010] [Figure 1] 1 shows a configuration of a robot system in a first embodiment. [Diagram 2] The hardware configuration of the robot controller is shown below. [Diagram 3] (A) shows an example of the data structure of application information, and (B) shows an example of the data structure of study-related information. [Figure 4] 4 is an example of a functional block diagram showing an overview of processing performed by the constraint condition learning device. [Diagram 5] 1 is a first example of a workspace for a task performed by a mobile robot. [Figure 6] FIG. 11 is a diagram clearly showing the movement trajectory of the robot when the task is successful and when the task is unsuccessful. [Figure 7] 1 is a second example of a workspace for a task performed by a mobile robot. [Figure 8] FIG. 1 is a schematic diagram of the workspace of a task performed by a pick-and-place robot. [Figure 9] 5 is an example of a flowchart showing a processing procedure related to learning of constraint conditions executed by the constraint condition learning device in the first embodiment. [Figure 10] 13 is an example of a functional block of a constraint condition learning device in a modified example. [Figure 11] FIG. 13 shows a schematic configuration diagram of a constraint condition learning device according to a second embodiment. [Figure 12] 13 is an example of a flowchart executed by the constraint condition learning device in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of a constraint condition learning device, a constraint condition learning method, and a storage medium will be described with reference to the drawings.

[0012] First Embodiment (1) System Configuration 1 shows the configuration of a robot system 100 according to the first embodiment. The robot system 100 is a system that learns constraint conditions used in a motion plan for a robot, and mainly includes a constraint condition learning device 1, an instruction device 2, a storage device 4, a robot 5, and a measurement device 7.

[0013] The constraint learning device 1 learns constraints that represent conditions that must be satisfied when the robot 5 is made to execute a predetermined task. These constraints are used in the motion planning of the robot 5. In the motion planning, a process is performed to convert a task expressed by a logical expression such as temporal logic into a sequence for each time step (time interval) of the task that the robot 5 can accept (also called an "motion sequence").

[0014] Furthermore, the constraint learning device 1 performs data communication with the instruction device 2, the storage device 4, the robot 5, and the measurement device 7 via a communication network or by direct wireless or wired communication. For example, the constraint learning device 1 transmits a display control signal related to candidates for constraints generated by learning to the instruction device 2, and receives an input signal (external input signal) by a user related to the candidate from the instruction device 2. Furthermore, when learning constraints, the constraint learning device 1 performs data communication with the storage device 4 to refer to and update various information stored in the storage device 4. Furthermore, the constraint learning device 1 transmits a control signal "S1" related to control of the robot 5 to the robot 5. Furthermore, the constraint learning device 1 receives a measurement signal "S2" from the measurement device 7.

[0015] The instruction device (display device) 2 is a device that receives instructions regarding updating of constraint conditions. The instruction device 2 may be a tablet terminal equipped with an input unit and a display unit, or may be a stationary personal computer.

[0016] The storage device 4 stores various information necessary for learning by the constraint learning device 1. The storage device 4 stores, for example, application information 41, learning-related information 42, proposition time-series data 43, and constraint candidate information 44. The application information 41 is information necessary for motion planning of the robot 5. The learning-related information 42 is information (including parameters) necessary for learning constraints. The proposition time-series data 43 is time-series data expressing, by propositions, the transition of the state and input (more specifically, the control input to the robot 5) of the robot system 100 during the execution period of a task executed by the robot 5 (also simply referred to as the "task execution period"). The constraint candidate information 44 is information expressing candidates for constraints generated by the constraint learning device 1.

[0017] The storage device 4 may be an external storage device such as a hard disk connected to or built into the constraint learning device 1, or may be a storage medium such as a flash memory. The storage device 4 may also be a server device that performs data communication with the constraint learning device 1 via a communication network. In this case, the storage device 4 may be composed of multiple server devices.

[0018] The robot 5 performs work related to a task based on a control signal S1 supplied from the constraint learning device 1. The robot 5 is a robot that operates in, for example, various factories such as an assembly factory or a food factory, or at a logistics site. The robot 5 may be a vertical articulated robot, a horizontal articulated robot, or any other type of robot. The robot 5 may supply the constraint learning device 1 with a signal indicating the progress of the operation sequence specified by the control signal S1, or the like.

[0019] The measurement device 7 is one or more sensors, such as a camera, a range sensor, a sonar, or a combination of these, that detect the state in the workspace in which the task is performed. The measurement device 7 may include a sensor provided on the robot 5, or may include a sensor provided in the workspace. For example, the measurement device 7 may include an external sensor such as a camera provided on the robot 5, an internal sensor that senses the state of the robot 5 (the state of the entire robot 5 or a specific part such as a joint), and the like. In another example, the measurement device 7 may include a self-propelled or flying sensor (including a drone) that moves in the workspace of the robot 5. The measurement device 7 may also include a sensor that detects sounds or the tactile sensation of an object in the workspace. In this way, the measurement device 7 may include various sensors that detect the state in the workspace and are provided at any location.

[0020] The configuration of the robot system 100 shown in FIG. 1 is an example, and various modifications may be made to the configuration. For example, there may be a plurality of robots 5, and the robot system may have a plurality of control objects, such as robot arms, each of which operates independently. The robot 5 may also work in collaboration with other robots, workers, or machine tools operating in the work space. The measuring device 7 may be a part of the robot 5. The instruction device 2 may be configured as the same device as the constraint learning device 1. The constraint learning device 1 may be configured from a plurality of devices. In this case, the plurality of devices constituting the constraint learning device 1 transmit and receive information required to execute a process assigned in advance between these plurality of devices. The constraint learning device 1 and the robot 5 may also be configured as one unit.

[0021] (2) Hardware Configuration 2(A) shows a hardware configuration of the constraint learning device 1. The constraint learning device 1 includes, as hardware, a processor 11, a memory 12, and an interface 13. The processor 11, the memory 12, and the interface 13 are connected via a data bus 10.

[0022] The processor 11 executes a program stored in the memory 12 to function as a controller (arithmetic device) that performs overall control of the constraint learning device 1. The processor 11 is, for example, a processor such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a TPU (Tensor Processing Unit). The processor 11 may be composed of multiple processors. The processor 11 is an example of a computer.

[0023] The memory 12 is composed of various types of volatile and non-volatile memories such as a RAM (Random Access Memory), a ROM (Read Only Memory), and a flash memory. The memory 12 also stores programs for executing the processes executed by the constraint learning device 1. Note that part of the information stored in the memory 12 may be stored in one or more external storage devices (e.g., the storage device 4) capable of communicating with the constraint learning device 1, or may be stored in a storage medium that is detachable from the constraint learning device 1.

[0024] The interface 13 is an interface for electrically connecting the constraint learning device 1 to other devices. These interfaces may be wireless interfaces such as a network adapter for wirelessly transmitting and receiving data to and from other devices, or may be hardware interfaces for connecting to other devices via cables or the like.

[0025] The hardware configuration of the constraint learning device 1 is not limited to the configuration shown in Fig. 2(A). For example, the constraint learning device 1 may be connected to or built-in at least one of a display device, an input device, and a sound output device. The constraint learning device 1 may also be configured to include at least one of an instruction device 2 and a storage device 4.

[0026] 2(B) shows a hardware configuration of the instruction device 2. The instruction device 2 includes, as hardware, a processor 21, a memory 22, an interface 23, an input unit 24a, a display unit 24b, and a sound output unit 24c. The processor 21, the memory 22, and the interface 23 are connected via a data bus 20. The interface 23 is also connected to the input unit 24a, the display unit 24b, and the sound output unit 24c.

[0027] The processor 21 executes a predetermined process by executing a program stored in the memory 22. The processor 21 is a processor such as a CPU or a GPU. The processor 21 generates an input signal by receiving a signal generated by the input unit 24a via the interface 23, and transmits the input signal to the constraint condition learning device 1 via the interface 23. The processor 21 also controls at least one of the display unit 24b and the sound output unit 24c via the interface 23 based on an output control signal received from the constraint condition learning device 1 via the interface 23.

[0028] The memory 22 is configured with various types of volatile and non-volatile memories such as RAM, ROM, flash memory, etc. The memory 22 also stores programs for executing the processes executed by the instruction device 2.

[0029] The interface 23 is an interface for electrically connecting the instruction device 2 to other devices. These interfaces may be wireless interfaces such as network adapters for wirelessly transmitting and receiving data to and from other devices, or may be hardware interfaces for connecting to other devices via cables or the like. The interface 23 also performs interface operations for the input unit 24a, the display unit 24b, and the sound output unit 24c. The input unit 24a is an interface for accepting user input, and corresponds to, for example, a touch panel, a button, a keyboard, a voice input device, or the like. The display unit 24b is, for example, a display, a projector, or the like, and performs display under the control of the processor 21. The sound output unit 24c is, for example, a speaker, and performs sound output under the control of the processor 21.

[0030] 2B. For example, at least one of the input unit 24a, the display unit 24b, and the sound output unit 24c may be configured as a separate device electrically connected to the instruction device 2. The instruction device 2 may be connected to various devices such as a camera, or may have these built-in.

[0031] (3) Data Structure Fig. 3(A) shows an example of the data structure of application information 41. As shown in Fig. 3(A), the application information 41 includes abstract state designation information I1, constraint condition information I2, operation limit information I3, subtask information I4, abstract model information I5, and object model information I6.

[0032] The abstract state designation information I1 is information that designates an abstract state that needs to be defined when generating an action sequence. This abstract state is an abstract state of an object in the working space, and is defined as a proposition to be used in a goal formula, which will be described later. For example, the abstract state designation information I1 designates an abstract state that needs to be defined for each type of task.

[0033] The constraint condition information I2 is information indicating constraint conditions when executing a task. For example, when the task is pick-and-place, the constraint condition information I2 indicates a constraint condition that the robot 5 (robot arm) must not come into contact with an obstacle, a constraint condition that the robots 5 (robot arms) must not come into contact with each other, and the like. Note that the constraint condition information I2 may be information recording constraint conditions appropriate for each type of task. As will be described later, the constraint condition information I2 is updated by candidates for constraint conditions learned by the constraint condition learning device 1.

[0034] The motion limit information I3 indicates information on the motion limits of the robot 5 controlled by the constraint learning device 1. The motion limit information I3 is, for example, information that specifies the upper limit of the speed, acceleration, or angular velocity of the robot 5. Note that the motion limit information I3 may be information that specifies the motion limits for each movable part or joint of the robot 5.

[0035] The subtask information I4 indicates information on subtasks that are components of an operation sequence. A "subtask" is a task that is broken down into units that the robot 5 can accept, and indicates the subdivided operation of the robot 5. For example, if the task is pick-and-place, the subtask information I4 specifies reaching, which is the movement of the robot arm of the robot 5, and grasping, which is the gripping by the robot arm, as subtasks. The subtask information I4 may indicate information on subtasks that can be used for each type of task.

[0036] The abstract model information I5 is information on a model that abstracts dynamics in the working space. The model represented by the abstract model information I5 may be, for example, a model in which real dynamics is abstracted by a hybrid system. In this case, the abstract model information I5 includes information indicating a condition for switching dynamics in the above-mentioned hybrid system. For example, in the case of pick-and-place in which the robot 5 grasps an object to be worked on and moves it to a predetermined position, the condition for switching corresponds to a condition that the object cannot be moved unless it is grasped by the robot 5. The abstract model information I5 includes, for example, information on a model abstracted for each type of task.

[0037] The object model information I6 is information on the object model of each object in the working space to be recognized from the measurement signal S2 generated by the measuring device 7. The above-mentioned objects include, for example, the robot 5, an obstacle, a tool handled by the robot 5 or other object, and a work body other than the robot 5. The object model information I6 includes, for example, information required for the constraint condition learning device 1 to recognize the type, position, posture, and currently executed operation of each of the above-mentioned objects, and three-dimensional shape information such as CAD (Computer Aided Design) data for recognizing the three-dimensional shape of each object. The former information includes parameters of an inference device obtained by learning a learning model in machine learning such as a neural network. For example, when an image is input, this inference device is trained in advance to output the type, position, posture, and the like of an object that is a subject in the image. In addition, when an AR marker for image recognition is attached to a main object such as a target object, information required for recognizing the object by the AR marker may be stored as the object model information I6.

[0038] In addition to the above-mentioned information, the application information 41 may store various information necessary for the constraint learning device 1 to generate the control signal S1. For example, the application information 41 may include information that specifies the workspace of the robot 5.

[0039] Figure 3(B) shows Learning-related information 42 As shown in FIG. 3(B), Learning-related information 42 has proposition definition information I7, operator information I8, constraint condition restriction information I9, and task success determination information I10.

[0040] The proposition definition information I7 is information on the definition of a proposition. For example, the proposition definition information I7 is information in which, for each symbol representing a proposition, a corresponding state and input of the robot system 100 are associated with each other. Note that the proposition definition information I7 may be provided for each type of task executed by the robot 5.

[0041] The operator information I8 is information that specifies operators that can be used in logical expressions (also called "constraint candidate logical expressions") that the constraint learning device 1 generates as candidates for constraints. For example, in the case of linear temporal logic (LTL), the operators specified in the operator information I8 are selected from logical product "∧", logical sum "∨", negation "￢", logical inclusion "⇒", next "○", until "U", eventually "◇", always "□", and the like. Note that the constraint candidate logical expressions may be expressed based on any temporal logic, such as MTL (Metric Temporal Logic) or STL (Signal Temporal Logic), not limited to linear logic, and the operators specified in the operator information I8 may also be operators based on any of these temporal logics.

[0042] The constraint condition limit information I9 is ​​information that specifies limits on the constraint candidate logical formulas generated as constraint condition candidates by the constraint condition learning device 1. The constraint condition limit information I9 is, for example, information that indicates an upper limit on the length of the constraint candidate logical formula (for example, the number of propositions included in the constraint candidate logical formula).

[0043] The task success determination information I10 is information necessary for determining the success or failure of a task performed by the robot 5. In this case, the task success determination information I10 may be, for example, information indicating the success condition of the task, information indicating the failure condition of the task, or information indicating both. For example, the task success determination information I10 is information indicating the time limit for performing the task and the state when the task is successful (or the state when the task is unsuccessful).

[0044] In addition to the above-mentioned information, the proposition time-series data 43 may store various information necessary for the constraint learning device 1 to learn the constraints.

[0045] (4) Processing Overview Next, an overview of the processing of the constraint learning device 1 will be described. The constraint learning device 1 estimates constraints necessary for a task by self-supervised learning. At this time, the constraint learning device 1 converts time-series data representing the state and input of the robot system 100 during the task execution period into proposition time-series data, and estimates constraint candidate logical expressions based on the information on the success / failure of the task and the proposition time-series data. In this way, the constraint learning device 1 appropriately generates constraint candidate logical expressions that are candidates for constraints, and updates the constraint information I2.

[0046] Fig. 4 is an example of a functional block showing an overview of the processing of the constraint learning device 1. The processor 11 of the constraint learning device 1 functionally has a first time-series data acquisition unit 14, a conversion unit 15, a constraint condition estimation unit 16, a learning constraint condition setting unit 17, a robot control unit 18, and an update unit 19. Note that Fig. 4 shows an example of data exchanged between each block, but is not limited to this. The same applies to other functional block diagrams described later.

[0047] The first time-series data acquisition unit 14 acquires time-series data (also referred to as "first time-series data") representing the state and input of the robot system 100 during the task execution period based on the measurement signal S2, etc., and supplies the acquired first time-series data to the conversion unit 15. The first time-series data includes time-series data (also referred to as "time-series input data") of control inputs (e.g., subtasks) to the robot 5 during the task execution period, and time-series data of the state regarding the robot system 100 (also referred to as "time-series state data"). The first time-series data acquisition unit 14 acquires the time-series input data, for example, from the robot 5 or a robot control unit 18 that controls the robot 5. In addition, the first time-series data acquisition unit 14 recognizes the state of an object (including the robot 5) existing in the working space by analyzing the measurement signal S2 using a technology for recognizing the environment of the working space (such as an image processing technology, an image recognition technology, a voice recognition technology, a technology using an RFID (Radio Frequency Identifier)), with reference to the object model information I6, etc. Then, the first time-series data acquisition unit 14 generates the time-series state recognition result as time-series state data. Note that the first time-series data may be continuous values ​​or discrete values.

[0048] Further, the first time series data acquisition unit 14 judges the success or failure of the task executed by the robot 5 based on the task success determination information I10 and the first time series data. In this case, for example, when the task success determination information I10 indicates the time limit of the task and the state (success state) when the task is successful, the first time series data acquisition unit 14 judges whether the state after the time limit from the start of execution of the task has elapsed is the success state based on the first time series data. Note that in this case, the first time series data acquisition unit 14 may judge the success or failure of the task based on any pattern recognition processing technique. Then, the first time series data acquisition unit 14 supplies information indicating the judgment result of the success or failure of the task to the conversion unit 15 together with the corresponding first time series data. The first time series data acquisition unit 14 is an example of the "task success determination means" in the present disclosure.

[0049] The conversion unit 15 converts the first time series data into proposition time series data based on the proposition definition information I7, and stores the converted data in the storage device 4 as proposition time series data 43. In this case, the conversion unit 15 divides the first time series data into state and input data for each time step, and converts the state and input data for each time step into a proposition based on the proposition definition information I7 indicating the correspondence between the proposition and the state and input data. There may be one or more propositions for each time step. Note that the conversion unit 15 may determine whether the task is successful or unsuccessful, instead of the first time series data acquisition unit 14. In this case, the conversion unit 15 may perform the above-mentioned determination based on the proposition time series data 43, instead of the first time series data.

[0050] The constraint condition estimation unit 16 estimates constraint conditions necessary for the robot 5 to succeed in the task executed based on the proposition time series data 43, the operator information I8, and the constraint condition limiting information I9. Specifically, the constraint condition estimation unit 16 generates a constraint candidate logical formula, which is a logical formula that is a candidate for the constraint condition, and supplies information on the constraint candidate logical formula to the update unit 19. In this case, the constraint condition estimation unit 16 may estimate the constraint condition using any method such as an algorithm for a satisfiability problem (SAT) of a logical formula, a decision tree, or Bayesian estimation. In this case, the constraint condition estimation unit 16 may also supply a probability indicating the likelihood of the calculated constraint candidate logical formula as a constraint condition to the update unit 19 together with the constraint candidate logical formula. The constraint condition estimated by the constraint condition estimation unit 16 is a constraint condition that satisfies a criterion for determining whether it has an effect on the success or failure of the task.

[0051] For example, when estimating constraint conditions using Bayesian estimation, if a set of constraint conditions is "Φ" (φ∈Φ), the constraint condition estimation unit 16 estimates "φ" which is the maximum posterior probability. * " is obtained as a candidate constraint formula. In this case, φ * is expressed by the following formula using a set of propositional time series data “X” (π∈X).

[0052]

number

[0053]

number

[0054] The learning constraint condition setting unit 17 sets constraint conditions (also called "learning constraint conditions") to be used in the motion plan of the robot 5 for executing the next task to newly obtain the first time-series data, based on the proposition time-series data 43 and the constraint condition candidate information 44. In this case, the learning constraint condition setting unit 17 selects a constraint condition (e.g., a constraint condition with a probability equal to or less than a predetermined value) according to the probability included in the constraint condition candidate information 44, and sets a constraint condition that contradicts the selected constraint condition (i.e., a constraint condition that always violates the selected constraint condition) as the learning constraint condition. Then, the learning constraint condition setting unit 17 supplies information on the set learning constraint conditions to the robot control unit 18. In this case, the robot control unit 18 formulates a motion plan that satisfies the learning constraint conditions set by the learning constraint condition setting unit 17 in addition to the constraint conditions indicated by the constraint condition information I2. Then, if the task based on the above-mentioned operation plan is successful, the constraint condition estimation unit 16 determines that the constraint condition selected by the above-mentioned learning constraint condition setting unit 17 is unnecessary, and deletes the constraint condition from the constraint condition candidate information 44. The learning constraint condition setting unit 17 may set time-series data of a proposition, or may set a proposition expressed by temporal logic, as the learning constraint condition.

[0055] In addition, even if the probability representing the likelihood is not linked to the constraint candidate logical formula, the learning constraint condition setting unit 17 may randomly (or sequentially select one by one) from the constraint candidate logical formulas registered in the constraint condition candidate information 44 and set a constraint condition that contradicts the selected constraint condition as the learning constraint condition.

[0056] The learning constraint condition setting unit 17 may also perform information amount evaluation and set candidates (hypotheses) of ambiguous and independent constraint conditions as learning constraint conditions. In this case, as an index of ambiguity, for example, the average information amount "H φ " is used.

[0057]

number

[0058]

number

[0059]

number

[0060]

number

[0061] The robot control unit 18 generates an operation sequence to be executed by the robot 5 based on the application information 41 and the learning constraint conditions set by the learning constraint condition setting unit 17, and supplies the robot 5 with a control signal S1 representing the operation sequence. In this case, the robot control unit 18 expresses, for example, a task to be executed by the robot 5 by a logical formula of temporal logic representing a final achievement state, and determines a control input to the robot 5 for each time step by solving an optimization problem based on the logical formula, the constraint conditions, and an abstract model of dynamics represented by the abstract model information I5. Then, the robot control unit 18 converts the control input for each time step into a sequence of subtasks based on the subtask information I4, and supplies the robot 5 with a control signal S1 that specifies the sequence of the subtasks as an operation sequence. In this case, the constraint conditions used by the robot control unit 18 for the operation plan include the constraint conditions indicated by the constraint condition information I2 and the learning constraint conditions set by the learning constraint condition setting unit 17.

[0062] The update unit 19 updates the constraint condition candidate information 44 based on the estimation result by the constraint condition estimation unit 16. In this case, the update unit 19 stores the constraint condition candidate logical formula (and the probability representing the likelihood) generated by the constraint condition estimation unit 16 in the storage device 4 as the constraint condition candidate information 44. The update unit 19 also compares the constraint condition candidate logical formula already stored as the constraint condition candidate information 44 with the constraint condition candidate logical formula newly supplied by the constraint condition estimation unit 16, and if these are substantially the same, includes only one of them in the constraint condition candidate information 44. In this case, the update unit 19 may include the latest constraint condition candidate logical formula supplied by the constraint condition estimation unit 16 in the constraint condition candidate information 44, may include the logical formula with the shorter length in the constraint condition candidate information 44, or may include the logical formula with the higher probability representing the associated likelihood in the constraint condition candidate information 44. In addition, even if the constraint candidate logical formula already stored as the constraint condition candidate information 44 and the constraint candidate logical formula supplied by the constraint condition estimation unit 16 are incompatible conditions (i.e., mutually contradictory conditions), the update unit 19 includes only one of them in the constraint condition candidate information 44.

[0063] Furthermore, when the learning of the constraint condition is completed, the update unit 19 performs a process of integrating the constraint condition candidate information 44 into the constraint condition information I2. In this case, when the constraint condition candidate information 44 contains a constraint condition candidate logical formula that is inconsistent with the confirmed constraint condition (also called "existing constraint condition") indicated by the constraint condition information I2, the update unit 19 accepts an input that allows the user to select whether to adopt the existing constraint condition or the constraint candidate logical formula. In this case, the update unit 19 transmits an output control signal to the instruction device 2 to cause the instruction device 2 to display the existing constraint condition and the constraint candidate logical formula to be selected, and receives an input signal indicating the user's selection result from the instruction device 2. Then, the update unit 19 determines whether to include the existing constraint condition or the constraint candidate logical formula in the constraint condition information I2 based on the input signal received from the instruction device 2. Similarly, the update unit 19 may accept an input that allows the user to select whether to adopt the existing constraint condition or the constraint candidate logical formula even when the constraint condition candidate information 44 contains a constraint condition candidate logical formula similar to the existing constraint condition.

[0064] Here, each of the components of the first time-series data acquisition unit 14, the conversion unit 15, the constraint condition estimation unit 16, the learning constraint condition setting unit 17, the robot control unit 18, and the update unit 19 can be realized, for example, by the processor 11 executing a program. Also, each component may be realized by recording a necessary program in an arbitrary non-volatile storage medium and installing it as necessary. Note that at least a part of each of these components may be realized by any combination of hardware, firmware, and software, without being limited to being realized by software by a program. Also, at least a part of each of these components may be realized by using an integrated circuit that is programmable by a user, such as an FPGA (Field-Programmable Gate Array) or a microcontroller. In this case, the integrated circuit may be used to realize a program composed of each of the above components. Also, at least a part of each of the components may be configured by an ASSP (Application Specific Standard Produce), an ASIC (Application Specific Integrated Circuit), or a quantum computer control chip. In this way, each of the components may be realized by various hardware. The above is the same in other embodiments described later. Furthermore, each of these components may be realized through the cooperation of multiple computers, for example, using cloud computing technology.

[0065] (5) Examples 5 is a first example of a workspace for a task in which the robot 5 is a mobile robot. Here, a task in which the robot 5 reaches a goal area 72 will be considered in a workspace in which the robot 5, obstacles 70 and 73, and a switch 71 exist. Here, the obstacle 73 surrounds the goal area 72 so that the robot 5 cannot reach the goal area 72 when the switch 71 is in its initial off state, and when the switch 71 is in its on state, the obstacle 73 transitions to a state in which the robot 5 can reach the goal area 72 (for example, becomes housed in the floor or wall).

[0066] In this case, information defining propositions related to the goal region 72, the obstacles 70 and 73, and the switch 71 is included in the proposition definition information I7. As an example, the proposition definition information I7 defines a proposition "g" that the robot 5 exists in the goal region 72, a proposition "h1" that the robot 5 interferes with the obstacle 70, a proposition "h2" that the robot 5 interferes with the obstacle 73, and a proposition "p" that the robot 5 presses the switch 71. The proposition definition information I7 may further include a proposition related to the starting point of the robot 5. The operator information I8 is also assumed to specify eventually "◇" and always "□" as usable operators. The task success determination information I10 is also assumed to specify that the robot 5 arrives at the goal region 72 within four time steps (i.e., by t=4 when t=0 is the task start time) as a success condition of the task. Also, it is assumed that the constraint information I2 includes a constraint condition that "the robot 5 does not interfere with the obstacle 70 at any time (□￢h1)."

[0067] Then, the robot control unit 18 of the constraint learning device 1 formulates a motion plan for the robot 5. In this case, when the robot 5 generates a plurality of candidate motion sequences in the motion plan, the robot 5 is caused to execute the motion sequences in sequence.

[0068] Fig. 6 is a diagram showing a movement trajectory "R1" of the robot 5 when the task is successful and a movement trajectory "R2" of the robot 5 when the task is unsuccessful in the task workspace shown in Fig. 5. In Fig. 6, the task start time is t=0, and the positions of the robot 5 at each of t=1 to 4 are shown by circles.

[0069] In this case, the first time series data acquisition unit 14 generates first time series data for the task execution period (the period from t=0 to t=4) based on the control signal S1 and the measurement signal S2, and judges the success or failure of the task based on the task success judgment information I10 and the first time series data. Then, the conversion unit 15 generates proposition time series data when the task is successful, which corresponds to the movement trajectory R1, and proposition time series data when the task is unsuccessful, which corresponds to the movement trajectory R2. Here, the proposition time series data when the task is successful is a sequence of propositions [{}, {p}, {}, {g}] ({} indicates no corresponding proposition) at each time step from t=0 to t=4, and the proposition time series data when the task is unsuccessful is a sequence of propositions [{}, {}, {}, {h2}] at each time step from t=0 to t=4.

[0070] For the sake of simplicity, the proposition time series data includes a maximum of one proposition at each time step, but may include multiple propositions. Examples of task failures are not limited to the pattern corresponding to the movement trajectory R2, and may include multiple patterns. Similarly, examples of task successes are not limited to the pattern corresponding to the movement trajectory R1, and may include multiple patterns.

[0071] Then, the constraint estimation unit 16 of the constraint learning device 1 generates a constraint candidate logical formula based on the proposition time series data when the task is successful and the proposition time series data when the task is unsuccessful. Here, as an example, the constraint estimation unit 16 calculates, as the constraint candidate logical formula, "◇p" representing pressing the switch 71, "□￢h2" representing not interfering with the obstacle 73 at any time, and the like.

[0072] Fig. 7 is a second example of a task workspace when the robot 5 is a mobile robot. In the example of Fig. 7, instead of the proposition h1 regarding the obstacle 70, a proposition "h3" is defined that the robot 5 exists in area A (see dashed frame 75), which is a rectangular area surrounding the obstacle 70. The constraint condition estimation unit 16 then generates the constraint candidate logical formulas "the robot 5 never interferes with the obstacle 70 (□￢h1)" and "the robot 5 never enters area A (□￢h3)" together with probabilities representing the plausibility.

[0073] In this case, the learning constraint condition setting unit 17 sets a constraint condition "◇h3" that contradicts (is not intentionally satisfied by) the constraint candidate logical formula "the robot 5 never enters area A (□￢h3)" based on the above-mentioned probability, and causes the robot control unit 18 to formulate an operation plan for passing through area A. The learning constraint condition setting unit 17 may set time-series data of propositions as the learning constraint conditions based on the above-mentioned information amount evaluation, etc. In this case, for example, the learning constraint condition setting unit 17 sets [{}, {p}, {h3}, {g}] as the learning constraint conditions.

[0074] Then, when the execution of the task based on the operation plan formulated by the robot control unit 18 on the basis of the learning constraint conditions set by the learning constraint condition setting unit 17 is successful, the constraint condition estimation unit 16, since there is a successful example of the task that does not satisfy the constraint condition of □￢h3, deletes the constraint candidate logical formula corresponding to □￢h3 from the constraint condition candidate information 44. This enables the constraint condition learning device 1 to try out the task while changing the constraint conditions, and appropriately update the constraint condition candidate information 44.

[0075] 8 is a schematic diagram of a workspace of a task in which the robot 5 is a robot that performs pick-and-place. Here, a task in which the robot 5 moves the object 75 to be grasped to a goal area 72A by pick-and-place in a workspace in which the robot 5, obstacles 70A and 73A, a switch 71A, and an object 75 to be grasped exist will be considered. Here, the obstacle 73A surrounds the goal area 72A so that the end effector of the robot 5 cannot move onto the goal area 72A when the switch 71A is in the initial off state, and when the switch 71A is in the on state, the state transition is made so that the end effector of the robot 5 can move to the goal area 72A.

[0076] In this case, the proposition definition information I7 includes definitions of propositions related to the goal region 72A, the obstacles 70A and 73A, the switch 71A, and the grasped object 75. As an example, the proposition definition information I7 defines a proposition "g" that the grasped object 75 exists within the goal region 72A, a proposition "h1" that the robot 5 interferes with the obstacle 70A, a proposition "h2" that the robot 5 interferes with the obstacle 73A, and a proposition "p" that the robot 5 presses the switch 71A. The task success determination information I10 also includes a task success condition that the grasped object 75 arrives at the goal region 72A within four time steps. The constraint condition information I2 also includes a constraint condition that "the robot 5 does not interfere with the obstacle 70A at any time (□￢h1)."

[0077] 5 to 7, after formulating an operation plan for the robot 5, the constraint learning device 1 generates first time series data for the task execution period (the period from t=0 to t=4) based on the control signal S1 and the measurement signal S2, etc., and generates proposition time series data when the task is successful and proposition time series data when the task fails. Then, the constraint learning device 1 calculates constraint candidate logical expressions (for example, "◇p", "□￢h2", etc.) based on the proposition time series data when the task is successful and the proposition time series data when the task fails.

[0078] In this way, the constraint learning device 1 can suitably calculate the constraint candidate logical formula based on self-supervised learning, regardless of the type of robot 5 and the type of task.

[0079] (6) Processing Flow FIG. 9 is an example of a flowchart showing a processing procedure related to learning of constraint conditions executed by the constraint condition learning device 1 in the first embodiment.

[0080] First, the first time-series data acquiring unit 14 of the constraint learning device 1 acquires first time-series data representing a state and an input related to the robot 5 during a task execution period (step S11). Then, the first time-series data acquiring unit 14 judges the success or failure of the task executed by the robot 5 based on the first time-series data (step S12).

[0081] Then, the conversion unit 15 converts the first time series data acquired in step S11 into proposition time series data expressed by propositions (step S13). Then, the constraint condition estimation unit 16 generates a constraint candidate logical formula based on the result of the task success / failure judged in step S12 and the proposition time series data generated in step S13, and the update unit 19 updates the constraint condition candidate information 44 based on the constraint candidate logical formula (step S14). In this case, the constraint condition estimation unit 16 may generate probability information representing the likelihood of the constraint candidate logical formula, and the update unit 19 may include the probability information in the constraint condition candidate information 44.

[0082] Then, the update unit 19 determines whether the learning is completed or not (step S15). In this case, the update unit 19 determines that the learning should be ended when, for example, an input signal instructing the end of the learning is received from the instruction device 2. In another example, the update unit 19 may refer to a learning end condition stored in advance in the memory 12 or the like, and determine that the learning should be ended when the end condition is satisfied.

[0083] If it is determined that the learning should not be ended (step S15; No), the learning constraint condition setting unit 17 sets learning constraint conditions to be used in the execution of the next task (step S17). Then, the constraint condition learning device 1 returns the process to step S11 again. In this case, the robot control unit 18 formulates an operation plan for the robot 5 by including the learning constraint conditions set in step S17 in the constraint conditions, and causes the robot 5 to execute the task.

[0084] On the other hand, when the update unit 19 determines that the learning should be terminated (step S15; Yes), it compares the constraint condition information I2 with the constraint condition candidate information 44 (step S16). Then, the update unit 19 determines whether or not there is a contradiction between the constraint condition information I2 and the constraint condition candidate information 44 based on the comparison result between the constraint condition information I2 and the constraint condition candidate information 44 (step S18). Then, when there is a contradiction between the constraint condition information I2 and the constraint condition candidate information 44 (i.e., a contradictory constraint condition is included) (step S18; Yes), the update unit 19 causes the instruction device 2 to present the comparison result between the constraint condition information I2 and the constraint condition candidate information 44, and updates the constraint condition information I2 based on the user input accepted by the instruction device 2 (step S19). In this case, the update unit 19 transmits a display control signal for displaying the contradictory constraint condition in a selectable manner to the instruction device 2, and recognizes the constraint condition to be adopted by receiving from the instruction device 2 an input signal related to the above selection accepted by the input unit 24a.

[0085] On the other hand, if there is no contradiction between the constraint condition information I2 and the constraint condition candidate information 44 (step S18; No), the update unit 19 updates the constraint condition information I2 (step S20). In this case, the update unit 19 registers the constraint conditions included in the constraint condition candidate information 44 in the constraint condition information I2. Note that, if there are constraint conditions that are substantially the same in the constraint condition information I2 and the constraint condition candidate information 44, the update unit 19 may leave only one of them in the constraint condition information I2.

[0086] As described above, the constraint condition learning device 1 in the first embodiment can autonomously learn the constraint conditions by self-supervised learning.

[0087] (7) Variations Next, modifications of the above-described embodiment will be described. The following modifications may be applied to the above-described embodiment in any combination.

[0088] (First Modification) When learning constraints, the constraint learning device 1 may obtain the first time series data by constructing a virtual robot system through simulation, instead of obtaining the first time series data by actually moving the robot 5.

[0089] Fig. 10 is an example of a functional block of the constraint condition learning device 1 in the modified example. The constraint condition learning device 1 shown in Fig. 10 has a simulation execution unit 18A instead of the robot control unit 18. After formulating an operation plan, the simulation execution unit 18A causes a simulation model of the robot to execute an operation sequence represented by the formulated operation plan, instead of transmitting a control signal S1 based on the operation plan to the robot 5. The simulation execution unit 18A generates time series data of the state of the robot system and input to the robot obtained in the simulation as first time series data, and supplies the generated first time series data to the first time series data acquisition unit 14.

[0090] In this way, the constraint learning device 1 can suitably acquire data necessary for learning the constraints and execute learning of the constraints, even by constructing a virtual robot system through simulation.

[0091] (Second Modification) The constraint learning device 1 is not limited to formulating an operation plan for the robot 5 based on temporal logic, and may formulate an operation plan for the robot 5 by any method.

[0092] For example, the application information 41 may include design information such as a flowchart for designing a control input or a subtask sequence corresponding to a task, and the constraint learning device 1 may generate the control input or the subtask sequence by referring to the design information. A specific example of executing a task based on a task sequence designed in advance is disclosed in, for example, JP 2017-39170 A.

[0093] <Second embodiment> 11 shows a schematic configuration diagram of a constraint condition learning device 1X in the second embodiment. The constraint condition learning device 1X mainly includes a conversion means 15X and a constraint condition estimation means 16X. The constraint condition learning device 1X may be composed of a plurality of devices. The constraint condition learning device 1X may be, for example, the constraint condition learning device 1 in the first embodiment.

[0094] The conversion means 15X converts first time series data related to a state and an input during the execution period of a task of a robot system that executes a task into second time series data expressed by a proposition. Here, the robot system may be a system realized by an actual machine, or may be a system virtually constructed by a simulation in a computer. The conversion means 15X may be, for example, the conversion unit 15 in the first embodiment.

[0095] The constraint condition estimation means 16X estimates the constraint condition related to the task as a logical formula based on the second time series data and information on whether the task is successful. The second time series data can be, for example, the proposition time series data in the first embodiment. The constraint condition estimation means 16X can be, for example, the constraint condition estimation unit 16 in the first embodiment.

[0096] 12 is an example of a flowchart executed by the constraint learning device 1X in the second embodiment. First, the conversion means 15X converts first time series data related to a state and an input during the execution period of a task of a robot system that executes a task into second time series data expressed by a proposition (step S21). The constraint estimation means 16X estimates a constraint related to the task as a logical expression based on the second time series data and information on whether the task was successful or not (step S22).

[0097] According to the second embodiment, the constraint learning device 1X can suitably learn the constraints necessary to successfully complete a task.

[0098] In each of the above-described embodiments, the program can be stored using various types of non-transitory computer readable media and can be supplied to a computer, such as a processor. The non-transitory computer readable medium includes various types of tangible storage media. Examples of the non-transitory computer readable medium include magnetic storage media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage media (e.g., magneto-optical disks), CD-ROM (Read Only Memory), CD-R, CD-R / W, and semiconductor memory (e.g., mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, and RAM (Random Access Memory)). The program may also be supplied to a computer by various types of transitory computer readable media. Examples of the transitory computer readable medium include electric signals, optical signals, and electromagnetic waves. The transitory computer readable medium can supply the program to a computer via a wired communication path such as an electric wire and an optical fiber, or via a wireless communication path.

[0099] In addition, some or all of the above-described embodiments (including modified examples, the same applies below) can be described as, but are not limited to, the following notes.

[0100] [Appendix 1] A conversion means for converting first time series data relating to a state and an input during an execution period of a task of a robot system that executes a task into second time series data expressed by a proposition; a constraint condition estimation means for estimating a constraint condition related to the task as a logical expression based on the second time-series data and information on whether the task has been successful; A constraint learning device having the following. [Appendix 2] 2. The constraint learning device according to claim 1, further comprising a task success determination means for determining whether the task has been successful or not based on the first time-series data or the second time-series data. [Appendix 3] A constraint condition learning device as described in Appendix 1 or 2, further comprising a learning constraint condition setting means for setting, based on the logical formula, learning constraint conditions which are constraint conditions to be applied when the robot system next executes the task. [Appendix 4] 4. The constraint condition learning device according to claim 3, wherein the learning constraint condition setting means sets a logical expression that is contradictory to the logical expression as the learning constraint condition. [Appendix 5] The constraint condition learning device according to claim 3, wherein the learning constraint condition setting means sets the learning constraint conditions based on an evaluation index calculated using the second time-series data. [Appendix 6] The constraint condition learning device described in Appendix 5, wherein the learning constraint condition setting means sets the learning constraint condition based on the evaluation index that integrates an average information amount for the second time series data and an average information amount for the constraint condition. [Appendix 7] 7. The constraint condition learning device according to any one of claims 1 to 6, further comprising an update means for updating the existing constraint conditions based on a comparison result between the logical formula and existing constraint conditions that have already been determined. [Appendix 8] The update means includes: storing an estimation result of the constraint condition estimation means including the logical formula as constraint condition candidate information representing candidates of the constraint condition; when an estimation result is obtained by the constraint condition estimation means, updating the constraint condition candidate information based on the estimation result; 8. The constraint condition learning device according to claim 7, wherein when learning of the constraint conditions is completed, the existing constraint conditions are updated based on a comparison result between the constraint condition candidate information and the existing constraint conditions. [Appendix 9] 9. The constraint learning device according to claim 7, wherein the update means displays the comparison result on a display device and updates the existing constraints based on an external input by accepting an external input related to updating the existing constraints. [Appendix 10] The apparatus further includes a first time series data acquisition means for acquiring the first time series data, The first time series data acquisition means acquiring the first time series data based on a measurement result of a workspace of the task during the execution period; Or, 8. The constraint learning device according to claim 1, wherein the first time-series data is acquired based on a simulation result of execution of the task in the robot system virtually constructed by simulation. [Appendix 11] The computer converting first time series data relating to a state and an input of a robot system executing a task during the execution of the task into second time series data expressed by a proposition; estimating a constraint condition for the task as a logical expression based on the second time-series data and information on whether the task has been successful; Constraint learning methods. [Appendix 12] converting first time series data relating to a state and an input of a robot system executing a task during the execution of the task into second time series data expressed by a proposition; A storage medium storing a program for causing a computer to execute a process of estimating constraint conditions related to the task as a logical expression based on the second time series data and information on whether the task was successful.

[0101] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-mentioned embodiments. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. In other words, the present invention naturally includes various modifications and corrections that a person skilled in the art could make in accordance with the entire disclosure, including the claims, and the technical ideas. In addition, the disclosures of the above cited patent documents and the like are incorporated herein by reference. [Industrial Applicability]

[0102] It is ideally suited for robot system integration business, UGVs (Unmanned Ground Vehicles), UAVs (Unmanned Aerial Vehicles), UUVs (Unmanned Underwater Vehicles), robots inside satellites (space stations), and autonomous robot solutions such as planetary probes. [Explanation of symbols]

[0103] 1. 1X Constraint Learning Device 2 Indicating device 4 Storage device 5. Robot 7. Measurement Equipment 41 Application Information 42 Learning-related information 43 Propositional Time Series Data 44 Constraint candidate information 100 Robot System

Claims

1. a conversion means for converting first time series data relating to a state and an input during an execution period of a task of a robot system that executes a task into second time series data expressed by a proposition; a constraint condition estimation means for estimating, as a logical expression, a constraint condition required for the task to be successful, based on the second time-series data and information on whether the task has been successful; A constraint learning device having the following.

2. 2. The constraint learning device according to claim 1, further comprising a task success determination means for determining whether or not the task has been successful based on the first time-series data or the second time-series data.

3. 3. The constraint learning device according to claim 1, further comprising a learning constraint setting means for setting, based on the logical formula, a learning constraint which is a constraint to be applied when the robot system next executes the task.

4. 4. The constraint condition learning device according to claim 3, wherein said learning constraint condition setting means sets a logical expression contradictory to said logical expression as said learning constraint condition.

5. 4. The constraint condition learning device according to claim 3, wherein the learning constraint condition setting means sets the learning constraint condition based on an evaluation index calculated using the second time-series data.

6. 6. The constraint condition learning device according to claim 5, wherein the learning constraint condition setting means sets the learning constraint condition based on the evaluation index obtained by integrating an average information amount for the second time series data and an average information amount for the constraint condition.

7. 7. The constraint condition learning device according to claim 1, further comprising an update means for updating the existing constraint conditions based on a comparison result between the logical expression and existing constraint conditions that have already been determined.

8. The update means includes: storing an estimation result of the constraint condition estimation means including the logical expression as constraint condition candidate information representing candidates of the constraint condition; when an estimation result is obtained by the constraint condition estimation means, updating the constraint condition candidate information based on the estimation result; 8. The constraint condition learning device according to claim 7, wherein when learning of the constraint conditions is completed, the existing constraint conditions are updated based on a result of comparison between the constraint condition candidate information and the existing constraint conditions.

9. The computer converting first time series data relating to a state and an input of a robot system executing a task during the execution of the task into second time series data expressed by a proposition; estimating, as a logical expression, constraint conditions necessary for the task to be successful, based on the second time-series data and information on whether the task has been successful. Constraint learning methods.

10. converting first time series data relating to a state and an input of a robot system executing a task during the execution of the task into second time series data expressed by a proposition; A program that causes a computer to execute a process of estimating, as a logical expression, constraint conditions necessary for the task to be successful, based on the second time series data and information on whether the task has been successful.

Images