Control device, control method, and program

Abstract

A control device 1X mainly includes an abstract state setting means 31X, an environment map generating means 34X, an abstract model generating means 35X, and a control input generating means 36X. The abstract state setting means 31X sets an abstract state of an object in a workspace where a robot performs work. The environment map generating means 34X generates an environment map expressing the certainty of information inside the workspace. The abstract model generating means 35X generates an abstract model expressing the dynamics of the abstract state and the change in the environment map over time. The control input generating means 36X generates control input for the robot on the basis of the abstract model.

Description

[Technical field]

[0001] The present disclosure relates to the technical fields of a control device, a control method, and a storage medium for controlling the operation of a robot. [Background technology]

[0002] A control method has been proposed for controlling a robot required to execute a task given to the robot. For example, Patent Document 1 discloses a controller that learns changes in behavior and environmental state, predicts changes in the environmental state for a given behavior based on the learned changes, and plans a behavior sequence for an autonomous agent from the current state to reach a goal state based on the prediction result. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2007-018490 A Summary of the Invention [Problem to be solved by the invention]

[0004] When planning the motion of a robot to execute a given task, it is sometimes impossible to measure an object or the like that needs to be recognized in order to complete the task in the initial state. Even in this case, it is necessary to properly plan the motion and start the robot's operation.

[0005] In view of the above-mentioned problems, one object of the present disclosure is to provide a control device, a control method, and a storage medium that are capable of suitably controlling a robot. [Means for solving the problem]

[0006] One aspect of the control device is an abstract state setting means for setting an abstract state which is an abstract state of an object in a work space in which the robot performs a task; an environment map generating means for generating an environment map which is a map representing the accuracy of information in the working space; an abstract model generating means for generating an abstract model representing the dynamics of the abstract state and the time change of the environment map; a control input generating means for generating a control input for the robot based on the abstract model; an abstract state comparison means for determining whether or not it is necessary to regenerate the abstract model based on a change in the abstract state during the operation of the robot by the control input; The control device is provided with:

[0007] One aspect of the control method includes: The computer Set an abstract state, which is an abstract state of an object in the workspace where the robot performs the work; generating an environment map that represents the accuracy of information within the workspace; generating an abstract model representing the dynamics of said abstract state and the evolution of said environment map over time; generating control inputs for the robot based on the abstract model; death , During the operation of the robot by the control input, a determination is made as to whether or not the abstract model needs to be regenerated based on a change in the abstract state. It is a control method.

[0008] One aspect of the program is Set an abstract state, which is an abstract state of an object in the workspace where the robot performs the work; generating an environment map which is a map representing the accuracy of information within the working space; generating an abstract model representing the dynamics of said abstract state and the evolution of said environment map over time; generating control inputs for the robot based on the abstract model; death , During the operation of the robot by the control input, a determination is made as to whether or not the abstract model needs to be regenerated based on a change in the abstract state. The computer is caused to execute the process. Effect of the Invention

[0009] A control input for moving the robot can be suitably generated. [Brief description of the drawings]

[0010] [Figure 1] 1 shows a configuration of a robot control system in a first embodiment. [Diagram 2] The hardware configuration of the robot controller is shown below. [Diagram 3] 13 shows an example of a data structure of application information. [Figure 4] 2 is an example of a functional block of a robot controller. [Diagram 5] 1 shows an overhead view of a workspace when a target task is pick-and-place. [Figure 6] 1 shows an overhead view of a workspace of a robot when the robot is a moving body. [Figure 7] 1 is a bird's-eye view of a working space, which diagrammatically illustrates the relationship between an unmeasured space in an initial state and a measured space. [Figure 8] FIG. 13 is an overhead view of the working space, which shows a schematic relationship between an unmeasured space and a measured space after the robot has moved. [Figure 9] 4 is an example of a flowchart showing an overview of a robot control process executed by a robot controller in the first embodiment. [Figure 10] FIG. 13 shows a schematic configuration diagram of a control device according to a second embodiment. [Figure 11] 13 is an example of a flowchart executed by the control device in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of a control device, a control method, and a storage medium will be described with reference to the drawings.

[0012] First Embodiment (1) System Configuration 1 shows the configuration of a robot control system 100 according to the first embodiment. The robot control system 100 mainly includes a robot controller 1, an instruction device 2, a storage device 4, a robot 5, and a measuring device 7.

[0013] When a task (also called a "target task") to be executed by the robot 5 is specified, the robot controller 1 converts the target task into a sequence of time steps (time increments) of simple tasks that the robot 5 can accept, and controls the robot 5 based on the generated sequence.

[0014] The robot controller 1 also performs data communication with the instruction device 2, the storage device 4, the robot 5, and the measurement device 7 via a communication network or by direct wireless or wired communication. For example, the robot controller 1 receives input signals from the instruction device 2 regarding the specification of a target task, the generation or update of application information, etc. The robot controller 1 also transmits a predetermined output control signal to the instruction device 2, causing the instruction device 2 to execute a predetermined display or sound output. The robot controller 1 also transmits a control signal "S1" regarding the control of the robot 5 to the robot 5. The robot controller 1 also receives a measurement signal "S2" from the measurement device 7.

[0015] The instruction device 2 is a device that receives instructions from a worker to the robot 5. The instruction device 2 performs a predetermined display or sound output based on an output control signal supplied from the robot controller 1, and supplies an input signal generated based on an input by the worker to the robot controller 1. The instruction device 2 may be a tablet terminal equipped with an input unit and a display unit, or may be a stationary personal computer.

[0016] The storage device 4 has an application information storage unit 41. The application information storage unit 41 stores application information required to generate an operation sequence, which is a sequence to be executed by the robot 5, from a target task. Details of the application information will be described later with reference to FIG. 3. The storage device 4 may be an external storage device such as a hard disk connected to or built into the robot controller 1, or may be a storage medium such as a flash memory. The storage device 4 may also be a server device that performs data communication with the robot controller 1 via a communication network. In this case, the storage device 4 may be composed of multiple server devices.

[0017] The robot 5 performs work related to the target task based on a control signal S1 supplied from the robot controller 1. The robot 5 is a robot that operates, for example, in various factories such as an assembly factory or a food factory, or at a logistics site. The robot 5 may be a vertical articulated robot, a horizontal articulated robot, or any other type of robot. The robot 5 may supply a status signal indicating the status of the robot 5 to the robot controller 1. This status signal may be an output signal of a sensor (internal sensor) that detects the status (position, angle, etc.) of the entire robot 5 or a specific part such as a joint, or may be a signal indicating the progress status of the sequence of the operation plan of the robot 5 represented by the control signal S1.

[0018] The measurement device 7 is one or more sensors (external sensors) such as a camera, a range sensor, a sonar, or a combination of these that detect the state in the workspace in which the target task is executed. The measurement device 7 may include a sensor provided on the robot 5, or may include a sensor provided in the workspace. In FIG. 1, the measurement device 7 includes an external sensor such as a camera provided on the robot 5, and the measurement range changes according to the operation of the robot 5. In another example, the measurement device 7 may include a self-propelled or flying sensor (including a drone) that moves in the workspace of the robot 5. The measurement device 7 may also include a sensor that detects sounds or the tactile sensation of an object in the workspace. In this way, the measurement device 7 may include various sensors that detect the state in the workspace and are provided at any location.

[0019] The configuration of the robot control system 100 shown in FIG. 1 is an example, and various modifications may be made to the configuration. For example, there may be a plurality of robots 5, and there may be a plurality of control objects, such as robot arms, each of which operates independently. Even in these cases, the robot controller 1 transmits a control signal S1 representing a sequence that specifies the operation of each robot 5 or each control object to the target robot 5 based on the target task. The robot 5 may also perform a collaborative work with other robots, workers, or machine tools operating in the workspace. The measuring device 7 may also be a part of the robot 5. The instruction device 2 may also be configured as the same device as the robot controller 1. The robot controller 1 may also be configured from a plurality of devices. In this case, the plurality of devices that configure the robot controller 1 transmit and receive information required to execute a process assigned in advance between these plurality of devices. The robot controller 1 and the robot 5 may also be configured as one unit.

[0020] (2) Hardware Configuration 2(A) shows the hardware configuration of the robot controller 1. The robot controller 1 includes, as hardware, a processor 11, a memory 12, and an interface 13. The processor 11, the memory 12, and the interface 13 are connected via a data bus 10.

[0021] The processor 11 executes a program stored in the memory 12 to function as a controller (arithmetic device) that performs overall control of the robot controller 1. The processor 11 is, for example, a processor such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a TPU (Tensor Processing Unit). The processor 11 may be composed of multiple processors. The processor 11 is an example of a computer.

[0022] The memory 12 is composed of various types of volatile and non-volatile memories such as a RAM (Random Access Memory), a ROM (Read Only Memory), and a flash memory. The memory 12 also stores programs for executing the processes executed by the robot controller 1. Note that part of the information stored in the memory 12 may be stored in one or more external storage devices (e.g., the storage device 4) capable of communicating with the robot controller 1, or may be stored in a storage medium that is detachable from the robot controller 1.

[0023] The interface 13 is an interface for electrically connecting the robot controller 1 to other devices. These interfaces may be wireless interfaces such as a network adapter for wirelessly transmitting and receiving data to and from other devices, or may be hardware interfaces for connecting to other devices via cables or the like.

[0024] The hardware configuration of the robot controller 1 is not limited to the configuration shown in Fig. 2(A). For example, the robot controller 1 may be connected to or built-in at least one of a display device, an input device, and a sound output device. The robot controller 1 may also be configured to include at least one of an instruction device 2 and a storage device 4.

[0025] 2(B) shows a hardware configuration of the instruction device 2. The instruction device 2 includes, as hardware, a processor 21, a memory 22, an interface 23, an input unit 24a, a display unit 24b, and a sound output unit 24c. The processor 21, the memory 22, and the interface 23 are connected via a data bus 20. The interface 23 is also connected to the input unit 24a, the display unit 24b, and the sound output unit 24c.

[0026] The processor 21 executes a predetermined process by executing a program stored in the memory 22. The processor 21 is a processor such as a CPU or a GPU. The processor 21 generates an input signal by receiving a signal generated by the input unit 24a via the interface 23, and transmits the input signal to the robot controller 1 via the interface 23. The processor 21 also controls at least one of the display unit 24b and the sound output unit 24c via the interface 23 based on an output control signal received from the robot controller 1 via the interface 23.

[0027] The memory 22 is configured with various types of volatile and non-volatile memories such as RAM, ROM, flash memory, etc. The memory 22 also stores programs for executing the processes executed by the instruction device 2.

[0028] The interface 23 is an interface for electrically connecting the instruction device 2 to other devices. These interfaces may be wireless interfaces such as network adapters for wirelessly transmitting and receiving data to and from other devices, or may be hardware interfaces for connecting to other devices via cables or the like. The interface 23 also performs interface operations for the input unit 24a, the display unit 24b, and the sound output unit 24c. The input unit 24a is an interface for accepting user input, and corresponds to, for example, a touch panel, a button, a keyboard, a voice input device, or the like. The display unit 24b is, for example, a display, a projector, or the like, and performs display under the control of the processor 21. The sound output unit 24c is, for example, a speaker, and performs sound output under the control of the processor 21.

[0029] 2B. For example, at least one of the input unit 24a, the display unit 24b, and the sound output unit 24c may be configured as a separate device electrically connected to the instruction device 2. The instruction device 2 may be connected to various devices such as a camera, or may have these built-in.

[0030] (3) Application Information Next, the data structure of the application information stored in the application information storage unit 41 will be described.

[0031] Fig. 3 shows an example of the data structure of the application information. As shown in Fig. 3, the application information includes abstract state designation information I1, constraint condition information I2, operation limit information I3, subtask information I4, dynamics model information I5, object model information I6, map update model information I7, and map and object prior information I8.

[0032] The abstract state designation information I1 is information that designates an abstract state that needs to be defined when generating an action sequence. This abstract state is an abstract state of an object in the working space, and is defined as a proposition to be used in a goal logical formula, which will be described later. For example, the abstract state designation information I1 designates an abstract state that needs to be defined for each type of target task.

[0033] The constraint information I2 is information indicating constraint conditions when executing a target task. For example, when the target task is pick-and-place, the constraint information I2 indicates a constraint condition that the robot 5 (robot arm) must not come into contact with an obstacle, a constraint condition that the robots 5 (robot arms) must not come into contact with each other, etc. Note that the constraint information I2 may be information recording constraint conditions appropriate for each type of target task.

[0034] The motion limit information I3 indicates information related to the motion limit of the robot 5 controlled by the robot controller 1. The motion limit information I3 is, for example, information that specifies the upper limit of the speed, acceleration, or angular velocity of the robot 5. Note that the motion limit information I3 may be information that specifies the motion limit for each movable part or joint of the robot 5.

[0035] The subtask information I4 indicates information on subtasks that are components of an operation sequence. A "subtask" is a task obtained by breaking down a target task into units that the robot 5 can accept, and indicates a subdivided operation of the robot 5. For example, if the target task is pick-and-place, the subtask information I4 specifies reaching, which is the movement of the robot arm of the robot 5, and grasping, which is the gripping by the robot arm, as subtasks. The subtask information I4 may indicate information on subtasks that can be used for each type of target task.

[0036] The dynamics model information I5 is information on a dynamics model that abstracts the dynamics in the working space. For example, the dynamics model may be a model in which real dynamics is abstracted by a hybrid system. In this case, the dynamics model information I5 includes information indicating the conditions for switching the dynamics in the above-mentioned hybrid system. For example, in the case of pick-and-place in which the robot 5 grasps an object (also called an "object") to be worked on by the robot 5 and moves it to a predetermined position, the switching condition corresponds to a condition that the object cannot be moved unless it is grasped by the robot 5. The dynamics model information I5 has information on a dynamics model suitable for each type of target task.

[0037] The object model information I6 is information on the object model of each object in the working space to be recognized from the measurement signal S2 generated by the measuring device 7. The above-mentioned objects include, for example, the robot 5, an obstacle, a tool handled by the robot 5 or other object, and a work body other than the robot 5. The object model information I6 includes, for example, information required for the robot controller 1 to recognize the type, position, posture, and currently executed operation of each of the above-mentioned objects, and three-dimensional shape information such as CAD (Computer Aided Design) data for recognizing the three-dimensional shape of each object. The former information includes parameters of an inference device obtained by learning a learning model in machine learning such as a neural network. For example, when an image is input, this inference device is trained in advance to output the type, position, posture, and the like of an object that is a subject in the image. In addition, when an AR marker for image recognition is attached to a main object such as a target object, information required for recognizing the object by the AR marker may be stored as the object model information I6.

[0038] The map update model information I7 is a model ("map update model b") for updating a map (also called "environment map b") that represents the accuracy of information collected in the working space (environment) of the robot 5. + " (also called "information about the

[0039] The environment map b is generated by the robot controller 1. The environment map b may be, for example, information that discretely represents the accuracy of information for each grid when the target space (two-dimensional or three-dimensional space) is divided into a grid, or may be information that continuously represents the accuracy of information in the target space. The environment map b may also be information of accuracy represented in a space in which an axis representing a concept such as time is added to the axis of a physical two-dimensional or three-dimensional space. The environment map b may also be one in which the accuracy attenuates over time. For example, when the measurement device 7 is a mobile robot that explores a maze or the like, the situation may change over time even at the measured location, and the accuracy of the information decreases over time after the measurement. In consideration of the above, the robot controller 1 may update the environment map b so that the accuracy represented by the environment map b is attenuated according to the elapsed time after the measurement.

[0040] Map update model b + is a model for predicting the time change of the environment map b, and is expressed by a function that takes the predicted input to the robot 5, the set abstract state, and the current environment map b as arguments. + is generated in advance, taking into consideration, for example, the installation position, installation direction (angle) of the measurement device 7 installed on the robot 5, the viewing angle of the measurement device 7, the measurable distance, etc. + This will be described in detail with reference to FIGS. 7 and 8 below.

[0041] The map / object prior information I8 represents prior information on the environment map b and the object for which the abstract state is to be set. The map / object prior information I8 is information generated before the robot controller 1 plans the operation of the robot 5, and may include, for example, information representing the initial state of any object, or may include information representing the initial state of the environment map b. The map update model information I7 and the map / object prior information I8 may be information generated based on an external input by a user using the instruction device 2.

[0042] In addition to the above-mentioned information, the application information storage unit 41 may store various information necessary for the robot controller 1 to generate the control signal S1. For example, the application information storage unit 41 may store information that specifies the working space of the robot 5.

[0043] (4) Processing Overview Next, an overview of the process of the robot controller 1 will be described. In summary, the robot controller 1 has an environment map b and a map update model b + and performs a motion plan for the robot 5 taking into account the uncertainty of the environment. Then, while the robot 5 is moving based on the control signal S1, the robot controller 1 updates the environment map b based on the measurement signal S2, and resets the abstract model and regenerates the control signal S1 in response to changes in the abstract state. In this way, the robot controller 1 appropriately controls the robot 5 so that the target task is completed even in an uncertain environment.

[0044] Fig. 4 is an example of a functional block showing an overview of the processing of the robot controller 1. The processor 11 of the robot controller 1 functionally includes an abstract state setting unit 31, a target logical formula generating unit 32, a time step logical formula generating unit 33, an environment map updating unit 34, an abstract model generating unit 35, a control input generating unit 36, a robot control unit 37, and an abstract state comparing unit 38. Note that Fig. 4 shows an example of data exchanged between each block, but is not limited to this. The same applies to other functional block diagrams described later.

[0045] The abstract state setting unit 31 sets an abstract state in the working space based on the measurement signal S2, the abstract state designation information I1, and the object model information I6 supplied from the measurement device 7. In this case, when the abstract state setting unit 31 receives the measurement signal S2, it refers to the object model information I6 and the like to recognize the attributes such as the type of each object in the working space and the state such as the position and posture that need to be considered when executing the target task. The state recognition result is expressed as, for example, a state vector. Then, based on the recognition result for each object, the abstract state setting unit 31 defines a proposition for expressing each abstract state that needs to be considered when executing the target task as a logical formula. The abstract state setting unit 31 supplies information indicating the set abstract state (also called "abstract state setting information IS") to the target logical formula generating unit 32.

[0046] Furthermore, during a period during which the robot 5 is operating based on the control signal S1 (also referred to as a "robot operating period Tw"), the abstract state setting unit 31 recognizes the latest abstract state of the object in the working space based on the latest measurement signal S2 at predetermined time intervals (e.g., at predetermined time steps).Then, the abstract state setting unit 31 supplies information representing the abstract state recognized based on the latest measurement signal S2 (also referred to as a "measured abstract state Stm") to the abstract state comparing unit 38.

[0047] Based on the abstract state setting information IS, the target logical formula generation unit 32 converts the specified goal task into a logical formula of temporal logic (also called "target logical formula Ltag") that represents the final achieved state. In this case, the target logical formula generation unit 32 adds a constraint condition that should be satisfied in the execution of the goal task to the target logical formula Ltag by referring to the constraint condition information I2 from the application information storage unit 41. Then, the target logical formula generation unit 32 supplies the generated target logical formula Ltag to the time-step logical formula generation unit 33.

[0048] The time-step logical formula generation unit 33 converts the target logical formula Ltag supplied from the target logical formula generation unit 32 into a logical formula (also called a "time-step logical formula Lts") that represents a state at each time step. Then, the time-step logical formula generation unit 33 supplies the generated time-step logical formula Lts to the control input generation unit 36.

[0049] The environment map update unit 34 generates an environment map b in an initial state based on the map / object prior information I8 and the measurement signal S2 at the time of motion planning before the robot 5 is operated. The environment map update unit 34 also updates the environment map b based on the measurement signal S2 at a predetermined time interval (for example, at each predetermined time step) during the robot motion period Tw. The environment map update unit 34 supplies the generated or updated environment map b to the abstract model generation unit 35 and the control input generation unit 36. Note that the map / object prior information I8 does not have to exist. In this case, the environment map update unit 34 generates an environment map b representing the accuracy of information in the initial state of the working space based on the measurement signal S2. The environment map update unit 34 may also update the environment map b so that the accuracy represented by the environment map b is attenuated according to the elapsed time since the measurement. In other words, the environment map update unit 34 may update the environment map b so that the accuracy in the space measured by the measurement device 7 is attenuated based on the elapsed time since the measurement.

[0050] The abstract model generation unit 35 generates a dynamics model that abstracts the real dynamics in the working space and a map update model b based on the dynamics model information I5, the map update model information I7, and the abstract state setting information IS. + The method of generating the abstract model Σ will be described later. The abstract model generation unit 35 supplies the generated abstract model Σ to the control input generation unit 36.

[0051] The control input generation unit 36 ​​determines a control input to the robot 5 for each time step that satisfies the time step logical formula Lts supplied from the time step logical formula generation unit 33 and the abstract model Σ supplied from the abstract model generation unit 35 and optimizes the evaluation function. Then, the control input generation unit 36 ​​supplies information on the control input to the robot 5 for each time step (also referred to as "control input information Icn") to the robot control unit 37 and the abstract state comparison unit 38.

[0052] The robot control unit 37 generates a control signal S1 representing a sequence of subtasks that can be interpreted by the robot 5, based on the control input information Icn supplied from the control input generation unit 36 ​​and the subtask information I4 stored in the application information storage unit 41. Then, the robot control unit 37 supplies the control signal S1 to the robot 5 via the interface 13. Note that the robot 5 may have a function equivalent to the robot control unit 37, instead of the robot controller 1. In this case, the robot 5 executes a planned operation for each time step, based on the control input information Icn supplied from the robot controller 1.

[0053] The abstract state comparison unit 38 judges whether or not it is necessary to regenerate the abstract model Σ based on the change in the abstract state during the robot operation period Tw. In this case, the abstract state comparison unit 38 compares the measured abstract state Stm (i.e., the measured current abstract state) supplied from the abstract state setting unit 31 with the current abstract state predicted based on the control input information Icn (also called the "predicted abstract state Stp"). Then, when the abstract state comparison unit 38 judges that there is a substantial difference between the measured abstract state Stm and the predicted abstract state Stp, it judges that it is necessary to regenerate the abstract model Σ, and supplies the measured abstract state Stm to the abstract model generation unit 35.

[0054] Here, each of the components of the abstract state setting unit 31, the target logical formula generating unit 32, the time step logical formula generating unit 33, the environment map updating unit 34, the abstract model generating unit 35, the control input generating unit 36, the robot control unit 37, and the abstract state comparing unit 38 can be realized, for example, by the processor 11 executing a program. Also, each component may be realized by recording a necessary program in an arbitrary non-volatile storage medium and installing it as necessary. Note that at least a part of each of these components may be realized by any combination of hardware, firmware, and software, without being limited to being realized by software by a program. Also, at least a part of each of these components may be realized by using an integrated circuit programmable by a user, such as an FPGA (Field-Programmable Gate Array) or a microcontroller. In this case, the integrated circuit may be used to realize a program composed of each of the above components. Also, at least a part of each of the components may be configured by an ASSP (Application Specific Standard Produce), an ASIC (Application Specific Integrated Circuit), or a quantum computer control chip. In this way, each component may be realized by various hardware. The above is the same in other embodiments described later. Furthermore, each of these components may be realized through the cooperation of multiple computers, for example, using cloud computing technology.

[0055] (5) Details of each processing section Next, the details of the processes executed by each processing unit described with reference to FIG. 4 will be described in order.

[0056] (5-1) Abstract State Setting Part First, the abstract state setting unit 31 refers to the object model information I6 and analyzes the measurement signal S2 using a technology for recognizing the environment of the work space (image processing technology, image recognition technology, voice recognition technology, technology using RFID (Radio Frequency Identifier), etc.), thereby recognizing the state and attributes (type, etc.) of the object present in the work space. The image recognition technology described above includes semantic segmentation based on deep learning, model matching, or recognition using an AR marker, etc. The above recognition result includes information such as the type, position, and posture of the object in the work space. In addition, the object in the work space is, for example, the robot 5, objects such as tools or parts handled by the robot 5, obstacles, and other work bodies (people or other objects performing work other than the robot 5), etc.

[0057] Next, the abstract state setting unit 31 sets an abstract state in the workspace based on the object recognition result based on the measurement signal S2 etc. and the abstract state designation information I1 acquired from the application information storage unit 41. In this case, the abstract state setting unit 31 first refers to the abstract state designation information I1 and recognizes the abstract state to be set in the workspace. The abstract state to be set in the workspace differs depending on the type of goal task. Therefore, when the abstract state to be set for each type of goal task is specified in the abstract state designation information I1, the abstract state setting unit 31 refers to the abstract state designation information I1 corresponding to the specified goal task and recognizes the abstract state to be set.

[0058] Fig. 5 shows an overhead view of a workspace when a pick-and-place is set as a target task. In the workspace shown in Fig. 5, there are two robot arms 52a and 52b, four objects 61 (61a to 61d), an obstacle 62, and an area G which is the destination of the object 61.

[0059] In this case, first, the abstract state setting unit 31 recognizes the state of each object in the working space. Specifically, the abstract state setting unit 31 recognizes the state of the target object 61, the state of the obstacle 62 (here, the existence range, etc.), the state of the robot 5, the state of the area G (here, the existence range, etc.), etc.

[0060] Here, the abstract state setting unit 31 recognizes the position vectors "x1" to "x4" of the centers of the objects 61a to 61d as the positions of the objects 61a to 61d. In addition, the abstract state setting unit 31 recognizes the position vector "x r1 " and the position vector of the robot hand 53b "x r2 " are recognized as the positions of the robot arms 52a and 52b. Note that these position vectors x1 to x4, x r1 ,x r2 may be defined as a state vector including various elements related to the state, such as an element related to the attitude (angle) of the corresponding object and an element related to the velocity.

[0061] Similarly, the abstract state setting unit 31 recognizes the existence range of the obstacle 62, the existence range of the area G, etc. Note that, for example, when the abstract state setting unit 31 regards the obstacle 62 as a rectangular parallelepiped and the area G as a rectangle, it recognizes the position vectors of each vertex of the obstacle 62 and the area G.

[0062] The abstract state setting unit 31 also determines the abstract state to be defined in the target task by referring to the abstract state designation information I1. In this case, the abstract state setting unit 31 determines a proposition indicating an abstract state based on the recognition results regarding objects present in the workspace (e.g., the number of each type of object) and the abstract state designation information I1.

[0063] In the example of FIG. 5, the abstract state setting unit 31 assigns identification labels "1" to "4" to the objects 61a to 61d recognized based on the measurement signal S2, etc. Also, the abstract state setting unit 31 assigns the proposition "g i In addition, the abstract state setting unit 31 assigns an identification label "O" to the obstacle 62 and defines a proposition "o i Furthermore, the abstract state setting unit 31 defines a proposition "h" that the robot arms 52 interfere with each other.

[0064] In this way, the abstract state setting unit 31 recognizes the abstract state to be defined, and determines the proposition that represents the abstract state (in the above example, g i ,o i , h, etc.) are defined according to the number of objects 61, the number of robot arms 52, the number of obstacles 62, the number of robots 5, etc. Then, the abstract state setting unit 31 supplies information representing the set abstract state (including the proposition and state vector representing the abstract state) to the target logical formula generation unit 32 as abstract state setting information IS.

[0065] Fig. 6 shows an overhead view of the working space (operating range) of the robot 5 when the robot 5 is a moving body. In the working space shown in Fig. 6, there are two robots 5A and 5B, an obstacle 72, and an area G which is the destination of the robots 5A and 5B.

[0066] In this case, first, the abstract state setting unit 31 recognizes the state of each object in the working space. Specifically, the abstract state setting unit 31 recognizes the positions, postures, and moving speeds of the robots 5A and 5B, the range of the obstacle 72, and the area G. Then, the abstract state setting unit 31 sets a state vector "x1" representing the position, posture (and moving speed) of the robot 5A, and a state vector "x2" representing the position, posture (and moving speed) of the robot 5B. In addition, the abstract state setting unit 31 represents the robots 5A and 5B by robot "i" (i=1 to 2), and sets the proposition "g" that the robot i is present in the area G, which is the target location where the robot i is to be placed. i In addition, the abstract state setting unit 31 assigns an identification label "O" to the obstacle 72 and defines the proposition "o i Furthermore, the abstract state setting unit 31 defines a proposition "h" that the robots i interfere with each other.

[0067] In this way, even when the robot 5 is a moving object, the abstract state setting unit 31 recognizes the abstract state to be defined and determines the proposition that represents the abstract state (in the above example, g i ,o i, h, etc.) can be suitably set. Then, the abstract state setting unit 31 supplies information indicating the proposition expressing the abstract state to the target logical formula generating unit 32 as abstract state setting information IS.

[0068] The set task may be one in which the robot 5 moves and also performs pick-and-place (i.e., a task equivalent to a combination of the examples in Fig. 5 and Fig. 6). Even in this case, the abstract state setting unit 31 generates abstract state setting information IS that represents an abstract state including both the examples in Fig. 5 and Fig. 6 and a proposition that represents the abstract state.

[0069] Here, a case will be described in which, among the objects (including areas such as area G) for which an abstract state should be set, there is an object (also called an "unset object") for which an abstract state cannot be set due to reasons such as not being able to be measured by the measuring device 7. The object for which an abstract state should be set is, for example, an object designated in the abstract state designation information I1 as an object for which an abstract state should be set. The unset object may be an object on which the robot 5 is to work, or may be the goal point of the object or the robot 5 (area G in FIG. 5 or FIG. 6).

[0070] In this case, the abstract state setting unit 31 defines a proposition regarding the undefined object while leaving the state vector representing the position, etc., of the undefined object undetermined. For example, in the example of FIG. 6, if the area G is in the blind spot of the measuring device 7 due to an obstacle 72 and cannot be measured by the measurement signal S2, the abstract state setting unit 31 defines a proposition regarding the area G (e.g., g i etc.)

[0071] The unspecified object may be an object in a blind spot formed by an obstacle, an object that is farther away than the measurement distance of the measurement device 7, or an object contained in a case or the like. For example, when a target task that requires an action to take an object out of a box is set, the object cannot be measured by the measurement device 7 until the lid of the box is opened. In this case, the object is treated as an unspecified object until the robot 5 opens the lid of the box and the robot controller 1 recognizes the object based on the measurement signal S2.

[0072] In addition, when the position, etc. of an object is specified in advance in the map / object advance information I8, the abstract state setting unit 31 may set the abstract state (state vector) of an object that cannot be measured by the measurement device 7 based on the map / object advance information I8.

[0073] In this way, even if it is not possible to measure the objects necessary for carrying out the target task in the initial state (i.e., the motion planning stage), the abstract state setting unit 31 can appropriately determine the propositions necessary for formulating the motion plan for the robot 5.

[0074] (5-2) Target formula generator In the following, first, the target logical formula Ltag when the environment map b is not taken into consideration will be explained.

[0075] For example, in the pick-and-place example shown in FIG. 5, suppose that a goal task of "eventually all objects are in area G" is given. In this case, the goal logical formula generation unit 32 generates a linear logical formula (LTL: Linear Temporal Logic) operator "◇" corresponding to "eventually", an operator "□" corresponding to "always", and a proposition "g i " to generate the following logical formula that represents the goal state of the target task: ∧ i ◇□g i

[0076] The target logical expression generating unit 32 may express a logical expression using any temporal logic operator other than the operators "◇" and "□" (logical product "∧", logical sum "∨", negation "￢", logical inclusion "⇒", next "○", until "U", etc.). Furthermore, the logical expression corresponding to the target task may be expressed using any temporal logic such as MTL (Metric Temporal Logic) or STL (Signal Temporal Logic), not limited to linear temporal logic.

[0077] Next, the target logical expression generating unit 32 generates a target logical expression Ltag by adding the constraint condition indicated by the constraint condition information I2 to the logical expression representing the objective task.

[0078] For example, when the constraint condition information I2 includes two constraint conditions corresponding to the pick-and-place shown in FIG. 5, namely, "the robot arms 52 do not interfere with each other" and "the target object i does not interfere with the obstacle O", the target logical formula generating unit 32 converts these constraint conditions into logical formulas. Specifically, the target logical formula generating unit 32 converts the proposition "o i Using " and proposition "h", the two constraints above are converted into the following logical formulas, respectively. □￢h ∧ i □￢o i

[0079] Therefore, in this case, the goal logical formula generating unit 32 generates the logical formula "∧ i ◇□g i By adding the logical expressions of these constraint conditions to ", the following target logical expression Ltag is generated. (∧ i ◇□g i )∧(□￢h)∧(∧ i □￢o i )

[0080] In reality, the constraint conditions corresponding to pick-and-place are not limited to the above two, and there are other constraint conditions such as "the robot arm 52 does not interfere with the obstacle O," "the multiple robot arms 52 do not grab the same object," and "the objects do not come into contact with each other." These constraint conditions are also stored in the constraint condition information I2 and reflected in the target logical formula Ltag.

[0081] 6, in which the robot 5 is a moving object, will be described. In this case, the goal logical formula generation unit 32 sets the following logical proposition, which expresses "all robots will eventually exist in the area G," as a logical formula expressing the target task. ∧ i ◇□g i

[0082] In addition, when the constraint information I2 includes two constraints, "robots do not interfere with each other" and "robot i does not interfere with obstacle O at any time," the target logical formula generating unit 32 converts these constraints into logical formulas. i Using " and proposition "h", the two constraints above are converted into the following logical formulas, respectively. □￢h ∧ i □￢o i

[0083] Therefore, in this case, the goal logical formula generating unit 32 generates the logical formula "∧ i ◇□g i By adding the logical expressions of these constraint conditions to ", the following target logical expression Ltag is generated. (∧ i ◇□g i )∧(□￢h)∧(∧ i □￢o i )

[0084] In this way, the target logical formula generating unit 32 can suitably generate the target logical formula Ltag based on the processing result of the abstract state setting unit 31 even when the robot 5 is a moving body.

[0085] Next, the goal formula Ltag that takes into account the environment map b will be further explained using as an example the case where the objective task "finally all robots are in the area G" is set.

[0086] In this case, for example, if the proposition based on the environment map b is "ρ", the target logical formula generation unit 32 sets the target logical formula Ltag as follows. (∧ i ◇□g i ∨ρ)∧(□￢h)∧(∧ i □￢o i ) Here, the proposition ρ is a proposition that becomes true when a value evaluating the accuracy represented by the environment map b (also called the "environment evaluation value y") is equal to or greater than a certain threshold. This threshold is, for example, a predetermined compatibility value, and is stored in the memory 12 or the storage device 4.

[0087] The environment evaluation value y is expressed as follows, for example, using a function “g” that takes the environment map b as an argument: y = g(b) Here, the function g expresses the accuracy of each grid represented by the environment map b as "p i " (i = 1 to n, "n" is the number of grids), the probability p i (i.e. g(b) = Σp i ), or a function that has a positive correlation with the sum. i " may be a value obtained by converting the environment map b, in which the accuracy is continuously defined, into discrete values ​​for each grid.

[0088] The above target formula Ltag is "∧ i ◇□g i" is satisfied or "ρ" is satisfied (i.e., the environment evaluation value y is equal to or greater than a threshold). Note that, when the abstract state of an object related to a target task is not set (i.e., an unset object exists), the robot controller 1 performs the logical expression "∧ i ◇□g i " is regarded as false. For example, in the example of FIG. 6, if area G cannot be measured by the measurement device 7 in the initial state and information on area G is not stored in the map / object prior information I8, the state vector for area G is not set. Therefore, in this case, the robot controller 1 determines that the logical expression "∧ i ◇□g i " is considered false.

[0089] And then, "∧ i ◇□g i If "ρ" is deemed false, the control input generating unit 36 ​​optimizes the operation of the robot 5 so that "ρ" becomes true (i.e., the environment evaluation value y becomes equal to or greater than the threshold). Therefore, in this case, an operation plan for the robot 5 is made so that the area with low accuracy in the environment map b is measured, and as a result, the measurement of the unset object is performed with priority. For example, in the example of FIG. 6, if area G exists in the blind spot of the measuring device 7 formed by the obstacle 72, an operation plan for the robot i is made so that the area in the blind spot of the measuring device 7 with low accuracy in the environment map b is measured.

[0090] (5-3) Time step formula generator The time-step logical formula generation unit 33 determines the number of time steps (also called the "target number of time steps") to complete the target task, and determines a combination of propositions that express the state at each time step that satisfies the target logical formula Ltag in the target number of time steps. Since there are usually multiple combinations, the time-step logical formula generation unit 33 generates a logical formula that combines these combinations with a logical sum as the time-step logical formula Lts. The above combinations become candidates for a logical formula that expresses a sequence of operations to be commanded to the robot 5, and are hereinafter also called "candidates φ".

[0091] Here, a specific example of the process of the time-step logical expression generating unit 33 in the explanation of pick-and-place shown in FIG. 5 will be described.

[0092] For the sake of simplicity, it is assumed here that a goal task of "finally, the target object (i=2) exists in the region G" has been set, and the following goal logical formula Ltag corresponding to this goal task has been supplied from the goal logical formula generation unit 32 to the time-step logical formula generation unit 33. It is assumed here that there is no unset object, and the description of the proposition ρ is omitted for the sake of convenience. (◇□g2)∧(□￢h)∧(∧ i □￢o i ) In this case, the time step formula generator 33 generates the proposition "g i " is extended to include the concept of time steps, i,k " is used. Proposition "g i,k " is the proposition that "object i exists in region G at time step k."

[0093] Here, if the target time step number is set to "3", the target logical formula Ltag can be rewritten as follows: (◇□g 2,3 )∧(∧ k=1,2,3 □￢h k )∧(∧ i,k=1,2,3 □￢o i,k ) Also, ◇□g 2,3 can be rewritten as shown in the following equation (1).

[0094]

number

[0095] At this time, the above-mentioned target logical formula Ltag is expressed by the logical sum (φ1∨φ2∨φ3∨φ4) of the four candidates “φ1” to “φ4” shown in the following formulas (2) to (5).

[0096]

number

[0097] Therefore, the time-step logical formula generating unit 33 determines the logical sum of the four candidates φ1 to φ4 as the time-step logical formula Lts. In this case, the time-step logical formula Lts is true when at least one of the four candidates φ1 to φ4 is true. Note that the part "(∧" corresponding to the constraint condition of each candidate φ1 to φ4 is k=1,2,3 □￢h k )∧(∧ i,k=1,2,3 □￢o i,k Regarding "φ1" and "φ2" in the optimization process by the control input generating unit 36, instead of incorporating them into the candidates φ1 to φ4, they may be combined with the candidates φ1 to φ4 by logical AND.

[0098] Next, an example in which the robot 5 shown in Fig. 6 is a moving body will be described. Here, for the sake of simplicity, it is assumed that a goal task of "finally the robot (i = 2) exists in the area G" has been set, and the following goal logical formula Ltag corresponding to this goal task has been supplied from the goal logical formula generation unit 32 to the time step logical formula generation unit 33. In this case, the following goal logical formula Ltag is supplied from the goal logical formula generation unit 32 to the time step logical formula generation unit 33. It is assumed here that there is no unset object, and a description of the proposition ρ is omitted for the sake of convenience. ( ◇ □g2)∧(□￢h)∧(∧ i □￢o i )

[0099] In this case, the time step formula generator 33 generates the proposition "g i " is extended to include the concept of time steps, i,k " is used. Here, the proposition "g i,k " is a proposition that "robot i exists in area G at time step k." If the target number of time steps is set to "3," the target logical formula Ltag can be rewritten as follows: (◇□g 2,3 )∧(∧ k=1,2,3 □￢h k )∧(∧i,k=1,2,3 □￢o i,k )

[0100] Also, ◇□g 2,3 can be rewritten as equation (1) in the same way as in the pick-and-place example. The target logical formula Ltag is expressed by the logical sum (φ1 ∨ φ2 ∨ φ3 ∨ φ4) of the four candidates "φ1" to "φ4" shown in equations (2) to (5) in the same way as in the pick-and-place example. Therefore, the time-step logical formula generation unit 33 determines the logical sum of the four candidates φ1 to φ4 as the time-step logical formula Lts. In this case, the time-step logical formula Lts is true when at least one of the four candidates φ1 to φ4 is true.

[0101] If an unset object exists, the time-step logical formula generation unit 33 determines one or more candidates φ that satisfy the proposition ρ, and generates a logical formula by combining the combinations of the candidates φ by logical sum as the time-step logical formula Lts.

[0102] Next, a method for setting the target number of time steps will be explained in more detail.

[0103] The time step logical formula generating unit 33 determines the target number of time steps based on, for example, the expected time of the task specified by the input signal supplied from the instruction device 2. In this case, the time step logical formula generating unit 33 calculates the target number of time steps from the expected time based on information on the time width per time step stored in the memory 12 or the storage device 4. In another example, the time step logical formula generating unit 33 stores in advance in the memory 12 or the storage device 4 information that associates a target number of time steps appropriate for each type of target task, and determines the target number of time steps according to the type of target task to be executed by referring to the information.

[0104] Preferably, the time-step logical formula generating unit 33 sets the target number of time steps to a predetermined initial value. Then, the time-step logical formula generating unit 33 gradually increases the target number of time steps until a time-step logical formula Lts that allows the control input generating unit 36 ​​to determine the control input is generated. In this case, if the control input generating unit 36 ​​performs optimization processing using the set target number of time steps and is unable to derive an optimal solution, the time-step logical formula generating unit 33 adds a predetermined number (an integer equal to or greater than 1) to the target number of time steps.

[0105] At this time, the time-step logical formula generating unit 33 may set the initial value of the target number of time steps to a value smaller than the number of time steps corresponding to the working time of the target task expected by the user. This allows the time-step logical formula generating unit 33 to preferably prevent setting an unnecessarily large target number of time steps.

[0106] (5-4) Environment Map Update Section First, at the time of motion planning before the robot motion period Tw, the environment map update unit 34 generates the environment map b used to generate the control signal S1. In this case, the environment map update unit 34 generates the environment map b based on the measurement signal S2 using a technique such as Occupancy Grid Maps that can generate a probabilistic map in a target space or area. In this case, the accuracy of the space measured by the measurement device 7 (also called "measured space") is set to a higher value than the accuracy of the space not measured by the measurement device 7 (also called "unmeasured space"). Furthermore, when the map / object prior information I8 includes information specifying the initial state of the environment map b, the environment map update unit 34 generates the environment map b based on both the map / object prior information I8 and the measurement signal S2.

[0107] Fig. 7 is an overhead view of the working space of the robot 5, which shows a schematic relationship between an unmeasured space 50 in an initial state and a measured space 51. In the example of Fig. 7, as an example, the measurement device 7 is fixed to the mobile robot 5, and the measurement range is a semicircle with the front of the robot 5 as the front direction.

[0108] In this case, the environment map update unit 34 recognizes the measured space 51 based on the measurement signal S2, and recognizes the working space other than the measured space 51 as an unmeasured space 50. The environment map update unit 34 may specify the measured space 51 based on the position and posture of the robot 5, the installation position and installation angle of the measurement device 7 relative to the robot 5, the viewing angle and measurable distance of the measurement device 7, etc.

[0109] Then, the environment map update unit 34 sets the accuracy of the space (two-dimensional space, i.e., including regions, the same applies below) of the environment map b corresponding to the measured space 51 to a value higher than the accuracy of the space of the environment map b corresponding to the unmeasured space 50. In a simple example, the environment map update unit 34 may set the accuracy of the space of the environment map b corresponding to the measured space 51 to the maximum value "1" and the accuracy of the space of the environment map b corresponding to the unmeasured space 50 to the minimum value "0". In addition to this example, the environment map update unit 34 may set the accuracy of the space of the environment map b to any real value between 0 and 1 based on a technique such as the above-mentioned Occupancy Grid Maps. Furthermore, the environment map b may have accuracy information for each two-dimensional or three-dimensional grid as a map of discrete values.

[0110] Furthermore, the environment map update unit 34 updates the environment map b at predetermined time intervals based on the latest measurement signal S2 output by the measuring device 7 during the robot operation period Tw in which the robot 5 operates.

[0111] FIG. 8 is an overhead view of the working space of the robot 5, which shows a schematic relationship between the unmeasured space 50 and the measured space 51 after the robot 5 has moved along the movement trajectory 54. In the example of FIG. 8, the robot 5 moves straight a predetermined distance from the initial position shown in FIG. 7. In this case, the environment map update unit 34 recognizes the measured space 51 based on the control signal S1. In FIG. 8, the measured space 51 in the initial state is displayed as the existing measured space 510, and the measured space 51 newly measured by the movement of the robot 5 is displayed as the new measured space 520. In this case, the environment map update unit 34 updates the environment map b so that the accuracy of the environment map b corresponding to the measured space 51 becomes higher than the accuracy of the environment map b corresponding to the unmeasured space 50.

[0112] Here, the map update model b + We will provide a supplementary explanation on map update model b. + is a model for predicting the time change of the environment map b from the environment map b, state vector, and predicted input vector to the robot 5 at the time of motion planning. For example, the map update model b + When a state vector and an environment map b corresponding to the initial state shown in FIG. 7 and an input vector for moving the robot 5 along the movement trajectory 54 are input, the map update model b outputs an environment map b with a high accuracy of the area corresponding to the new measurement space 520 shown in FIG. + Regarding "(5-5) Abstract Model Generation This is explained further in the " section.

[0113] (5-5) Abstract Model Generation The abstract model generation unit 35 receives the dynamics model information I5, the abstract state setting information IS, the environment map b, and the map update model b. + Here, the abstract model Σ is generated based on the dynamics model and the map update model b + In the following, we first explain the dynamics model of the abstract model Σ.

[0114] For example, a dynamics model in the case where the target task is pick-and-place will be described. In this case, a generic abstract model that does not specify the position or number of objects, the position of the area where the objects are placed, the number of robots 5 (or the number of robot arms 52), etc. is recorded in the dynamics model information I5. The abstract model generation unit 35 then generates a dynamics model by reflecting the object recognition result by the abstract state setting unit 31 in the generic model including the dynamics of the robot 5 recorded in the dynamics model information I5. As a result, the abstract model Σ becomes a model that abstractly represents the state of the objects in the workspace and the dynamics of the robot 5. In the case of pick-and-place, the state of the objects in the workspace indicates the position and number of objects, the position of the area where the objects are placed, the number of robots 5, etc.

[0115] Here, during the task of picking and placing, the dynamics in the workspace frequently changes. For example, in the picking and placing example shown in Fig. 5, when the robot arm 52 holds the object i, the object i can be moved, but when the robot arm 52 does not hold the object i, the object i cannot be moved.

[0116] Considering the above, in the present embodiment, in the case of pick-and-place, the action of grabbing the target object i is expressed as a logical variable “δ i In this case, for example, the abstract model generation unit 35 can determine the dynamics model of the abstract model Σ to be set for the working space in the pick-and-place example of FIG. 5 by the following formula (6).

[0117]

number

[0118] Here, "u j" indicates the control input for controlling the robot hand j ("j=1" is the robot hand 53a, and "j=2" is the robot hand 53b), "I" indicates the unit matrix, and "0" indicates the zero matrix. Note that the control input is assumed to be velocity here as an example, but it may be acceleration. Also, "δ j,i " is a logical variable that is "1" when the robot hand j is holding the object i, and is "0" otherwise. Also, "x r1 ", "x r2 " indicates the position vector of robot hand j (j = 1, 2), "x1" to "x4" indicate the position vector of object i (i = 1 to 4). "h(x)" is a variable that satisfies "h(x) ≧ 0" when a robot hand is close enough to an object to grasp it, and satisfies the following relationship with logical variable δ. δ=1 ⇔ h(x)≧0 In this equation, if the robot hand is close enough to an object to grasp it, it is assumed that the robot hand is grasping the object, and the logical variable δ is set to 1.

[0119] Here, equation (6) is a differential equation that shows the relationship between the state of the object at time step k and the state of the object at time step k + 1. In the above equation (6), the grip state is represented by a logical variable that is a discrete value, and the movement of the object is represented by a continuous value, so equation (6) shows a hybrid system.

[0120] Moreover, in equation (6), only the dynamics of the robot hand, which is the end effector of the robot 5 that actually grasps an object, is taken into consideration, rather than the detailed dynamics of the entire robot 5. This makes it possible to suitably reduce the amount of calculation in the optimization process performed by the control input generation unit 36.

[0121] Furthermore, the dynamics model information I5 records logical variables corresponding to the action at which the dynamics is switched (the action of gripping the object i in the case of pick-and-place), and information for deriving the differential equation of formula (6) from the object recognition results based on the measurement signal S2, etc. Therefore, even if the positions and number of objects, the area where the objects are placed (area G in FIG. 5), the number of robots 5, etc. vary, the abstract model generation unit 35 can determine a dynamics model of the abstract model Σ that is suited to the environment of the target working space, based on the dynamics model information I5 and the object recognition results.

[0122] When other work objects exist, information on the abstracted dynamics of the other work objects may be included in the dynamics model information I5. In this case, the dynamics model of the abstract model Σ is a model in which the state of the object in the work space, the dynamics of the robot 5, and the dynamics of the other work objects are abstractly represented. Moreover, the abstract model generation unit 35 may generate a model of a mixed logical dynamical (MLD) system or a hybrid system that combines a Petri net, an automaton, and the like, instead of the model shown in equation (6).

[0123] Next, a dynamics model of the abstract model Σ when the robot 5 shown in Fig. 6 is a moving body will be described. In this case, the abstract model generation unit 35 determines the dynamics model of the abstract model Σ to be set for the working space shown in Fig. 6, for example, by the following formula (7) using a state vector x1 for the robot (i=1) and a state vector x2 for the robot (i=2).

[0124]

number

[0125] Here, "u1" represents an input vector for the robot (i=1), and "u2" represents an input vector for the robot (i=2). Additionally, "A1", "A2", "B1", and "B2" are matrices and are determined based on the dynamics model information I5.

[0126] In another example, when there are multiple operation modes of the robot i, the abstract model generation unit 35 may represent the dynamics model of the abstract model Σ to be set for the work space shown in Fig. 6 by a hybrid system in which dynamics are switched according to the operation mode of the robot i. In this case, when the operation mode of the robot i is "mi", the abstract model generation unit 35 determines the dynamics model of the abstract model Σ to be set for the work space shown in Fig. 6 by the following formula (8).

[0127]

number

[0128] In this way, the abstract model generation unit 35 can suitably determine the dynamics model of the abstract model Σ even when the robot 5 is a moving body. Note that the abstract model generation unit 35 may generate a model of an MLD system or a hybrid system that combines a Petri net, an automaton, etc., instead of the model shown in equation (7) or equation (8).

[0129] In addition, the vector x representing the state of the object or robot 5 in the abstract model Σ shown in equations (6) to (8) is i and input u i can be discrete. Vector x i and input u i Even when is expressed discretely, the abstract model generation unit 35 can set an abstract model Σ that appropriately abstracts the real dynamics. In addition, when the robot 5 moves and a target task of pick-and-place is set, the abstract model generation unit 35 sets a dynamics model that assumes switching of the operation mode as shown in, for example, equation (8).

[0130] Next, we consider the map update model b + This article explains:

[0131] Map update model b + is a function that predicts the environment map b at the next time point based on the environment map b at a certain time point, the state vector x, and the input vector u to the robot 5. Therefore, the environment map b is expressed as the state vector x, the input vector u to the robot 5, the map update model b + The function “f” corresponds to the time step k, and the error vector “nv” is expressed by the following difference equation: b k+1 =f(b k ,x k ,u k )+nv k

[0132] Information on the function f and the error vector nv is stored, for example, in map update model information I7. Also, the state vector x, the input vector u, and the environment map b are expressed as follows. Here, it is assumed that the number of robots 5 is "M", and the environment map b is a discrete map with the number of grids "n". x=[x1 T ,x2 T …,x M T ] T u=[u1 T ,u2 T …,u M T ] T b = [p1, p2…, p n ] T Each element x1,x2…,x M , u1,u2…,u M is a vector representing the state or input of each robot, and p i ∈[0,1] (i=1 to n) represents the accuracy of each grid. In addition, x and u are expressed as follows in the case of the dynamics model shown in equation (7), for example. xk+1 =Ax k +Bu k

[0133] In the above-mentioned differential equation regarding the environment map b, the environment map b is determined probabilistically using the error vector nv. Alternatively, the environment map b may be determined deterministically without using the error vector nv.

[0134] (5-6) Control Input Generation Unit The control input generation unit 36 ​​determines an optimal control input for the robot 5 for each time step based on the time step logical formula Lts supplied from the time step logical formula generation unit 33 and the abstract model Σ supplied from the abstract model generation unit 35. In this case, the control input generation unit 36 ​​defines an evaluation function for the target task, and solves an optimization problem of minimizing the evaluation function with the abstract model Σ and the time step logical formula Lts as constraints. The evaluation function is, for example, predetermined for each type of target task, and stored in the memory 12 or the storage device 4.

[0135] For example, the control input generating unit 36 ​​receives the control input “u k " and the environmental evaluation value y. In this case, the control input generating unit 36 ​​sets an evaluation function based on the control input u k The evaluation function is minimized so that the smaller the smaller the φ (i.e., the smaller the energy consumed by the robot 5), and the larger the environmental evaluation value y (i.e., the higher the accuracy of information in the entire working space) when an unspecified object exists. Specifically, the control input generation unit 36 ​​minimizes the evaluation function so that the smaller the φ (i.e., the smaller the energy consumed by the robot 5), the larger the environmental evaluation value y (i.e., the higher the accuracy of information in the entire working space). i The constrained mixed integer optimization problem shown in the following equation (9) is solved, where the constraints are a logical expression based on the logical sum of

[0136]

number

[0137] Here, the evaluation function term "-αy k " and the logical formula "ρ T ∧(□￢h)∧(∧ i □￢o i )" are provided in consideration of the case where an unspecified object exists. By providing these, "h" and "o" i The control input u is adjusted to improve the accuracy of the environment map b while satisfying the general constraints specified by k It becomes possible to determine

[0138] Preferably, when an unspecified object exists, the control input generating unit 36 ​​selects a candidate φ corresponding to the time step logical formula Lts. i Logical OR of "∨ i φ i On the other hand, when there is no undefined object, the control input generating unit 36 ​​preferably determines that the term "-αy k " and the logical formula "ρ T ∧(□￢h)∧(∧ i □￢o i )" and control input u k It is a good idea to optimize the following:

[0139] In this way, the control input generator 36 calculates the environment map b and the map update model b + The optimal control input u is obtained by solving an optimization problem including the environmental evaluation value y and proposition ρ based on k In this way, when an unspecified object exists, the control input generating unit 36 ​​determines the control input u so as to increase the accuracy of the information in the working space and measure the specified object. k If there is no unspecified object, the control input u is determined so as to achieve the target task. k can be determined.

[0140] Preferably, the control input generation unit 36 ​​approximates the logical variables to continuous values ​​(continuous relaxation problem). This allows the control input generation unit 36 ​​to reduce the amount of calculations. If STL is used instead of linear logic (LTL), it is possible to describe it as a nonlinear optimization problem. In addition, the environment map b and the map update model b + may be incorporated into the optimization problem based on any reinforcement learning method in addition to the above-mentioned method.

[0141] Furthermore, when the target number of time steps is long (for example, larger than a predetermined threshold), the control input generation unit 36 ​​may set the number of time steps used for optimization to a value smaller than the target number of time steps (for example, the above-mentioned threshold). In this case, the control input generation unit 36 ​​sequentially calculates the control input u by solving the above-mentioned optimization problem, for example, every time a predetermined number of time steps elapses. k In this case, the control input generation unit 36 ​​solves the above optimization problem for each predetermined event corresponding to an intermediate state with respect to the achievement state of the target task, and determines the control input u k may be determined. In this case, the control input generating unit 36 ​​sets the number of time steps until the next event occurs to the number of time steps used for optimization. The above-mentioned event is, for example, an event in which dynamics in the workspace is switched. For example, when pick-and-place is set as the target task, the robot 5 grasps an object, the robot 5 finishes carrying one of a plurality of objects to be carried to a destination point, and the like are defined as events. The events are, for example, defined in advance for each type of target task, and information for identifying the event for each type of target task is stored in the storage device 4.

[0142] (5-7) Robot Control Unit The robot control unit 37 generates a sequence of subtasks based on the control input information Icn supplied from the control input generation unit 36 ​​and the subtask information I4 stored in the application information storage unit 41. In this case, the robot control unit 37 recognizes subtasks that the robot 5 can accept by referring to the subtask information I4, and converts the control input for each time step indicated by the control input information Icn into a subtask.

[0143] For example, in the subtask information I4, functions indicating two subtasks, namely, moving (reaching) of the robot hand and grasping (grasping) of the robot hand, are defined as subtasks that the robot 5 can accept when the target task is pick-and-place. In this case, the function "Move" representing reaching is a function that takes as arguments, for example, the initial state of the robot 5 before the execution of the function, the final state of the robot 5 after the execution of the function, and the time required to execute the function. Also, the function "Grasp" representing grasping is a function that takes as arguments, for example, the state of the robot 5 before the execution of the function, the state of the object to be grasped before the execution of the function, and the logical variable δ. Here, the function "Grasp" represents a grasping action when the logical variable δ is "1", and represents a releasing action when the logical variable δ is "0". In this case, the robot control unit 37 determines the function "Move" based on the trajectory of the robot hand determined by the control input for each time step indicated by the control input information Icn, and determines the function "Grasp" based on the transition of the logical variable δ for each time step indicated by the control input information Icn.

[0144] Then, the robot control unit 37 generates a sequence consisting of the function "Move" and the function "Grasp", and supplies a control signal S1 representing the sequence to the robot 5. For example, when the target task is "finally the object (i=2) is present in the area G", the robot control unit 37 generates a sequence of the function "Move", the function "Grasp", the function "Move", and the function "Grasp" for the robot hand closest to the object (i=2). In this case, the robot hand closest to the object (i=2) moves to the position of the object (i=2) by the first function "Move", grasps the object (i=2) by the first function "Grasp", moves to the area G by the second function "Move", and places the object (i=2) in the area G by the second function "Grasp".

[0145] (5-8) Abstract State Comparison During the robot operation period Tw, the abstract state comparison unit 38 compares the measured abstract state Stm supplied from the abstract state setting unit 31 with the predicted abstract state Stp based on the control input information Icn, and determines whether or not the abstract model Σ needs to be regenerated. Here, a method for acquiring the predicted abstract state Stp will be described in more detail. The abstract state comparison unit 38 recognizes the trajectory of the state of each object predicted for each time step based on, for example, the control input information Icn representing the processing result of the optimization process executed by the control input generation unit 36. Then, the abstract state comparison unit 38 determines the predicted state at the time step corresponding to the current time as the predicted abstract state Stp.

[0146] Here, when the abstract state comparison unit 38 judges that there is a substantial difference between the measured abstract state Stm and the predicted abstract state Stp, it judges that it is necessary to regenerate the abstract model Σ. The case where "there is a substantial difference between the measured abstract state Stm and the predicted abstract state Stp" refers to, for example, a case where the number of objects defining the abstract state is different, a case where the position of the same object differs by a predetermined distance or more, or a case where other predetermined conditions are satisfied. In this case, the abstract state comparison unit 38 suitably judges that it is necessary to regenerate the abstract model Σ when the measurement device 7 detects an unset object, or when it detects that there is a difference between the abstract state of an object set based on the map / object advance information I8 and the measurement result of the object. In addition to the above cases, the abstract state comparison unit 38 also judges that it is necessary to regenerate the abstract model when an unexpected obstacle occurs, an object that moves unexpectedly exists, or the like.

[0147] Then, when the abstract state comparison unit 38 determines that it is necessary to regenerate the abstract model Σ, it supplies the measured abstract state Stm to the abstract model generation unit 35. The measured abstract state Stm is the latest information representing an abstract state such as the position and posture of each object (including a detected unset object) in the working space. The abstract model generation unit 35 generates an abstract model Σ reflecting the current abstract state based on the measured abstract state Stm and the latest environment map b generated by the environment map update unit 34. The control input generation unit 36 ​​then constructs an optimization problem shown in Equation (9) using the generated abstract model Σ as a constraint condition, and solves the constructed optimization problem. This allows the control input generation unit 36 ​​to suitably generate a control input representing an optimal motion plan for the robot 5 taking into account the latest abstract state. This allows the motion of the robot 5 to be replanned, and the sequence of the motion of the robot 5 to be suitably determined based on the latest measurement result.

[0148] (6) Processing Flow FIG. 9 is an example of a flowchart showing an outline of the robot control process executed by the robot controller 1 in the first embodiment.

[0149] First, the abstract state setting unit 31 of the robot controller 1 sets the abstract state of an object existing in the working space (step S11). Here, the abstract state setting unit 31 executes step S11, for example, when it receives an external input from the instruction device 2 or the like instructing the execution of a predetermined target task. In step S11, the abstract state setting unit 31 sets a proposition and a state vector such as a position and orientation regarding an object related to the target task, for example, based on the abstract state designation information I1, the object model information I6, the map and object prior information I8, and the measurement signal S2. Note that, when the abstract state setting unit 31 determines that an undefined object exists based on the measurement signal S2 and the abstract state designation information I1, it sets a proposition regarding the undefined object while leaving a state vector such as a position for the undefined object undefined.

[0150] Next, the target logical formula generation unit 32 determines the target logical formula Ltag based on the abstract state setting information IS representing the processing result of step S11 (step S12). In this case, the target logical formula generation unit 32 adds a constraint condition for the execution of the objective task to the target logical formula Ltag by referring to the constraint condition information I2.

[0151] Then, the time-step logical formula generating unit 33 converts the target logical formula Ltag into a time-step logical formula Lts that represents a state at each time step (step S13). In this case, the time-step logical formula generating unit 33 determines a target number of time steps, and generates, as the time-step logical formula Lts, a logical sum of candidates φ that represent a state at each time step that satisfies the target logical formula Ltag in the target number of time steps. In this case, preferably, the time-step logical formula generating unit 33 judges the feasibility of each candidate φ by referring to the operational limit information I3, and excludes candidates φ that are judged to be infeasible from the time-step logical formula Lts.

[0152] Next, the environment map update unit 34 generates the environment map b (step S14). In this case, the environment map update unit 34 generates the environment map b based on, for example, the measurement signal S2 and the map / object prior information I8. Then, the abstract model generation unit 35 generates an abstract model Σ (step S15). In this case, the abstract model generation unit 35 generates the dynamics model and the map update model b based on the environment map b, the abstract state setting information IS, the dynamics model information I5, the map update model information I7, etc. + An abstract model Σ is generated that represents

[0153] Then, the control input generating unit 36 ​​constructs an optimization problem based on the processing results of steps S11 to S15, and determines the control input by solving the constructed optimization problem (step S16). In this case, for example, the control input generating unit 36 ​​constructs an optimization problem as shown in equation (9), and determines the control input that minimizes the evaluation function set based on the control input and the environmental evaluation value y.

[0154] Then, the robot control unit 37 controls the robot 5 based on the control input determined in step S16 (step S17). In this case, for example, the robot control unit 37 converts the control input determined in step S16 into a sequence of subtasks that can be interpreted by the robot 5 with reference to the subtask information I4, and supplies a control signal S1 representing the sequence to the robot 5.

[0155] Then, the robot controller 1 determines whether the target task is completed (step S18). For example, the robot controller 1 determines that the target task is completed when it receives a signal indicating normal completion from the robot 5, or when it detects based on the measurement signal S2 that an object in the working space has reached a goal state in which the target task has been achieved. If the robot controller 1 determines that the target task is completed (step S18; Yes), it ends the processing of the flowchart. On the other hand, if the target task is not completed (step S18; No), the environment map update unit 34 updates the environment map b based on the latest measurement signal S2 (step S19).

[0156] Then, the abstract state comparison unit 38 monitors changes in the abstract state (step S20). Specifically, the abstract state comparison unit 38 recognizes a predicted abstract state Stp representing a predicted abstract state at the time of comparison, based on the control input information Icn representing the processing result of the optimization processing executed by the control input generation unit 36. Then, the abstract state comparison unit 38 compares the measured abstract state Stm representing the abstract state set by the abstract state setting unit 31 based on the latest measurement signal S2 with the predicted abstract state Stp.

[0157] Then, the abstract state comparison unit 38 judges whether or not there has been a substantial change in the abstract state (step S21). If the abstract state comparison unit 38 judges that there has been a substantial change in the abstract state (step S21; Yes), the process proceeds to step S15. Thereafter, the abstract model Σ and the control input are generated, and the motion plan of the robot 5 is redetermined based on the latest abstract state. On the other hand, if the abstract state comparison unit 38 judges that there has been no substantial change in the abstract state (step S21; No), the process returns to step S18. In this case, the control of the robot 5 continues based on the original motion plan.

[0158] (7) Variations The functional block configuration of the processor 11 shown in FIG. 4 is an example, and various modifications may be made.

[0159] For example, information on candidates φ for the sequence of operations to be commanded to the robot 5 is stored in advance in the storage device 4, and the processor 11 executes the optimization process of the control input generation unit 36 ​​based on the information. As a result, the processor 11 selects the optimal candidate φ and determines the control input of the robot 5. In this case, the processor 11 does not need to have functions corresponding to the target logical formula generation unit 32 and the time step logical formula generation unit 33 in generating the control signal S1. In this way, information on the execution results of some of the functional blocks of the processor 11 shown in FIG. 4 may be stored in advance in the application information storage unit 41.

[0160] In another example, the application information may include design information such as a flow chart for designing a control input or a subtask sequence corresponding to a target task in advance, and the robot controller 1 may generate the control input or the subtask sequence by referring to the design information. In this case, the robot controller 1 executes a process flow branched based on, for example, the presence or absence of an unset object and / or threshold processing for the environmental evaluation value y. A specific example of executing a task based on a task sequence designed in advance is disclosed in, for example, JP 2017-39170 A.

[0161] <Second embodiment> FIG. 10 shows a schematic configuration diagram of a control device 1X in the second embodiment. The control device 1X mainly includes an abstract state setting means 31X, an environment map generating means 34X, an abstract model generating means 35X, and a control input generating means 36X. The control device 1X may be composed of a plurality of devices. The control device 1X may be, for example, the robot controller 1 in the first embodiment.

[0162] The abstract state setting means 31X sets an abstract state, which is an abstract state of an object in a work space where the robot performs work. The abstract state setting means 31X may set the abstract state based on the measurement result of a sensor that measures the work space, or may set the abstract state based on advance information generated in advance. For example, it may be the abstract state setting unit 31 in the first embodiment.

[0163] The environment map generating means 34X generates an environment map which is a map representing the accuracy of information in the working space. The environment map generating means 34X may generate the environment map based on the measurement results of a sensor that measures the working space, or may set the environment map based on prior information generated in advance. The environment map generating means 34X may be, for example, the environment map updating unit 34 in the first embodiment.

[0164] The abstract model generating means 35X generates an abstract model that represents the dynamics of the abstract state and the time change of the environment map. The abstract model generating means 35X can be, for example, the abstract model generating unit 35 in the first embodiment.

[0165] The control input generating means 36X generates a control input for the robot based on the abstract model. The control input generating means 36X can be, for example, the control input generating unit 36 ​​in the first embodiment.

[0166] FIG. 11 is an example of a flowchart executed by the control device 1X in the second embodiment. First, the abstract state setting means 31X sets an abstract state, which is an abstract state of an object in a work space where the robot performs work (step S31). The environment map generating means 34X generates an environment map in the work space (step S32). Note that steps S31 and S32 may be performed in any order, and may be performed simultaneously or in the reverse order. The abstract model generating means 35X generates an abstract model that represents the dynamics of the abstract state and the time change of the environment map (step S33). The control input generating means 36X generates a control input for the robot based on the abstract model (step S34).

[0167] According to the second embodiment, the control device 1X can suitably generate a control input for suitably operating the robot, taking into account the uncertainty of information in the workspace.

[0168] In each of the above-described embodiments, the program can be stored using various types of non-transitory computer readable media and can be supplied to a computer, such as a processor. The non-transitory computer readable medium includes various types of tangible storage media. Examples of the non-transitory computer readable medium include magnetic storage media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage media (e.g., magneto-optical disks), CD-ROM (Read Only Memory), CD-R, CD-R / W, and semiconductor memory (e.g., mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, and RAM (Random Access Memory)). The program may also be supplied to a computer by various types of transitory computer readable media. Examples of the transitory computer readable medium include electric signals, optical signals, and electromagnetic waves. The transitory computer readable medium can supply the program to a computer via a wired communication path such as an electric wire and an optical fiber, or via a wireless communication path.

[0169] In addition, some or all of the above embodiments may be described as, but are not limited to, the following supplementary notes.

[0170] [Appendix 1] an abstract state setting means for setting an abstract state which is an abstract state of an object in a work space in which the robot performs a task; an environment map generating means for generating an environment map which is a map representing the accuracy of information in the working space; an abstract model generating means for generating an abstract model representing the dynamics of the abstract state and the time change of the environment map; a control input generating means for generating a control input for the robot based on the abstract model; A control device comprising: [Appendix 2] The control device described in Appendix 1, further comprising an abstract state comparison means for determining whether or not the abstract model needs to be regenerated based on a change in the abstract state during operation of the robot by the control input. [Appendix 3] The control device according to claim 2, wherein the abstract state comparison means determines whether or not the abstract model needs to be regenerated based on a change in at least one of the number or position of the objects during operation of the robot due to the control input. [Appendix 4] The robot is equipped with a measuring device, the measurement range of the measuring device changes in response to the movement of the robot; The control device according to claim 2 or 3, wherein the abstract state setting means identifies a change in the abstract state based on a measurement signal generated by the measurement device during operation of the robot. [Appendix 5] The control device described in Appendix 4, wherein the abstract state comparison means determines whether or not the abstract model needs to be regenerated based on a difference between the current abstract state set based on the measurement signal and the current abstract state predicted based on the control input. [Appendix 6] The control device according to any one of claims 1 to 5, wherein the control input generation means generates the control input based on the abstract model and an environment evaluation value that evaluates a degree of accuracy represented by the environment map. [Appendix 7] The control device described in Appendix 6, wherein the control input generation means sets an evaluation function including the control input and the environmental evaluation value, and constraint conditions to be satisfied in executing a target task, which is a task to be performed by the robot, and generates the control input by optimization based on the evaluation function and the constraint conditions. [Appendix 8] a target formula generating means for generating a target formula which is a formula of temporal logic representing a final goal; a time step logical expression generating means for generating, from the logical expressions, a time step logical expression which is a logical expression representing a state for each time step for executing a target task which is a task to be performed by the robot; and 8. The control device according to claim 1, wherein the control input generating means generates the control input based on the abstract model and the time-step logical formula. [Appendix 9] The control device described in Appendix 8, wherein the target logical formula generation means generates the target logical formula including a logical sum of a logical formula corresponding to the target task and a proposition based on an environmental evaluation value that evaluates the certainty represented by the environmental map. [Appendix 10] The control device according to any one of claims 1 to 9, further comprising a robot control means for converting the control input into a sequence of subtasks executable by the robot and supplying the subtask sequence to the robot. [Appendix 11] The control device according to any one of appendices 1 to 10, wherein the environmental map generating means updates the environmental map so as to attenuate the accuracy in the space in which the measurement was performed based on the elapsed time since the measurement. [Appendix 12] The computer Set an abstract state, which is an abstract state of an object in the workspace where the robot performs the work; generating an environment map which is a map representing the accuracy of information within the working space; generating an abstract model representing the dynamics of said abstract state and the evolution of said environment map over time; generating control inputs for the robot based on the abstract model; Control methods. [Appendix 13] Set an abstract state, which is an abstract state of an object in the workspace where the robot performs the work; generating an environment map which is a map representing the accuracy of information within the working space; generating an abstract model representing the dynamics of said abstract state and the evolution of said environment map over time; A storage medium storing a program that causes a computer to execute a process for generating a control input for the robot based on the abstract model.

[0171] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-mentioned embodiments. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. In other words, the present invention naturally includes various modifications and corrections that a person skilled in the art could make in accordance with the entire disclosure, including the claims, and the technical ideas. In addition, the disclosures of the above cited patent documents and the like are incorporated herein by reference. [Explanation of symbols]

[0172] 1 Robot Controller 1X Control Unit 2 Indicating device 4 Storage device 5. Robot 7. Measurement Equipment 41 Application information storage unit 100 Robot Control System

Claims

1. an abstract state setting means for setting an abstract state which is an abstract state of an object in a work space in which the robot performs a task; an environment map generating means for generating an environment map which is a map representing the accuracy of information in the working space; an abstract model generating means for generating an abstract model representing the dynamics of the abstract state and the time change of the environment map; a control input generating means for generating a control input for the robot based on the abstract model; an abstract state comparison means for determining whether or not it is necessary to regenerate the abstract model based on a change in the abstract state during the operation of the robot by the control input; A control device comprising:

2. 2. The control device according to claim 1, wherein the abstract state comparison means determines whether or not the abstract model needs to be regenerated based on a change in at least one of the number and positions of the objects during an operation of the robot based on the control input.

3. The robot is equipped with a measuring device, the measurement range of the measuring device changes in response to the movement of the robot; The control device according to claim 1 , wherein the abstract state setting means identifies a change in the abstract state based on a measurement signal generated by the measurement device during operation of the robot.

4. 4. The control device according to claim 3, wherein the abstract state comparison means determines whether or not the abstract model needs to be regenerated based on a difference between the current abstract state set based on the measurement signal and the current abstract state predicted based on the control input.

5. The control device according to any one of claims 1 to 4, wherein the control input generating means generates the control input based on the abstract model and an environmental evaluation value that evaluates the accuracy represented by the environmental map.

6. 6. The control device according to claim 5, wherein the control input generation means sets an evaluation function including the control input and the environmental evaluation value, and constraint conditions to be satisfied in the execution of a target task which is a task to be performed by the robot, and generates the control input by optimization based on the evaluation function and the constraint conditions.

7. a target formula generating means for generating a target formula which is a formula of temporal logic representing a final goal; a time step logical expression generating means for generating, from the logical expressions, a time step logical expression which is a logical expression representing a state for each time step for executing a target task which is a task to be performed by the robot; The control device according to any one of claims 1 to 6, wherein the control input generating means generates the control input based on the abstract model and the time step logical expression.

8. The computer Set an abstract state, which is an abstract state of an object in the workspace where the robot performs the work; generating an environment map which is a map representing the accuracy of information within the working space; generating an abstract model representing the dynamics of said abstract state and the evolution of said environment map over time; generating control inputs for the robot based on the abstract model; During the operation of the robot by the control input, a determination is made as to whether or not the abstract model needs to be regenerated based on a change in the abstract state. Control methods.

9. Set an abstract state, which is an abstract state of an object in the workspace where the robot performs the work; generating an environment map which is a map representing the accuracy of information within the working space; generating an abstract model representing the dynamics of said abstract state and the evolution of said environment map over time; generating control inputs for the robot based on the abstract model; A program that causes a computer to execute a process of determining whether or not it is necessary to regenerate the abstract model based on a change in the abstract state while the robot is operating due to the control input.

10. An abstract state setting means for setting an abstract state which is an abstract state of an object in a work space in which a robot performs work; an environment map generating means for generating an environment map which is a map representing the accuracy of information in the working space; an abstract model generating means for generating an abstract model representing the dynamics of the abstract state and the time change of the environment map; a control input generating means for generating a control input for the robot based on the abstract model and an environment evaluation value for evaluating the accuracy of the environment map; A control device comprising:

Images