Robot control system, robot control method, and robot control program

Abstract

A robot control system comprising: a storage unit that stores a behavior tree, including a plurality of sub-trees that correspond to a plurality of tasks and parallel nodes that are parent nodes connecting to the plurality of sub-trees, and resource information, including the status of shared resources used in the plurality of tasks; a tree management unit that, in individual cycles, calls each of the plurality of sub-trees in sequence from the parallel nodes on the basis of the behavior tree; and a task control unit that causes tasks corresponding to the called sub-trees to be executed. The shared resources include a robot. The status includes at least an in-use state and a usage-possible state. With respect to each of the sub-trees, the task control unit refers to the resource information in cases where the sub-tree is called, and causes the task corresponding to the sub-tree to be executed using the shared resource in cases where the status of the shared resource used in the task is the usage-possible state.

Description

ROBOT CONTROL SYSTEM, ROBOT CONTROL METHOD, AND ROBOT CONTROL PROGRAM

[0001] One aspect of the present disclosure relates to a robot control system, a robot control method, and a robot control program.

[0002] US Patent No. 6,299,949 describes a method for controlling a robot that involves the use of a behavior tree architecture for tasks performed by the robot.

[0003] Patent No. 6943364

[0004] It is desirable to efficiently execute multiple tasks.

[0005] A robot control system according to one aspect of the present disclosure includes a memory unit that stores a behavior tree including multiple subtrees corresponding to multiple tasks and parallel nodes that are parent nodes connected to the multiple subtrees, and resource information including the status of common resources used by the multiple tasks, wherein the common resources include robots and the status includes at least an in-use state and an available state; a tree management unit that calls each of the multiple subtrees from the parallel node in turn based on the behavior tree in each cycle; and a task control unit that executes a task corresponding to the called subtree, wherein the task control unit refers to the resource information for each of the multiple subtrees when the subtree is called, and if the status of the common resource used by the task corresponding to the subtree is available, executes the task using the common resource.

[0006] According to one aspect of the present disclosure, multiple tasks can be executed efficiently.

[0007] FIG. 1 is a diagram illustrating an example of an application of a robot control system; FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer used for a robot control system; FIG. 3 is a diagram illustrating an example of a behavior tree; FIG. 4 is a state transition diagram illustrating an example of a change in the status of a shared resource; FIG. 5 is a diagram illustrating an example of a release batch; FIG. 6 is a flowchart illustrating an example of a process executed by a robot control system; FIG. 7 is a flowchart illustrating an example of a process related to a parallel node; and FIG. 8 is a diagram illustrating an example of a change in the order in which subtrees are called.

[0008] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or equivalent elements are designated by the same reference numerals, and redundant description will be omitted.

[0009] [System Overview] An example robot control system is a computer system that causes agents to execute multiple tasks using shared resources, including robots, in a real workspace. A task is a process performed to achieve a certain goal. An agent is an entity that executes a task. An agent may be a robot or another type of device or machine. In a robot control system, a robot may be managed as at least one of an agent and a shared resource. A shared resource is a resource (e.g., a hardware resource) that may be used by multiple tasks. Examples of shared resources other than robots include various sensors such as cameras and transport devices such as conveyors. The robot control system manages at least one shared resource.

[0010] Executing multiple tasks using a common resource corresponds to concurrent processing, parallel processing, or a combination thereof. The use of a common resource by a task means that the common resource is occupied by that task. In these processes, phenomena called deadlock and resource starvation are taken into consideration. Deadlock refers to a phenomenon in which two or more processes (tasks in this disclosure) mutually request a resource reserved by another process, preventing any process from proceeding. Resource starvation refers to a phenomenon in which a process (task) is permanently unable to acquire a resource and is unable to execute. Generally, it is not easy to design a system for executing multiple tasks in parallel or in parallel while avoiding these problems.

[0011] A robot control system uses a behavior tree to manage multiple tasks and shared resources in order to process multiple tasks in parallel or in parallel while avoiding or reducing deadlock and starvation. As a result, it is expected that multiple tasks will be executed efficiently. Furthermore, by introducing a behavior tree, it is possible to easily design and manage a mechanism for flexibly operating agents in response to changes in the environment of the real workspace.

[0012] A behavior tree is a method of representing an agent's behavior using a tree structure. A behavior tree includes a root node, a control node, and an execution node. A node is connected to another node by a directed edge. The node at the start of a directed edge is called a "parent node," and the node at the end of a directed edge is called a "child node." Each node has at most one parent node and zero or more child nodes. The root node is the node at the top of the behavior tree. The root node has no parent node and typically has one child node. A control node has one parent node and one or more child nodes. A control node sequentially invokes one or more child nodes in response to being invoked. An execution node has one parent node and no child nodes. An execution node is also called a "leaf" of the behavior tree. Each of multiple tasks is associated with an execution node.

[0013] In this disclosure, when focusing on a certain node, a set of one child node of the node and zero or more nodes located below the child node is also referred to as a "subtree." Each subtree corresponds to a task. Because subtrees can be defined at each layer of the behavior tree, the relationship between subtrees can be said to be a nested structure. Corresponding to this structure, a certain task can be realized by a set of multiple subtasks.

[0014] The root node calls its child nodes at predetermined cycle intervals. This call is also called a "tick." Generally, the cycle interval is a very short time and can be set, for example, based on the frame rate of the video captured by the camera 4. In response to the call by the root node, each node in the subtree connected to the root node is called in a predetermined order based on the tree structure, with priority given to the left. The call (tick) propagates from the root node to each execution node, thereby executing the entire behavior tree. Each node returns one of the following execution results: True, indicating successful execution; False, indicating unsuccessful execution; or Running, indicating that the node is currently running.

[0015] [System Configuration] FIG. 1 is a diagram illustrating an application of a robot control system 1 according to an example. In this example, a robot 2 placed in a real workspace is shown as an example of an agent. The robot control system 1 is connected to a robot controller 3, and the robot controller 3 is connected to the robot 2. A camera 4 is provided in the workspace to capture images of the environment including the robot 2. The camera 4 is connected to the robot control system 1. Each communication network connecting devices may be a wired network or a wireless network. The communication network may include at least one of the Internet and an intranet. Alternatively, the communication network may be realized simply by a single communication cable.

[0016] In this example, the robot control system 1 causes the robot 2 to perform multiple tasks using common resources including the robot 2 and a camera 4. The robot control system 1 executes a given calculation to generate a command signal for controlling the robot 2. In one example, the command signal includes data for controlling the robot 2, such as a path indicating the trajectory of the robot 2. The trajectory of the robot 2 refers to the path of movement of the robot 2 or its components. For example, the trajectory of the robot 2 may be the trajectory of the tip of the robot. The robot control system 1 transmits the generated command signal to the robot controller 3.

[0017] The robot controller 3 is a device that operates the robot 2 in accordance with command signals from the robot control system 1. In one example, the robot controller 3 calculates joint angle target values ​​for matching the position and posture of the tip of the robot 2 with the target values ​​indicated by the command signals, and controls the robot 2 in accordance with the joint angle target values. The joint angle target values ​​are the angle target values ​​of each joint of the robot 2.

[0018] The robot 2 operates under the control of the robot control system 1 to perform multiple tasks. At least one task is set to process some kind of workpiece. A workpiece refers to a tangible object processed by the robot 2. In one example, the robot 2 is a multi-axis, serial-link, vertical, articulated robot. The robot 2 includes a manipulator 2a and an end effector 2b, which is a tool attached to the tip of the manipulator 2a. The robot 2 can perform various processes using the end effector 2b. The robot 2 can freely change the position and orientation of the end effector 2b within a given range. The robot 2 may be a six-axis vertical, articulated robot, or a seven-axis vertical, articulated robot that adds one redundant axis to the six axes.

[0019] The camera 4 is an imaging device that captures images of the workspace, for example, the area around the robot 2. In the example of Fig. 1, the camera 4 is provided in a location different from the robot 2, but the camera 4 may be disposed on the manipulator 2a, for example, attached near the tip of the manipulator 2a.

[0020] 1 also shows an example of the functional configuration of the robot control system 1. In this example, the robot control system 1 includes a storage unit 11, a tree management unit 12, and a task control unit 13 as functional components.

[0021] The memory unit 11 is a functional module that stores a behavior tree representing the operation of the robot 2 and resource information including the status of a shared resource. The status of a shared resource is information indicating the current state of the shared resource, for example, information indicating the relationship between the shared resource and a task. The memory unit 11 may further store history information regarding the execution of at least one task. The history information indicates the usage status of the shared resource for each of multiple tasks in at least two consecutive cycles. In one example, the history information corresponds to each control node (e.g., each parallel node). For example, the history information corresponding to a certain parallel node indicates, for each of multiple tasks corresponding to multiple subtrees connected to the parallel node, whether the task used the shared resource or whether the task released the shared resource.

[0022] The tree management unit 12 is a functional module that calls each node in the behavior tree in sequence, starting from the root node, in each cycle. When focusing on the root node or a certain control node, the tree management unit 12 calls each of the one or more subtrees connected to that node in sequence.

[0023] The task control unit 13 is a functional module that, in response to a subtree being called, controls the execution of a task corresponding to the subtree based on the status of the common resource.

[0024] The robot control system 1 can be realized by any type of computer. The computer may be a general-purpose computer such as a personal computer or a business server, or may be incorporated into a dedicated device that executes a specific process.

[0025] 2 is a diagram showing an example of the hardware configuration of a computer 100 used for the robot control system 1. In this example, the computer 100 includes a main body 110, a monitor 120, and an input device .

[0026] The main body 110 is a device having a circuit 160. The circuit 160 includes a processor 161, a memory 162, a storage 163, an input / output port 164, and a communication port 165. The number of each hardware component may be one or more. The storage 163 records programs for configuring each functional module of the main body 110. The storage 163 is a computer-readable recording medium such as a hard disk, a non-volatile semiconductor memory, a magnetic disk, or an optical disk. The memory 162 temporarily stores programs loaded from the storage 163, calculation results of the processor 161, and the like. The processor 161 configures each functional module by executing programs in cooperation with the memory 162. The input / output port 164 inputs and outputs electrical signals to and from the monitor 120 or the input device 130 in response to instructions from the processor 161. The input / output port 164 may also input and output electrical signals to and from other devices such as the robot controller 3 and the camera 4. The communication port 165 performs data communication with other devices via the communication network N in response to instructions from the processor 161.

[0027] The monitor 120 is a device for displaying information output from the main body 110. For example, the monitor 120 is a device capable of displaying graphics, such as a liquid crystal panel.

[0028] The input device 130 is a device for inputting information to the main body 110. Examples of the input device 130 include operation interfaces such as a keypad, a mouse, and an operation controller.

[0029] The monitor 120 and the input device 130 may be integrated as a touch panel. For example, the main body 110, the monitor 120, and the input device 130 may be integrated as a tablet computer.

[0030] Each functional module of the robot control system 1 is realized by loading a robot control program onto the processor 161 or memory 162 and having the processor 161 execute the program. The robot control program includes code for realizing each functional module of the robot control system 1. The processor 161 operates the input / output port 164 and the communication port 165 in accordance with the robot control program, and reads and writes data from and to the memory 162 or the storage 163.

[0031] The robot control program may be provided in the form of a non-transitory recording medium such as a CD-ROM, a DVD-ROM, or a semiconductor memory, or may be provided via a communications network as a data signal superimposed on a carrier wave.

[0032] [Behavior Tree] The behavior tree will be described with reference to Fig. 3. Fig. 3 is a diagram showing a behavior tree 200 stored in the storage unit 11.

[0033] FIG. 3 shows a fallback node 21, a sequence node 22, a parallel node 23, a fallback node with memory 24, and a sequence node with memory 25 as types of control nodes.

[0034] The fallback node 21 is used when it is sufficient for one of multiple child nodes to succeed. The fallback node 21 calls two or more child nodes in order from left to right. In response to any child node returning True or Running, the fallback node 21 returns the return value to the parent node as the execution result of the fallback node 21, and does not call the remaining child nodes. In response to all child nodes returning False, the fallback node 21 returns False to the parent node as the execution result of the fallback node 21.

[0035] The sequence node 22 is used when the success of all child nodes is required. The sequence node 22 calls two or more child nodes in order from left to right. In response to any child node returning False or Running, the sequence node 22 returns the return value to the parent node as the execution result of the sequence node 22, and does not call the remaining child nodes. In response to all child nodes returning True, the sequence node 22 returns True to the parent node as the execution result of the sequence node 22.

[0036] The parallel node 23 is used to execute all child nodes, essentially executing those child nodes in parallel or in parallel. The parallel node 23 calls two or more child nodes in order from left to right. The parallel node 23 returns True to the parent node in response to all child nodes returning True, returns False to the parent node in response to at least one child node returning False, and returns Running to the parent node in other cases.

[0037] The fallback node with memory 24 is an extension of the fallback node 21. The fallback node with memory 24 differs from the fallback node 21 in that, if the execution result in the previous cycle is Running, the fallback node with memory 24 calls the child nodes in order starting from the last child node called in that cycle.

[0038] The sequence node with memory 25 is an extension of the sequence node 22. The sequence node with memory 25 differs from the sequence node 22 in that, if the execution result in the previous cycle was Running, the sequence node with memory 25 calls the child nodes in order starting from the last child node called in that cycle.

[0039] FIG. 3 shows condition nodes 26 and action nodes 27 as types of execution nodes.

[0040] A condition node 26 is used to check whether a specific condition is met. The condition node 26 executes a function corresponding to the condition. The condition node 26 returns either True, indicating that the condition is met, or False, indicating that the condition is not met, to the parent node.

[0041] An action node 27 is used to execute a task. The action node 27 executes a function corresponding to the task. The action node 27 returns one of the following to the parent node: True, which indicates that the task was successful; False, which indicates that the task failed; or Running, which indicates that the task is in another state (e.g., running).

[0042] The robot control system 1 manages parallel processing and concurrent processing by using at least a parallel node. The following description will focus on the parallel node.

[0043] In behavior tree 200, root node 201 has a single child node, parallel node 202. Parallel node 202 has three child nodes, memory-attached sequence nodes 211, 231, and 251. Parallel node 202 is connected to subtree 210 having memory-attached sequence node 211 as its apex, subtree 230 having memory-attached sequence node 231 as its apex, and subtree 250 having memory-attached sequence node 251 as its apex.

[0044] Subtree 210 includes parallel node 212 located two levels below memory-equipped sequence node 211. Parallel node 212 connects to a subtree whose apex is sequence node 213 and a subtree whose apex is fallback node 214. Parallel node 215, located one level below fallback node 214, connects to a subtree whose apex is sequence node 216 and a subtree whose apex is sequence node 217.

[0045] Subtree 230 includes a parallel node 232 located two levels below memory-equipped sequence node 231. Parallel node 232 connects to a subtree having sequence node 233 as its apex and a subtree having sequence node 234 as its apex.

[0046] The behavior tree 200 includes four parallel nodes 202, 212, 215, and 232. The parallel node 202 is the highest-level parallel node, which is located at the highest level among these parallel nodes. The parallel nodes 212, 215, and 232 are lower-level parallel nodes, which are located at a lower level than the highest-level parallel node.

[0047] In one example, at least one of the multiple subtrees connected to a parallel node includes a occupy node, which is an action node that requests the parallel node to change the status of a common resource to in-use, and an execute node, which is an action node that uses the common resource. The tree manager 12 calls the occupy node and the execute node, in this order, in that subtree. At least one of the multiple subtrees connected to a parallel node may further include, in addition to the occupy node and the execute node, a release node, which is an action node that requests the parallel node to return the status of the common resource to available. The tree manager 12 calls the occupy node, the execute node, and the release node, in this order, in that subtree. In FIG. 3, the occupy node, the execute node, and the release node are represented by "P," "A," and "R," respectively. For example, the subtree connected to the parallel node 212 and having the sequence node 213 as its apex includes the occupy node 218, the execute node 219, and the release node 220.

[0048] In one example, the occupancy node, execution node, and release node return values ​​to their parent nodes as follows: the occupancy node returns True if the common resource is available, and Running otherwise; the execution node returns True if the task using the common resource is successful, False if the task fails, and Running if the task is in any other state (e.g., Running); the release node returns True if the common resource is releasable, and Running otherwise. The ordering of these nodes, occupancy node, execution node, and release node, controls whether the task runs using the common resource or waits to run until the common resource is available.

[0049] In one example, the storage unit 11 stores resource information in association with at least one parallel node. For example, the storage unit 11 may store resource information in association with the highest parallel node. Alternatively, the storage unit 11 may store resource information in association with each of the highest parallel node and the lower parallel node. In FIG. 3, the resource information is represented as "M." In the behavior tree 200, the storage unit 11 stores resource information in association with each of the parallel nodes 202, 212, 215, and 232. In this example, the parallel nodes 202, 212, 215, and 232 correspond to resource information 291, 291, 293, and 294, respectively.

[0050] In one example, resource information associated with a parallel node indicates the status of common resources that can be used by multiple tasks corresponding to multiple subtrees connected to the parallel node. Resource information 293 indicates the status of common resources that can be used in the lower layer of parallel node 215. Resource information 292 indicates the status of common resources that can be used in the lower layer of parallel node 212. Because parallel node 215 is included in one of the multiple subtrees connected to parallel node 212, resource information 292 includes resource information 293. Resource information 294 indicates the status of common resources that can be used in the lower layer of parallel node 232. Because parallel nodes 212 and 215 are included in subtree 210 and parallel node 232 is included in subtree 230, resource information 291 includes resource information 292 to 294. Therefore, resource information associated with the top-level parallel node may indicate the status of all common resources that can be used under management by the behavior tree.

[0051] The resource information may be associated with a control node other than a parallel node. For example, the resource information may be associated with at least one of the fallback node 21, the sequence node 22, the fallback node with memory 24, and the sequence node with memory 25. The resource information may be associated with each control node of the behavior tree 200.

[0052] [Status of a Common Resource] Figure 4 is a state transition diagram showing an example of changes in the status of a common resource. This example shows two types of state transition patterns: weak occupancy and strong occupancy. In one example, the tree manager 12 dynamically sets the state transition of the common resource based on the strength of occupancy associated with the execution node (i.e., task) that requests use of the common resource. Therefore, while multiple tasks are being executed based on the behavior tree, there may be a common resource whose state transition pattern changes between weak occupancy and strong occupancy.

[0053] When the state transition of a common resource is weak occupancy, the tree management unit 12 changes the status of the common resource among available, in-use, semi-released, and releasing states. That is, in weak occupancy, the status includes available, in-use, semi-released, and releasing states. The available state indicates that the common resource is not being used by any task and any task can start using the common resource. The in-use state indicates that the common resource is being used by a task and other tasks cannot use the common resource. The semi-released state is a state defined to avoid deadlock of the common resource. The releasing state is a state defined to release the common resource from a task that is occupying the common resource.

[0054] If a first task requests use of a shared resource when the shared resource status is in the available state, the tree management unit 12 updates the status to in use in response to the request. The task control unit 13 then causes the agent (robot 2) to execute the first task. If the shared resource status is in use, other tasks, including the second task, cannot use the shared resource, and therefore the task control unit 13 does not cause the agent to execute the second task.

[0055] When the cycle transitions to the next cycle, that is, when tree manager 12 calls the root node again, tree manager 12 updates the status from in use to provisionally released in response to the transition, which allows any task, including the first task and the second task, to potentially use the shared resource.

[0056] When the first task requests the use of the temporarily released shared resource, the tree management unit 12 responds to the request by updating the status to "in use," allowing the first task to continue using the shared resource.

[0057] On the other hand, if a second task requests the use of the provisionally released shared resource, the tree management unit 12 updates the status to "releasing" in response to the request, which creates the possibility that the second task will be able to use the shared resource.

[0058] When the status of the shared resource is updated to a releasing state, the task control unit 13 causes the agent (robot 2) to execute a release batch for releasing the shared resource. This release batch is pre-associated with the subtree corresponding to the first task in the behavior tree. When the shared resource is the robot 2, the release batch may be, for example, a process for causing the robot 2 holding a workpiece to temporarily place the workpiece in a predetermined location. In response to the completion of the release batch, the tree management unit 12 updates the status to a usable state. As a result, any task can use the shared resource.

[0059] Thus, in weak seizure, a common resource used by one task (first task) may be pre-empted by another task (second task) through provisional release and release batches.

[0060] When the state transition of a common resource is strong occupancy, the tree management unit 12 changes the status of the common resource between available and in use. That is, in strong occupancy, the status includes available and in use. The available state has the same meaning as weak occupancy. The in use state is the same as weak occupancy in that the common resource is in use by a task and cannot be used by other tasks, but differs from weak occupancy in that the common resource is provisionally released and not released. That is, in strong occupancy, a common resource being used by a task cannot be preempted by another task.

[0061] If a task requests the use of a shared resource when the shared resource status is in the available state, the tree management unit 12 updates the status to in use in response to the request. The task control unit 13 then causes the agent (robot 2) to execute the task. If the shared resource status is in use, another task cannot use the shared resource, and therefore the task control unit 13 does not cause the agent to execute the other task.

[0062] When a task that has been using a shared resource is terminated, the tree management unit 12 updates the status to available in response to the termination, so that any task can use the shared resource.

[0063] [Release Batches] FIG. 5 is a diagram showing an example of a release batch. This diagram shows a subtree with a parallel node 301 as its apex as part of a behavior tree. The parallel node 301 connects to three subtrees Sa, Sb, and Sc. In this example, robots Ra and Rb are both managed as agents and common resources. Subtree Sa corresponds to task Ta, which includes an action Ac by robot Ra and an action Ad by robot Rb. Subtree Sa also corresponds to release batch α for releasing robot Ra, which is being used to execute task Ta. Subtree Sb corresponds to task Tb, which includes an action Ap by robot Rb and an action Aq by robot Ra. Subtree Sb also corresponds to release batch β for releasing robot Rb, which is being used to execute task Tb. Subtree Sc corresponds to task Tc, which includes an action Ax. Subtree Sc does not include a release batch.

[0064] When the status of robot Ra is updated to released, the task control unit 13 causes robot Ra to execute released batch α. Robot Ra executes actions Bc, Bd, and Be constituting released batch α in this order. In response to the completion of released batch α, the tree management unit 12 updates the status of robot Ra to available. As a result, there is a possibility that task Tb will be able to use robot Ra.

[0065] When the status of robot Rb is updated to released, the task control unit 13 causes robot Rb to execute released batch β. Robot Rb executes the actions Bp, Bq, and Br that make up released batch β in this order. In response to the completion of released batch β, the tree management unit 12 updates the status of robot Rb to available. As a result, there is a possibility that task Ta will be able to use robot Rb.

[0066] The resource information 302 indicates the status of each of the robots Ra and Rb, which are shared resources. If the status of the shared resource is in use, the resource information 302 further indicates a release batch for releasing the shared resource.

[0067] [Robot Control Method] As an example of a robot control method according to the present disclosure, processing executed by the robot control system 1 will be described with reference to Figs. 6 and 7. Fig. 6 is a flowchart showing one example of the processing as processing flow S1. That is, the robot control system 1 executes processing flow S1. Fig. 7 is a flowchart showing one example of processing related to a parallel node, which is part of processing flow S1. In the examples of Figs. 6 and 7, the agent is robot 2.

[0068] As described above, the tree manager 12 calls each node of the behavior tree in turn in each cycle. Figure 6 shows the cycle using a variable i. In step S11, the tree manager 12 initializes the variable i to 1.

[0069] In step S12, the robot control system 1 calls each node based on the behavior tree and controls task execution as processing in the i-th cycle. The tree management unit 12 calls each node in order starting from the root node based on the behavior tree stored in the storage unit 11. If the called node corresponds to a task, the task control unit 13 refers to the resource information stored in the storage unit 11 and cooperates with the tree management unit 12 to control task execution based on the status of the shared resource.

[0070] With reference to FIG. 7, processing related to one parallel node will be described as processing flow S120. In this description, the parallel node will be referred to as the "target parallel node" for convenience. Processing flow S120 is part of step S12 and is executed for each of one or more parallel nodes in the behavior tree. As described above, a parallel node connects to multiple subtrees corresponding to multiple tasks. FIG. 7 uses the variable k to distinguish between individual subtrees.

[0071] In steps S1201 and S1202, the tree management unit 12 calls the first subtree connected to the target parallel node. The first subtree is the subtree located at the leftmost position one level below the target parallel node.

[0072] In step S1203, in response to the invocation of the subtree, the task control unit 13 refers to the resource information and obtains the status of the common resource used by the task corresponding to the subtree. In one example, the task control unit 13 refers to the resource information associated with any parallel node in the behavior tree. For example, the task control unit 13 may refer to the resource information associated with the target parallel node. If the target parallel node is a lower-level parallel node, the task control unit 13 may refer to the resource information associated with the highest-level parallel node. Alternatively, the task control unit 13 may refer to the resource information associated with the highest-level parallel node and the target parallel node (lower-level parallel node). Alternatively, the task control unit 13 may refer to the resource information associated with each of these two parallel nodes, as well as the resource information associated with another control node (e.g., another parallel node) located on the path from the target parallel node to the highest-level parallel node. When the task control unit 13 refers to the resource information for two or more control nodes, including the highest-level parallel node and the target parallel node (lower-level parallel node), the task control unit 13 obtains multiple statuses from the resource information of those nodes.

[0073] In step S1204, the task control unit 13 controls task execution based on the status of the shared resource in cooperation with the tree management unit 12. Alternatively, the task control unit 13 controls execution of the released batch based on the status of the shared resource. In connection with these controls, the tree management unit 12 updates the status as necessary.

[0074] (In the case of a usable state) When the status is a usable state, the task control unit 13 causes the robot 2 (agent) to execute a task using a shared resource. When multiple statuses are acquired, the task control unit 13 may cause the robot 2 to execute a task in response to each of the statuses being a usable state.

[0075] In response to the execution of a task, the tree management unit 12 updates the status to "in use." If multiple statuses are acquired, the tree management unit 12 updates each status to "in use." In this case, the status of the shared resource is updated to "in use" in the resource information of each of two or more control nodes, including at least the top parallel node and the lower parallel node. In this way, when the task control unit 13 causes the robot 2 to execute a task (first task), the tree management unit 12 stores in the storage unit 11 resource information indicating that the task (first task) is using the shared resource and that another task (second task) cannot use the shared resource. The tree management unit 12 further stores in the storage unit 11 history information indicating that the task used the shared resource in the i-th cycle.

[0076] The task control unit 13 generates a path for executing the task by path planning, and outputs a command signal indicating the path to the robot controller 3. The robot controller 3 controls the robot 2 according to the command signal. The robot 2 operates along the path to execute the task.

[0077] (In case of busy state) If the status is in busy state, the task control unit 13 does not cause the robot 2 to execute the task. In other words, the task control unit 13 does not generate or output an instruction signal. If multiple statuses are acquired, the task control unit 13 responds that at least one status is in busy state and does not cause the robot 2 to execute the task. The tree management unit 12 stores in the memory unit 11 history information indicating that the task does not use the shared resource in the i-th cycle.

[0078] (In the case of a provisionally released state) When the status is a provisionally released state, the task control unit 13 executes processing according to the task corresponding to the called subtree. The task is either a first task that used the shared resource before being provisionally released, or a second task that was unable to use the shared resource before being provisionally released.

[0079] When the first task requires the use of a shared resource, the task control unit 13 causes the robot 2 (agent) to execute the first task using the shared resource. When multiple statuses are acquired, the task control unit 13 may cause the robot 2 to execute the first task in response to each status being a provisional release state.

[0080] In response to the execution of the first task, the tree management unit 12 updates the status to "in use." If multiple statuses are acquired, the tree management unit 12 updates each status to "in use." In this case, the status of the shared resource is updated to "in use" in the resource information of each of two or more control nodes, including at least the top parallel node and the lower parallel node. Thus, when the task control unit 13 causes the robot 2 to execute the first task, the tree management unit 12 stores in the storage unit 11 resource information indicating that the first task is using the shared resource and that another task (second task) cannot use the shared resource. In other words, the tree management unit 12 updates the status of the shared resource to "in use" so that the first task can continue to use the shared resource. The tree management unit 12 also stores in the storage unit 11 history information indicating that the first task used the shared resource in the i-th cycle.

[0081] The task control unit 13 generates a path for executing the first task by path planning, and outputs a command signal indicating the path to the robot controller 3. The robot controller 3 controls the robot 2 in accordance with the command signal. The robot 2 operates along the path to execute the first task.

[0082] If the second task requests the use of the shared resource, the tree manager 12 updates the status to "releasing." If multiple statuses are acquired, the tree manager 12 updates each status to "releasing." In this case, the status of the shared resource is updated to "releasing" in the resource information of each of two or more control nodes, including at least the top parallel node and the lower parallel node.

[0083] The task control unit 13 does not cause the robot 2 to execute either the first task or the second task. That is, the task control unit 13 does not generate or output an instruction signal. The tree management unit 12 stores in the storage unit 11 history information indicating that the first task is scheduled to release the shared resource in the i-th cycle.

[0084] (Releasing State) When the status is the releasing state, the task control unit 13 executes processing in response to the fact that the task corresponding to the called subtree is the first task that used the shared resource before being provisionally released. The task control unit 13 causes the robot 2 (agent) to execute a release batch for releasing the shared resource used in the first cycle. When multiple statuses are acquired, the task control unit 13 may cause the robot 2 to execute a release batch in response to each status being the releasing state.

[0085] On the other hand, if the task corresponding to the called subtree is a task different from the first task, the task control unit 13 does not cause the robot 2 to execute the different task. In other words, the task control unit 13 does not generate or output an instruction signal. The tree management unit 12 stores in the storage unit 11 history information indicating that the different task does not use the shared resource in the i-th cycle.

[0086] When a release batch is executed, the tree manager 12 updates the status to available. When multiple statuses are acquired, the tree manager 12 updates each status to available. In this case, the status of the shared resource is updated to available in the resource information of each of two or more control nodes including at least the top parallel node and the lower parallel node. The tree manager 12 further stores in the storage unit 11 history information indicating that the first task released the shared resource in the i-th cycle.

[0087] The task control unit 13 generates a path for executing the released batch by path planning, and outputs a command signal indicating the path to the robot controller 3. The robot controller 3 controls the robot 2 in accordance with the command signal. The robot 2 operates along the path to execute the released batch.

[0088] In step S1205, the tree management unit 12 determines whether all subtrees connected to the target parallel node have been called. If all subtrees have been called (YES in step S1205), the tree management unit 12 ends processing for the target parallel node and executes processing for the next node based on the behavior tree. If there is an uncalled subtree (NO in step S1205), the processing proceeds to step S1206, where the tree management unit 12 selects the (k+1)th subtree, i.e., the next subtree. Then, the processing returns to S1202, where the tree management unit 12 calls that subtree. Thereafter, the robot control system 1 executes the processing from step S1203 onwards for that subtree.

[0089] 6, in step S13, the tree management unit 12 determines whether or not to terminate the entire processing managed by the behavior tree. For example, the tree management unit 12 may terminate the processing in response to the completion of multiple tasks to be executed, or in response to the arrival of the operation end time of the robot 2 as an agent (YES in step S13). If the processing is not to be terminated (NO in step S13), the processing proceeds to step S14. In step S14, the tree management unit 12 increments the variable i, which indicates the cycle, by 1.

[0090] In step S15, the tree management unit 12 updates the status of the shared resource that is currently in use with weak occupancy to the provisionally released status. The tree management unit 12 accesses the storage unit 11 and references the resource information associated with at least one control node in the behavior tree. Next, the tree management unit 12 updates the status of the shared resource that is currently in use with weak occupancy to the provisionally released status for each piece of resource information.

[0091] In step S16, the tree management unit 12 changes the order in which subtrees are called for each parallel node as needed. For each parallel node, the tree management unit 12 changes the order in which multiple subtrees connected to the parallel node are called based on the history information corresponding to the parallel node. For example, if a task corresponding to a subtree located first relative to a parallel node has not used a shared resource for at least two consecutive cycles, the tree management unit 12 may move the subtree to the second or later position. Alternatively, if a task corresponding to a subtree located first relative to a parallel node releases a shared resource used in a certain cycle (first cycle) in the next cycle (second cycle), the tree management unit 12 may move the subtree located first to the second or later position.

[0092] Changing the order in which subtrees are called will be described with reference to Figure 8. Figure 8 shows an example of such a change. In this example, a parallel node 401 connects to subtrees 411, 412, and 413. Subtree 411 corresponds to a task realized by a combination of actions A1, A2, and A3. Subtree 412 corresponds to a task realized by a combination of actions A4, A5, and A6. Subtree 413 corresponds to a task realized by action A7.

[0093] Assume that in a certain cycle, subtrees 411, 412, and 413 are arranged in this order. That is, subtree 411 is the first subtree, subtree 412 is the second subtree, and subtree 413 is the last subtree. In the example of FIG. 8 , the tree manager 12 moves the subtree located in the first position to the last position. As a result, the next time the tree manager 12 invokes parallel node 401, it invokes three subtrees in the order of subtrees 412, 413, and 411. The tree manager 12 may also move the subtree located in the first position between any two other subtrees; for example, it may move subtree 411 between subtrees 412 and 413.

[0094] 6, the process returns to step S12 after step S16. In the repeated step S12, the robot control system 1 again calls each node based on the behavior tree and controls the execution of tasks as the process for the next cycle. Then, in step S13, the tree management unit 12 again determines whether or not to terminate the entire process.

[0095] As described above with reference to Figures 6 and 7, the tree management unit 12 calls each of the multiple subtrees from the target parallel node in turn based on the behavior tree in each cycle. The task control unit 13 executes a task corresponding to the called subtree. For each of the multiple subtrees, the task control unit 13 references resource information when the subtree is called. If the status of the common resource used by the task corresponding to that subtree is available, the task control unit 13 causes the robot 2 (agent) to execute the task using the common resource. The tree management unit 12 updates the status of the common resource indicated by the resource information from in use to temporarily released, and after this update, calls the multiple subtrees from the parallel node.

[0096] [Modifications] Various examples of the present disclosure have been described above in detail. However, the technical matters of the present disclosure are not limited to the above examples. Various modifications are possible to the technical matters of the present disclosure without departing from the gist thereof.

[0097] The hardware configuration of the system is not limited to a configuration in which each functional module is realized by executing a program. For example, at least some of the functional modules may be configured by logic circuits specialized for the functions, or may be configured by an ASIC (Application Specific Integrated Circuit) that integrates the logic circuits.

[0098] The processing steps of the method executed by at least one processor are not limited to the above examples. For example, some of the steps or processes described above may be omitted, or the steps may be executed in a different order. Furthermore, any two or more of the steps described above may be combined, or some of the steps may be modified or deleted. Alternatively, other steps may be executed in addition to the steps described above.

[0099] When comparing the magnitude of two numbers within a computer system or computer, either of the two criteria "greater than or equal to" and "greater than" can be used, or either of the two criteria "less than or equal to" and "under".

[0100] [Supplementary Note] As can be seen from the various examples above, the present disclosure includes the following aspects: (Supplementary Note 1) A robot control system comprising: a storage unit that stores a behavior tree including a plurality of subtrees corresponding to a plurality of tasks and parallel nodes that are parent nodes connected to the plurality of subtrees, and resource information including statuses of common resources used by the plurality of tasks, wherein the common resources include robots, and the statuses include at least a busy state and a usable state; a tree management unit that calls each of the plurality of subtrees in turn from the parallel node based on the behavior tree in each cycle; and a task control unit that executes the task corresponding to the called subtree, wherein the task control unit: for each of the plurality of subtrees, when the subtree is called, refers to the resource information, and when the status of the common resource used by the task corresponding to the subtree is the usable state, executes the task using the common resource. (Supplementary Note 2) The robot control system according to Supplementary Note 1, wherein the storage unit stores the resource information in association with the parallel node, and the task control unit, for each of the plurality of subtrees, refers to the resource information associated with the parallel node when the subtree is called. (Supplementary Note 3) The robot control system according to Supplementary Note 2, wherein the behavior tree includes, as the plurality of parallel nodes, a top-level parallel node located at the top of the plurality of parallel nodes and a lower-level parallel node located below the top-level parallel node, the storage unit stores the resource information in association with the top-level parallel node, and the task control unit, for each of the plurality of subtrees connected to the lower-level parallel node, refers to the resource information associated with the top-level parallel node when the subtree is called.(Supplementary Note 4) The robot control system according to Supplementary Note 3, wherein the storage unit stores the resource information in association with at least the top-level parallel node and each of the lower-level parallel nodes, and the task control unit, for each of the plurality of subtrees connected to the lower-level parallel node, when the subtree is called, references the resource information associated with the top-level parallel node and the resource information associated with the lower-level parallel node, and executes the task if the status of the common resource used by the task corresponding to the subtree is in the available state in at least both the resource information associated with the top-level parallel node and the resource information associated with the lower-level parallel node. (Supplementary Note 5) The robot control system according to any one of Supplementary Notes 1 to 4, wherein, when the task control unit executes the task, the tree management unit saves in the storage unit the resource information indicating that the task is using the common resource and that other tasks cannot use the common resource. (Supplementary Note 6) The robot control system according to Supplementary Note 5, wherein the individual cycles include a first cycle and a second cycle following the first cycle, the plurality of tasks include a first task and a second task, the tree management unit saves the resource information indicating that the first task is using the common resource and that the second task cannot use the common resource in the first cycle, and when transitioning to the second cycle, the tree management unit updates the status of the common resource indicated by the saved resource information to a provisionally released state that avoids deadlock of the common resource, and calls the plurality of subtrees from the parallel node after updating the resource information.(Supplementary Note 7) The robot control system according to Supplementary Note 6, wherein, in the second cycle, when the status of the common resource is the provisionally released state, if the first task requests use of the common resource, the tree management unit updates the status of the common resource to an in-use state so that the first task can continue to use the common resource, and when the second task requests use of the common resource, the tree management unit updates the status of the common resource to a releasing state so that the common resource is released from the first task. (Supplementary Note 8) The robot control system according to Supplementary Note 7, wherein a subtree corresponding to the first task further corresponds to a release batch for releasing the common resource used in the first cycle, and the task control unit executes the release batch when the status of the common resource is updated to the releasing state. (Supplementary Note 9) The robot control system according to any one of Supplements 5 to 8, wherein at least one of the plurality of subtrees has an occupancy node that requests the parallel node to change the status of the common resource to the in-use state, and an execution node that uses the common resource, and the tree management unit calls the occupancy node and the execution node in this order when calling at least one of the plurality of subtrees. (Supplementary Note 10) The robot control system according to Supplementary Note 9, wherein at least one of the plurality of subtrees further has a release node that requests the parallel node to return the status of the common resource to the available state, and the tree management unit calls the occupancy node, the execution node, and the release node in this order when calling at least one of the plurality of subtrees. (Supplementary Note 11) The robot control system according to any one of Supplements 1 to 10, wherein the storage unit further stores history information regarding the execution of at least one of the plurality of tasks, and the tree management unit changes the order in which the plurality of subtrees are called based on the history information.(Supplementary Note 12) The robot control system according to Supplementary Note 11, wherein the tree management unit moves the subtree located first with respect to the parallel node to a second or later position when the task corresponding to the subtree located first with respect to the parallel node has not used the common resource for at least two consecutive cycles. (Supplementary Note 13) The robot control system according to Supplementary Note 11 or 12, wherein the individual cycles include a first cycle and a second cycle following the first cycle, and the tree management unit moves the subtree located first with respect to the parallel node to a second or later position when the task corresponding to the subtree located first with respect to the parallel node releases the common resource used in the first cycle in the second cycle. (Supplementary Note 14) A robot control method executed by a robot control system having at least one processor, comprising: a step of storing in a storage unit a behavior tree including a plurality of subtrees corresponding to a plurality of tasks and parallel nodes that are parent nodes connected to the plurality of subtrees, and resource information including statuses of common resources used by the plurality of tasks, wherein the common resources include robots, and the statuses include at least a busy state and a usable state; a step of calling each of the plurality of subtrees in turn from the parallel node based on the behavior tree in each cycle; and a step of having the robot execute the task corresponding to the called subtree, wherein in the step of having the robot execute the task, for each of the plurality of subtrees, when the subtree is called, the resource information is referenced, and when the status of the common resource used by the task corresponding to the subtree is the usable state, the task is executed using the common resource.(Supplementary Note 15) A robot control program that causes a computer to execute the following steps: storing in a storage unit a behavior tree including a plurality of subtrees corresponding to a plurality of tasks and parallel nodes that are parent nodes connected to the plurality of subtrees, and resource information including statuses of common resources used by the plurality of tasks, wherein the common resources include a robot, and the statuses include at least an in-use state and an available state; calling each of the plurality of subtrees in turn from the parallel node based on the behavior tree in each cycle; and causing the robot to execute the task corresponding to the called subtree, wherein in the step of causing the robot to execute the task, for each of the plurality of subtrees, when the subtree is called, the resource information is referenced; and when the status of the common resource used by the task corresponding to the subtree is the available state, the task is executed using the common resource.

[0101] According to Supplementary Notes 1, 14, and 15, both a behavior tree showing multiple tasks using a shared resource and the status of the shared resource are managed in an integrated manner, and individual tasks are executed based on that status. This mechanism allows multiple tasks to be executed efficiently.

[0102] According to Supplementary Note 2, resource information is associated with parallel nodes, making the correspondence between shared resources and multiple tasks clearer. As a result, it is easier to implement a process for referencing the status of a shared resource corresponding to a task.

[0103] According to Supplementary Note 3, resource information is associated with the highest level parallel node, making it possible to centrally manage one or more common resources that may be used.

[0104] According to Supplementary Note 4, resource information is associated with at least both the top-level parallel node and the lower-level parallel node, making it easier to manage the status of common resources even when the behavior tree becomes large or complex.

[0105] According to Supplementary Note 5, when a task is executed using a shared resource, resource information is saved so that the task occupies the shared resource. By managing resource information in this manner, it is possible to more reliably manage the occupation of the shared resource by a task.

[0106] According to Supplementary Note 6, when moving to the next cycle, the status of the shared resource occupied by the first task is updated to a provisionally released state, so that the possibility of using the shared resource is provided not only to the first task but also to the second task (i.e., other tasks). This mechanism allows multiple tasks to be executed efficiently as a whole.

[0107] According to Supplementary Note 7, the status of a shared resource is updated according to the task that requested the use of the shared resource. This mechanism allows multiple tasks to be executed efficiently as a whole according to the situation in which the tasks are invoked.

[0108] According to Supplementary Note 8, when a task other than the first task that has been using the shared resource attempts to use the shared resource, the shared resource is released by the release batch. This mechanism makes it possible for the other task to use the shared resource more reliably.

[0109] According to Supplementary Note 9, common resources can be more easily designed or managed, since the request and use of the common resources are explicitly represented by occupancy nodes and execution nodes, respectively, in the behavior tree.

[0110] According to Supplementary Note 10, the release of a common resource is explicitly represented by a release node in the behavior tree, making it easier to design or manage the common resource.

[0111] According to Supplementary Note 11, the order of calling subtrees is changed based on the task execution history, so that resource starvation can be more reliably eliminated or suppressed.

[0112] According to Supplementary Note 12, the subtree corresponding to a task that is not using the shared resource is moved down in the order, allowing another task to request the use of the shared resource first. This mechanism allows multiple tasks to be executed more efficiently.

[0113] According to Supplementary Note 13, the subtree corresponding to the task that released the shared resource is moved down in the order, allowing another task to request the use of the shared resource first. This mechanism allows multiple tasks to be executed more efficiently.

[0114] 1... robot control system, 2... robot, 3... robot controller, 4... camera, 11... memory unit, 12... tree management unit, 13... task control unit, 200... behavior tree.

Claims

1. a storage unit that stores a behavior tree including a plurality of subtrees corresponding to a plurality of tasks and parallel nodes that are parent nodes connected to the plurality of subtrees, and resource information including a status of a common resource used by the plurality of tasks, the common resource including a robot, and the status including at least an in-use state and an available state; a tree manager that sequentially invokes each of the plurality of subtrees from the parallel node based on the behavior tree in each cycle; a task control unit that executes the task corresponding to the called subtree; Equipped with The task control unit performs the following for each of the plurality of subtrees: Referencing the resource information when the subtree is called; If the status of the common resource used by the task corresponding to the subtree is in the available state, execute the task using the common resource. Robot control system.

2. the storage unit stores the resource information in association with the parallel node; the task control unit refers to the resource information associated with the parallel node for each of the plurality of subtrees when the subtree is called; The robot control system of claim 1 .

3. the behavior tree includes, as the plurality of parallel nodes, a top-level parallel node located at the top of the plurality of parallel nodes, and a lower-level parallel node located below the top-level parallel node; the storage unit stores the resource information in association with the highest-level parallel node; the task control unit refers to the resource information associated with the highest-level parallel node for each of the plurality of subtrees connected to the lower-level parallel node when the subtree is called; The robot control system of claim 2 .

4. the storage unit stores the resource information in association with at least the top parallel node and the lower parallel node, The task control unit, for each of the plurality of subtrees connected to the lower parallel node, When the subtree is invoked, refer to the resource information associated with the highest parallel node and the resource information associated with the lower parallel node; executes a task when the status of the common resource used by the task corresponding to the subtree is in the available state in at least both the resource information associated with the top parallel node and the resource information associated with the lower parallel node; The robot control system of claim 3 .

5. when the task control unit executes the task, the tree management unit stores in the storage unit the resource information indicating that the task is using the shared resource and that other tasks cannot use the shared resource; The robot control system according to any one of claims 1 to 4.

6. each of the cycles includes a first cycle and a second cycle following the first cycle; the plurality of tasks include a first task and a second task; the tree management unit stores the resource information indicating that the first task is using the common resource and that the second task is unable to use the common resource in the first cycle; When transitioning to the second cycle, the tree management unit: updating the status of the common resource indicated by the saved resource information to a provisionally released state that avoids a deadlock of the common resource; after updating the resource information, invoking the plurality of subtrees from the parallel node; The robot control system of claim 5 .

7. When the status of the shared resource is the provisionally released state in the second cycle, the tree management unit: If the first task requests use of the common resource, updating the status of the common resource to a busy state so that the first task can continue to use the common resource; if the second task requests use of the common resource, updating the status of the common resource to a releasing state, which releases the common resource from the first task; The robot control system of claim 6.

8. the subtree corresponding to the first task further corresponds to a release batch for releasing the common resource used in the first cycle; the task control unit executes the released batch when the status of the common resource is updated to the releasing state. The robot control system of claim 7.

9. At least one of the plurality of subtrees has an occupancy node that requests the parallel node to change the status of the common resource to the in-use state, and an execution node that uses the common resource; the tree management unit calls the occupation node and the execution node in this order in the invocation of the at least one of the plurality of subtrees; The robot control system of claim 5 .

10. the at least one of the plurality of subtrees further comprises a release node that requests the parallel node to return the status of the common resource to the available state; the tree management unit calls the occupancy node, the execution node, and the release node in this order in the invocation of at least one of the plurality of subtrees; The robot control system of claim 9.

11. the storage unit further stores history information relating to the execution of at least one of the plurality of tasks; the tree management unit changes the order in which the plurality of subtrees are called based on the history information. The robot control system according to any one of claims 1 to 4.

12. the tree management unit moves the subtree located first with respect to the parallel node to a second or subsequent position when the task corresponding to the subtree has not used the common resource for at least two consecutive cycles; The robot control system of claim 11.

13. each of the cycles includes a first cycle and a second cycle following the first cycle; the tree management unit moves the subtree located first with respect to the parallel node to a second or subsequent position when the task corresponding to the subtree used in the first cycle releases the common resource in the second cycle. The robot control system of claim 11.

14. 1. A robot control method executed by a robot control system having at least one processor, comprising: a step of storing in a storage unit a behavior tree including a plurality of subtrees corresponding to a plurality of tasks and parallel nodes that are parent nodes connected to the plurality of subtrees, and resource information including statuses of common resources used by the plurality of tasks, the common resources including robots, and the statuses including at least an in-use state and an available state; In each cycle, sequentially invoking each of the plurality of subtrees from the parallel node based on the behavior tree; executing the task corresponding to the invoked subtree; Including, In the step of executing the task, for each of the plurality of subtrees, Referencing the resource information when the subtree is called; If the status of the common resource used by the task corresponding to the subtree is in the available state, execute the task using the common resource. Robot control method.

15. a step of storing in a storage unit a behavior tree including a plurality of subtrees corresponding to a plurality of tasks and parallel nodes that are parent nodes connected to the plurality of subtrees, and resource information including statuses of common resources used by the plurality of tasks, the common resources including robots, and the statuses including at least an in-use state and an available state; In each cycle, sequentially invoking each of the plurality of subtrees from the parallel node based on the behavior tree; executing the task corresponding to the invoked subtree; on the computer, In the step of executing the task, for each of the plurality of subtrees, Referencing the resource information when the subtree is called; If the status of the common resource used by the task corresponding to the subtree is in the available state, execute the task using the common resource. Robot control program.