Robot control system, robot control method, and robot control program

Abstract

A robot control system includes: a storage unit that stores a behavior tree including a plurality of sub-trees corresponding to a plurality of tasks and parallel nodes that are parent nodes connected to the plurality of sub-trees, and resource information including a state of a common resource used by the plurality of tasks; a tree management unit that, in each cycle, sequentially calls each of the plurality of sub-trees from the parallel node on the basis of the behavior tree; and a job control unit that executes a job corresponding to the called sub-tree. The public resources comprise robots. The state at least comprises an in-use state and an available state. For each sub-tree, the task control unit refers to the resource information when the sub-tree is called, and executes the task by using the common resource when the state of the common resource used by the task corresponding to the sub-tree is a usable state.

Description

Technical Field

[0001] One aspect of the present disclosure relates to a robot control system, a robot control method, and a robot control program. Background Art

[0002] Patent Document 1 describes a method for controlling a robot that includes utilizing a behavior tree framework for tasks performed by the robot.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent No. 6943364 Summary of the Invention

[0006] Problems to be solved by the invention

[0007] Expected to perform multiple tasks efficiently.

[0008] Means for solving problems

[0009] A robot control system according to one aspect of the present disclosure comprises: a storage unit storing a behavior tree and resource information, wherein the behavior tree includes a plurality of subtrees corresponding to a plurality of tasks and a parallel node serving as a parent node connected to the plurality of subtrees, and the resource information includes the status of a common resource used by the plurality of tasks, the common resource including a robot, and the status including at least an in-use state and a usable state; a tree management unit sequentially calling each of the plurality of subtrees from the parallel node based on the behavior tree in each loop; and a task control unit executing the task corresponding to the called subtree, the task control unit referring to the resource information for each of the plurality of subtrees when the subtree is called, and using the common resource to execute the task when the status of the common resource used by the task corresponding to the subtree is a usable state.

[0010] Effects of the Invention

[0011] According to one aspect of the present disclosure, multiple tasks can be performed efficiently. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is a diagram showing an example of application of a robot control system.

[0013] Figure 2 This is a diagram showing an example of the hardware configuration of a computer used in a robot control system.

[0014] Figure 3 This is a diagram showing an example of a behavior tree.

[0015] Figure 4This is a state transition diagram showing an example of state changes of a common resource.

[0016] Figure 5 This is a diagram showing an example of releasing a batch.

[0017] Figure 6 This is a flowchart showing an example of processing executed by the robot control system.

[0018] Figure 7 This is a flowchart showing an example of processing related to a parallel node.

[0019] Figure 8 This is a diagram showing an example of a change in the order of calling subtrees. DETAILED DESCRIPTION

[0020] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or equivalent elements are denoted by the same reference numerals, and repeated descriptions are omitted.

[0021] [System Overview]

[0022] One example of a robot control system is a computer system for allowing agents to perform multiple tasks in a real workspace using common resources including robots. A task is a process performed to achieve a certain purpose. An agent is a subject that performs a task. An agent can be a robot or another type of device or machine. In a robot control system, a robot can be managed as at least one of an agent and a common resource. Common resources are resources (such as hardware resources) that may be used by multiple tasks. Examples of common resources other than robots include various sensors such as cameras and conveying devices such as conveyors. The robot control system manages at least one common resource.

[0023] Executing multiple tasks using common resources is equivalent to concurrent processing, parallel processing, or a combination thereof. The use of common resources by a task means that the common resources are occupied by the task. Among these processes, phenomena such as deadlock and resource starvation are considered. Deadlock refers to the following phenomenon: because two or more processes (tasks in this disclosure) request resources that are being secured by other processes, neither process can proceed. Resource starvation refers to the phenomenon that a process (task) cannot permanently obtain resources and cannot be executed. Generally, it is not easy to design a system to avoid these problems and execute multiple tasks in parallel or in parallel.

[0024] Robot control systems use behavior trees to manage multiple tasks and shared resources, avoiding or reducing deadlocks and resource starvation while processing multiple tasks in parallel or parallel. This results in efficient execution of multiple tasks. Furthermore, by incorporating behavior trees, it's easy to design and manage mechanisms that allow agents to flexibly operate in response to changes in the real-world workspace environment.

[0025] A behavior tree is a method of representing agent actions using a tree structure. A behavior tree consists of a root node, control nodes, and execution nodes. Each node is connected to other nodes via directed edges. The node at the starting point of a directed edge is called a "parent node," and the node at the end point of a directed edge is called a "child node." Each node has at most one parent node and zero or more child nodes. The root node is the topmost node in the behavior tree. The root node has no parent node and typically has one child node. A control node has one parent node and one or more child nodes. In response to being called, a control node sequentially calls one or more child nodes. An execution node has one parent node and no child nodes. Execution nodes are also called "leaf nodes" of the behavior tree. Multiple tasks are associated with each execution node.

[0026] In this disclosure, when focusing on a specific node, the collection of a child node and zero or more nodes below it is also referred to as a "subtree." Each subtree corresponds to a task. Because subtrees can be defined at various levels of the behavior tree, the relationship between subtrees can be described as a nested structure. With this structure, a task can be implemented through a collection of multiple subtasks.

[0027] The root node calls its child nodes at a predetermined, recurring interval. This call is also called a "tick." Typically, this recurring interval is very short, set, for example, based on the frame rate of the video captured by camera 4. In response to the root node's call, each node in the subtree connected to the root node is called in a predetermined order, starting from the left, based on the tree structure. This call (tick) propagates from the root node to each execution node, thereby executing the entire behavior tree. As an execution result, each node returns either True indicating success, False indicating failure, or Running, indicating that the execution is in progress.

[0028] [System Structure]

[0029] Figure 1This diagram illustrates an example application of a robot control system 1. In this example, a robot 2 is shown as an agent, located in a real workspace. The robot control system 1 is connected to a robot controller 3, which is in turn connected to the robot 2. A camera 4 is provided in the workspace to capture the environment containing the robot 2. The camera 4 is connected to the robot control system 1. The communication networks connecting the devices can be either wired or wireless. The communication network can also include at least one of the Internet and an intranet. Alternatively, the communication network can be implemented simply via a single communication cable.

[0030] In this example, robot control system 1 uses common resources, including robot 2 and camera 4, to cause robot 2 to perform multiple tasks. Robot control system 1 performs predetermined operations and generates command signals for controlling robot 2. In one example, the command signals include data for controlling robot 2, such as a path indicating the trajectory of robot 2. The trajectory of robot 2 refers to the path along which robot 2 or its components move. For example, the trajectory of robot 2 can be a track on the end device. Robot control system 1 transmits the generated command signals to robot controller 3.

[0031] The robot controller 3 is a device that operates the robot 2 in accordance with command signals from the robot control system 1. In one example, the robot controller 3 calculates target joint angle values ​​for aligning the position and posture of the robot 2's end portion with the target values ​​indicated by the command signals, and controls the robot 2 in accordance with these target joint angle values. The target joint angle values ​​are target angle values ​​for each joint of the robot 2.

[0032] The robot 2 performs a plurality of tasks by operating under the control of the robot control system 1. At least one task is set to process certain workpieces. The workpiece refers to a tangible object processed by the robot 2. In one example, the robot 2 is a multi-axis serial link type vertical multi-joint robot. The robot 2 has a manipulator 2a and a tool, namely an end effector 2b, mounted at the end of the manipulator 2a. The robot 2 can perform various processes using the end effector 2b. The robot 2 can freely change the position and posture of the end effector 2b within a given range. The robot 2 can be a 6-axis vertical multi-joint robot or a 7-axis vertical multi-joint robot obtained by adding a redundant axis to the 6 axes.

[0033] The camera 4 is a camera device that captures the work space, for example, the surroundings of the robot 2. Figure 1 In the example shown, the camera 4 is set at a location different from the robot 2, but the camera 4 may be arranged on the manipulator 2a, for example, it may be installed near the end of the manipulator 2a.

[0034] Figure 1 The following also shows an example of the functional configuration of the robot control system 1. In one example, the robot control system 1 includes a storage unit 11, a tree management unit 12, and a task control unit 13 as functional components.

[0035] The storage unit 11 is a functional module that stores a behavior tree that reflects the operation of the robot 2 and resource information including the status of common resources. The status of common resources refers to information that represents the current status of common resources, and for example, refers to information that represents the relationship between common resources and tasks. The storage unit 11 may also store historical information related to the execution of at least one task. The historical information represents the usage status of common resources for each of a plurality of tasks in at least two consecutive cycles. In one example, the historical information corresponds to each control node (e.g., each parallel node). For example, the historical information corresponding to a parallel node indicates, for a plurality of tasks corresponding to a plurality of subtrees connected to the parallel node, whether the task has used common resources or whether the task has released common resources.

[0036] The tree management unit 12 is a functional module that sequentially calls each node from the root node based on the behavior tree in each loop. When focusing on the root node or a control node, the tree management unit 12 sequentially calls each of the one or more subtrees connected to the node.

[0037] The task control unit 13 is a functional module that controls the execution of a task corresponding to a subtree based on the state of a common resource in response to a subtree being called.

[0038] The robot control system 1 can be realized by any type of computer. The computer may be a general-purpose computer such as a personal computer or a business server, or may be incorporated into a dedicated device that executes specific processing.

[0039] Figure 2 1 is a diagram showing an example of the hardware configuration of a computer 100 used in the robot control system 1. In this example, the computer 100 includes a main body 110, a monitor 120, and an input device 130.

[0040] The main body 110 is a device having a circuit 160. The circuit 160 includes a processor 161, memory 162, storage 163, input / output ports 164, and a communication port 165. The number of each hardware component can be one or two or more. The memory 163 stores the programs used to configure the various functional modules of the main body 110. The memory 163 is a computer-readable recording medium such as a hard disk, non-volatile semiconductor memory, magnetic disk, or optical disk. The memory 162 temporarily stores programs loaded from the memory 163, calculation results of the processor 161, and the like. The processor 161 executes the programs in cooperation with the memory 162 to configure the various functional modules. The input / output ports 164 input and output electrical signals to and from the monitor 120 or input device 130 in accordance with instructions from the processor 161. The input / output ports 164 can also input and output electrical signals to and from other devices such as the robot controller 3 and the camera 4. The communication port 165 communicates data with other devices via the communication network N in accordance with instructions from the processor 161.

[0041] The monitor 120 is a device for displaying information output from the main body 110. For example, the monitor 120 is a device capable of displaying graphics such as a liquid crystal panel.

[0042] The input device 130 is a means for inputting information to the main body 110. Examples of the input device 130 include a keyboard, a mouse, and an operation interface of an operation controller.

[0043] The monitor 120 and the input device 130 may be integrated as a touch panel. For example, the main body 110, the monitor 120, and the input device 130 may be integrated as in a tablet computer.

[0044] Each functional module of the robot control system 1 is implemented by loading a robot control program into the processor 161 or the memory 162 and executing the program on the processor 161. The robot control program includes code for implementing each functional module of the robot control system 1. The processor 161 operates the input / output port 164 and the communication port 165 according to the robot control program, and reads and writes data from the memory 162 or the storage 163.

[0045] The robot control program may be provided after being recorded on a non-transitory recording medium such as a CD-ROM, DVD-ROM, or semiconductor memory, or provided as a data signal superimposed on a carrier wave via a communication network.

[0046] [Behavior Tree]

[0047] Reference Figure 3 Describes the behavior tree. Figure 32 is a diagram showing the behavior tree 200 stored in the storage unit 11 .

[0048] Figure 3 2 , a fallback node 21 , a sequence node 22 , a parallel node 23 , a fallback node with memory 24 , and a sequence node with memory 25 are shown as types of control nodes.

[0049] Fallback node 21 is used when only one of multiple child nodes needs to succeed. Fallback node 21 calls two or more child nodes sequentially, starting from the left. If any child node returns True or Running, fallback node 21 returns its return value to the parent node as the execution result of fallback node 21. The remaining child nodes are not called. If all child nodes return False, fallback node 21 returns False as the execution result of fallback node 21 to the parent node.

[0050] Sequence node 22 is used when all child nodes must succeed. Sequence node 22 calls two or more child nodes sequentially, starting from the left. If any child node returns False or Running, Sequence node 22 returns its return value to the parent node as the execution result of Sequence node 22. The remaining child nodes are not called. If all child nodes return True, Sequence node 22 returns True to the parent node as the execution result of Sequence node 22.

[0051] Parallel nodes 23 are used to execute all child nodes, essentially executing them in parallel or parallel. Parallel nodes 23 call all of their two or more child nodes sequentially, starting from the left. If all child nodes return True, parallel node 23 returns True to the parent node. If at least one child node returns False, parallel node 23 returns False to the parent node. Otherwise, it returns "Running" to the parent node.

[0052] The fallback node 24 with memory is an extension of the fallback node 21. The fallback node 24 with memory differs from the fallback node 21 in that, when the execution result in the previous loop is Running, the child nodes are called in sequence starting from the last child node called in the loop.

[0053] The sequence node with memory 25 is an extension of the sequence node 22. The difference between the sequence node with memory 25 and the sequence node 22 is that, when the execution result in the previous loop is Running, the child nodes are called in order from the last child node called in the loop.

[0054] Figure 32 shows a condition node 26 and an action node 27 as types of execution nodes.

[0055] The condition node 26 is used to check whether a specified condition is met. The condition node 26 executes the function corresponding to the condition. The condition node 26 returns either True, indicating that the condition is met, or False, indicating that the condition is not met, to the parent node.

[0056] Action node 27 is used to execute a task. Action node 27 executes the function corresponding to the task. Action node 27 returns one of the following to the parent node: True, indicating task success; False, indicating task failure; or Running, indicating that the task is in another state (e.g., executing).

[0057] The robot control system 1 manages parallel processing and parallel processing using at least parallel nodes. The following description focuses on the parallel nodes.

[0058] In behavior tree 200, root node 201 has a single child node, parallel node 202. Parallel node 202 has three child nodes, sequence nodes with memory 211, 231, and 251. Parallel node 202 is connected to subtree 210, which has sequence node with memory 211 as its vertex; subtree 230, which has sequence node with memory 231 as its vertex; and subtree 250, which has sequence node with memory 251 as its vertex.

[0059] Subtree 210 includes parallel node 212 located two levels below sequence node 211 with memory. Parallel node 212 is connected to a subtree with sequence node 213 as its vertex and a subtree with fallback node 214 as its vertex. Parallel node 215 located one level below fallback node 214 is connected to a subtree with sequence node 216 as its vertex and a subtree with sequence node 217 as its vertex.

[0060] Subtree 230 includes parallel nodes 232 located two levels below sequence node 231 with memory. Parallel node 232 is connected to a subtree with sequence node 233 as a vertex and a subtree with sequence node 234 as a vertex.

[0061] Behavior tree 200 includes four parallel nodes: 202, 212, 215, and 232. Parallel node 202 is the highest parallel node among these parallel nodes. Parallel nodes 212, 215, and 232 are lower parallel nodes located below the highest parallel node.

[0062] In one example, at least one of the multiple subtrees connected to the parallel node has: an occupying node, which is an action node requesting the parallel node to change the state of the common resource to an in-use state; and an executing node, which is an action node using the common resource. The tree management unit 12 calls the occupying node and the executing node in the subtree in that order. At least one of the multiple subtrees connected to the parallel node may also have a releasing node in addition to the occupying node and the executing node, which is an action node requesting the parallel node to restore the state of the common resource to a usable state. The tree management unit 12 calls the occupying node, the executing node, and the releasing node in the subtree in that order. Figure 3 In FIG, occupied nodes, executed nodes, and released nodes are represented as “P”, “A”, and “R”, respectively. For example, a subtree connected to parallel node 212 and having sequence node 213 as a vertex has occupied node 218, executed node 219, and released node 220.

[0063] In one example, the occupy node, execute node, and release node return values ​​to the parent node as follows. That is, the occupy node returns True if the public resource is available, otherwise, it returns Running. The execute node returns True if the task using the public resource succeeds, returns False if the task fails, and returns Running if the task is in another state (such as being executed). If the public resource is releasable, the release node returns True, otherwise it returns Running. By arranging these nodes in the order of occupy node, execute node, and release node, it is controlled whether to use public resources to execute tasks or wait for the execution of tasks until public resources can be used.

[0064] In one example, the storage unit 11 stores resource information in association with at least one parallel node. For example, the storage unit 11 may store resource information in association with the highest parallel node. Alternatively, the storage unit 11 may store resource information in association with the highest parallel node and the lower parallel nodes. Figure 3 In the example, resource information is represented by "M". In behavior tree 200, storage unit 11 stores resource information in association with parallel nodes 202, 212, 215, and 232. In this example, parallel nodes 202, 212, 215, and 232 correspond to resource information 291, 291, 293, and 294, respectively.

[0065] In one example, the resource information associated with a parallel node indicates the status of common resources available to multiple tasks corresponding to the multiple subtrees connected to the parallel node. Resource information 293 indicates the status of common resources available to the lower layers of parallel node 215. Resource information 292 indicates the status of common resources available to the lower layers of parallel node 212. Parallel node 215 is included in one of the multiple subtrees connected to parallel node 212, so resource information 292 includes resource information 293. Resource information 294 indicates the status of common resources available to the lower layers of parallel node 232. Parallel nodes 212 and 215 are included in subtree 210, and parallel node 232 is included in subtree 230, so resource information 291 includes resource information 292-294. Thus, the resource information associated with the highest-level parallel node can indicate the status of all common resources available under the management of the behavior tree.

[0066] Resource information can also be associated with control nodes other than parallel nodes. For example, resource information can be associated with at least one of fallback node 21, sequence node 22, fallback node with memory 24, and sequence node with memory 25. Resource information can also be associated with each control node in behavior tree 200.

[0067] [Status of public resources]

[0068] Figure 4 This is a state transition diagram illustrating an example of state changes for a common resource. This example shows two state transition patterns: weak occupation and strong occupation. In one example, the tree management unit 12 dynamically sets the state transitions for a common resource based on the occupation intensity associated with the execution node (i.e., task) requesting use of the common resource. Therefore, while multiple tasks are being executed based on a behavior tree, there may be common resources whose state transition patterns change between weak occupation and strong occupation.

[0069] When the state transition of a common resource is weak occupation, the tree management unit 12 changes the state of the common resource between the usable state, the in-use state, the temporarily released state, and the releasing state. That is, in weak occupation, the states include the usable state, the in-use state, the temporarily released state, and the releasing state. The usable state indicates that the common resource is not in use by any task and any task can begin using the common resource. The in-use state indicates that the common resource is currently in use by a task and cannot be used by other tasks. The temporarily released state is defined to avoid deadlock in the common resource. The releasing state is defined to release the common resource from the task currently occupying the common resource.

[0070] If a public resource is in the available state and a first task requests use of the public resource, the tree management unit 12 updates the state to "in use" in response to the request. The task control unit 13 then instructs the agent (robot 2) to execute the first task. If the public resource is in the available state, other tasks, including the second task, cannot use the public resource. Therefore, the task control unit 13 prevents the agent from executing the second task.

[0071] When the next loop is transitioned, that is, when the tree management unit 12 calls the root node again, the tree management unit 12 updates the state from the in-use state to the temporarily released state in response to the transition. As a result, any task including the first task and the second task may be able to use the common resource.

[0072] When the first task requests to use the temporarily released common resource, the tree management unit 12 updates the state to the in-use state in response to the request. As a result, the first task can continue to use the common resource.

[0073] On the other hand, when the second task requests to use the temporarily released common resource, the tree management unit 12 updates the state to the releasing state in response to the request. As a result, there is a possibility that the second task can use the common resource.

[0074] When the public resource's status is updated to the releasing state, the task control unit 13 instructs the agent (Robot 2) to execute a release batch process to release the public resource. This release batch process is pre-associated with the subtree corresponding to the first task in the behavior tree. For example, if the public resource is Robot 2, the release batch process could be one that causes Robot 2, currently holding a workpiece, to temporarily place the workpiece in a specified location. Upon completion of the release batch process, the tree management unit 12 updates the status to the usable state. As a result, any task can now use the public resource.

[0075] Thus, in weak occupation, the common resources being used by a task (the first task) may be preempted by other tasks (the second task) through temporary release and release batching.

[0076] If the state transition of a public resource is strong occupation, the tree management unit 12 changes the state of the public resource between the usable state and the in-use state. That is, in strong occupation, the state includes both the usable state and the in-use state. The usable state has the same meaning as weak occupation. The in-use state is similar to weak occupation in that the public resource is currently in use by a task and cannot be used by other tasks, but differs from weak occupation in that no temporary release or release is performed. In other words, in strong occupation, a public resource currently in use by a task cannot be preempted by other tasks.

[0077] If a task requests the use of a public resource while the public resource is in the available state, the tree management unit 12 updates the state to available in response to the request. The task control unit 13 then instructs the agent (robot 2) to execute the task. If the public resource is in use, other tasks cannot use the public resource, so the task control unit 13 prevents the agent from executing the other task.

[0078] When a task using a common resource ends, the tree management unit 12 updates the state to an available state in response to the end. As a result, any task can use the common resource.

[0079] [Release batch]

[0080] Figure 5 This is a diagram showing an example of a release batch. The diagram shows a subtree with a parallel node 301 as a vertex as part of a behavior tree. Parallel node 301 is connected to three subtrees Sa, Sb, and Sc. In this example, it is assumed that robots Ra and Rb are managed as both agents and public resources. Subtree Sa corresponds to task Ta, which includes action Ac of robot Ra and action Ad of robot Rb. Subtree Sa also corresponds to release batch α for releasing robot Ra, which is currently being used to execute task Ta. Subtree Sb corresponds to task Tb, which includes action Ap of robot Rb and action Aq of robot Ra. Subtree Sb also corresponds to release batch β for releasing robot Rb, which is currently being used to execute task Tb. Subtree Sc corresponds to task Tc, which includes action Ax. Subtree Sc does not include a release batch.

[0081] When the status of robot Ra is updated to the releasing state, the task control unit 13 instructs robot Ra to execute the release batch α. Robot Ra executes the actions Bc, Bd, and Be that comprise release batch α in the order they appear. In response to the completion of release batch α, the tree management unit 12 updates the status of robot Ra to the available state. As a result, it is possible that robot Ra can be used in task Tb.

[0082] When the status of robot Rb is updated to the releasing state, the task control unit 13 instructs robot Rb to execute the release batch β. Robot Rb executes the actions Bp, Bq, and Br that make up release batch β in the order they appear. Upon completion of release batch β, the tree management unit 12 updates the status of robot Rb to the usable state. This results in the possibility that robot Rb can be used for task Ta.

[0083] The resource information 302 indicates the status of each of the robots Ra and Rb as common resources. If the common resource is in use, the resource information 302 also indicates a release batch for releasing the common resource.

[0084] [Robot control method]

[0085] As an example of the robot control method disclosed in the present invention, refer to Figure 6 and Figure 7 The processing executed by the robot control system 1 will be described. Figure 6 1 is a flowchart showing an example of this process as a process flow S1. That is, the robot control system 1 executes the process flow S1. Figure 7 This is a flowchart showing an example of processing related to parallel nodes as part of the processing flow S1. Figure 6 as well as Figure 7 In the example, assume that the agent is robot 2.

[0086] As described above, the tree management unit 12 sequentially calls each node of the behavior tree in each loop. Figure 6 This loop is represented by a variable i. In step S11, the tree management unit 12 initially sets the variable i to 1.

[0087] In step S12, as part of the i-th loop, the robot control system 1 calls each node based on the behavior tree to control task execution. The tree management unit 12 calls each node sequentially, starting from the root node, based on the behavior tree stored in the storage unit 11. If the called node corresponds to a task, the task control unit 13 references the resource information stored in the storage unit 11 and, in collaboration with the tree management unit 12, controls task execution based on the status of shared resources.

[0088] Reference Figure 7 The processing associated with a particular parallel node is described as process flow S120. For convenience, this parallel node is referred to as the "target parallel node." Process flow S120 is part of step S12 and is executed for one or more parallel nodes within the behavior tree. As described above, parallel nodes connect to multiple subtrees corresponding to multiple tasks. Figure 7 Use the variable k to distinguish between subtrees.

[0089] In steps S1201 and S1202, the tree management unit 12 calls the first subtree connected to the target parallel node. The first subtree is the leftmost subtree in the next layer of the target parallel node.

[0090] In step S1203, the task control unit 13, in response to the subtree being called, references resource information and obtains the status of common resources used by the tasks corresponding to the subtree. In one example, the task control unit 13 references resource information associated with any parallel node within the behavior tree. For example, the task control unit 13 may also reference resource information associated with the target parallel node. If the target parallel node is a lower-level parallel node, the task control unit 13 may also reference resource information associated with the highest-level parallel node. Alternatively, the task control unit 13 may reference resource information associated with both the highest-level parallel node and the target parallel node (lower-level parallel node). Alternatively, in addition to the resource information for each of these two parallel nodes, the task control unit 13 may also reference resource information associated with other control nodes (e.g., other parallel nodes) located on the path from the target parallel node to the highest-level parallel node. When resource information is referenced for two or more control nodes, including the highest-level parallel node and the target parallel node (lower-level parallel node), the task control unit 13 obtains multiple statuses from the resource information of these nodes.

[0091] In step S1204, the task control unit 13 controls the execution of tasks based on the state of the common resource in cooperation with the tree management unit 12. Alternatively, the task control unit 13 controls the execution of the release batch based on the state of the common resource. Regarding these controls, the tree management unit 12 updates its state as needed.

[0092] (When in usable condition)

[0093] If the status is usable, the task control unit 13 may cause the robot 2 (agent) to execute the task using the common resource. If multiple statuses are obtained, the task control unit 13 may cause the robot 2 to execute the task in response to each status being usable.

[0094] In response to the execution of the task, the tree management unit 12 updates the state to the in-use state. When multiple states are obtained, the tree management unit 12 updates each state to the in-use state. In this case, the state of the common resource is updated to the in-use state in the resource information of each of two or more control nodes, including at least the highest parallel node and the lower parallel node. In this way, when the task control unit 13 instructs the robot 2 to execute a task (the first task), the tree management unit 12 saves the resource information indicating that the task (the first task) is using the common resource and that other tasks (the second task) cannot use the common resource in the storage unit 11. The tree management unit 12 also saves historical information indicating that the task used the common resource in the i-th cycle in the storage unit 11.

[0095] The task control unit 13 generates a path for executing the task through path planning and outputs a command signal indicating the path to the robot controller 3. The robot controller 3 controls the robot 2 according to the command signal. The robot 2 moves along the path to execute the task.

[0096] (When in use)

[0097] If the status is "in use," the task control unit 13 prevents the robot 2 from executing the task. Specifically, the task control unit 13 does not generate or output an instruction signal. If multiple statuses are obtained, the task control unit 13 prevents the robot 2 from executing the task in response to at least one status being "in use." The tree management unit 12 stores historical information in the storage unit 11 indicating that the task did not use common resources in the i-th cycle.

[0098] (In the case of temporary release)

[0099] When the state is temporarily released, the task control unit 13 executes processing according to the task corresponding to the called subtree, which is either the first task that used the common resource before being temporarily released or the second task that could not use the common resource before being temporarily released.

[0100] If the first task requires the use of public resources, the task control unit 13 may cause the robot 2 (agent) to execute the first task using the public resources. If multiple states are acquired, the task control unit 13 may cause the robot 2 to execute the first task in response to each state being temporarily released.

[0101] In response to the execution of the first task, the tree management unit 12 updates the state to the in-use state. When multiple states are obtained, the tree management unit 12 updates each state to the in-use state. In this case, in the resource information of each of two or more control nodes including at least the highest parallel node and the lower parallel node, the state of the common resource is updated to the in-use state. In this way, when the task control unit 13 allows the robot 2 to perform the first task, the tree management unit 12 saves the resource information indicating that the first task is using the common resource and other tasks (second tasks) cannot use the common resource in the storage unit 11. That is, the tree management unit 12 updates the state of the common resource to the in-use state so that the first task can continue to use the common resource. The tree management unit 12 also saves the historical information indicating that the first task used the common resource in the i-th cycle in the storage unit 11.

[0102] The task control unit 13 generates a path for executing the first task through path planning and outputs a command signal indicating the path to the robot controller 3. The robot controller 3 controls the robot 2 according to the command signal. The robot 2 moves along the path to execute the first task.

[0103] If the second task requests the use of a common resource, the tree management unit 12 updates the state to the releasing state. If multiple states are obtained, the tree management unit 12 updates each state to the releasing state. In this case, the common resource state is updated to the releasing state in the resource information of at least two control nodes, including at least the highest parallel node and the lower parallel nodes.

[0104] The task control unit 13 instructs the robot 2 not to execute either the first task or the second task. That is, the task control unit 13 does not generate or output an instruction signal. The tree management unit 12 stores in the storage unit 11 historical information indicating that the first task is scheduled to release the common resource in the i-th cycle.

[0105] (Released state)

[0106] If the status is "releasing," the task control unit 13 executes processing in response to the fact that the task corresponding to the called subtree is the first task that used the common resource before it was temporarily released. The task control unit 13 instructs the robot 2 (agent) to execute a batch release process to release the common resource used in the first cycle. If multiple statuses are obtained, the task control unit 13 may also instruct the robot 2 to execute the batch release process in response to each status being "releasing."

[0107] On the other hand, if the task corresponding to the called subtree is different from the first task, task control unit 13 prevents robot 2 from executing the different task. That is, task control unit 13 does not generate or output an instruction signal. Tree management unit 12 stores historical information in storage unit 11 indicating that other tasks did not use common resources in the i-th cycle.

[0108] When executing a batch release, the tree management unit 12 updates the status to usable. If multiple statuses are obtained, the tree management unit 12 updates each status to usable. In this case, the status of the common resource is updated to usable in the resource information of at least two control nodes, including at least the highest parallel node and the lower parallel nodes. The tree management unit 12 also stores historical information in the storage unit 11 indicating that the first task released the common resource in the i-th cycle.

[0109] The task control unit 13 generates a path for executing the release batch process through path planning and outputs a command signal indicating the path to the robot controller 3. The robot controller 3 controls the robot 2 according to the command signal. The robot 2 moves along the path to execute the release batch process.

[0110] In step S1205, the tree management unit 12 determines whether all subtrees connected to the target parallel node have been called. If all subtrees have been called (yes in step S1205), the tree management unit 12 ends processing for the target parallel node and executes processing related to the next node based on the behavior tree. If there are uncalled subtrees (no in step S1205), the process proceeds to step S1206, where the tree management unit 12 selects the (k+1)th subtree, the next subtree. The process then returns to step S1202, and the tree management unit 12 calls the subtree. The robot control system 1 then executes the processing from step S1203 onwards for that subtree.

[0111] return Figure 6 In step S13, the tree management unit 12 determines whether to terminate the entire process managed by the behavior tree. For example, the tree management unit 12 may terminate the process in response to the completion of multiple tasks to be executed or the arrival of the end time of operation of the agent robot 2 (Yes in step S13). If the process is not terminated (No in step S13), the process proceeds to step S14. In step S14, the tree management unit 12 increments the variable i indicating the loop by 1.

[0112] In step S15, the tree management unit 12 updates the weakly occupied, in-use state to a temporarily released state. The tree management unit 12 accesses the storage unit 11 and references the resource information associated with at least one control node in the behavior tree. Next, for each piece of resource information, the tree management unit 12 updates the state of the public resource in the weakly occupied, in-use state to a temporarily released state.

[0113] In step S16, the tree management unit 12 changes the order of calling subtrees for each parallel node as needed. The tree management unit 12 changes the order of calling multiple subtrees connected to each parallel node based on the historical information corresponding to the parallel node. For example, it is also possible that when the task corresponding to the subtree located at the first position relative to the parallel node does not use common resources continuously in at least two cycles, the tree management unit 12 moves the subtree to a position after the second position. Alternatively, it is also possible that when the task corresponding to the subtree located at the first position relative to the parallel node releases the common resources used in a certain cycle (the first cycle) in the next cycle (the second cycle), the tree management unit 12 moves the subtree located at the first position to a position after the second position.

[0114] Reference Figure 8 , explaining the change in the order of calling subtrees. Figure 8 This diagram illustrates an example of this change. In this example, parallel node 401 is connected to subtrees 411, 412, and 413. Subtree 411 corresponds to the task implemented by the combination of actions A1, A2, and A3. Subtree 412 corresponds to the task implemented by the combination of actions A4, A5, and A6. Subtree 413 corresponds to the task implemented by action A7.

[0115] In a certain cycle, suppose subtrees 411, 412, and 413 are arranged in this order. That is, subtree 411 is the first subtree, subtree 412 is the second subtree, and subtree 413 is the last subtree. Figure 8 In the example, tree management unit 12 moves the subtree at the first position to the last position. As a result, when tree management unit 12 subsequently calls parallel node 401, it calls three subtrees in the order of subtrees 412, 413, and 411. Tree management unit 12 can also move the subtree at the first position between any two other subtrees. For example, it can move subtree 411 between subtrees 412 and 413.

[0116] return Figure 6 After step S16, the process returns to step S12. In the repeated step S12, as part of the next cycle, the robot control system 1 again calls each node based on the behavior tree to control the execution of the task. Then, in step S13, the tree management unit 12 again determines whether to terminate the entire process.

[0117] Reference Figure 6 and Figure 7 As described above, in each loop, the tree management unit 12 sequentially calls each of the multiple subtrees, starting from the target parallel node, based on the behavior tree. The task control unit 13 executes the task corresponding to the called subtree. For each of the multiple subtrees, the task control unit 13 refers to the resource information when the subtree is called. If the public resource used by the task corresponding to the subtree is in a usable state, the task control unit 13 uses the public resource and has the robot 2 (agent) execute the task. The tree management unit 12 updates the state of the public resource indicated by the resource information from the in-use state to the temporarily released state. After this update, the multiple subtrees are called from the parallel node.

[0118] [Modification]

[0119] The various examples of the present disclosure have been described in detail above. However, the technical matters of the present disclosure are not limited to the above examples. The technical matters of the present disclosure can be modified in various ways without departing from the scope of the present disclosure.

[0120] The hardware structure of the system is not limited to implementing each functional module through program execution. For example, at least a portion of the functional module group may be composed of logic circuits dedicated to that function, or may be composed of an ASIC (Application Specific Integrated Circuit) that integrates such logic circuits.

[0121] The processing of the method executed by at least one processor is not limited to the above examples. For example, some of the above steps or processes may be omitted, or the steps may be performed in a different order. Furthermore, any two or more of the above steps may be combined, or some steps may be modified or deleted. Alternatively, other steps may be performed in addition to the above steps.

[0122] When comparing the magnitude of two numerical values ​​in a computer system or within a computer, either of the two criteria of "above" and "greater than" or "below" and "less than" may be used.

[0123] [Note]

[0124] As is apparent from the various examples described above, the present disclosure includes the following aspects.

[0125] (Note 1)

[0126] A robot control system comprising:

[0127] a storage unit configured to store a behavior tree and resource information, wherein the behavior tree includes a plurality of subtrees corresponding to a plurality of tasks and a parallel node serving as a parent node connected to the plurality of subtrees, and the resource information includes a state of a common resource used by the plurality of tasks, the common resource including a robot, the state including at least an in-use state and an available state;

[0128] a tree management unit that, in each loop, sequentially calls each of the plurality of subtrees starting from the parallel node based on the behavior tree; and

[0129] a task control unit that executes the task corresponding to the called subtree,

[0130] The task control unit refers to the resource information for each subtree of the plurality of subtrees when the subtree is called, and uses the common resource to execute the task when the state of the common resource used by the task corresponding to the subtree is the usable state.

[0131] (Note 2)

[0132] According to the robot control system described in Note 1, the storage unit stores the resource information in association with the parallel node, and the task control unit refers to the resource information associated with the parallel node for each subtree among the multiple subtrees when the subtree is called.

[0133] (Note 3)

[0134] According to the robot control system described in Note 2, wherein, as a plurality of parallel nodes, the behavior tree includes a top-level parallel node located at the top and lower-level parallel nodes located below the top-level parallel node among the plurality of parallel nodes, the storage unit stores the resource information in association with the top-level parallel node, and the task control unit refers to the resource information associated with the top-level parallel node for each subtree among the plurality of subtrees connected to the lower-level parallel node when the subtree is called.

[0135] (Note 4)

[0136] According to the robot control system described in Note 3, the storage unit stores the resource information in association with at least the top parallel node and the lower parallel node, respectively, and the task control unit refers to the resource information associated with the top parallel node and the resource information associated with the lower parallel node for each subtree of the multiple subtrees connected to the lower parallel node when the subtree is called, and executes the task when the state of the common resource used by the task corresponding to the subtree is the usable state, at least in both the resource information associated with the top parallel node and the resource information associated with the lower parallel node.

[0137] (Note 5)

[0138] A robot control system according to any one of Notes 1 to 4, wherein, when the task control unit executes the task, the tree management unit saves the resource information indicating that the task is using the common resource and other tasks cannot use the common resource in the storage unit.

[0139] (Note 6)

[0140] A robot control system according to Note 5, wherein the respective loops include a first loop and a second loop following the first loop, the plurality of tasks include a first task and a second task, and in the first loop, the tree management unit saves the resource information indicating that the first task is using the common resource and the second task cannot use the common resource, and in the case of transferring to the second loop, the tree management unit updates the state of the common resource represented by the saved resource information to a temporarily released state to avoid deadlock of the common resource, and after updating the resource information, calls the plurality of subtrees starting from the parallel node.

[0141] (Note 7)

[0142] According to the robot control system described in Note 6, wherein, in the second loop, when the state of the common resource is the temporarily released state, the tree management unit updates the state of the common resource to the in-use state when the first task requests to use the common resource so that the first task can continue to use the common resource, and when the second task requests to use the common resource, updates the state of the common resource to the releasing state to release the common resource from the first task.

[0143] (Note 8)

[0144] According to the robot control system described in Note 7, the subtree corresponding to the first task also corresponds to a release batch for releasing the common resources used in the first loop, and the task control unit executes the release batch when the state of the common resources is updated to the releasing state.

[0145] (Note 9)

[0146] A robot control system according to any one of Notes 5 to 8, wherein at least one of the multiple subtrees has: an occupied node, which requests the parallel node to change the state of the common resource to the in-use state; and an execution node, which uses the common resource, and the tree management unit calls the at least one subtree among the multiple subtrees in the order of the occupied node and the execution node.

[0147] (Note 10)

[0148] According to the robot control system described in Note 9, at least one of the multiple subtrees also has a release node, and the release node requests the parallel node to return the state of the common resource to the usable state. The tree management unit calls at least one of the multiple subtrees in the order of the occupied node, the execution node and the release node.

[0149] (Note 11)

[0150] A robot control system according to any one of Notes 1 to 10, wherein the storage unit further stores historical information related to the execution of at least one of the multiple tasks, and the tree management unit changes the order of calling the multiple subtrees based on the historical information.

[0151] (Note 12)

[0152] According to the robot control system described in Note 11, when the task corresponding to the subtree located at the first position relative to the parallel node has not used the common resource continuously for at least 2 cycles, the tree management unit moves the subtree located at the first position to a position after the second position.

[0153] (Note 13)

[0154] A robot control system according to Note 11 or 12, wherein each loop includes a first loop and a second loop following the first loop, and when the task corresponding to the subtree located at the first position relative to the parallel node releases the common resources used in the first loop in the second loop, the tree management unit moves the subtree located at the first position to a position after the second position.

[0155] (Note 14)

[0156] A robot control method is executed by a robot control system having at least one processor, wherein the robot control method includes the following steps: a step of storing a behavior tree and resource information in a storage unit, the behavior tree including a plurality of subtrees corresponding to a plurality of tasks and a parallel node as a parent node connected to the plurality of subtrees, the resource information including the status of a common resource used by the plurality of tasks, the common resource including a robot, the status including at least an in-use state and a usable state; a step of sequentially calling each of the plurality of subtrees from the parallel node based on the behavior tree in each loop; and a step of executing the task corresponding to the called subtree, wherein in the step of having the robot execute the task, for each of the plurality of subtrees, the resource information is referred to when the subtree is called, and when the status of the common resource used by the task corresponding to the subtree is the usable state, the common resource is used to execute the task.

[0157] (Note 15)

[0158] A robot control program causes a computer to execute the following steps: a step of storing a behavior tree and resource information in a storage unit, the behavior tree including a plurality of subtrees corresponding to a plurality of tasks and a parallel node as a parent node connected to the plurality of subtrees, the resource information including the status of a common resource used by the plurality of tasks, the common resource including a robot, the status including at least an in-use state and a usable state; a step of sequentially calling each of the plurality of subtrees from the parallel node based on the behavior tree in each loop; and a step of executing the task corresponding to the called subtree, wherein in the step of causing the robot to execute the task, for each of the plurality of subtrees, the resource information is referred to when the subtree is called, and when the status of the common resource used by the task corresponding to the subtree is the usable state, the common resource is used to execute the task.

[0159] According to Supplementary Notes 1, 14, and 15, both the behavior trees representing multiple tasks using a common resource and the state of the common resource are comprehensively managed, and each task is executed based on the state. This mechanism enables efficient execution of multiple tasks.

[0160] According to Supplementary Note 2, since resource information is associated with parallel nodes, the correspondence between common resources and multiple tasks becomes clearer, thereby making it easier to implement processing that refers to the status of common resources corresponding to tasks.

[0161] According to Supplementary Note 3, since the resource information is associated with the highest-level parallel node, it is possible to centrally manage one or more common resources that may be used.

[0162] According to Supplementary Note 4, resource information is associated with at least both the highest parallel node and the lower parallel nodes. Therefore, even when the behavior tree is large or complex, the state of common resources can be managed more simply.

[0163] According to Supplementary Note 5, when executing a task using a public resource, resource information is saved so that the task occupies the public resource. By managing resource information in this way, the occupation of the public resource by the task can be managed more reliably.

[0164] According to Supplementary Note 6, when transitioning to the next cycle, the state of the shared resource occupied by the first task is updated to a temporarily released state, thereby providing not only the first task but also the second task (i.e., other tasks) with the possibility of using the shared resource. This mechanism enables efficient execution of multiple tasks as a whole.

[0165] According to Supplementary Note 7, the state of the common resource is updated in accordance with the task that requests to use the common resource. This mechanism enables efficient execution of a plurality of tasks as a whole in accordance with the situation in which the task is called.

[0166] According to Supplementary Note 8, when a task other than the first task using the common resource wants to use the common resource, the common resource is released through the release batch process. This mechanism allows other tasks to use the common resource more reliably.

[0167] According to Note 9, the request and use of public resources are explicitly represented in the behavior tree by an occupation node and an execution node, respectively. This makes it easier to design or manage public resources.

[0168] According to Supplementary Note 10, the release of a public resource is explicitly represented by a release node in a behavior tree, thereby making it easier to design or manage public resources.

[0169] According to Supplementary Note 11, the calling order of subtrees is changed based on the execution history of tasks, so that resource starvation can be eliminated or suppressed more reliably.

[0170] According to Supplementary Note 12, since the order of the subtrees corresponding to the tasks that do not use the common resources descends, other tasks can first request to use the common resources. This mechanism allows multiple tasks to be executed more efficiently.

[0171] According to Supplementary Note 13, the order of the subtrees corresponding to the tasks that released the common resources is lowered, so other tasks can request to use the common resources first. This mechanism allows multiple tasks to be executed more efficiently.

[0172] Description of labels

[0173] 1 robot control system; 2 robot; 3 robot controller; 4 camera; 11 storage unit; 12 tree management unit; 13 task control unit; 200 behavior tree.

Claims

1. A robot control system comprising: a storage unit configured to store a behavior tree and resource information, wherein the behavior tree includes a plurality of subtrees corresponding to a plurality of tasks and a parallel node serving as a parent node connected to the plurality of subtrees, and the resource information includes a state of a common resource used by the plurality of tasks, the common resource including a robot, the state including at least an in-use state and an available state; a tree management unit, which, in each loop, sequentially calls each of the plurality of subtrees starting from the parallel node based on the behavior tree; as well as a task control unit that executes the task corresponding to the called subtree, The task control unit refers to the resource information for each subtree of the plurality of subtrees when the subtree is called, and uses the common resource to execute the task when the state of the common resource used by the task corresponding to the subtree is the usable state.

2. The robot control system according to claim 1, wherein: The storage unit stores the resource information in association with the parallel node. The task control unit refers to the resource information associated with the parallel node for each of the plurality of subtrees when the subtree is called.

3. The robot control system according to claim 2, wherein: As the plurality of parallel nodes, the behavior tree includes a top-level parallel node located at the top of the plurality of parallel nodes and lower-level parallel nodes located below the top-level parallel node. The storage unit stores the resource information in association with the highest-level parallel node. The task control unit refers to the resource information associated with the highest parallel node for each of the plurality of subtrees connected to the lower parallel node when the subtree is called.

4. The robot control system according to claim 3, wherein: The storage unit stores the resource information in association with at least the highest parallel node and the lower parallel nodes, The task control unit controls each of the plurality of subtrees connected to the lower parallel node. When the subtree is called, the resource information associated with the highest parallel node and the resource information associated with the lower parallel nodes are referred to. The task is executed when the state of the common resource used by the task corresponding to the subtree is the usable state in at least both of the resource information associated with the highest parallel node and the resource information associated with the lower parallel node.

5. The robot control system according to any one of claims 1 to 4, wherein: When the task control section executes the task, the tree management section stores the resource information indicating that the task is using the common resource and other tasks cannot use the common resource in the storage section.

6. The robot control system according to claim 5, wherein: Each cycle includes a first cycle and a second cycle following the first cycle, The plurality of tasks include a first task and a second task, In the first loop, the tree management unit stores the resource information indicating that the first task is using the common resource and the second task cannot use the common resource. When transitioning to the second loop, the tree management unit updates the state of the common resource represented by the saved resource information to a temporarily released state to avoid deadlock of the common resource, and after updating the resource information, calls the multiple subtrees from the parallel node.

7. The robot control system according to claim 6, wherein: In the second cycle, when the state of the common resource is the temporarily released state, the tree management unit When the first task requests to use the public resource, updating the state of the public resource to an in-use state so that the first task can continue to use the public resource; When the second task requests to use the common resource, the state of the common resource is updated to a releasing state in which the common resource is released from the first task.

8. The robot control system according to claim 7, wherein: The subtree corresponding to the first task also corresponds to a release batch for releasing the common resources used in the first loop. The task control unit executes the release batch process when the state of the common resource is updated to the releasing state.

9. The robot control system according to claim 5, wherein: At least one subtree among the plurality of subtrees has: an occupying node requesting the parallel node to change the state of the common resource to the in-use state; and an execution node that uses the common resource, The tree management unit calls at least one of the plurality of subtrees in the order of the occupied node and the execution node.

10. The robot control system according to claim 9, wherein: At least one of the plurality of subtrees further includes a release node, the release node requesting the parallel node to return the state of the common resource to the usable state, The tree management unit calls at least one of the plurality of subtrees in the order of the occupied node, the executed node, and the released node.

11. The robot control system according to any one of claims 1 to 4, wherein: The storage unit further stores historical information related to the execution of at least one task among the plurality of tasks. The tree management unit changes the order of calling the plurality of subtrees based on the history information.

12. The robot control system according to claim 11, wherein: When the task corresponding to the subtree located at the first position relative to the parallel node does not use the common resource continuously for at least two cycles, the tree management unit moves the subtree located at the first position to a position subsequent to the second position.

13. The robot control system according to claim 11, wherein: Each cycle includes a first cycle and a second cycle following the first cycle, When the task corresponding to the subtree located at the first position relative to the parallel node releases the common resource used in the first cycle in the second cycle, the tree management unit moves the subtree located at the first position to a position after the second position.

14. A robot control method, which is executed by a robot control system having at least one processor, wherein: The robot control method comprises the following steps: The step of storing a behavior tree and resource information in a storage unit, wherein the behavior tree includes a plurality of subtrees corresponding to a plurality of tasks and a parallel node serving as a parent node connected to the plurality of subtrees, and the resource information includes the status of a common resource used by the plurality of tasks, the common resource including a robot, and the status including at least an in-use state and an available state; In each loop, a step of sequentially calling each subtree of the plurality of subtrees starting from the parallel node based on the behavior tree; as well as executing the task corresponding to the subtree of the call, In the step of having the robot perform the task, for each subtree among the multiple subtrees, the resource information is referred to when the subtree is called, and when the state of the common resource used by the task corresponding to the subtree is the usable state, the common resource is used to perform the task.

15. A robot control program, which causes a computer to execute the following steps: The step of storing a behavior tree and resource information in a storage unit, wherein the behavior tree includes a plurality of subtrees corresponding to a plurality of tasks and a parallel node serving as a parent node connected to the plurality of subtrees, and the resource information includes the status of a common resource used by the plurality of tasks, the common resource including a robot, and the status including at least an in-use state and an available state; In each loop, a step of sequentially calling each subtree of the plurality of subtrees starting from the parallel node based on the behavior tree; as well as executing the task corresponding to the subtree of the call, In the step of having the robot perform the task, for each subtree among the multiple subtrees, the resource information is referred to when the subtree is called, and when the state of the common resource used by the task corresponding to the subtree is the usable state, the common resource is used to perform the task.

Images