Synchronized multi-agent visual planning method and tool based on hierarchical, concurrent and conditional agent action flow diagrams

Abstract

The invention relates to a visual planning method and tool with a focus on enabling collective, synchronised planning and re-planning for distributed or hierarchical organisations composed of different types of agents, including autonomous systems, artificial intelligence, humans or simulations thereof. The invention particularly relates to a visual planning method and tool that allows creating agent actions / missions / tasks in the form of several conditional, concurrent and synchronised flow diagrams (plans) consisting of lower-level sub-agent action (sub-action), function and condition blocks and similarly using these compound agent actions as sub-actions in higher level, concurrent flow diagrams to define actions of a superior agent, which creates an action hierarchy based on the ontological hierarchy between the agents forming an organisation / compound agent, providing flexibility that allows defining all kinds of actions of any agent / organisation, while preventing over- or under-loading of the sub-agents (resources) during the plan construction, increasing planning efficiency. A sub-method / tool allowing the definition of various function and condition contents complements this ability to define all kinds of agent actions.

Description

[0001] SYNCHRONIZED MULTI-AGENT VISUAL PLANNING METHOD AND TOOL BASED ON HIERARCHICAL, CONCURRENT AND CONDITIONAL AGENT ACTION FLOW DIAGRAMS

[0002] Technical Field

[0003] The invention relates to a visual planning method and tool with a focus on enabling collective, synchronised planning and re-planning for distributed or hierarchical organisations composed of different types of agents, including autonomous systems, artificial intelligence, humans or simulations thereof.

[0004] The invention particularly relates to a visual planning method and tool that allows creating agent actions / missions / tasks in the form of several conditional, concurrent and synchronised flow diagrams (plans) consisting of lower-level sub-agent action (sub-action), function and condition blocks and similarly using these compound agent actions as sub-actions in higher level, concurrent flow diagrams to define actions of a superior agent, which creates an action hierarchy based on the ontological hierarchy between the agents forming an organisation / compound agent, providing flexibility that allows defining all kinds of actions of any agent / organisation, while preventing over- or under-loading of the sub-agents (resources) during the plan construction, increasing planning efficiency. A sub-method / tool allowing the definition of various function and condition contents complements this ability to define all kinds of agent actions.

[0005] State of the Art

[0006] One of the key requirements of modern end-user products (such as robots, simulation systems or video games) is re-programmability. This flexibility requirement, which was previously satisfied through high-level programming, has led to the development of visual programming languages over time, with the emerging necessity that reprogramming should be done by the end user instead of the modeller. Basically, high-level programming is achieved through using reusable actions to construct different plans. The early forms used for visual programming are behaviour trees (BTs) and finite state machines (FSMs), which were developed for modellers rather than the users.

[0007] For example, the Unreal Game Engine developed by Epic Games provides game developers with high-level visual programming opportunity via the Blueprints Visual Scripting System, where general purpose coding can be done visually and lower-level actions can be triggered by behaviour trees drawn inside the so called Blueprint pages. However, BTs are basically reactive behaviour definitions and are not real plans. A hierarchy can be created in accordance with the hierarchical task networking (HTN) approach, where certain, especially repeated parts of a compound action are wrapped up as sub-actions. However, this is not in accordance with the real, ontological hierarchy between the objects that perform the actions (instead, there is an action-oriented, conceptual hierarchy in question).

[0008] The new techniques of visual programming make use of sequence and flow diagrams, which are already used in areas such as business process modelling, scenario preparation and software design. (Besides, plans as outputs of automatic planning processes are almost always produced as action sequences.) In particular, flow diagrams -which are frequently encountered in describing business processes and military operations- allow the preparation of concurrent, synchronised and conditional plans. Flow diagrams can be organised in a hierarchical structure reflecting the hierarchy in organisations and their work plans. Nevertheless, this hierarchy is a hierarchy between plans and has no direct relation to the object hierarchy / ontology. This situation can cause the focus to be on the plan / action and especially in the preparation of multi-agent plans, some agents can be overlooked or overloaded on the contrary.

[0009] Even in swim-lane flow diagrams within the scope of the UML software design language, in the state of the art, when an action in one lane (associated with an agent) ends, an action in another lane (belonging to another agent) is triggered and the next action of the agent performing the previous action is not necessarily determined. Since there is no warning for completing this missing part of the plan, the agent can easily be ignored. This approach is more suitable for processes and organisations where agents perform sequential functions. (Again, sequence diagrams within the UML language similarly focus on defining sequential operations.) However, real agents generally act concurrently in joint operations, and the correct modelling of such operations requires the ability to define complete and concurrent plans. Sequential action planning for concurrent work may be sufficient at the conceptual level. However, in order to prepare a plan that will actually and automatically be executed , an approach that makes it easier to consider all of the agents during plan construction (thus preventing possible errors caused by ignored units) is needed.

[0010] The method and tool described in the patent numbered US2022244688A1 ("Method of Developing and Using a Toll, and the Resulting Tool, for Automating Complex Processes") in the state of the art, were developed for the automation of complex processes and easy updating. The invention aims to meet the need for correctly defining all possible flows before the algorithm is delivered to the software developer who will perform the coding for providing autonomy in real processes. It is stated that this need is especially due to the distance of the coder from real processes and the importance of developing a common (human readable) language for the software developer and the field expert -who defines the process- is emphasised. The proposed solution is inspired by the visualisation technique of plans known in the literature. The existence of tools that visualise workflows and actors together on flow diagrams is stated. Flows expressed in visual language can be directly converted to code without requiring any programming knowledge, including scripting. Thus, since programming and reprogramming also become easier, the need to predict all possible flows and exceptions at the beginning can be eliminated.

[0011] The method and tool within the scope of the invention allow plans to be defined as sequences consisting of steps performed by different actors. The outputs of previous actions / steps performed by some actors can be the inputs of the following steps or their success can be the preconditions for the latters. (Thus, just a sequential process definition can be realised.) Visualisation is performed through program tables -defining workflows- at different hierarchical levels. The tables contain the preconditions and outputs for each action together with the next action data. Thus the process plan can be executed by agents reading the table. The lower-level plans referenced in a higher-level flow plan are executed in an order appropriate to the plan. It has been emphasised that the tool and method can be used to automate any process in any industry. In the state of the art, there is no solution that allows the programming of individual agents or hierarchical agent organisations by the end-user in a way that they can carry out all their potential behaviours, nor a solution that allows the preparation of complete plans that can be directly loaded to agents / agent groups and executed right after). Providing such a flexibility can be achieved with the method we propose, which allows the visual preparation of concurrent and synchronous action plans of all sub-agents of any agent together.

[0012] A similar process programming method and tool is also mentioned in the patent numbered CN107073710A and titled "Software Interface for Authoring Robotic Manufacturing Process" in the state of the art. However, the focus here is on programming and reprogramming robotic systems within a factory. This time, visual programming is provided by connecting the nodes representing robot actions to each other in a more similar way to flow chart preparation. Again, the actions (nodes) are connected to each other according to the execution order and thus are executed sequentially. Only by multiple triggering after any action, different robots can perform concurrent activities on certain parts of the plans. Again, the ease of programming and reprogramming provided by the method is emphasised.

[0013] A similar approach and tool focusing on the programming of factory robots -but offered as a cloud-based simulation service- is defined in the patent numbered US2021316459A1 and named "Cloud Based Computer-Implemented System and Method for Computer-Assisted Planning and Simulation of Robot Motions in Construction" in the state of the art. Via this tool, which has an inherent physics engine, and offers visual programming opportunities, complex production processes can be planned and tested in a virtual environment before real implementation. Again, planning / re-planning is carried out through the construction of behaviour sequences for a large number of robots. Apart from this, no novel visual planning technique is proposed.

[0014] In the patent numbered US2023214191A1 and named "Workflow for Computer Game Development" in the state of the art, a visual programming method and tool that will facilitate communication between non-programmer video game designers and software engineers (for going beyond the currently used verbal communication) has been defined / proposed with a focus on the video game industry. Although the sector is somewhat different, the visual tool again benefits from action / state nodes that are connected to each other, the completion of one of which sometimes can trigger multiple others. The main innovation here is that the visual program and the corresponding script program are displayed together on the same tool. This mainly aims at facilitating communication between the software engineer and the game developer who prefer different (visual or verbal) representations. Both bots and real players can be supported as actors. Rather than a plan-oriented structure, the focus is on diagrams that express event-based, reactive behaviour, which is more convenient for computer games.

[0015] The patent numbered WO2023062362A1 and named "Visual Programming Environment for Developing Interactive Media Programs" in the state of the art, comprises the idea of developing a similar visual, executable software generating tool that enables both game programming and the preparation of interactive movie scripts. Again, visual planning is done through the interface via creating nodes and connecting them with transition lines. Nodes can contain actions and conditions that can be automatically converted into code. Thus, it is stated that at the end of each node, the plan can be operated towards different directions according to conditions fired depending on the values of some, related variables (which means that writing conditional plans is supported). It is emphasised that the process is divided into parts as nodes and the transitions between them allow for multi-option / multi- path plots. Nevertheless, it is not possible to create different plans that work concurrently, considering different actors. As in other patents examined, a single flow chart is created for all actors.

[0016] As a result, the need for a new efficient, user-friendly, synchronised visual planning and replanning method to solve the above-mentioned problems in the state of the art and the inadequacy of existing solutions have made it necessary to make a development in the relevant technical field.

[0017] The Aim of the Invention The present invention is a method related to a visual planning technique developed to eliminate the above-mentioned disadvantages and bring new advantages to the relevant technical field, to meet the need for user-friendly multi-autonomous agent plan creation and reuse of the created plans in different scenarios that arise in training and analysis simulators.

[0018] The most important aim of the invention is to meet the need for autonomous operation planning of hierarchical multi-agent organisations in a virtual environment, in a reprogrammable and synchronised manner, with a user-friendly technique. It can also be used for planning the collective behaviour of real organisations consisting of humans, organisations without humans or a mixture thereof, and for updating and implementing these plans. The method and tool within the scope of the invention mainly focuses on visual planning via hierarchical flow diagrams, but the innovation brought is in the way complex behaviours are sorted out. Behaviour plans are produced and executed by writing and operating separate conditional plans for each agent that can be synchronised with each other. This concurrent plan structure is hierarchically repeated for each level. Thus, for example, any action that can be performed by a platform through synchronously using its multiple actuators can easily be defined, and the same structure allows for writing any scenario that can be defined as concurrent action flows for scenario platforms at a higher level. The same approach can also allow for the insertion of an agent group layer between scenarios and platforms. In this way, all actions that organisations can perform in real life can be correctly / completely planned / replanned by non-programmer users and loaded into virtual or real autonomous systems to be executed automatically. The main advantage of the invention -in addition to concurrent planning capability- is that a full flexibility can be achieved , through correct organisation and movement analysis and reflecting this analysis into visual programming. A sub-tool and method developed to define the functions and conditions, which are some of the keystones of the plans, providing high genericity are also within the scope of the invention.

[0019] The structural and characteristic properties and all advantages of the invention will be more clearly understood with the help of the detailed description given. Therefore, the assessment should be made by taking this detailed description also into account. Reference Numbers

[0020] 10. Superior agent action plan

[0021] 20. Agent action plan

[0022] 30. Sub-agent action plan: Action 3

[0023] 40. Sub-agent 1 action plan

[0024] 50. Sub-agent 2 action plan

[0025] 60. Function / Condition name

[0026] 70. Function / Condition writing virtual keyboard

[0027] 80. Condition / function text window

[0028] 90. Save key

[0029] 100. Number, operator, ready (hard-coded) function and time variable keys

[0030] 110. Keys that provide access to inputs, events and agent parameters

[0031] 120. Server

[0032] 130. Computer

[0033] 140. Screen

[0034] 150. Virtual reality glasses

[0035] 160. Simulator

[0036] 170. Autonomous Platform / Robot

[0037] Detailed Description of the Invention

[0038] In this detailed description, preferred embodiments of the visual planning method and tool are explained only for a better understanding of the subject and without any limiting effect.

[0039] The invention comprises a visual programming method and a tool developed based on this method that will allow the plans of hierarchical or distributed multi-agent organisations to be defined completely in a user-friendly way and to be executed on a server (120), a computer (130) or an information processing unit (embedded processor card etc.) integrated into an autonomous platform / robot (150) . The structure of the plans is a graph including conditional, concurrent and synchronous flow diagrams consisting of action, function and condition blocks. Each plan defines an action belonging to a relevant agent (virtual or real) and each action can be used as a sub-agent action (30) in concurrent flow diagrams that similarly define the action of a superior agent (10). In this way, an action hierarchy based on the ontological hierarchy between the agents forming the organisation has been created. This approach provides genericity that allows all kinds of actions of the agents to be defined individually or collectively, and also facilitates the prevention of overloading or overlooking of any agent as a resource during the plan writing phase. The developed tool increases planning efficiency by allowing the agent actions produced as concurrent sub-agent action plans to be recorded, updated and reused. The tool also allows the use of function and condition blocks beside action nodes to allow planning all types of actions and has an interface that allows the definition of functions and conditions used in the relevant plan blocks.

[0040] The ability to collectively model the behaviours of agents within a hierarchical organisation can allow the behaviours of agents to be simulated in a realistic way in almost any type of simulation. The image above shows hierarchical action plans based on flowcharts that allow for the representation and planning of collective actions. For each agent action plan (20), there can be a superior agent plan (10) consisting of concurrent action plans of this agent and other agents at the same hierarchical level, and a synchronised sub-agent collective plan (30) that describes / plans how the actions of the agent are carried out by its sub-agents. The synchronised sub-agent collective plan consists of sub-agent action plans (40,50). The action plan at each level (20,40,50) is a conditional plan consisting of at least one action node and conditions that trigger -when fired- the transition from one action to another. The condition blocks include the relevant condition functions. Not only actions but also functions can be assigned to action nodes. This allows synchronisation between plans and enables the description of repeatedly executed parts inside a plan. Thus, for example, for an action that is desired to be repeated N times, two parameters (reserved for that plan) can be selected and one of them can be assigned to the value 1, which is the beginning of the count, and the other to the value N, which is the limit, in a node containing an initialisation function. After each execution of the action during the plan, the initial value can be increased by 1 through another function, and the resulting value can be compared with the limit value (N) in a subsequent condition block. The action can be applied or not -anymore- be applied according to the result of this comparison. As a result, it is provided that the action is repeated N times. Similarly, synchronisation can be achieved between plans by assigning another plan parameter (S) reserved for synchronisation purposes to a certain value in a function at a certain point of the action plan belonging to an agent, and by checking this S value in a condition in another agent plan at the same level and acting according to this value.

[0041] The above image shows a representative high-level function writing interface that allows the user to write condition functions assigned to the condition blocks between plan nodes or functions that can be assigned to plan nodes directly. This interface, which can be accessed from any condition block or any action / function node, contains the name (60) of the function assigned to the relevant node / condition block and allows this name to be changed. Writing the function (or the condition function) is carried out via a virtual keyboard / keypad (70) located on the user interface and containing symbols suitable for the user's needs and areas of use. In this way, any function or condition function that may be needed in the preparation of any plan can be created. If necessary, two different user interfaces can be used for writing conditions and condition functions for simplicity. The function / condition function written via the keys is displayed in a text window (80). The prepared or changed function / condition function can be saved via a save key (90).

[0042] With the developed agent action planning method and tool, single (20, 40, 50) or collective (10, 30) action plans can be defined for agent groups (superior agents), agents and sub- agents, which can be saved and reused. The approach allows planning for any desired number of hierarchical levels. For example, N group and M agent action plans can be written for N groups and M agents within the scope of a scenario via a monitor (140) or a virtual reality glass (150), for a simulator (160) running on a server (120) or a computer (130). These plans are automatically executed by the agents and agent groups after being written by the user as concurrent, conditional and synchronised flow diagrams (20, 40, 50). Each action within the scope of the agent action plans (20) that constitute the superior agent collective action plan (10), is similarly created and recorded as a collective plan (30) consisting of the conditional and synchronised sub-agent action plans (40, 50) of the sub-agents that constitute the agent. The number of these hierarchical levels increases to three for groups. (Because each group action also consists of the concurrent and synchronised action plans of the agents / platforms that constitute the group.)

[0043] The scenario stands at the highest level of behaviour hierarchy. The action plans of the lowest-level agents are at the lowest level. Since these plans generally represent reactive actions, they can be designed as behaviour trees instead of flow charts and recorded in this form on servers (120), computers (130) or on the integrated information processing units (embedded processor card, etc.) of autonomous platforms / robots (150). Thus, a hybrid structure can be created. The actions of the lowest-level agents can be called atomic actions in accordance with the atomicity assumption of the agent accepted as the lowest level. Atomic action is a primitive, basic action that cannot be created as the actions of the subagents of the relevant agent. For example, if the systems of the platforms (the engine of a car etc.) are considered as the lowest level agents (which cannot be divided into lower level agents or do not need to be divided due to requirements) within the scope of a simulation scenario or a real plan, the rotation of a shaft by this engine can be considered as an atomic action. (The driving behaviour of the car, which is the superior agent of the engine, will be created as a synchronised plan of this action and similar atomic actions - the rotation of the steering wheel for example, which is another system.)

[0044] The technique is based on the ontological hierarchy of the relevant multi-agent organisation for the action planning hierarchy. For this reason, it is flexible enough to allow the design of all kinds of integrated movements, and it also provides user convenience in terms of planning, reducing errors. In addition, the hierarchy prevents action plans from becoming complex, allowing planning at every level without increasing complexity.

[0045] Through this visual programming approach, not only scenarios but also reusable agent actions -consisting of collective sub-agent plans- called vignettes can be designed and recorded, which then can be used repeatedly in different scenarios () For any agent, inputs that generalise / genericise such an action (consisting of synchronised and conditional simultaneous plans of its own child agents) can be defined. (For example, for the shaft rotation behaviour of the motor mentioned above, the power applied or speed of the rotation are such inputs that determine how this behaviour will be performed. Thus, the rotation behaviour of the motor becomes a more generalized, generic behaviour that can be executed at the desired speed by the user.) These inputs can automatically include the inputs of the lower-level actions that make up the action (for example, the cruising action of a car can take the rotation speed, which is one of the lower-level actions that make up this action, as input to be transmitted to the engine) or these inputs can be defined internally as constant or variable values within the scope of the action definition.

[0046] These actions can also be written and recorded as role-based plans for cases where the subagents of the agent are not a priori determined. In other words, a "role action plan" is written instead of the sub-agent action plan and during the action assignment, the action plan can be transferred to the sub-agent by assigning the appropriate sub-agents to the relevant roles. (During operation, this sub-agent will apply the action plan written for the relevant role. Due to the suitability of the sub-agent to this specific role, the actions within the scope of the plan are executable by the relevant sub-agent.) (For example, a role might be defined for a leader platform in a task group. The task plan that the leader is asked to implement would be written to this role, and when the time comes to the assignment of the task to any task group, this plan is assigned onto the platform that has the features appropriate to and is selected for this leader role.)

[0047] Through such kind of joint plan creation, it is prevented that any agent at any level is overlooked or overloaded. In addition, in order to further increase flexibility and especially to fully meet tactical needs, the capability of assigning multiple plans -that can be operated concurrently- to any agent is also provided to the user within the scope of the method and tool. For this reason, while an agent at any level is operating any action, another action can be requested -from that agent- by another plan. Such conflicts are resolved through priority mechanisms. In this way, any agent can apply multiple action plans -as in real world- provided that they do not use the same actuator / sub-agent resources at the same time. In addition, writing high priority reflex plans that do not have a place in the main action plan and are expected to be executed only when certain conditions occur by interrupting the main plan, can be done easily and without unnecessarily increasing the complexity of the main plan, thanks to this approach in which the agents can apply multiple plans concurrently. (Thus, for example, a high priority collision prevention plan can be written in addition to a main cruise plan for a car, and when a collision risk occurs, this plan would completely take over steering and engine control and the cruise plan would be interrupted for a while.)

[0048] None of the agents always have to execute an action (during some joint plan). Nevertheless, for a case where that agent should stay idle in some time, at least the "idle" behaviour must be assigned to the agent, although this might be just a conceptual action.

[0049] Nodes within the scope of plans usually include agent actions at different levels. On the other hand, in order to maximize flexibility (for the implementation of all types of action plans) and to ensure synchronisation between plans, nodes can also include user-defined functions in addition to actions. These functions usually do not have temporal dimension (which means that they do not consume time in simulation or consume negligible time in real operation). For example, if an action is to be applied repeatedly, the value of an agent plan parameter can be altered in a function node and then this parameter can be compared with some limit number in a following condition block to repeat an operation as many times as desired. Similarly, synchronisation can be achieved by assigning a parameter value in an action plan and checking this parameter value in the condition blocks inside another agent's action plan. In our invention, the preparation of scenarios consisting of the highest-level plans for analysis, training and real operations is the responsibility of the user. As a result, the user / human is already in the loop at the top level of the hierarchy -as the preparer of the top-level scenario. However, the approach can also be used for collective action planning for any hybrid organisation where humans are sub-agents at any level. These synchronised plans can be applied by humans and virtual / real autonomous agents (170) in virtual or real environments.

[0050] The method can also be used for the healthy planning and monitoring of collective actions of organisations consisting only of people -free of autonomous systems. In this case, the planned high-level action may not be a scenario, but an operation, integrated operation, production process, project, or even a program consisting of projects managed concurrently. In this context, the advantages of the approach over Gannt diagram techniques can be utilised. Because Gannt diagrams cannot directly show the workloads of people involved in the processes within the scope of the workplans. This makes it difficult to detect low or excessive workload for any team member and to re-plan for load balancing. On the other hand, in the proposed approach (namely, in a project / program / operation planning tool using this approach) where separate, concurrent and synchronised work plans are prepared for each employee, such errors will be prevented at the stage of planning. In this way, monitoring and plan updating will also be facilitated.

[0051] In the new joint planning technique defined, the practice of separation of concerns is also applied. Within the scope of the plan, lower-level behaviours and upper-level coordination are already separated through sub-agent actions and plans that manage them from the upper level. In addition, communication between agents and the creation of situational awareness of any agent are excluded from the plans in the proposed method. These functions -as reactive functions- are provided to be operated concurrently and continuously alongside any joint plan. The cognitive data created and updated as a result of these functions constitute the necessary live / up-to-date inputs to the plans. While the joint plans of real agents can be prepared with the proposed approach, plans developed and tested in a virtual environment can be loaded or assigned to real systems / agents, as well.

[0052] 120 . 160

[0053] The basic architecture of a sample plan preparation and loading system for real agents is shown in the image above. In such a system, a simulator (160) running on a server (120) or a computer (130) with screen (140) or virtual reality glasses (150) can be used to prepare and test a mission plan for an autonomous platform / robot (170) in a virtual environment. Then, with the help of the computer (130) and screen (140) (or virtual reality glasses (150)), this plan can be loaded onto the relevant autonomous platform / robot (170) for real world tests.

[0054] A condition / function writing feature completes the planning capability of the proposed approach / tool. Any condition can be defined through the tool and saved with a special name (60) via the save key (90). The tool consists of two basic parts: a virtual keypad / keyboard (70) and a text window (80). The condition / function definition written via the keypad (70) is reflected in the text window (80). (Conditions are special functions that give Boolean output Boolean is a variable type that can only have one of the values of "true" or "false" and is used in logic operations. Node functions, on the other hand, do not produce output. They can only alter the values of some parameters.)

[0055] This condition / function writing interface can be accessed via any node or decision block within the scope of the flowcharts. Here, a new condition / function can be written, an existing condition / function can be updated, and another / ready condition / function that has been recorded can be assigned to the corresponding node. If necessary, inputs can also be defined for the functions / conditions, and the response of these conditions / functions can be changed according to the alternating input values assigned to them during the plan construction (in this way, generic / generalised functions / conditions can be created).

[0056] The number, operator, ready (hard-coded) function and time keys (100) that can be found on the keyboard (on the user-interface) of the tool allow the use of any numbers including some irrational numbers, arithmetic and logic operators, some basic, hard-coded functions such as functions that give the distance between two agents or generate random numbers, and some basic simulation parameters such as simulation time inside the functions or condition formulations. The inputs, events and agent / action parameter keys (110) provide access to the inputs of the function / condition, the inputs of the upper-level, collective synchronised plan (for example: a task belonging to a superior platform / group), the events defined in the simulation, all parameters of the agents (and their sub-agents) inserted into the simulation scenario, and parameters specific to the action plans, during the function / condition construction.

[0057] The written function or condition function is executed when necessary during the simulation run and the desired parameter / state change is performed, or the agent plan is continued according to the Boolean value produced by calculating the output of the subsequent condition. In order to achieve this, the relevant function / condition operating module is provided with access to all agent parameters during the run.

[0058] The invention can be used in military training and analysis simulators for the aim of writing, updating and applying organised task plans at different levels varying from systems' up to group and side levels, and also for defining new tasks / tactics.

[0059] Our invention -or versions thereof adapted for related sectors- can be used for the aim of preparing / updating / applying or testing -in real or virtual worlds- the collective task plans of all organisations in which their agents, including people and autonomous systems, execute joint / synchronised actions together. An example of such applications is the planning, testing and execution of synchronised operations of robots in a factory for any production process. Not only joint operations of autonomous agents but also collective operations of people in any institution and sector can be planned, tracked and re-planned as easily and completely as possible with the proposed planning method. The invention can as well be used in the video games sector, which is quite close to the simulation world. The operation of all collective or individual agents within the scope of any game in accordance with the plot can be planned with the proposed user-friendly method and tool or a version thereof, adapted to the relevant sector. The scope of protection of this application is determined in the claims section and cannot be limited to those explained above for the aim of example. It is clear that a person skilled in the art can reveal the innovation presented in the invention by using similar structures and / or apply this structure to other areas with similar purposes used in the relevant technique. Therefore, it is also obvious that such structures will lack the criteria of novelty and especially surpassing the known state of the art.

Claims

CLAIMS1. A visual planning method and tool focused on enabling collective, synchronised planning and replanning for distributed or hierarchical organisations consisting of different types of agents, wherein conditional, concurrent and synchronised flowcharts consisting of sub-action, function and condition blocks are defined on a server (120), computer (130), virtual reality glasses (130) or autonomous platform / robot (170), each of which describes an action of an agent and each such action is used similarly as a sub-action (30) in concurrent flowcharts defining the action of a superior agent (10).

2. A visual planning method and tool according to Claim 1, wherein agent actions generated from concurrent sub-agent action plans are saved, updated and reused.

3. A visual planning method and tool according to Claim 1, comprising an interface that allows the use of function and condition blocks aside from sub-action nodes for planning actions and allows the definition of these functions and conditions used in plan blocks.

4. A visual planning method and tool according to Claim 1, comprising an interface that comprises a virtual keyboard / keypad (70) with keys (100) that allow using rational and some irrational numbers, logical and arithmetic operators, some basic hard-coded (sub) functions and time data for writing the functions and conditions to be used inside a conditional plan, as well as with keys (110) that provide access to the inputs, if any, of the function / condition in construction, the inputs and parameters of the corresponding collective action plan, simulation events and parameters of the agents within the scope of the scenario and thus allows the creation of any function or condition that may be needed in the preparation of plans.

5. A visual planning method and tool according to Claim 1, wherein each action within the scope of the action plans (20) that constitute the superior agent collective sub-action plan (10) is also similarly created and recorded as a collective plan (30) consisting of the conditional and synchronised action plans (40, 50) of the sub-agents that constitute that agent.

6. A visual planning method and tool according to Claim 1, wherein the action plans of the lowest-level, possibly reactive agents might as well be designed as behaviour trees instead of flow charts, creating a hybrid structure.

7. A visual planning method and tool according to Claim 1, wherein inputs that generalize / genericize any agent action, which consists of synchronised, conditional and concurrent plans of its sub-agents, can be defined during the action (visual) scripting and the inputs of the sub-actions that make up the action are automatically included in those inputs, or these sub- action inputs are set as fixed or variable values internally.

8. A visual planning method and tool according to Claim 1, wherein, by writing a role action plan instead of a sub-agent action plan, an action plan might be assigned to the subagent by assigning the appropriate sub-agent to the corresponding role during the action assignment to the superior agent.

9. A visual planning method and tool according to Claim 3, comprising a tool that allows the condition / function typed via the keypad (70) to be reflected in the text window (80), saved with the save key (90) and named (60).

10. A visual planning method and tool according to Claim 1, wherein multiple actions are assigned to the agent to run concurrently.

Images