A multi-agent cooperative confrontation deduction method and system based on a hierarchical behavior tree control framework

Abstract

The present application belongs to the technical field of decision-making behavior modeling and intelligent simulation, and relates to a multi-agent cooperative confrontation deduction method and system based on a hierarchical behavior tree control framework. The method comprises: constructing an editable behavior tree, wherein the parameters of the action nodes can be edited; constructing a multi-level intelligent decision-making model based on the editable behavior tree; and using the multi-level intelligent decision-making model to perform cooperative confrontation deduction of multi-agents. The present application can solve the difference between human and machine thinking, significantly improve the interpretability and artificial intervention ability of strategies, realize parameter configuration combining dynamic and static, greatly enhance the adaptability to dynamic unstructured environment, break through the bottleneck of large-scale agent cooperation, and realize multi-scale efficient management and control of "high layer-middle layer-low layer".

Description

Technical Field

[0001] This invention belongs to the field of decision behavior modeling and intelligent simulation technology, specifically involving a multi-agent collaborative adversarial inference method and system based on a hierarchical behavior tree control framework, which is applicable to multi-party dynamic game scenarios. Background Technology

[0002] Multi-Agent (MA): Refers to a system consisting of multiple individuals with autonomous decision-making capabilities, capable of interacting with the environment and other intelligent agents. Each agent can independently perceive the environment and execute actions, while simultaneously completing complex tasks through collaboration. Cooperative adversarial simulation is a dynamic simulation scenario involving multiple intelligent agents. The participants are divided into different camps, requiring both internal agent cooperation and strategic adversarial interactions with opposing camps. The simulation aims to achieve its objectives by mimicking real-world situational changes. Behavior Tree (BT): A tree-structured model used to describe the behavioral logic of intelligent agents. It consists of a root node, control nodes, decorator nodes, and task nodes. Orderly execution and flexible transitions of behavior are achieved through the transmission of node states (success / failure / running), featuring modularity and visualization.

[0003] A Finite State Machine (FSM) is a model that describes the behavior of a system through discrete states and transition rules between states. Its core components include a set of states, a set of events, transition functions, an initial state, and a final state. During operation, the system starts from the initial state. When an event in the event set is triggered, the system transitions from the current state to a new state according to the transition function, forming a loop of "state → event → new state" until it reaches the final state (if it exists). This describes the system's behavioral logic under different conditions. The core technologies of FSMs revolve around the modeling and efficient processing of states and transitions: state modeling requires discretizing and abstracting system behavior, balancing the granularity of the division to avoid ambiguity or redundancy; transition rule design needs to clearly define the binding relationship between events and states, supporting multiple condition combinations and action embedding during transitions; state compression optimization alleviates state explosion through equivalent state merging and hierarchical nesting (such as HFSM); the execution engine relies on event-driven or polling mechanisms to ensure the real-time performance and correctness of state transitions, while supporting concurrent management of multiple FSMs. Finite state machines have significant limitations: First, they are prone to state explosion. As the dimensions of system behavior increase (such as multiple roles or multiple conditions), the number of states will increase dramatically, making the rules complex and difficult to maintain. Second, they have poor scalability. When adding new entity behaviors, due to the continuity of state transitions, existing transition rules need to be structurally modified, resulting in high remodeling costs and difficulty in adapting to the dynamic adjustment needs of complex scenarios.

[0004] Reinforcement learning-based decision intelligence models are machine learning frameworks that learn optimal policies from feedback through agent-environment interaction. The core concept is "trial and error learning," encompassing two main roles—the agent and the environment—and three key elements—state, action, and reward. The model operates on a "perception-decision-feedback-update" cycle, with the ultimate goal of learning a policy that maximizes long-term cumulative rewards. The core technologies of this model focus on policy optimization efficiency and learning stability: first, policy representation and optimization, divided into value-based (e.g., Q-Learning) and policy-based (e.g., policy gradient method), adapting to different action scenarios; second, value function estimation, using TD, MC, etc., to reduce bias, and combining it with advantage functions to highlight the merits of actions; third, balancing exploration and exploitation, achieved through ε-greedy algorithms, UCB, etc.; furthermore, experience replay and target networks can reduce data correlation and alleviate training oscillations. The model has several obvious drawbacks: First, it has low sample efficiency, requires massive environmental interactions, and trial and error in complex scenarios is time-consuming and costly. Second, the reward design relies on manual intervention, and sparse or unreasonable rewards can easily lead to learning stagnation and "reward bias". Third, it has weak generalization ability, the training strategy is prone to overfitting, and the performance deteriorates when transferring to new environments. Fourth, it has security risks, as it may perform destructive actions during the exploration phase, making it difficult to adapt to scenarios with high security requirements.

[0005] Machine learning-based decision intelligence models are a technical framework that utilizes data-driven learning to achieve automated decision-making. Their core is built around "extracting patterns from data to guide decision-making," encompassing three core elements: data (input foundation), model (learning vehicle), and decision objectives (such as classification, regression, and clustering). Their working principle follows a closed loop of "data preprocessing - model training - inference and decision-making": first, the data is cleaned, labeled, and feature-processed; then, the model (such as a classifier or regressor) learns the mapping relationship between inputs and decision results from the data; after training, the model receives new inputs and outputs decision results based on the learned patterns, ultimately meeting specific decision-making needs such as classification, prediction, and optimization. The core technology of machine learning-based decision intelligence models focuses on "improving the accuracy and applicability of decisions." First, model architecture selection encompasses supervised learning (such as logistic regression and random forests, suitable for classification / prediction of labeled data), unsupervised learning (such as K-means and PCA, for clustering / dimensionality reduction of unlabeled data), and semi-supervised learning (combining two types of data to address scenarios with scarce labels), adapting to different data conditions and decision-making tasks. Second, feature engineering techniques optimize input data and improve model learning efficiency through feature extraction (such as CNN for image feature extraction), feature selection (removing redundant features), and feature transformation (standardization / normalization). Third, optimization and evaluation techniques use loss functions (such as cross-entropy and MSE) to measure decision error, iteratively optimize model parameters using algorithms such as gradient descent, and evaluate decision performance using metrics such as accuracy, F1 score, and AUC to ensure model reliability. However, machine learning-based decision intelligence models have significant application limitations. First, it heavily relies on data quality and scale. If the data sample is insufficient, biased (such as gender / regional bias), or has too much noise, the model is prone to learning incorrect patterns, leading to unfair or wrong decisions. Second, interpretability is generally weak. Complex models (such as deep neural networks) have a "black box" problem, making the decision-making process difficult to trace. In scenarios such as healthcare and finance where clear decision-making criteria are required, this can easily reduce user trust. Third, it lacks generalization and robustness. The model is prone to decision failure in new scenarios outside the training data distribution (such as sudden abnormal data), and when faced with adversarial examples (such as tampered images or text), the decision results are easily interfered with, making it difficult to cope with complex and ever-changing real-world environments.

[0006] The existing technology has the following problems: (1) Differences between human and machine thinking lead to insufficient adaptability: Current rule-based intelligent agents are mainly designed and developed based on human thinking. Computer algorithms differ from human thinking and it is difficult to perfectly replicate human thinking. (2) Poor adaptability to situational changes: The operation relies entirely on pre-set fixed rules and decision-making frameworks. During the design phase, such agents will clearly define the triggering conditions and corresponding actions based on known situational scenarios. However, situational changes in the actual environment are often dynamic, complex, and uncertain. New scenarios may arise that are not covered by the pre-set rules (such as sudden abnormal information input, situational trends influenced by multiple factors), or the original rules may only be able to match some situational details and cannot adapt to the complete change logic. (3) Humans cannot participate in the decision-making process: The decision-making process of rule-based intelligent agents is highly automated and closed, making it difficult for humans to actively intervene and exert influence during their operation. The decision-making logic of such intelligent agents relies entirely on the rule base and triggering conditions preset in the design phase. From situational information input and feature matching to strategy generation, the entire process is automatically executed by the program according to a fixed link, and there is no interactive interface for real-time human intervention. Summary of the Invention

[0007] To address the aforementioned problems, this invention provides a multi-agent cooperative adversarial inference method and system based on a hierarchical behavior tree control framework.

[0008] The technical solution adopted in this invention is as follows: A multi-agent cooperative adversarial inference method based on a hierarchical behavior tree control framework includes the following steps: Construct an editable behavior tree, where the parameters of action nodes can be edited; A multi-level intelligent decision-making model is built based on editable behavior trees; Multi-level intelligent decision-making models are used to conduct collaborative adversarial simulations of multiple agents.

[0009] Furthermore, the editable behavior tree includes a root node, control nodes, action nodes, and condition nodes; control nodes are used to manage the execution logic of child nodes, including sequential nodes, selection nodes, and parallel nodes; condition nodes are used to control the execution conditions of child nodes; action nodes are responsible for executing specific actions; each node returns one of three states after execution: runnable, inoperable, and running; each node is displayed and categorized in an icon-based format, and various types of nodes can be combined by dragging and dropping.

[0010] Furthermore, the parameters of the action node are edited through a front-end visual editing layer, an intermediate parameter processing layer, and a back-end execution layer.

[0011] Furthermore, the parameters of the action node include static parameters, blackboard data reference parameters, and expression parameters; the blackboard data reference parameters are dynamically updated according to changes in the blackboard data, and the functions of the blackboard data management mechanism include: data access interface, scope management, change monitoring mechanism, type checking and conversion.

[0012] Furthermore, the multi-level intelligent decision-making model includes a first-level decision-making model, a second-level decision-making model, and a third-level decision-making model; the first-level decision-making model includes at least one action node, each action node performing a specific atomic action; the second-level decision-making model includes at least one control node and at least one action node, combining the atomic actions of the first-level decision-making model into behaviors with specific objectives; the third-level decision-making model includes at least one condition node, at least one control node, and at least one action node, selecting or switching the combined behaviors of the second-level decision-making model based on the environmental situation to determine the overall strategy direction of the agent.

[0013] Furthermore, the multi-level intelligent decision-making model employs a task allocation mechanism, a status reporting mechanism, a coordinated decision-making mechanism, and an exception handling mechanism for communication and task scheduling.

[0014] Furthermore, the above method provides situational information, database data, and model use case library as basic inputs through the environmental data layer, and supports the creation of new models, node selection, node parameter setting, and viewing of model use cases through the user interaction layer, forming a closed-loop process of "data input - execution scheduling - decision modeling - user interaction".

[0015] A multi-agent cooperative adversarial inference system based on a hierarchical behavior tree control framework, comprising: The behavior tree building module is used to build editable behavior trees, where the parameters of action nodes can be edited; The intelligent decision-making model building module is used to build multi-level intelligent decision-making models based on editable behavior trees; The collaborative adversarial simulation module is used to conduct collaborative adversarial simulations of multiple agents using a multi-level intelligent decision-making model.

[0016] This invention constructs a hierarchical behavior tree control framework for multi-agent cooperative adversarial inference, and combines visual editing, a dynamic blackboard mechanism, and a three-level hierarchical control strategy. Compared with existing technologies, it produces the following significant beneficial effects: 1. Address the differences in human-machine thinking, significantly improving the interpretability of strategies and the ability for human intervention.

[0017] Key features: Editable behavior tree model, visual node drag and drop.

[0018] Technical Principles and Effects: Existing deep learning models suffer from a "black box" problem, making it difficult to trace the decision-making logic. This invention employs a behavior tree structure to decouple complex decision-making logic into a tree-like "condition-action" sequence. The logical structure is clear and intuitive, fully conforming to human logical reasoning habits.

[0019] Specific benefits include: Through a visual editor, experts without a programming background can directly participate in the construction and modification of decision-making logic. In experiments or simulations, when an agent exhibits unexpected behavior, operators can quickly pinpoint the specific behavior tree node (e.g., an incorrect condition judgment at a selection node), achieving fault location and logic correction within seconds, thus solving the pain points of traditional algorithms being "unintelligible and uninterventional."

[0020] 2. It enables the configuration of parameters that combine static and dynamic elements, greatly enhancing its adaptability to dynamic and unstructured environments.

[0021] Key point: Action node parameters are editable (static parameters and blackboard data references can be configured together).

[0022] Technical Principles and Effects: Traditional rule systems typically have fixed parameters, making it difficult to cope with battlefield fog or sudden situations. The parameter configuration mechanism designed in this invention allows only parameter references (such as referencing {target_distance}) to be defined when a node is created, while environmental awareness data is acquired in real time through the Blackboard at runtime.

[0023] Specific benefits: This "separation of logic and data" design allows the same behavior tree to be reused in completely different scenarios. For example, in adversarial simulations, when the enemy changes tactics, causing a drastic change in the situation, the agent does not need to rewrite the code. It only needs to update the perception data through the blackboard, and the behavior tree can automatically switch to a better branch (such as switching from "attack" to "flanking") based on the new values ​​(such as distance and health threshold), which significantly improves the robustness of the system.

[0024] 3. Break through the bottleneck of large-scale intelligent agent collaboration and achieve efficient multi-scale control at the "high-level-middle-low-level" levels.

[0025] Key points: multi-level intelligent agents (three-level hierarchical architecture), dynamic resource allocation.

[0026] Technical Principles and Effects: Addressing the state explosion problem that easily arises in multi-agent collaboration using FSMs (Finite State Machines), this invention innovatively proposes a three-tiered architecture: "Action Layer - Composite Behavior Layer - Decision Planning Layer." The upper layers focus on the overall intent, while the lower layers focus on specific execution. The layers are decoupled through instruction issuance and state reporting.

[0027] Specific beneficial effects: 1) Improved computational efficiency: The decision search space is confined to each level, avoiding the exponential computational explosion caused by global search, and the supported agent scale can be increased from dozens to thousands. 2) Coordination and consistency: Through unified utility-based task auction and allocation at the top level (decision planning layer), goal conflicts (such as multiple units vying for the same shelter) or resource waste during task execution by multiple agents are effectively avoided, ensuring the emergent and orderly nature of group behavior.

[0028] 4. Improve development and reuse efficiency and reduce system maintenance costs.

[0029] Key points: modular node library, node connection and topology management.

[0030] Technical principle and effect: This invention encapsulates basic atomic actions into a standard library.

[0031] Specific benefits include: When constructing new simulation scenarios, developers do not need to write code from scratch; they can simply reuse existing action layer nodes (such as "move" and "reconnaissance") and generate new action logic by combining different permutations of the behavior layer. Theoretical analysis shows that this reuse mechanism can shorten the adaptation time to new scenarios by more than 50%, greatly reducing the maintenance and expansion costs of the system. Attached Figure Description

[0032] Figure 1 This is a flowchart of the steps of a multi-agent cooperative adversarial inference method based on a hierarchical behavior tree control framework in one embodiment of the present invention.

[0033] Figure 2 This is a technical framework diagram of a multi-agent cooperative adversarial inference method based on a hierarchical behavior tree control framework in one embodiment of the present invention.

[0034] Figure 3 This is a schematic diagram of a three-level hierarchical architecture for a multi-agent collaborative adversarial inference method based on a hierarchical behavior tree control framework in one embodiment of the present invention.

[0035] Figure 4 This is a block diagram of a multi-agent collaborative adversarial inference system based on a hierarchical behavior tree control framework in one embodiment of the present invention. Detailed Implementation

[0036] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0037] Based on the deficiencies of existing technologies, this invention aims to specifically address the following technical problems. All problems can be practically solved through the technical means of this invention, and there is a clear logical correspondence between them and the technical solutions: 1. The problem of insufficient adaptability caused by differences in human and machine thinking: Although rule-based intelligent agents are designed according to human thinking, computer algorithms cannot perfectly replicate human thinking, directly resulting in insufficient adaptability and a lack of decision-making flexibility at the human level. This invention, through an editable behavior tree and a dynamic parameter configuration mechanism, enables humans to adjust the intelligent agent's behavioral logic in real time, compensating for differences in human and machine thinking and improving adaptability.

[0038] 2. The problem of poor adaptability of rule-based models to changing situations: Rule-based agents rely on preset fixed rules, while actual adversarial scenarios are dynamic, complex, and uncertain, making it easy for preset rules to be incomplete or insufficiently adapted. This invention achieves dynamic adjustment of the agent's behavioral logic through a hierarchical behavior tree architecture and a real-time blackboard data update mechanism, thereby improving its adaptability to changing situations.

[0039] 3. The problem of a closed decision-making process that prevents human participation: Existing intelligent agent decision-making processes are highly automated and closed, lacking real-time intervention interfaces, preventing human intervention during the runtime phase. This invention constructs a human-machine collaborative decision-making mechanism through a visual editing interface, real-time intervention interface, and parameter adjustment functions, supporting human participation in decision-making during runtime.

[0040] 4. Insufficient Multi-Agent Collaborative Management Capabilities: Existing solutions struggle to achieve task allocation, coordination, and hierarchical management of multiple agents, easily leading to action conflicts and inconsistent goals. This invention, through a three-level hierarchical behavior tree architecture and a dynamic resource allocation mechanism, achieves multi-scale collaborative management of "decision planning - combined behavior - action execution," improving the robustness and collaborative efficiency of multi-agent systems.

[0041] The key points of this invention are as follows: (1) Editable behavior tree model. An intelligent decision-making model is constructed using an editable behavior tree. The behavior tree nodes are visualized, and nodes can be added and combined freely by dragging and dropping. The parameters of the action nodes are editable, and it can read both blackboard data and the data passed in when the node was created.

[0042] Node library and drag-and-drop system: Preset basic node types, control nodes such as selection, sequence, and parallel, action nodes such as "move" and "detect", and condition nodes such as "distance judgment". Each node is displayed in an icon-based form and stored in categories (such as "control type", "action type", and "condition type"). Various types of nodes can be combined by dragging and dropping to build a behavior tree model.

[0043] Node connection and topology management: Design node ports (control nodes have "entry + multiple exits", leaf nodes only have "entry"), users establish parent-child relationships between nodes by dragging and dropping port connections, the editing engine verifies the legality of node connections in real time (e.g., control nodes must be connected to child nodes, leaf nodes cannot have child nodes), and automatically maintains the JSON description of the behavior tree topology structure (including node ID, type, parent node ID, and location coordinates).

[0044] Parameter editing panel: When each action node is selected, a parameter configuration form pops up, supporting mixed configuration of static parameters (passed in during creation) and blackboard data references (dynamically read); the form automatically recognizes the parameter type (string / number / boolean / blackboard variable) and provides input validation (such as coordinate format check and blackboard variable existence validation).

[0045] The intelligent decision-making model of this invention has an intuitive and visible logic, and the parameters of the action nodes are editable, which can improve human participation in the decision-making process.

[0046] (2) Multi-level intelligent agent. The behavior tree-based intelligent decision-making model is divided into three levels: multiple first-level decision-making models form a second-level decision-making model, and multiple second-level decision-making models form a third-level decision-making model. The first-level decision-making model contains at least one action node, which encapsulates reusable basic execution units (such as "move to target point", "data detection", "instruction sending"). Each action node performs only a single, explicit operation and outputs a "success / failure" status. The second-level decision-making model contains at least one control node and at least one action node. Through the sequence (sequential execution), selection (conditional branch), and parallel (synchronous execution) nodes of the behavior tree, the atomic actions of the first-level decision-making model are combined into behaviors with specific goals. The third-level decision-making model contains at least one condition node, at least one control node, and at least one action node. Based on the environmental situation (such as "opponent distance" and "task priority"), it selects or switches the combined behaviors of the second-level decision-making models to determine the overall strategy direction of the intelligent agent (such as "perform patrol when there is a low threat, switch to alert when there is a high threat").

[0047] The various levels of the aforementioned intelligent decision-making model can operate independently or be combined to form multi-level intelligent agents. For example, each level-one decision-making model can be an intelligent agent, each level-two decision-making model can be an intelligent agent, and each level-three decision-making model can be an intelligent agent. Level-one, level-two, and level-three decision-making models can be combined to form new intelligent agents, etc. The intelligent decision-making model of this invention can meet the needs of application scenarios of different scales.

[0048] The first-level, second-level, and third-level decision-making models of the aforementioned intelligent decision-making model can also be referred to as first-level behavior trees, second-level behavior trees, and third-level behavior trees, or as action layers, combined behavior layers, and decision planning layers, respectively. Essentially, the behavior trees are divided into three levels according to function. The lower-level behavior trees provide basic execution capabilities for the upper-level behavior trees, and the upper-level behavior trees guide the behavior logic of the lower-level behavior trees, forming a hierarchical control architecture of "atomic actions - combined behaviors - global decisions" to achieve flexible management and control of multi-scale behaviors.

[0049] In one embodiment, the multi-agent cooperative adversarial inference method based on a hierarchical behavior tree control framework of the present invention, such as Figure 1 As shown, it includes the following steps: Step S11: Construct an editable behavior tree, in which the parameters of action nodes can be edited; Step S12: Construct a multi-level intelligent decision-making model based on an editable behavior tree; Step S13: Use a multi-level intelligent decision-making model to conduct collaborative adversarial simulations of multiple agents.

[0050] In one embodiment, the editable behavior tree includes a root node, control nodes, action nodes, and condition nodes; control nodes are used to manage the execution logic of child nodes, including sequential nodes, selection nodes, and parallel nodes; condition nodes are used to control the execution conditions of child nodes; action nodes are responsible for executing specific actions; each node returns one of three states after execution: runnable, inoperable, and running; each node is displayed and categorized in an icon-based format, and various types of nodes can be combined by dragging and dropping.

[0051] In one embodiment, the parameters of the action node are edited through a front-end visual editing layer, an intermediate parameter processing layer, and a back-end execution layer.

[0052] In one embodiment, the parameters of the action node include static parameters, blackboard data reference parameters, and expression parameters; the blackboard data reference parameters are dynamically updated according to changes in the blackboard data, and the functions of the blackboard data management mechanism include: data access interface, scope management, change monitoring mechanism, type checking and conversion.

[0053] In one embodiment, the multi-level intelligent decision-making model includes a first-level decision-making model, a second-level decision-making model, and a third-level decision-making model; the first-level decision-making model includes at least one action node, each action node performing a specific atomic action; the second-level decision-making model includes at least one control node and at least one action node, combining the atomic actions of the first-level decision-making model into behaviors with specific objectives; the third-level decision-making model includes at least one condition node, at least one control node, and at least one action node, selecting or switching the combined behaviors of the second-level decision-making model based on the environmental situation to determine the overall strategy direction of the agent.

[0054] In one embodiment, the multi-level intelligent decision-making model employs a task allocation mechanism, a status reporting mechanism, a coordination decision-making mechanism, and an exception handling mechanism for communication and task scheduling.

[0055] In one embodiment, the technical framework of the multi-agent cooperative adversarial inference method based on a hierarchical behavior tree control framework of the present invention is as follows: Figure 2 As shown, it is specifically divided into four layers from bottom to top. The environmental data layer provides situational information, database data, and model use case library as basic inputs; the communication and task scheduling layer receives data and provides execution support through task allocation mechanism, status reporting mechanism, coordination decision-making mechanism, and exception handling mechanism; the decision model layer includes a first-level decision model (tree structure), a second-level decision model (multi-level branches), and a third-level decision model (large-scale network), which is responsible for core logic modeling; the user interaction layer supports creating new models, selecting nodes, setting node parameters, and viewing model use cases, and is the user operation entry point. Each layer realizes data / instruction transmission through bidirectional interaction, forming a closed-loop process of "data input - execution scheduling - decision modeling - user interaction".

[0056] In one embodiment, the present invention employs the following steps to construct nodes of an editable behavior tree: 1) Behavior tree basic structure.

[0057] The behavior tree consists of nodes of various types, each returning one of three states as its outcome: Success, Failure, and Running. Parent node To child nodes The process of passing control flow and checking execution results is denoted as ,in Indicates the execution status of the control flow. Indicates the trigger node Child nodes provide feedback upwards, and parent nodes control the tree's direction by checking the feedback results. From a node type perspective, behavior tree nodes can be divided into four main categories: root nodes, composite nodes, decorator nodes, and leaf nodes. The root node is the starting point of the behavior tree, has no input, can only have one child node, and is responsible for executing the behavior tree's entry logic. Composite nodes, as the "branching structure" of the behavior tree, manage the execution logic of child nodes, including sequential nodes, selection nodes, and parallel nodes. Decorator nodes control the execution conditions of child nodes. Action nodes are the leaf nodes of the behavior tree, responsible for executing specific actions, such as patrolling, attacking, and dodging.

[0058] 2) Definitions and characteristics of the three states.

[0059] Each node in the behavior tree returns one of three states after execution: Success (success / runnable), Failure (failure / not runnable), and Running (running). These three states form the basis of the behavior tree state mechanism and have the following characteristics: The Success state indicates that the node has successfully completed the task, informing the parent node that its operation was successful. For conditional nodes, Success is returned when the condition test result is true; for action nodes, Success is returned when the specific operation is successfully executed. The Failure state indicates that the node failed to execute or the task could not be completed, informing the parent node that its operation failed. Conditional nodes return Failure when the condition test result is false; action nodes return Failure when an error is encountered during execution or when the task cannot be completed. The Running state indicates that the execution of a node has not yet been completed and needs to continue. The node will be selected for execution again the next time the tree is activated. The introduction of the Running state enables behavior trees to handle tasks that require multiple frames or long periods of time to complete, such as waiting, navigation, and persistent attacks.

[0060] 3) Control node.

[0061] Sequential nodes execute their child nodes sequentially from left to right, adhering to strict sequential execution logic. Their state handling mechanism is as follows: a) Execution logic: Sequential nodes execute their child nodes sequentially from left to right. The next child node only runs after the previous one returns a "complete" state (Success or Failure). This execution method ensures the orderliness and dependencies of tasks. b) State return rules: If all child nodes return Success, the sequential node returns Success; if any child node returns Failure during execution, the sequential node immediately returns Failure and stops the execution of subsequent child nodes.

[0062] The selection node, also known as the fallback node, executes its child nodes sequentially from left to right until a successful child node is found. Its state handling mechanism has the following characteristics: a) Execution logic: The selection node executes child nodes in left-to-right order. Once a child node succeeds, execution stops and returns success; if all child nodes fail, the selection node returns failure. This mechanism allows the selection node to choose the first feasible action from multiple alternative actions. b) Priority mechanism: Selection nodes have a clear priority order, with the leftmost child node having the highest priority and the rightmost child node having the lowest priority. This left-to-right priority order is the basis of the behavior tree execution mechanism. The execution of nodes in the tree follows a depth-first search order, with the left leaf node having the highest priority.

[0063] Parallel nodes execute all child nodes simultaneously, determining the outcome based on predefined success conditions. Their state handling mechanism is the most complex, requiring consideration of multiple execution strategies: a) Execution logic: Unlike sequential and selection nodes, parallel nodes execute child nodes in parallel, not one at a time, but all simultaneously. This mechanism enables parallel nodes to handle scenarios requiring the simultaneous execution of multiple tasks. b) Execution modes: Parallel nodes have two main execution modes: forced mode (all child nodes must succeed before returning success) and arbitrary mode (success is returned if any child node succeeds).

[0064] 4) Condition nodes.

[0065] Conditional nodes are used to control the execution conditions of child nodes or modify the behavior of child nodes. For example, a target health check node checks whether a certain condition is met, and only continues to execute child nodes if the condition is met. An application scenario is "retreat only if the enemy's health is below 50%".

[0066] In one embodiment, the present invention employs the following steps to edit the parameters of the action node: 1) Overall Architecture Overview. The technical architecture of the editable action node parameters adopts a layered design pattern, mainly including three core layers: a front-end visual editing layer, a middle parameter processing layer, and a back-end execution layer. The front-end layer is responsible for providing a user-friendly interface, supporting drag-and-drop node creation and parameter editing; the middle layer undertakes the core functions of parameter parsing, binding, and validation; and the back-end layer is responsible for the execution and state management of the behavior tree. In terms of data flow, the system adopts a bidirectional data flow architecture: during editing, the user inputs parameter configurations through the front-end interface, which are parsed by the middle layer and stored as standardized configuration data; during runtime, the execution engine reads parameters from the configuration data and dynamically calculates parameter values ​​based on the blackboard data. This architecture design ensures data consistency between the editing state and the runtime state, while supporting real-time hot updates of parameters.

[0067] 2) Parameter Type and Data Structure Design: To achieve a hybrid configuration of static parameters and blackboard data references, the system needs to design a unified parameter representation method. Based on the parameter source and characteristics, parameters are divided into three main types: a) Static parameters: Fixed values ​​directly specified when the node is created, such as "movement speed is 5m / s" or "attack cooldown time is 3 seconds". Static parameters remain unchanged during runtime and are stored as specific numerical values ​​or strings. b) Blackboard data reference parameters: References dynamic data stored in the blackboard, such as "target position" or "target status". These parameters are dynamically updated based on changes in the blackboard data during runtime and are represented using a specific syntax format (e.g., {blackboard_key}). c) Expression parameters: Parameters that support complex logical operations, such as "movement speed is 8m / s if the target distance is less than 10 meters, otherwise it is 5m / s". Expression parameters are parsed and calculated through an integrated expression engine, supporting complex operations such as mathematical operations, logical judgments, and function calls.

[0068] 3) Blackboard Data Management Mechanism. The blackboard, as a shared data storage center for all nodes in the behavior tree, is the core infrastructure for implementing dynamic parameters. The blackboard uses a key-value pair storage structure and supports multiple data types, including numbers, strings, booleans, vectors, and objects. The blackboard data management mechanism includes the following core functions: a) Data access interface: Provides get(key) and set(key, value) methods for reading and writing data. Nodes can access shared data in the blackboard through these interfaces. b) Scope Management: Supports multi-level scopes including global blackboard, tree-level blackboard, and node-level blackboard. Global blackboard data is visible to all behavior tree instances; tree-level blackboard data is shared only within the current behavior tree instance; node-level blackboard data is private to specific nodes. c) Change Monitoring Mechanism: When the blackboard data changes, it can automatically notify relevant nodes to re-evaluate, i.e., assess the impact of the data change on the node state. This mechanism is implemented using the observer pattern, with the blackboard as the observed object and the nodes as observers. d) Type checking and conversion: Type checking is performed during data writing to ensure data type consistency. If a type mismatch occurs, a corresponding error message will be thrown. In one embodiment, the multi-level behavior tree architecture of the intelligent decision-making model of the present invention adopts a hierarchical organization, wherein each level in the hierarchical structure performs different roles and responsibilities, such as... Figure 2 As shown. Core features include the decomposition of control and decision-making (higher levels focus on abstract planning and coordination, while lower levels handle specific actions and local decision-making) and a tree-like or holistic arrangement (agents can be organized into trees, clusters, or a holistic structure). Regarding information and task flow, top-down paths transmit goals, parameters, or commands, while bottom-up processes propagate state updates, feedback, or learned knowledge. Efficient communication between different levels of behavior trees is achieved using messages or commands; this communication mechanism is also the foundation of event- or message-driven behavior trees. This hierarchical design avoids redundancy in single-behavior-tree decision-making and improves the system's robustness. The Level 1 decision model (Level 1 behavior tree), as the basic execution unit of the system, is responsible for handling specific task execution details. The design focus of the Level 1 decision model is on precise motion control and real-time response capabilities, requiring specific functions such as processing sensor data, actuator control, and local path planning. The secondary decision-making model (secondary behavior tree) consists of multiple primary decision-making models, enabling more complex task coordination. For example, reconnaissance and strike tasks together form a secondary decision-making model, achieving objectives by coordinating the actions of different entity units. The secondary decision-making model needs to handle issues such as task allocation, resource coordination, and conflict resolution, while maintaining the ability to monitor and adjust the execution of the primary decision-making models. The Level 3 Decision Model (Level 3 Behavior Tree), as the highest decision-making layer of the system, is responsible for global planning, resource allocation, and long-term goal setting. It guides the entire system's actions by analyzing the environmental situation, assessing task priorities, and formulating plans. Since Level 3 decisions typically involve a larger time and spatial scope, its information processing focuses on trend analysis, risk assessment, and opportunity identification. Trend analysis is the foundation for the Level 3 Decision Model to set long-term goals and conduct global planning. It does not focus on the instantaneous execution details of Level 1 and Level 2 decision models, but rather extracts patterns and predicts trends from comprehensive, long-term, and multi-dimensional situational data. It aggregates comprehensive data reported by Level 1 and Level 2 decision models through message / command mechanisms, including environmental awareness, task execution, system status, and resource consumption, providing long-term predictive basis for global planning and task prioritization, avoiding decisions that only focus on short-term local factors. Risk assessment is the core capability of the Level 3 Decision Model to ensure system robustness and optimize resource allocation, focusing on global, cross-level risks rather than single-task conflicts. It covers major hidden dangers affecting overall objectives, such as global task failure risk, global resource imbalance / exhaustion risk, cross-level communication failure risk, macro-environmental deterioration risk, and multi-task collaboration collapse risk. The opportunity identification three-level decision-making model has the key capability to optimize decision-making and improve global task efficiency. Unlike the local action optimization of the first and second level decision-making models, opportunity identification captures high-value, short-term, and global opportunities from the overall situation. Through global command scheduling, the first and second level decision-making models respond quickly, maximizing task efficiency, resource utilization, and objective achievement.

[0069] The effective operation of the multi-level intelligent decision-making model of this invention relies on efficient node communication and state synchronization mechanisms. The blackboard mechanism is a component in the behavior tree used to store and access runtime data. It allows different nodes in the tree to share data and can also serve as a communication bridge between nodes within the behavior tree or between the behavior tree and external systems, exchanging information by reading and writing data on the blackboard. In the multi-level architecture, the communication mechanism adopts a hierarchical design. Higher-level agents act as coordinators, allocating tasks and aggregating results; this structure is efficient for tasks requiring top-down control and decision-making. Sub-goals are the core of this hierarchical communication architecture, requiring effective communication from higher levels and providing guidance to lower levels for coordination. In one embodiment, the specific communication protocol design of the multi-level intelligent decision-making model of the present invention includes the following key elements: task allocation mechanism, status reporting mechanism, coordination decision-making mechanism, and exception handling mechanism.

[0070] In one embodiment, the task allocation mechanism refers to a three-level decision model generating a task list based on a global objective and allocating tasks hierarchically to the second and first levels through a hierarchical structure. Each task includes information such as objective description, execution requirements, resource needs, and priority. The task allocation process employs a utility-based task allocation algorithm, comprehensively considering factors such as task urgency, resource matching, and execution difficulty. The utility-based task allocation algorithm includes the following steps: 1) Information collection: The three-level decision model reads information such as the current global situation, the load status of each second-level / first-level model, available resources, and historical execution success rate from the hierarchical blackboard.

[0071] 2) Utility factor calculation: For each task to be assigned, calculate key utility indicators such as task urgency, resource matching degree, execution difficulty, and contribution to the overall goal.

[0072] 3) Comprehensive utility score: Multiple utility indicators are weighted and summed according to preset weights to obtain the comprehensive utility value of each task-execution unit combination. The higher the utility value, the more reasonable the matching.

[0073] 4) Task sorting and allocation: Tasks are sorted from high to low according to their comprehensive utility value. High-value and high-urgency tasks are assigned to the secondary decision model with the best utility. The secondary decision model then assigns the tasks to the execution nodes of the primary decision model using the same logic.

[0074] 5) Allocation result synchronization: Write the final task allocation plan to the hierarchical blackboard to complete global state synchronization and avoid duplicate allocation and resource conflicts.

[0075] In one embodiment, the status reporting mechanism refers to the first- and second-level decision-making models periodically reporting the execution status to higher levels, including information such as task progress, resource usage, encountered problems, and environmental changes. The status reporting adopts an event-driven approach, reporting immediately when critical status changes, ensuring that higher management can promptly understand the system's operational status. In one embodiment, the coordination decision-making mechanism refers to making decisions through a negotiation protocol when multiple decision-making models (or multiple intelligent agents) need to collaboratively execute a task. The negotiation process, based on factors such as task importance, resource availability, and time constraints, uses an auction mechanism or a voting mechanism to determine the execution order and responsibility allocation. The negotiation protocol includes the following steps: 1) Negotiation Initiation: A collaborative negotiation request is initiated by a high-level decision-making model or a peer-level decision-making model where there is resource competition or overlapping tasks.

[0076] 2) Information announcement: Collaborative task information is broadcast through hierarchical blackboards, including task objectives, resource constraints, time requirements, and scope of participating units.

[0077] 3) Negotiated Decision-Making: If an auction mechanism is adopted: each execution unit submits bidding information based on its own capabilities, such as execution efficiency, resource consumption, and reliability, and senior management selects the best bidding unit. If a voting mechanism is adopted: each participating unit is assigned a voting weight according to task relevance and execution capability, and votes on the execution order and division of responsibilities, with the final solution determined by the number of votes.

[0078] 4) Result Confirmation: The high-level decision-making model summarizes and confirms the negotiation results, and generates unified collaborative execution instructions.

[0079] 5) Execution binding: The negotiation results are synchronized to the blackboard, and the relevant decision-making models execute tasks in the agreed order and according to their responsibilities.

[0080] In one embodiment, the exception handling mechanism refers to handling abnormal situations (such as task failure, insufficient resources, sudden environmental changes, etc.) encountered during the execution of the behavior tree through an exception handling protocol. Exception handling includes steps such as error detection, cause analysis, recovery strategy selection, and execution adjustment. The exception handling protocol includes the following steps: 1) Anomaly detection: The first and second level decision models monitor the execution status in real time. By comparing the expected results with the blackboard data, problems such as task failure, insufficient resources, communication anomalies, and sudden environmental changes are identified.

[0081] 2) Anomaly reporting: Using an event-driven approach, the anomaly type, location, and scope of impact are immediately reported to the superior decision-making model and updated to the blackboard simultaneously.

[0082] 3) Anomaly classification and localization: The high-level decision-making model classifies anomalies into local anomalies, hierarchical anomalies, and global anomalies, and analyzes the root causes by combining historical data with the current situation.

[0083] 4) Recovery strategy generation: Generate corresponding handling strategies based on the anomaly level and cause, such as local retry, resource reallocation, task adjustment, global planning recalculation, and degraded execution.

[0084] 5) Strategy issuance and execution: The recovery command is issued to the corresponding nodes level by level, and the relevant decision model adjusts the execution logic according to the strategy and attempts to recover.

[0085] 6) Results closed loop: Continuously monitor the abnormal recovery status. If the recovery is successful, return to the normal process; if the abnormality cannot be resolved, continue to report to the superior and initiate a higher-level emergency decision.

[0086] Other embodiments of the present invention: 1. Alternatives to Behavior Tree Structures. While this invention primarily describes the standard Behavior Tree (BT) structure, in practical implementations, some control nodes (such as selection nodes and sequence nodes) can be replaced with **Hierarchical Finite State Machine (HFSM) or Goal-Oriented Action Plan (GOAP)** modules. For example, in the "Action Layer," a simple HFSM can be used to manage transient transitions in actions (such as start-accelerate-stop), while the upper layers still retain the behavior tree for decision-making. As long as the overall architecture retains the core ideas of "hierarchical control" and "modular decision-making," it falls within the scope of this patent.

[0087] 2. Alternatives to the Blackboard Communication Mechanism. The Blackboard mechanism can be replaced by a publish / subscribe message bus (such as DDS, MQTT) or distributed shared memory. If an explicit "blackboard" key-value pair storage is not used, but instead a point-to-point message broadcasting network is established between agents to achieve real-time data sharing and parameter updates, the essence is still to achieve "decoupling of logic and data" and "dynamic parameter referencing," which should be considered an equivalent alternative to this solution.

[0088] 3. Alternatives to Resource Allocation Algorithms. The auction-based negotiation decision-making and utility-based task algorithms mentioned in this invention can be replaced by schedulers based on deep reinforcement learning (DRL) or greedy algorithms. Even if the allocation algorithm changes from explicit formula calculation (utility function) to implicit neural network reasoning, as long as its function remains to distribute tasks at the "decision planning layer" and execute at the "combinatorial behavior layer," and serves the purpose of multi-agent collaboration, it should fall within the scope of protection.

[0089] 4. Alternatives to the hierarchical definition. The decision model layer of this invention defines a three-level architecture: "Level 1 Decision Model, Level 2 Decision Model, and Level 3 Decision Model." In other embodiments, it can also be modified into a two-level architecture or an N-level recursive architecture (more granular layering). As long as a solution to the complexity of multi-agent control adopts the hierarchical encapsulation idea of ​​"lower layers providing capabilities and upper layers providing guidance logic," regardless of the specific number of layers, it is a variation of this invention.

[0090] Another embodiment of the present invention provides a multi-agent cooperative adversarial inference system based on a hierarchical behavior tree control framework, such as... Figure 4 As shown, the system includes: Behavior tree building module 21 is used to build an editable behavior tree, in which the parameters of action nodes can be edited; The intelligent decision-making model construction module 22 is used to construct multi-level intelligent decision-making models based on editable behavior trees; The collaborative adversarial simulation module 23 is used to conduct collaborative adversarial simulations of multiple agents using a multi-level intelligent decision-making model.

[0091] The above division of modules is merely illustrative. In practical applications, the functions described above can be assigned to different functional modules as needed to complete all or part of the functions described in the aforementioned method. The specific working process of each module can be referred to the corresponding process in the aforementioned method embodiments, and will not be repeated here. Each of the above modules can be implemented entirely or partially through software, hardware, or a combination thereof.

[0092] Another embodiment of the present invention provides a computer device (computer, server, smartphone, etc.) including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing steps of the method of the present invention.

[0093] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk) that stores a computer program, which, when executed by a computer, implements the steps of the method of the present invention.

[0094] Another embodiment of the present invention provides a computer program product, the computer program product including a computer program, which, when executed by a computer, implements the steps of the method of the present invention.

[0095] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and to implement it accordingly. Those skilled in the art will understand that various substitutions, changes, and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the content disclosed in the embodiments of this specification; the scope of protection of the present invention is defined by the claims.

Claims

1. A multi-agent cooperative adversarial inference method based on a hierarchical behavior tree control framework, characterized in that, Includes the following steps: Construct an editable behavior tree, where the parameters of action nodes can be edited; A multi-level intelligent decision-making model is built based on editable behavior trees; Multi-level intelligent decision-making models are used to conduct collaborative adversarial simulations of multiple agents.

2. The method according to claim 1, characterized in that, The editable behavior tree includes a root node, control nodes, action nodes, and condition nodes; control nodes are used to manage the execution logic of child nodes, including sequential nodes, selection nodes, and parallel nodes; condition nodes are used to control the execution conditions of child nodes. Action nodes are responsible for executing specific actions; each node returns one of three states after execution: runnable, not runnable, or running; each node is displayed and categorized in an icon format, and various types of nodes can be combined by dragging and dropping.

3. The method according to claim 1, characterized in that, The parameters of the action node are edited through a front-end visual editing layer, an intermediate parameter processing layer, and a back-end execution layer.

4. The method according to claim 3, characterized in that, The parameters of the action node include static parameters, blackboard data reference parameters, and expression parameters; the blackboard data reference parameters are dynamically updated according to changes in the blackboard data, and the functions of the blackboard data management mechanism include: data access interface, scope management, change monitoring mechanism, type checking and conversion.

5. The method according to claim 1, characterized in that, The multi-level intelligent decision-making model includes a first-level decision-making model, a second-level decision-making model, and a third-level decision-making model; the first-level decision-making model includes at least one action node, and each action node completes a specific atomic action; The second-level decision model contains at least one control node and at least one action node, which combines the atomic actions of the first-level decision model into behaviors with specific objectives. The three-level decision model includes at least one condition node, at least one control node, and at least one action node. Based on the environmental situation, it selects or switches the combined behavior of the two-level decision model to determine the overall policy direction of the agent.

6. The method according to claim 1, characterized in that, The multi-level intelligent decision-making model employs a task allocation mechanism, a status reporting mechanism, a coordination decision-making mechanism, and an exception handling mechanism for communication and task scheduling.

7. The method according to claim 1, characterized in that, The environmental data layer provides situational information, database data, and model use case library as basic inputs, while the user interaction layer supports the creation of new models, node selection, node parameter setting, and viewing of model use cases, forming a closed-loop process of "data input - execution scheduling - decision modeling - user interaction".

8. A multi-agent cooperative adversarial inference system based on a hierarchical behavior tree control framework, characterized in that, include: The behavior tree building module is used to build editable behavior trees, where the parameters of action nodes can be edited; The intelligent decision-making model building module is used to build multi-level intelligent decision-making models based on editable behavior trees; The collaborative adversarial simulation module is used to conduct collaborative adversarial simulations of multiple agents using a multi-level intelligent decision-making model.

9. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the method according to any one of claims 1 to 7.

Images