An unmanned aerial vehicle cluster autonomous reasoning method and system based on a lightweight actor-critic architecture

Abstract

The application provides an unmanned aerial vehicle cluster autonomous reasoning method and system based on a lightweight Actor-Critic architecture. The method uses a decision tree strategy network as an Actor module to make strategy decisions, uses a value evaluation module based on a finite state machine as a Critic to perform value estimation and TD(0) updating, and introduces a lightweight consistency coordination module to ensure resource and behavior consistency of multi-unmanned aerial vehicle cooperative action through a reserved-rollback transaction process. The application replaces a complex deep neural network with a decision tree and a finite state machine, overcomes defects of a large model, high reasoning delay and difficulty in deployment on an onboard embedded device of an existing deep reinforcement learning algorithm, solves a problem of easy failure of cluster cooperation in an unstable communication environment through a distributed transaction mechanism, and finally realizes low-delay, high-consistency and strong-robustness autonomous reasoning of the unmanned aerial vehicle cluster under a resource-limited condition.

Description

Technical Field

[0001] This invention relates to the field of autonomous decision-making technology for unmanned swarms, and in particular to an autonomous reasoning method and system for unmanned swarms based on a lightweight Actor-Critic architecture. Background Technology

[0002] Currently, drone swarms face multiple challenges in autonomous decision-making, including high real-time requirements, limited communication, and resource constraints. Drone swarms are commonly used for surveillance, search and rescue, or formation operations in dynamic environments, requiring each drone to perceive the environment in real time and make rapid decisions. However, due to the limited bandwidth and susceptibility to interference in wireless communication between drones, centralized decision-making models struggle to meet real-time requirements: over-reliance on a central node or frequent communication not only increases network burden but may also lead to missed opportunities due to communication delays or interruptions. Furthermore, each drone has limited onboard computing resources and power capacity, making it difficult to run overly complex algorithms efficiently locally. This places higher demands on the reliability and robustness of swarm-based autonomous decision-making.

[0003] In existing technologies, autonomous decision-making schemes for drone swarms are shown in the table below: UAV strategy optimization PPO / DDPG equal-depth RL Relying on a large-scale ActorCritic deep learning network, edge inference requires GPU memory. Exceeding the onboard computing power / memory of a small UAV; communication jitter makes it difficult for the strategy to converge. Cluster Consistency Raft / Paxos consensus protocol Leader election + log replication ensures strong consistency Requires a stable, low-latency network; coarse transaction granularity. Microservice compensation transactions Saga Pattern Subtransaction failed → Rollback with reverse compensation Not customized for UAV scenarios; no task resource coupling. In addition, recent works such as Pang (2025)’s hierarchical MAPPO introduce a LeaderFollower mechanism in air combat decision-making, but still rely on a >20 MB LSTMActor network and do not handle weapon inventory consistency, and cannot guarantee that resources will not conflict when cooperating across clusters.

[0004] The aforementioned prior art has the following problems: The model is too large and the inference latency is high: the existing deep web ActorCritic cannot complete inference within ≤10 ms on a single SoC.

[0005] Collaboration fails when nodes lose connection: The lack of a fine-grained compensation mechanism and link jitter leading to inconsistent inventory counts result in task failure. The absence of a unified resource consistency maintenance mechanism during multi-drone collaboration makes it prone to inconsistencies or conflicts in decision-making during network interference or single-point failures. Strong consistency protocols have high communication volumes: Raft / Paxos is prone to timeouts under interference-resistant channels and cannot maintain cross-cluster resource non-conflict. Summary of the Invention

[0006] The technical problem to be solved by this invention is how to improve the reliability and robustness of autonomous decision-making in UAV swarms. This invention proposes an autonomous reasoning method, system and medium for UAV swarms based on a lightweight Actor-Critic architecture.

[0007] The autonomous inference method for UAV swarms based on a lightweight Actor-Critic architecture according to embodiments of the present invention includes: State awareness steps: The drone acquires environmental and its own state information through onboard sensors; Strategy decision-making steps: Input the state information into the decision tree policy network to obtain the action probability distribution, and select the action to be executed based on the action probability distribution; wherein, the nodes of the decision tree policy network are split based on the information gain criterion; Value assessment steps: The value of the current state of the UAV is estimated by the value assessment module based on the finite state machine, and after the action is executed, the value estimate of the current state is updated by the temporal difference TD(0) algorithm based on the immediate reward and the next state. Consistency Coordination Steps: When the selected action is a collaborative action requiring multi-drone coordination, a consistency transaction process is triggered, which includes: Reservation Phase: All drone nodes participating in the collaborative action attempt to reserve the required resources; Judgment phase: If all nodes are successfully reserved, the process proceeds to the commit phase; if any node fails to reserve or times out, the process proceeds to the rollback phase. Submission phase: All nodes synchronously execute the aforementioned collaborative actions and formally occupy the resources; Rollback phase: All nodes that have been successfully reserved perform compensation operations to release the reserved resources.

[0008] According to some embodiments of the present invention, in the policy decision-making step, the leaf nodes of the decision tree policy network store the action probability distribution, and update the action probability distribution based on the dominance function; wherein, when the dominance function value is positive, the probability of the corresponding action is increased, and when the dominance function value is negative, the probability of the corresponding action is decreased.

[0009] In some embodiments of the present invention, in the value assessment step, the value assessment module predefines multiple task states, including various ones such as search, approach, execution, and return; the value assessment module manages the current task state of the UAV and performs state transition when preset trigger conditions are met.

[0010] According to some embodiments of the present invention, in the value assessment step, the update formula of the TD(0) algorithm is: V(s) ← V(s) + α [r + γV(s') - V(s)]; Where V(s) is the value of the current state, V(s') is the value of the next state, r is the immediate reward, α is the learning rate, and γ is the discount factor.

[0011] In some embodiments of the present invention, the decision tree policy network performs incremental updates or pruning operations based on online collected experience data.

[0012] An autonomous inference system for drone swarms based on a lightweight Actor-Critic architecture, according to an embodiment of the present invention, wherein the system is deployed on a drone, and the system includes: The state awareness module is used to acquire environmental and self-state information through airborne sensors; The strategy decision module is used to input the state information into the decision tree strategy network to obtain the action probability distribution, and select the action to be executed based on the action probability distribution; wherein, the nodes of the decision tree strategy network are split based on the information gain criterion; The value assessment module is used to estimate the value of the current state of the UAV based on a finite state machine, and after the action is executed, it updates the value estimate of the current state using the temporal difference TD(0) algorithm based on the immediate reward and the next state. The consistency coordination module is used to trigger a consistency transaction process when the selected action is a collaborative action requiring multi-drone coordination. The consistency coordination module specifically includes: The reserved unit is used to enable all drone nodes participating in the collaborative action to attempt to reserve the required resources; The judgment unit is used to trigger the commit unit when it is determined that all nodes have been successfully reserved; and to trigger the rollback unit when it is determined that any node has failed to be reserved or has timed out. The submission unit is used to enable all nodes to synchronously execute the collaborative action and formally occupy the resources. The rollback unit is used to enable all nodes that have successfully reserved resources to perform compensation operations and release the reserved resources.

[0013] According to some embodiments of the present invention, the leaf nodes of the decision tree policy network in the policy decision module store the action probability distribution and are configured to update the action probability distribution based on the advantage function; wherein, when the advantage function value is positive, the probability of the corresponding action is increased, and when the advantage function value is negative, the probability of the corresponding action is decreased.

[0014] In some embodiments of the present invention, the value assessment module predefines multiple task states, including various types such as search, approach, execution, and return; the value assessment module is also used to manage the current task state of the UAV and to perform state transitions when preset trigger conditions are met.

[0015] According to an embodiment of the present invention, the electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the autonomous inference method for UAV swarm based on a lightweight Actor-Critic architecture as described above.

[0016] According to an embodiment of the present invention, a computer storage medium stores a computer program, which, when executed by a processor, implements the steps of the autonomous inference method for unmanned aerial vehicle swarms based on a lightweight Actor-Critic architecture as described above.

[0017] The present invention has the following beneficial effects: Deployment Feasibility: The model is small in size, with the decision tree and state value table requiring minimal storage and low inference computational complexity. A single drone can run the algorithm independently on its onboard embedded processor, eliminating the need for high-performance hardware such as GPUs, thus facilitating large-scale cluster deployment.

[0018] Low energy consumption and communication overhead: By eliminating complex neural network forward computation, the method of this invention significantly reduces the computational energy consumption of a single decision. Simultaneously, by employing a decentralized autonomous decision-making strategy, each UAV primarily relies on local perception and only engages in brief communication during necessary coordination phases. This significantly reduces the communication load and energy consumption of the swarm system, making it more suitable for application scenarios with unstable communication and limited energy resources.

[0019] Low decision response latency: This invention uses simple logical judgments (decision tree lookup and table update) instead of deep network inference, resulting in a shorter decision feedback path and significantly improved real-time performance. Compared to methods requiring cloud computing or complex planning, each UAV can complete the perception and decision-making closed loop in milliseconds, thereby greatly reducing decision latency and meeting the real-time response requirements in highly dynamic environments.

[0020] Strong Coordination and Consistency: Through the Saga transaction mechanism, this invention effectively avoids decision-making disagreements and resource conflicts in multi-UAV collaboration. When any node fails or experiences communication anomalies, the transaction compensation mechanism ensures that other nodes promptly suspend related actions, enabling the entire cluster to either consistently execute successfully or consistently and safely abort, greatly improving system consistency and reliability.

[0021] Explainability and Auditability: The method of this invention consists of transparent modules such as decision trees and finite state machines, and each step of the decision-making process is traceable, making it easy for humans to understand and verify. This good explainability makes the system's decision logic open to both developers and regulators, allowing for security assessment through review of rules and parameters. Compared to the "black box" deep neural network strategy, it is more audit-friendly and has greater security controllability.

[0022] Modular Scalability: The loosely coupled design of the three main modules—Actor, Critic, and Consistency Mechanism—allows for independent improvements or replacements based on different task requirements. For example, the decision tree depth or node partitioning criteria can be adjusted according to specific scenarios, the FSM state set can be expanded, or new compensation strategies can be introduced without affecting other parts of the overall architecture. This modular design improves the system's flexibility and scalability, facilitating future secondary development and functional enhancements for new unmanned cluster applications. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of an autonomous inference system architecture for a drone swarm based on a lightweight Actor-Critic architecture, according to an embodiment of the present invention. Figure 2 This is a flowchart of decision-making processes according to an embodiment of the present invention.

[0024] Figure 3 This is a state machine diagram of an FSM according to an embodiment of the present invention. Detailed Implementation

[0025] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments.

[0026] The steps described in the specification and the flowcharts in the accompanying drawings of this invention are not necessarily to be strictly followed according to the step numbers; the execution order of the steps can be changed. Furthermore, certain steps can be omitted, multiple steps can be combined into one step, and / or one step can be broken down into multiple steps.

[0027] To facilitate understanding of the present invention, the relevant technology is explained as follows: 1. Deep Reinforcement Learning (Deep RL) ActorCritic Framework: Represented by PPO, it outputs the probability distribution of actions through "Actor" and estimates the state value through "Critic". It has been verified to be superior to manual rule-based policies in scenarios such as UAV air combat simulation and task allocation.

[0028] In recent years, Actor-Critic methods based on deep reinforcement learning (such as the Proximal Policy Optimization (PPO) algorithm) have shown certain advantages in UAV swarm decision-making. These methods leverage deep neural networks to extract high-dimensional environmental features and can autonomously learn near-optimal policies through interaction with the environment, achieving better performance than manually designed algorithms in complex dynamic tasks. In particular, the PPO algorithm, as an advanced Actor-Critic variant, ensures training stability by limiting the policy update magnitude.

[0029] However, existing deep reinforcement learning methods also suffer from bottlenecks such as model complexity and difficulties in training and deployment. On the one hand, their policy and value networks often contain a large number of parameters, placing a heavy burden on airborne processors. On the other hand, such algorithms typically require a large amount of interactive data and long periods of offline training during the training phase, resulting in slow convergence. For drone swarms with frequently changing missions, long-term model training before each mission is impractical. Furthermore, the lack of transparency in deep neural network models makes the decision-making process difficult to interpret, which is detrimental to safety supervision.

[0030] 2. Decision Tree (DT): Features are recursively partitioned using information gain or Gini index to generate ifthen rule chains. It has low inference overhead, interpretable structure, and can complete inference within KB-level storage and µs-level computation.

[0031] Compared to learning-based algorithms, traditional rule-based decision-making methods still play a crucial role in unmanned systems. For example, decision tree algorithms recursively split features based on metrics such as information gain to construct a decision model based on if-then rules. Decision tree models are intuitive and interpretable, with the reasoning process involving only simple comparison operations, offering advantages such as good real-time performance and ease of implementation.

[0032] 3. Finite State Machine (FSM): It describes the sequence of system behavior using an explicit set of states and transition conditions. It is often used for UAV obstacle avoidance and task flow orchestration, and has the advantages of clear logic and strong real-time performance.

[0033] 4. Saga Transaction Pattern: A compensatory distributed transaction in a microservice architecture. It breaks down long transactions into local sub-transactions plus compensatory actions. If a sub-transaction fails, the compensatory actions are executed in reverse order to ensure eventual consistency. In the field of distributed transaction consistency, the Saga transaction mechanism is commonly used in microservice architectures to ensure consistency across nodes. The Saga pattern breaks down a long transaction into a series of sequentially executable local sub-transactions. Each sub-transaction triggers the next step via an event upon completion. If a step fails, compensatory actions are executed sequentially to undo previously completed sub-transactions, thus restoring the entire system to a consistent state. This compensatory transaction mechanism maintains data consistency across multiple nodes without requiring traditional distributed locks or two-phase commit protocols.

[0034] While the Saga pattern is primarily used in IT systems such as financial transactions and order processing, its principles are equally relevant to resource scheduling in drone swarms. In collaborative drone missions, different nodes need to maintain consistency in their use of shared resources (such as frequency bands, target tasks, and spatial locations); otherwise, conflicts or resource contention may occur. Introducing a "reservation-rollback" strategy similar to Saga—where resources are reserved beforehand when multiple drones collaborate on a task, and a rollback is implemented to compensate for any failure of a node—could potentially ensure cross-node resource consistency and improve the cluster's fault tolerance.

[0035] This invention provides an autonomous reasoning method for drone swarms based on a lightweight Actor-Critic architecture, called the Surrogate Actor-Critic (SAC-Lite) architecture. It replaces the deep network with a decision tree Actor + FSMCritic architecture, while maintaining the PPO strategy improvement approach. The SagaLite design reserves rollback transactions to ensure inventory consistency even when the link is unstable. This enables the swarm to achieve minimal decision latency and a transaction success rate of ≥98% in resource-constrained environments.

[0036] Figure 1 The diagram shows the overall architecture of SAC-Lite, which includes three core modules: the Actor strategy decision-making module, the Critic value assessment module, and the consistency coordination module (Saga transaction mechanism). Figure 1 The document demonstrates the three modules: Actor, Critic, and Saga-Lite, along with the data flow. The functions and implementation schemes of each module are as follows: The Actor module (a lightweight decision tree policy network based on information gain and advantage function) implements the UAV's policy decision-making using a decision tree structure. This module takes environmental state features as input and constructs a lightweight decision tree through recursive partitioning. The node splitting criterion selects the most discriminative features based on maximizing information gain. It also incorporates the advantage function evaluation from reinforcement learning to improve policy update efficiency. When the Critic module evaluates a state-action pair to have a positive advantage value, the corresponding leaf node of the Actor module increases the probability of selecting that action; otherwise, it decreases it.

[0037] This decision tree strategy network, based on information gain and advantage value, has a small number of parameters and an intuitive structure, allowing inference to run on embedded devices with only a few kilobytes of memory. The Actor module outputs the probability distribution corresponding to each candidate action, realizing probabilistic decision-making in the discrete action space, thus providing a basis for the UAV to select the next action.

[0038] Figure 2The flowchart for decision-making processes illustrates the complete sequence of Reserve → Commit / Rollback.

[0039] The Critic module (value estimation and state transition based on finite state machine): uses a finite state machine (FSM) to describe the state and evaluate the value of the UAV decision-making process. First, several finite states are predefined based on the UAV mission flow and environmental conditions, such as target search, approach formation, mission execution, obstacle avoidance, and return to base. Each state corresponds to a specific behavioral strategy and resource allocation. The Critic module tracks and manages the current FSM state, executing state transitions when trigger conditions are met to reflect the stage evolution of the decision-making process. For each state, Critic maintains a value estimation table, storing the value of that state (such as the expected long-term cumulative reward). After the UAV performs an action and receives reward feedback, Critic updates the state value table using the "Temporal Difference TD(0)" algorithm: that is, according to the formula... The value estimate of the current state is revised, where r is the immediate reward. This represents the new state after the action. Due to the use of tables for storage and updating, the Critic module has low computational cost and is easy to implement, eliminating the need to train complex value neural networks. Furthermore, the introduction of FSM makes the decision-making logic transparent and controllable. Developers can incorporate prior knowledge by adjusting state definitions and transition conditions to ensure the system's reliability and safety under known scenarios.

[0040] Figure 3 The FSM state machine diagram shows the transition relationships from Idle to Monitor, Aggregate, Engage, and Disperse, along with an example of reward labeling.

[0041] Consistency Coordination Module (Reservation / Compensation Consistency Strategy Based on Saga Mechanism): In multi-UAV collaborative decision-making, this invention ensures cross-node resource scheduling and behavior consistency by introducing the Saga transaction mechanism.

[0042] Specifically, when the Actor module decides to execute a collaborative action requiring multiple drones or an action that occupies shared resources, the system first enters the Saga reservation phase: Each participating node reserves the necessary resources or execution qualifications, such as communication channels and task allocation slots, and communicates the reservation results via message exchange. If all involved nodes successfully reserve, the system enters the execution phase: each drone synchronously executes the collaborative action and formally occupies the resources; at this point, the Saga transaction is successfully committed, and the cluster status remains consistent. If any node fails to confirm successful reservation during the reservation phase due to a fault or communication failure, a rollback compensation process is triggered: all nodes that have completed reservations perform compensation operations, releasing previously reserved resources and terminating the collaborative action attempt, thereby restoring the system to its consistent state before execution.

[0043] Through the aforementioned reservation-rollback mechanism, this invention can promptly correct inconsistencies caused by partial execution in the event of network interference or single node failure, avoiding resource conflicts and task disorder, and significantly improving the consistency and fault tolerance of cluster decision-making.

[0044] The modules described above can operate independently, yet they also interact through clearly defined interfaces to form a closed-loop autonomous decision-making process. Specifically, the decision-making process is as follows: S0, Initialization → S1, State awareness and discretization → S2, FSM-Critic TD(0) update and state transition → S3, Actor decision tree reasoning and (optional) incremental update / pruning based on ΔH and A → S4, If it is a cooperative behavior, execute Saga-Lite reservation-commit-execution / compensation → S5, Write back (s, a, r, s, A) to the buffer and enter the next loop.

[0045] State awareness: Each UAV acquires its own state and surrounding environment information through onboard sensors and internally maps it to a predefined FSM state. If necessary, UAVs can exchange a small amount of key information (such as neighboring UAV status and intent) to enhance environmental awareness, but this communication does not rely on central node control.

[0046] Strategic Decision-Making: The UAV inputs its current state into the decision tree policy network of the Actor module, calculates the probability distribution of executable actions, and selects an action plan accordingly. For tasks with high deterministic requirements, the action with the highest probability can be selected; for tasks with high exploration requirements, actions can be sampled according to probability, thereby balancing exploration and utilization.

[0047] Consistency Negotiation: If the selected action is an independent action of the local machine (not affecting others), it directly enters the execution phase; if the action involves the coordination of multiple drones (such as formation adjustment, task relay, etc.), all relevant nodes initiate the reservation phase of the Saga consistency mechanism. Each node attempts to reserve the resources or permissions required to execute the action and provides feedback on the reservation results within a limited time.

[0048] Transaction Commit or Rollback: If all participating drones successfully reserve resources (meaning everyone has the resources to execute the action), the collaborative action is executed uniformly, and the Saga commit phase is initiated to confirm resource allocation. Conversely, if any node fails to reserve resources or times out without responding, the cluster immediately abandons the action and triggers a rollback compensation—all nodes that have reserved resources cancel their previous resource reservation operations, ensuring the entire cluster maintains a consistent initial state. This "execute if all succeed, otherwise cancel" strategy ensures the cluster either unanimously completes the action or unanimously abandons it, avoiding the uncertainty caused by some nodes executing while others fail.

[0049] Execution Feedback: After an action is successfully executed, each UAV obtains feedback information from the environment, such as mission score, changes in energy consumption, and changes in position, and calculates an immediate reward value accordingly. Subsequently, the Critic module of each node updates the value of the old state involved in this decision using TD(0) based on the reward and the next state, improving its adaptability to environmental dynamics; at the same time, it updates the internal state representation according to the FSM state transition rules (e.g., the mission changes from "search" to "track").

[0050] Iterative Loop: The drone swarm repeats the perception-decision-execution-learning process described above, continuously reasoning and optimizing its decision-making strategies to form a closed-loop online learning and reasoning system until the mission ends or a predetermined stopping condition is met. Throughout the process, each drone primarily relies on its local lightweight model for decision-making and reasoning, communicating only briefly in necessary collaborative steps. This ensures both real-time response for individual drones and maintains consistency in multi-drone collaboration.

[0051] Through the above technical solution, this invention realizes a lightweight Actor-Critic decision-making architecture capable of operating independently under limited computing power conditions for UAVs. Each UAV utilizes a small-scale model to achieve autonomous decision-making, and a transaction mechanism ensures overall cluster coordination when needed. On the one hand, by leveraging the structured strategies of decision trees and finite state machines, the method of this invention possesses good interpretability and modularity. On the other hand, by combining the value assessment and policy optimization mechanisms of reinforcement learning, the system has online self-improvement capabilities, thus balancing the reliability of rule-based methods with the adaptability of learning methods.

[0052] In summary, this invention has made at least the following technical innovations: A lightweight decision tree policy network construction and update method is proposed: a method that uses decision trees to represent policies in the Actor module is proposed. The state feature space is divided by information gain criterion, and the policy probability of leaf nodes is adjusted in real time by combining advantage function feedback, realizing efficient policy inference and update with only KB-level memory.

[0053] FSM-Critic Design and its Integration with TD(0) Table Updates: A finite state machine is creatively introduced into the Critic value assessment module. A finite set of states is used to describe the stages of a UAV mission, and the TD(0) algorithm is used to update the value of each state in a tabular format. This design effectively integrates rule control and reinforcement learning, ensuring both the controllability and reliability of the value assessment process and endowing the system with the ability to learn and optimize online.

[0054] Saga Transaction Consistency Strategy for Drone Swarms: This strategy innovatively applies the Saga distributed transaction mechanism to collaborative decision-making in drone swarms, introducing a reservation / compensation-based resource consistency scheduling strategy. When multiple drones need to coordinate their actions, a scheme combining reservation confirmation and failure rollback compensation is adopted to ensure consistency in resource allocation and behavioral decisions across nodes, thereby improving the robustness and fault tolerance of swarm decision-making.

[0055] The three elements combine to form a lightweight Surrogate Actor-Critic closed loop.

[0056] The present invention has the following beneficial effects: Deployment Feasibility: The model is small in size, with the decision tree and state value table requiring minimal storage and low inference computational complexity. A single drone can run the algorithm independently on its onboard embedded processor, eliminating the need for high-performance hardware such as GPUs, thus facilitating large-scale cluster deployment.

[0057] Low energy consumption and communication overhead: By eliminating complex neural network forward computation, the method of this invention significantly reduces the computational energy consumption of a single decision. Simultaneously, by employing a decentralized autonomous decision-making strategy, each UAV primarily relies on local perception and only engages in brief communication during necessary coordination phases. This significantly reduces the communication load and energy consumption of the swarm system, making it more suitable for application scenarios with unstable communication and limited energy resources.

[0058] Low decision response latency: This invention uses simple logical judgments (decision tree lookup and table update) instead of deep network inference, resulting in a shorter decision feedback path and significantly improved real-time performance. Compared to methods requiring cloud computing or complex planning, each UAV can complete the perception and decision-making closed loop in milliseconds, thereby greatly reducing decision latency and meeting the real-time response requirements in highly dynamic environments.

[0059] Strong Coordination and Consistency: Through the Saga transaction mechanism, this invention effectively avoids decision-making disagreements and resource conflicts in multi-UAV collaboration. When any node fails or experiences communication anomalies, the transaction compensation mechanism ensures that other nodes promptly suspend related actions, enabling the entire cluster to either consistently execute successfully or consistently and safely abort, greatly improving system consistency and reliability.

[0060] Explainability and Auditability: The method of this invention consists of transparent modules such as decision trees and finite state machines, and each step of the decision-making process is traceable, making it easy for humans to understand and verify. This good explainability makes the system's decision logic open to both developers and regulators, allowing for security assessment through review of rules and parameters. Compared to the "black box" deep neural network strategy, it is more audit-friendly and has greater security controllability.

[0061] Modular Scalability: The loosely coupled design of the three main modules—Actor, Critic, and Consistency Mechanism—allows for independent improvements or replacements based on different task requirements. For example, the decision tree depth or node partitioning criteria can be adjusted according to specific scenarios, the FSM state set can be expanded, or new compensation strategies can be introduced without affecting other parts of the overall architecture. This modular design improves the system's flexibility and scalability, facilitating future secondary development and functional enhancements for new unmanned cluster applications.

[0062] Through the description of specific embodiments, a more in-depth and specific understanding should be gained of the technical means and effects adopted by the present invention to achieve the intended purpose. However, the accompanying drawings are only provided for reference and illustration and are not intended to limit the present invention.

Claims

1. A method for autonomous inference in UAV swarms based on a lightweight Actor-Critic architecture, characterized in that, include: State awareness steps: The UAV acquires environmental and its own state information through onboard sensors and internally maps it to a predefined finite state machine (FSM) state; Strategy decision-making steps: The state information is input into a decision tree policy network to obtain an action probability distribution, and an action is selected to be executed based on the action probability distribution; wherein, the nodes of the decision tree policy network are split based on the information gain criterion; in the strategy decision-making steps, the leaf nodes of the decision tree policy network store the action probability distribution, and the action probability distribution is updated based on the dominance function; wherein, when the dominance function value is positive, the probability of the corresponding action is increased, and when the dominance function value is negative, the probability of the corresponding action is decreased; Value assessment steps: The value of the current state of the UAV is estimated by a value assessment module based on a finite state machine. After the action is executed, the value estimate of the current state is updated using the temporal difference TD(0) algorithm based on the immediate reward and the next state. In the value assessment steps, the value assessment module predefines multiple task states, including various types such as search, approach, execution, and return. The value assessment module manages the current task state of the UAV and performs state transitions when preset trigger conditions are met. Specifically, a finite state machine is used to describe the state and assess the value of the UAV decision-making process. First, several finite states are predefined according to the UAV task flow and environmental conditions. Each state corresponds to a certain behavior strategy and resource configuration. The current FSM state is tracked and managed. When the trigger conditions are met, a state transition is performed to reflect the stage evolution of the decision-making process. For each state, a value estimation table is maintained to store the value of the state. After the UAV performs an action and receives reward feedback, the state value is updated in the table using the temporal difference TD(0) algorithm. Consistency Coordination Steps: When the selected action is a collaborative action requiring multi-drone coordination, a consistency transaction process is triggered, which includes: Reservation Phase: All drone nodes participating in the collaborative action attempt to reserve the required resources; Judgment phase: If all nodes are successfully reserved, the process proceeds to the commit phase; if any node fails to reserve or times out, the process proceeds to the rollback phase. Submission phase: All nodes synchronously execute the aforementioned collaborative actions and formally occupy the resources; Rollback phase: All nodes that have been successfully reserved perform compensation operations, release the reserved resources and abort collaborative action attempts, so that the system is restored to the consistent state before execution.

2. The autonomous inference method for UAV swarms based on a lightweight Actor-Critic architecture as described in claim 1, characterized in that, In the value assessment step, the update formula of the TD(0) algorithm is: V(s) ← V(s) + α [r + γV(s') - V(s)]; Where V(s) is the value of the current state, V(s') is the value of the next state, r is the immediate reward, α is the learning rate, and γ is the discount factor.

3. The autonomous inference method for UAV swarms based on a lightweight Actor-Critic architecture according to claim 1, characterized in that, The decision tree strategy network performs incremental updates or pruning operations based on online collected experience data.

4. A drone swarm autonomous inference system based on a lightweight Actor-Critic architecture, the system being used to implement the drone swarm autonomous inference method based on a lightweight Actor-Critic architecture as described in any one of claims 1-3, characterized in that, The system is deployed on a drone, and the system includes: The state awareness module is used to acquire environmental and self-state information through airborne sensors; The strategy decision module is used to input the state information into the decision tree strategy network to obtain the action probability distribution, and select the action to be executed based on the action probability distribution; wherein, the nodes of the decision tree strategy network are split based on the information gain criterion; The value assessment module is used to estimate the value of the current state of the UAV based on a finite state machine, and after the action is executed, it updates the value estimate of the current state using the temporal difference TD(0) algorithm based on the immediate reward and the next state. The consistency coordination module is used to trigger a consistency transaction process when the selected action is a collaborative action requiring multi-drone coordination. The consistency coordination module specifically includes: The reserved unit is used to enable all drone nodes participating in the collaborative action to attempt to reserve the required resources; The judgment unit is used to trigger the commit unit when it is determined that all nodes have been successfully reserved; and to trigger the rollback unit when it is determined that any node has failed to be reserved or has timed out. The submission unit is used to enable all nodes to synchronously execute the collaborative action and formally occupy the resources. The rollback unit is used to enable all nodes that have successfully reserved resources to perform compensation operations and release the reserved resources.

5. The UAV swarm autonomous inference system based on a lightweight Actor-Critic architecture according to claim 4, characterized in that, The leaf nodes of the decision tree policy network in the policy decision module store the action probability distribution and are configured to update the action probability distribution based on the dominance function; wherein, when the dominance function value is positive, the probability of the corresponding action is increased, and when the dominance function value is negative, the probability of the corresponding action is decreased.

6. The UAV swarm autonomous inference system based on a lightweight Actor-Critic architecture according to claim 4, characterized in that, The value assessment module predefines multiple task states, including various types such as search, approach, execution, and return. The value assessment module is also used to manage the current task state of the UAV and to perform state transitions when preset trigger conditions are met.

7. An electronic device, characterized in that, The electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the autonomous inference method for unmanned aerial vehicle swarms based on a lightweight Actor-Critic architecture as described in any one of claims 1 to 3.

8. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which, when executed by a processor, implements the steps of the autonomous inference method for unmanned aerial vehicle swarms based on a lightweight Actor-Critic architecture as described in any one of claims 1 to 3.

Images