A method for UAV swarm patrol path decision-making under resource constraints

Abstract

The present application discloses a method for making patrol path decisions of a swarm of unmanned aerial vehicles (UAVs) under resource constraints, which relates to the technical field of UAV path planning. The method comprises: discretizing the actual physical locations to be patrolled to construct an undirected topological graph; generating a steady-state distribution of each patrol node according to the topological constraints and the importance of the nodes; generating a plurality of transfer matrices with the same steady-state distribution but different transfer characteristics according to a multi-stage entropy-driven random matrix optimization algorithm; initializing the position of a navigator and determining the path selected by it according to the transfer matrix; implementing adaptive active positioning decisions under positioning constraints according to the navigator's reference path to ensure path tracking effects; according to the navigator's path selection and tracking, the followers form a humanoid grouping cluster with the navigator through a reward function; according to the multi-state transfer matrix and the humanoid grouping, automatically switching to the next transfer matrix when a transfer number threshold is reached, thereby realizing intelligent patrol path decisions of the UAV swarm under resource constraints.

Description

Technical Field

[0001] The present application relates to the technical field of drone path planning, and in particular to a method for deciding a patrol path for a drone cluster under resource constraints. Background Art

[0002] Existing technologies use the Metropolis-Hastings algorithm to generate the state transition matrix of a Markov chain. Research is underway on multi-constrained trajectory planning for aircraft, taking into account positioning error correction. By constructing a grid map of the unmanned equipment swarm's operating area based on map information, the A* algorithm is used for path planning, and a leader-follower model is employed for formation control to create a swarm path.

[0003] First, existing technologies are fragmented, with no connections between independent systems. Second, the state transition matrices of the Markov chains generated by existing technologies are highly self-looping, making them easily learned by potential attackers to predict patrol paths. Furthermore, while much research has been conducted on the computational aspects of multi-constrained aircraft trajectory planning, no algorithms have been specifically developed to address path selection and tracking within positioning constraints. Finally, existing leader-follower models lack a framework for randomized path selection or for making formation path decisions within resource constraints. Summary of the Invention

[0004] The main purpose of the embodiments of the present application is to propose a method for deciding the patrol path of a drone cluster under resource constraints, so as to realize the selection and tracking of the path under resource constraints.

[0005] To achieve the above objectives, an embodiment of the present application provides a method for determining a patrol path for a swarm of drones under resource constraints, the method comprising the following steps:

[0006] The actual physical location to be patrolled is discretized to construct an undirected topological graph, where the nodes represent the locations to be patrolled and the edges between the nodes represent the flight paths of the drones.

[0007] The target steady-state distribution of each patrol node is generated according to the topological constraints of the constructed undirected topological graph, the importance of the node and the attack time;

[0008] Generate multiple state transfer matrices with the same target steady-state distribution but different transfer characteristics according to a multi-stage entropy-driven random matrix optimization algorithm, each state transfer matrix represents the transfer probability between each node of the drone cluster;

[0009] Initialize the position of the UAV based on the initial state transfer matrix and determine the path chosen by the navigator;

[0010] Based on the reference path selected by the navigator, the navigator's Transformer Q network model predictive control algorithm dynamically exchanges information between the TQ active positioning decision module and the MPC trajectory tracking control module to achieve adaptive active positioning decisions while satisfying positioning constraints, ensuring tracking of the selected path.

[0011] According to the path selection and tracking of the leader, the followers maintain a set distance from the leader through a reward function to form a humanoid formation cluster of drones;

[0012] According to each of the state transfer matrices and human-like groupings, when the initial transfer matrix reaches the transfer number threshold, it automatically switches to the next transfer matrix, executes random patrol path selection and tracking, and repeats the cycle to achieve unmanned cluster intelligent patrol path decision-making under resource constraints.

[0013] In some embodiments, generating a target steady-state distribution of patrol nodes according to the topological constraints of the constructed undirected topological graph, the importance of the nodes, and the attack time includes the following steps:

[0014] Different nodes Different drones need to patrol different locations. Different nodes have different values, and attackers need different amounts of time to attack different nodes. Target nodes with high value and short attack times are selected as target nodes. Target nodes need to obtain more patrol resources, and patrol drones are more likely to visit target nodes.

[0015] The expression of the target steady-state distribution is:

[0016] ;

[0017] ;

[0018] in, , represents the i-th node, V represents the set of each of the nodes, Represents the target node The value of represents the attack time, Representative Node The importance weight parameter, Representative Node The importance weight parameter, n is the total number of nodes, Represents the target node The target steady-state distribution represents the probability distribution of the patroller at each target node after the Markov chain runs for a long time.

[0019] In some embodiments, the method of generating a plurality of state transfer matrices having the same target steady-state distribution but different transfer characteristics according to a multi-stage entropy-driven random matrix optimization algorithm, wherein each state transfer matrix represents a transfer probability between each node of the drone cluster, includes the following steps:

[0020] Phase 1: Initialization and basic optimization:

[0021] Generate a Q matrix that satisfies the topological constraints of the undirected topological graph using a bitangent mapping function;

[0022] The self-loop probability is limited to a set value, and an iterative optimization method of adaptive learning rate with exponential decay and correction factor is applied to iteratively adjust to meet the steady-state equation;

[0023] Phase 2: Entropy maximization:

[0024] Calculate the maximum possible entropy, focus on optimizing rows with low entropy ratios, and redistribute off-diagonal probabilities while maintaining self-loop constraints;

[0025] Simulate quantum superposition states and perform steady-state fine-tuning every few iterations to balance entropy optimization and steady-state constraints;

[0026] Phase 3: Diversity Enhancement:

[0027] Calculate the MAD between matrices for heterogeneity assessment;

[0028] A GAN-like mechanism is introduced to identify high-probability and low-probability transitions for each generated matrix, analyze similarities in multiple dimensions, and identify the most critical differences for enhancement.

[0029] The average usage pattern of the existing matrix is ​​analyzed to identify the critical path. The frequently used transfers are regarded as more important strategic decision points. The cascading differentiation between unrelated transfers is created through threshold trigger adjustment, random direction binary decision and probability redistribution mechanism.

[0030] Stage 4: Final fine-tuning:

[0031] Reapply constraints, balance between equilibrium and diversity, optimize multiple objective functions simultaneously, dynamically adjust entropy optimization and differentiation intensity, resolve potential conflicts between entropy maximization and diversity maximization, and find the best balance between entropy and diversity;

[0032] When all preset constraints are met, the P matrix is ​​obtained as the candidate state transfer matrix.

[0033] In some embodiments, the method further includes determining a control action based on the target state transfer matrix and the reference trajectory, comprising the following steps:

[0034] Determining a control action based on the target state transfer matrix and a reference trajectory using model predictive control;

[0035] The objective function and constraints of the model predictive control are:

[0036] ;

[0037] in, : The control input sequence of the UAV from the current time k to the end of the prediction time domain;

[0038] : The estimated value of the physical state at the i-th time step, including position and velocity; MPC uses the estimated value to plan control actions, and in the case of passive positioning, the estimated value represents the current system state;

[0039] : physical control input at the i-th time step;

[0040] : physical control input at the i-1th time step;

[0041] : The reference trajectory state at the i-th time step, indicating the target state that the system should track;

[0042] : weighted quadratic norm of the state tracking error, Q is the state error weight, which is used to assign different weights to different state components;

[0043] : The weighted quadratic norm of the control input change, R is the control change weight, which is used to penalize the sudden change of the control input;

[0044] : End of time domain The estimated value of the physical state at time steps;

[0045] :End of time domain The reference trajectory state of the time step represents the target state that the system should track;

[0046] : The weighted quadratic norm of the terminal state error in the prediction time domain, P is the terminal state weight;

[0047] : Weight coefficients of the three objectives of equilibrium state tracking, control smoothing and terminal state constraint;

[0048] : The prediction time domain length of MPC, which indicates the number of time steps that MPC predicts forward;

[0049] : The current time step of the drone;

[0050] :Prediction time domain from k arrive k+H MPC The time step index.

[0051] In some embodiments, initializing the nodes of the drone cluster, the active positioning budget constraint, and the initial state transition matrix, and determining the path selected by the leader, includes the following steps:

[0052] The initial position of the drone and the active positioning budget constraint are input into Transformer Q. A random path is selected through the transition matrix. Transformer Q is used to optimize the positioning decision and learn where to actively position the selected path trajectory.

[0053] The architecture of Transformer Q is as follows:

[0054] State-action embedding design:

[0055] For continuous state , the embedding function is: ,in, s represents the continuous state vector, The weight matrix representing the state embedding; represents the embedding dimension, Represents the state dimension, Represents the bias vector of state embedding; GELU represents the Gaussian error linear unit activation function; LayerNorm represents the layer normalization function;

[0056] State-Action Fusion:

[0057] Fuse state and action embeddings into a unified representation: ,in represents the fusion weight matrix dimension, Row 2 List, Indicates that the fusion bias vector belongs to dimensional real space, h Represents the representation vector after the state and action are embedded into each other. represents the action vector, represents the state embedding vector, represents the action embedding vector, represents vector concatenation, Indicates the concatenation of the state embedding vector and the action embedding vector;

[0058] Rotational position encoding is applied before attention calculation, and a multi-head self-attention mechanism is adopted. Each temporal gated Transformer block includes a self-attention sublayer, a gated convolution sublayer, and a feedforward network sublayer;

[0059] Meta-training Transformer Q, different trajectories and different budget combinations form a task space, each specific task is a set Yes, among them, represents a specific reference trajectory sampled from the trajectory distribution, represents the specific active positioning budget value sampled from the budget distribution, sampling different tasks in the task space to train the TD algorithm to improve the generalization ability of the trained Transformer Q;

[0060] The state update equation is as follows:

[0061] ;

[0062] in, : The estimated value of the physical state at the next moment k+1;

[0063] : The real physical state value at the next moment k+1;

[0064] : Information reliability attenuation factor, reflecting the attenuation of sensor information quality with state estimation error;

[0065] : Information attenuation basic coefficient, which controls the basic strength of attenuation and has a value range of [0,1];

[0066] : Information decay rate adjustment parameter, which controls the sensitivity of decay to state error;

[0067] : estimated value of the physical state at time k;

[0068] : the real physical state value at time k;

[0069] : The square of the Euclidean distance between the current estimated state and the true state;

[0070] : System dynamics models that predict the next state based on the current state, control inputs, and random disturbances;

[0071] : physical control input at time k;

[0072] : random disturbance at time k;

[0073] : The remaining positioning budget constraint at time k, which represents the number of active positioning attempts remaining;

[0074] : Uncertainty growth function, which is related to the remaining positioning budget. The smaller the budget, the slower the uncertainty growth;

[0075] : The covariance matrix of the state estimate, representing the uncertainty of the state estimate;

[0076] : Positioning decision variable, which is also a binary decision action output by the Transformer Q network , Indicates that the patrol drone performs active positioning to obtain the real status of the system. Indicates that the patrol UAV does not perform active positioning and estimates the system state through average particle filtering.

[0077] In some embodiments, for a given patrol UAV reference trajectory and active positioning budget, the key component of TQMPC is the interaction between TQ and MPC, including the following steps:

[0078] TQ is based on status Determine the optimal positioning action ,This decision will affect the MPC’s belief state estimation and its subsequent control actions;

[0079] The trajectory deviation calculated by MPC is used to improve the TQ's decision-making process. The trajectory tracking performance is used to guide the learning of the patrol UAV's positioning strategy, encouraging the TQ to prioritize positioning decisions that can reduce trajectory deviation.

[0080] The dynamic exchange between the TQ localization strategy and the MPC control output establishes a feedback loop. By combining the budget-exceeding constraint and the trajectory tracking performance reward, the TQ learning is guided to learn to make adaptive active localization decisions while ensuring both the trajectory tracking performance and the state estimation accuracy of the patrol UAV.

[0081] The total reward function of the TQ algorithm is as follows:

[0082] ;

[0083] in, : The overall reward function of DQN, used to train the positioning decision network;

[0084] : Feedback reward obtained from MPC to evaluate the impact of localization strategy on trajectory tracking;

[0085] : The weight parameter that balances the MPC feedback and state estimation error, with a value range of [0,1];

[0086] : the real physical state value at time k;

[0087] : estimated value of the physical state at time k;

[0088] : The error between the current true state and the estimated state, using the Euclidean distance;

[0089] : The weight coefficient of the information gain term, which controls the proportion of long-term planning in the reward;

[0090] : Information gain function, which evaluates the potential value of the current positioning decision for future trajectory tracking;

[0091] : Total positioning budget, which indicates the maximum number of active positioning times allowed in the entire task;

[0092] : Remaining positioning budget, indicating the number of active positioning times currently available;

[0093] : The remaining trajectory length or remaining task complexity is used to adjust the information gain calculation;

[0094] : Large negative rewards when the budget is insufficient, forcing DQN to learn to abide by the budget constraints.

[0095] In some embodiments, the method further includes controlling each follower in the drone cluster to maintain a set distance from the corresponding leader to form a formation movement, including the following steps:

[0096] Controlling each follower in the drone cluster to maintain a set distance from the corresponding leader according to a distance-maintaining reward function to form a formation movement;

[0097] The reward function for maintaining distance is:

[0098] ;

[0099] in, Represents the reward given to the follower by the reward function; t represents the current moment, represents the distance between the follower and the leader at time t, denotes the distance between the follower and the leader at time t+1, D1 represents the lower threshold of the distance between the follower and the leader, D2 represents the upper threshold of the distance between the follower and the leader, and D1 < D2 is a positive constant. The thresholds C1 > C2 > C3 represent positive constant reward values obtained by the drone in its interaction with the environment under different conditions.

[0100] To achieve the above object, on the other hand, an embodiment of the present application proposes a drone swarm patrol path decision-making device under resource constraints, and the device includes:

[0101] A graph construction unit for discretizing the actual physical positions to be patrolled to construct an undirected topological graph, where nodes represent the positions to be patrolled, and the edges between nodes represent the flight paths of the drones;

[0102] A steady-state distribution generation unit for generating the target steady-state distribution of each patrol node according to the topological constraints of the constructed undirected topological graph, the importance degree of the nodes, and the attack time;

[0103] A transition matrix generation unit for generating multiple state transition matrices with the same target steady-state distribution but different transition characteristics according to the multi-stage entropy-driven stochastic matrix optimization algorithm, and each state transition matrix represents the transition probability of the drone swarm between each node;

[0104] A path selection unit for initializing the position of the drone according to the initial state transition matrix and determining the path selected by the leader;

[0105] A decision-making unit for, according to the reference path selected by the leader, the Transformer Q-network model prediction control algorithm of the leader realizes adaptive active positioning decision-making through the dynamic exchange of information between the TQ active positioning decision-making module and the MPC trajectory tracking control module under the condition of satisfying the positioning constraints, and ensures the tracking effect of the selected path;

[0106] A tracking unit for, according to the path selection and tracking of the leader, the follower forms a humanoid formation swarm of drones by maintaining a set distance from the leader through a reward function;

[0107] A formation control unit for, according to each state transition matrix and the humanoid formation, automatically switching to the next transition matrix when the initial transition matrix reaches the transition times threshold, performing the selection and tracking of random patrol paths, and realizing the intelligent patrol path decision-making of the unmanned swarm under resource constraints through repeated cycling.

[0108] To achieve the above object, on the other hand, an embodiment of the present application proposes an electronic device, and the electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the above method is realized.

[0109] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the above-mentioned method when executed by a processor.

[0110] The embodiments of the present application include at least the following beneficial effects:

[0111] This application discretizes the actual physical location to be patrolled to construct an undirected topological graph; generates the target steady-state distribution of each patrol node based on the topological constraints of the constructed undirected topological graph, the importance of the nodes and the attack time; generates multiple transfer matrices with the same steady-state distribution but different transfer characteristics based on the multi-stage entropy-driven random matrix optimization algorithm, which are more difficult to predict than the paths generated by the existing algorithms; initializes the position of the drone based on the initial state transfer matrix and determines the path selected by the navigator; based on the reference path selected by the navigator, the navigator's Transformer Q network model predictive control algorithm dynamically exchanges information between the TQ active positioning decision module and the MPC trajectory tracking control module to achieve adaptive active positioning decision-making under the condition of satisfying positioning constraints, ensuring the tracking of the selected path, and the tracking effect is better than the existing pure MPC The path tracking algorithm has higher accuracy; according to the navigator's path selection and tracking, the followers maintain a set distance from the navigator through a reward function to form a drone humanoid grouping cluster, thereby increasing the patrol visual range; according to each of the state transfer matrices and humanoid groupings, when the initial transfer matrix reaches the transfer number threshold, it automatically switches to the next transfer matrix, executes random patrol path selection and tracking, and repeats the cycle to realize unmanned cluster intelligent patrol path decision-making under resource constraints. BRIEF DESCRIPTION OF THE DRAWINGS

[0112] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0113] Figure 1 A flowchart of a method for determining a patrol path for a drone swarm under resource constraints provided in an embodiment of the present application;

[0114] Figure 2 An example diagram of a Markov chain provided in an embodiment of the present application;

[0115] Figure 3 Flowchart of the MEDRO algorithm provided in the embodiment of this application;

[0116] Figure 4Flowchart of the TQMPC algorithm provided in the embodiment of this application;

[0117] Figure 5 The basic configuration of the humanoid group provided in the embodiment of the present application;

[0118] Figure 6 A schematic diagram of an extended humanoid formation provided in an embodiment of the present application;

[0119] Figure 7 A schematic diagram of the structure of a UAV swarm patrol path decision-making device under resource constraints provided in an embodiment of the present application;

[0120] Figure 8 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0121] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are merely examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0122] It will be understood that the terms "first", "second", etc. used in this application may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0123] The terms "at least one", "plurality", "each", "any", etc. used in this application include "at least one", "two" or more, "plurality" or "each", "any" or "any one", "each" or "any one" as used herein.

[0124] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0125] Before describing the embodiments of the present application in detail, some of the related technologies involved in the embodiments of the present application are first described as follows:

[0126] Metropolis–Hastings algorithm: Metropolis–Hastings algorithm.

[0127] Transformer: A deep learning model building.

[0128] TQ: Transformer Q-Networks.

[0129] MPC: Model Predictive Control, model predictive control.

[0130] MEDRO: Multi-stage Entropy-Driven Random-matrix Optimization, multi-stage entropy-driven random matrix optimization.

[0131] TQMPC: Transformer Q-Networks Model Predictive Control, Transformer Q network model predictive control.

[0132] Reference Figure 1 The embodiment of the present application provides a method for determining a patrol path of a drone cluster under resource constraints. The method may include but is not limited to steps S100 to S160, as follows:

[0133] S100: Discretize the actual physical locations to be patrolled to construct an undirected topological graph, where nodes represent the locations to be patrolled and edges between nodes represent the flight paths of the drones;

[0134] S110: Generate a target steady-state distribution of each patrol node according to the topological constraints of the constructed undirected topological graph, the importance of the node, and the attack time;

[0135] S120: generating a plurality of state transfer matrices having the same target steady-state distribution but different transfer characteristics according to a multi-stage entropy-driven random matrix optimization algorithm, wherein each state transfer matrix represents a transfer probability between each node of the drone cluster;

[0136] S130: Initializing the position of the UAV according to the initial state transfer matrix and determining the path selected by the navigator;

[0137] S140: Based on the reference path selected by the navigator, the Transformer Q network model predictive control algorithm of the navigator implements adaptive active positioning decision-making under the condition of satisfying positioning constraints through dynamic information exchange between the TQ active positioning decision module and the MPC trajectory tracking control module, thereby ensuring the tracking effect of the selected path;

[0138] S150: Based on the path selection and tracking of the leader, the followers maintain a set distance from the leader through a reward function to form a drone humanoid formation cluster;

[0139] S160: According to each of the state transfer matrices and human-like groupings, when the initial transfer matrix reaches the transfer number threshold, it automatically switches to the next transfer matrix, executes random patrol path selection and tracking, and repeats the cycle to achieve unmanned cluster intelligent patrol path decision-making under resource constraints.

[0140] Optionally, generating a target steady-state distribution of each patrol node according to the topological constraints of the constructed undirected topological graph, the importance of the node, and the attack time comprises the following steps:

[0141] Different nodes Different drones need to patrol different locations. Different nodes have different values, and attackers need different amounts of time to attack different nodes. Target nodes with high value and short attack times are selected as target nodes. Target nodes need to obtain more patrol resources, and patrol drones are more likely to visit target nodes.

[0142] The expression of the target steady-state distribution is:

[0143] ;

[0144] ;

[0145] in, , represents the i-th node, V represents the set of each of the nodes, Represents the target node The value of represents the attack time, Representative Node The importance weight parameter, Representative Node The importance weight parameter, n is the total number of nodes, Represents the target node The target steady-state distribution represents the probability distribution of the patroller at each target node after the Markov chain runs for a long time.

[0146] Optionally, generating a plurality of state transfer matrices having the same target steady-state distribution but different transfer characteristics according to a multi-stage entropy-driven random matrix optimization algorithm, wherein each state transfer matrix represents a transfer probability between each node of the drone cluster, comprises the following steps:

[0147] Phase 1: Initialization and basic optimization:

[0148] Generate a Q matrix that satisfies the topological constraints of the undirected topological graph using a bitangent mapping function;

[0149] The self-loop probability is limited to a set value, and an iterative optimization method of adaptive learning rate with exponential decay and correction factor is applied to iteratively adjust to meet the steady-state equation;

[0150] Phase 2: Entropy maximization:

[0151] Calculate the maximum possible entropy, focus on optimizing rows with low entropy ratios, and redistribute off-diagonal probabilities while maintaining self-loop constraints;

[0152] Simulate quantum superposition states and perform steady-state fine-tuning every few iterations to balance entropy optimization and steady-state constraints;

[0153] Phase 3: Diversity Enhancement:

[0154] Calculate the MAD between matrices for heterogeneity assessment;

[0155] A GAN-like mechanism is introduced to identify high-probability and low-probability transitions for each generated matrix, analyze similarities in multiple dimensions, and identify the most critical differences for enhancement.

[0156] The average usage pattern of the existing matrix is ​​analyzed to identify the critical path. The frequently used transfers are regarded as more important strategic decision points. The cascading differentiation between unrelated transfers is created through threshold trigger adjustment, random direction binary decision and probability redistribution mechanism.

[0157] Stage 4: Final fine-tuning:

[0158] Reapply constraints, balance between equilibrium and diversity, optimize multiple objective functions simultaneously, dynamically adjust entropy optimization and differentiation intensity, resolve potential conflicts between entropy maximization and diversity maximization, and find the best balance between entropy and diversity;

[0159] When all preset constraints are met, the P matrix is ​​obtained as the candidate state transfer matrix.

[0160] Optionally, the method further includes determining a control action according to the target state transfer matrix and the reference trajectory, comprising the following steps:

[0161] Determining a control action based on the target state transfer matrix and a reference trajectory using model predictive control;

[0162] The objective function and constraints of the model predictive control are:

[0163] ;

[0164] in, : The control input sequence of the UAV from the current time k to the end of the prediction time domain;

[0165] : The estimated value of the physical state at the i-th time step, including position and velocity; MPC uses the estimated value to plan control actions, and in the case of passive positioning, the estimated value represents the current system state;

[0166] : physical control input at the i-th time step;

[0167] : physical control input at the i-1th time step;

[0168] : The reference trajectory state at the i-th time step, indicating the target state that the system should track;

[0169] : weighted quadratic norm of the state tracking error, Q is the state error weight, which is used to assign different weights to different state components;

[0170] : The weighted quadratic norm of the control input change, R is the control change weight, which is used to penalize the sudden change of the control input;

[0171] : End of time domain The estimated value of the physical state at time steps;

[0172] :End of time domain The reference trajectory state of the time step represents the target state that the system should track;

[0173] : The weighted quadratic norm of the terminal state error in the prediction time domain, P is the terminal state weight;

[0174] : Weight coefficients of the three objectives of equilibrium state tracking, control smoothing and terminal state constraint;

[0175] : The prediction time domain length of MPC, which indicates the number of time steps that MPC predicts forward;

[0176] : The current time step of the drone;

[0177] :Prediction time domain from k arrive k+H MPC The time step index.

[0178] Optionally, initializing the nodes of the drone cluster, the active positioning budget constraint, and the initial state transition matrix, and determining the path selected by the leader, includes the following steps:

[0179] The initial position of the drone and the active positioning budget constraint are input into Transformer Q. A random path is selected through the transition matrix. Transformer Q is used to optimize the positioning decision and learn where to actively position on the selected path trajectory.

[0180] The architecture of Transformer Q is as follows:

[0181] State-action embedding design:

[0182] For continuous state , the embedding function is: ,in, s represents the continuous state vector, The weight matrix representing the state embedding; represents the embedding dimension, Represents the state dimension, Represents the bias vector of state embedding; GELU represents the Gaussian error linear unit activation function; LayerNorm represents the layer normalization function;

[0183] State-Action Fusion:

[0184] Fuse state and action embeddings into a unified representation: ,in represents the fusion weight matrix dimension, Row 2 List, Indicates that the fusion bias vector belongs to dimensional real space, h Represents the representation vector after the state and action are embedded into each other. represents the action vector, represents the state embedding vector, represents the action embedding vector, represents vector concatenation, Indicates the concatenation of the state embedding vector and the action embedding vector;

[0185] Rotational position encoding is applied before attention calculation, and a multi-head self-attention mechanism is adopted. Each temporal gated Transformer block includes a self-attention sublayer, a gated convolution sublayer, and a feedforward network sublayer;

[0186] Meta-training Transformer Q, different trajectories and different budget combinations form a task space, each specific task is a set Yes, among them, represents a specific reference trajectory sampled from the trajectory distribution, represents the specific active positioning budget value sampled from the budget distribution, sampling different tasks in the task space to train the TD algorithm to improve the generalization ability of the trained Transformer Q;

[0187] The state update equation is as follows:

[0188] ;

[0189] in, : The estimated value of the physical state at the next moment k+1;

[0190] : The real physical state value at the next moment k+1;

[0191] : Information reliability attenuation factor, reflecting the attenuation of sensor information quality with state estimation error;

[0192] : Information attenuation basic coefficient, which controls the basic strength of attenuation and has a value range of [0,1];

[0193] : Information decay rate adjustment parameter, which controls the sensitivity of decay to state error;

[0194] : estimated value of the physical state at time k;

[0195] : the real physical state value at time k;

[0196] : The square of the Euclidean distance between the current estimated state and the true state;

[0197] : System dynamics models that predict the next state based on the current state, control inputs, and random disturbances;

[0198] : physical control input;

[0199] : random perturbation;

[0200] : The remaining positioning budget constraint at time k, which represents the number of active positioning attempts remaining;

[0201] : Uncertainty growth function, which is related to the remaining positioning budget. The smaller the budget, the slower the uncertainty growth;

[0202] : The covariance matrix of the state estimate, representing the uncertainty of the state estimate;

[0203] : Positioning decision variable, which is also a binary decision action output by the Transformer Q network , Indicates that the patrol drone performs active positioning to obtain the real status of the system. Indicates that the patrol UAV does not perform active positioning and estimates the system state through average particle filtering.

[0204] Alternatively, for a given patrol UAV’s reference trajectory and active positioning budget, the key component of TQMPC is the interaction between the two modules (TQ and MPC), which includes the following steps:

[0205] TQ is based on status Determine the optimal positioning action ,This decision will affect the MPC’s belief state estimation and its subsequent control actions;

[0206] The trajectory deviation calculated by MPC is used to improve the TQ's decision-making process. The trajectory tracking performance is used to guide the learning of the patrol UAV's positioning strategy, encouraging the TQ to prioritize positioning decisions that can reduce trajectory deviation.

[0207] The dynamic exchange between the TQ localization strategy and the MPC control output establishes a feedback loop. By combining the budget-exceeding constraint and the trajectory tracking performance reward, the TQ learning is guided to learn to make adaptive active localization decisions while ensuring both the trajectory tracking performance and the state estimation accuracy of the patrol UAV.

[0208] The total reward function of the TQ algorithm is as follows:

[0209] ;

[0210] in, : The overall reward function of DQN, used to train the positioning decision network;

[0211] : Feedback reward obtained from MPC to evaluate the impact of localization strategy on trajectory tracking;

[0212] : The weight parameter that balances the MPC feedback and state estimation error, with a value range of [0,1];

[0213] : the real physical state value at time k;

[0214] : estimated value of the physical state at time k;

[0215] : The error between the current true state and the estimated state, using the Euclidean distance;

[0216] : The weight coefficient of the information gain term, which controls the proportion of long-term planning in the reward;

[0217] : Information gain function, which evaluates the potential value of the current positioning decision for future trajectory tracking;

[0218] : Total positioning budget, which indicates the maximum number of active positioning times allowed in the entire task;

[0219] : Remaining positioning budget, indicating the number of active positioning times currently available;

[0220] : The remaining trajectory length or remaining task complexity is used to adjust the information gain calculation;

[0221] : Large negative rewards when the budget is insufficient, forcing DQN to learn to abide by the budget constraints.

[0222] Optionally, the method further includes controlling each follower in the drone cluster to maintain a set distance from the corresponding leader to form a formation movement, comprising the following steps:

[0223] Controlling each follower in the drone cluster to maintain a set distance from the corresponding leader according to a distance-maintaining reward function to form a formation movement;

[0224] The reward function for maintaining distance is:

[0225] ;

[0226] in, Represents the reward given to the follower by the reward function; t represents the current moment, represents the distance between the follower and the leader at time t, Let \(d_{t + 1}\) denote the distance between the follower and the leader at time \(t+1\), \(D_1\) denote the lower threshold of the distance between the follower and the leader, \(D_2\) denote the upper threshold of the distance between the follower and the leader, and \(D_1<D_2\) be positive constants. The thresholds \(C_1 > C_2 > C_3\) are positive constant reward values obtained by the UAV in its interaction with the environment under different conditions.

[0227] Next, specific application examples will be combined to introduce and illustrate the solutions of the embodiments of this application in detail.

[0228] This embodiment proposes an intelligent patrol path decision-making system and method for UAV swarms: The leader realizes the selection and tracking of a stochastic path under resource constraints through the Multi-stage Entropy-Driven Random-matrix Optimization-Transformer Q Network Model Predictive Control (MEDRO-TQMPC) algorithm. The two algorithms are connected through a system environment module. The system environment includes information such as a network topology graph, steady-state conditions, and budget constraints. The leader's Multi-stage Entropy-Driven Random-matrix Optimization (MEDRO) algorithm dynamically adjusts and solves the stochastic path selection / planning problem with specific constraints through a transition matrix initial generation module and a transition matrix optimization adjustment module. The leader's Transformer Q Network Model Predictive Control (TQMPC) algorithm dynamically exchanges information between a TQ active positioning decision module and an MPC trajectory tracking control module. The updated decision of TQ provides information for the control strategy of MPC, and the output result of MPC in turn optimizes the learning of TQ to ensure the tracking effect of the selected path under resource constraints. The follower forms a basic humanoid formation by maintaining a set distance from the leader through a reward function. On this basis, the formation is expanded in multiples of three to achieve intelligent patrol path decision-making for a cluster of 3 n ground unmanned systems.

[0229] The specific solution is as follows:

[0230] 1. System modeling.

[0231] Discretize the physical environment according to the actual patrol situation of the UAV and model it through an undirected topology graph, , where represents the set of nodes (patrol positions). Each node in the undirected topology graph represents a position that requires UAV patrol and surveillance in the real environment, represents the set of edges, represents the transition from node to node The probability of satisfying . and Not necessarily equal. Representative Node and Connected between, Representative Node and The path length between Representative Node The value of Attacker / drone attacking node The time required. Figure 2 Represents a Markov chain with five nodes and its corresponding transition matrix P, which is irreducible and non-periodic. The state transition matrix P is as follows:

[0232] ;

[0233] The transition probabilities between nodes are encoded in the transition matrix P. Each row of the matrix P represents a probability distribution (i.e., all values ​​in each row add up to 1). The patrol drone starts from any initial node and then samples according to the probability distribution in the row corresponding to its current node to determine which area to patrol / inspect next.

[0234] 2. Multi-stage entropy-driven random matrix optimization (MEDRO) algorithm.

[0235] The Multi-stage Entropy Driven Random Matrix Optimization (MEDRO) algorithm is a four-stage combinatorial algorithm that efficiently solves the problem of generating random Markov transition matrices with specific constraints by generating steady-state distributions that meet special requirements based on a specific network topology.

[0236] 2.1 Generate the optimal steady-state distribution.

[0237] like Figure 2 The Markov chain shown, different nodes It represents the locations that different drones need to patrol. Different nodes have different values, and attackers need different amounts of time to attack different nodes. Therefore, target nodes (including high-value nodes and nodes with short attack times) need to obtain more patrol resources, and patrol drones have a higher probability of visiting these nodes. is a general steady-state distribution that represents the probability distribution of drones patrolling each node after the Markov chain runs for a long time. Therefore, it is necessary to design a steady-state distribution with specific optimization properties. Formula (1) shows how to calculate the steady-state distribution considering the target value and attack time.

[0238] (1)

[0239] in, , , represents the i-th node, V represents the set of each of the nodes, Represents the target node The value of represents the attack time, Representative Node The importance weight parameter, Representative Node The importance weight parameter, n is the total number of nodes, Represents the target node The target steady-state distribution represents the probability distribution of the patroller at each target node after the Markov chain runs for a long time.

[0240] 2.2 MEDRO algorithm generates transfer matrix.

[0241] In order to increase the difficulty of predicting the path planned by the patrol drone, the MEDRO algorithm is used to generate multiple (Formula 1) but with different transfer characteristics, the transfer matrix P is used to achieve a random path selection strategy with time-space decoupling, such as Figure 3 Specifically, the Multi-stage Entropy Driven Random Matrix Optimization (MEDRO) algorithm has four stages, namely initialization and basic optimization, entropy maximization, diversity enhancement, and final fine-tuning. Entropy maximization is used as one of the core optimization objectives, and the matrix is ​​continuously improved through an iterative approach to meet the constraints of network topology, row sum to 1, and steady-state conditions.

[0242] The four stages are as follows:

[0243] Phase 1: Initialization and basic optimization.

[0244] 1. Random initialization:

[0245] Using the bitangent mapping function instead of the linear probability distribution, a more complex random Q matrix that satisfies the topological constraints is generated. The random initialization function is shown in formula (2), where, represents the element in the i-th row and j-th column of the initial transition probability matrix Q, and random() represents a function that generates random numbers in the interval [0,1]. The adjacency matrix representing the topological constraints, represents the element in the i-th row and k-th column of the adjacency matrix A, and n represents the total number of states.

[0246] (2)

[0247] 2. Self-loop constraint application:

[0248] Limit the self-loop probability to no more than the set value, ,in, represents the optimized self-loop probability (the probability of transferring from state i to itself), represents the self-loop probability of the initial state (the probability of transferring from state i to itself before optimization), The upper threshold of the self-loop probability. If the original self-loop probability , then keep the original value ; If the original self-loop probability , then keep the original value .

[0249] 3. Steady-state optimization:

[0250] (1) An iterative optimization method with adaptive learning rate, which uses exponential decay. When the error is large, a larger learning rate is used, and the learning rate is automatically reduced as the error decreases to avoid oscillation.

[0251] (2) Apply correction factors: ,in, represents the original transition probability, the transition probability from state i to state j; represents the target steady-state probability, the steady-state probability of the state j that we hope to achieve; represents the current estimated steady-state probability, the steady-state probability of state j under the current matrix; is the learning rate, iteratively adjusted to satisfy the steady-state equation , is the steady-state probability vector, It represents the probability distribution when the Markov chain reaches equilibrium; the steady-state equation indicates that when the system reaches steady state, the probability distribution no longer changes with time.

[0252] Phase 2: Entropy maximization.

[0253] 1. Balanced distribution:

[0254] (1) Calculate the maximum possible entropy: in, represents the maximum possible entropy value, It represents the number of feasible transitions from state i (the number of other states to which one can transition from state i).

[0255] (2) Moving towards uniform distribution: ,in, represents the probability value of the system transferring from state i to state j in a random process, represents the average value of the transition probability of a row in the process of balanced distribution, Represents the shift factor that controls the magnitude of the shift toward uniform distribution.

[0256] 2. Adaptive entropy enhancement:

[0257] (1) We focus on optimizing rows with low entropy ratios and redistribute off-diagonal probabilities while maintaining self-loop constraints.

[0258] (2) Simulate quantum superposition states, allowing the matrix to evolve in multiple directions simultaneously and retain the optimal result.

[0259] (3) Entropy enhancement is only applied to rows whose entropy ratio is lower than the target to avoid over-homogenization.

[0260] (4) Perform steady-state fine-tuning every few iterations to balance entropy optimization and steady-state constraints.

[0261] Phase 3: Increased diversity.

[0262] 1. Differentiation assessment:

[0263] Calculate the MAD (mean absolute difference) between matrices: ,in, It represents the average of the absolute value of the difference between the elements of matrix a and matrix b. represents the transition probability from state i to state j in the a-th matrix, represents the transition probability from state i to state j in the b-th matrix, K is the number of valid elements, which represents the total number of non-zero elements in the matrix involved in the calculation.

[0264] 2. Adversarial Adjustment:

[0265] (1) Adopting innovative adversarial technology, a GAN-like mechanism is introduced, where the generator creates a new matrix and the discriminator evaluates predictability.

[0266] (2) Identify the high-probability and low-probability transitions of each generated matrix, deliberately reduce the high-probability transitions of the existing matrix in the new matrix, and improve the low-probability transition paths in the existing matrix.

[0267] (3) Analyze similarities in multiple dimensions, identify the most critical differences and enhance them, and make reverse adjustments to highly similar elements:

[0268] if , further increase ;

[0269] if , further reducing .

[0270] in, represents the transition probability from state i to state j in the a-th matrix, represents the transition probability from state i to state j in the bth matrix.

[0271] (4) Add a random factor random() to the adjustment to avoid generating completely predictable adversarial patterns and ensure that even adversarial reinforcement will not produce predictable anti-patterns.

[0272] 3. Critical path diversification:

[0273] (1) History-based path importance assessment: Critical paths are identified by analyzing the average usage patterns of the existing matrix. Transfers with high frequency of use are considered to be more important strategic decision points.

[0274] (2) Threshold-triggered adjustment: Adjustment is triggered only when the current usage pattern is highly similar to the average pattern, ensuring that each critical path behaves differently in different matrices.

[0275] (3) Random directional binary decision: Randomly decide whether to increase or decrease the probability, maximize unpredictability, and adjust the range between 15% and 30%, which is enough to create a significant difference but not destroy the overall structure.

[0276] (4) Probability redistribution mechanism: When the probability of a certain transition is increased / decreased, the probabilities of other transitions are correspondingly reduced / increased, keeping the total probability sum to 1 and creating cascading differences between unrelated transitions.

[0277] Phase 4: Final fine-tuning.

[0278] 1. Reapply the constraints:

[0279] (1) Ensure that the self-loop probability remains within the limit.

[0280] (2) Normalize each row so that its sum is 1.

[0281] (3) Re-optimize the steady-state distribution.

[0282] 2. Balance between equilibrium and diversity:

[0283] Synchronously optimize multiple objective functions, dynamically adjust entropy optimization and differentiation intensity, resolve potential conflicts between entropy maximization and diversity maximization, and find the optimal balance between entropy and diversity.

[0284] 3. Transformer Q-network model predictive control (TQMPC) algorithm.

[0285] In order to enable the UAV to better track the selected path, an algorithm combining deep reinforcement learning (TransformerQ-Networks, TQ) and model predictive control (MPC) is used. The update decision of TQ provides information for the control strategy of MPC, and the output of MPC in turn optimizes the learning of TQ to ensure the tracking effect of the UAV on the patrol trajectory. The TQMPC algorithm process is as follows Figure 4 shown.

[0286] 3.1 Model Predictive Control (MPC) Trajectory Tracking Algorithm.

[0287] MPC minimizes a predefined cost function by optimizing future control actions within a limited time range. Considering the positioning constraints of the UAV, the objective function and constraints of MPC are shown in formula (3):

[0288] (3)

[0289] in, : The control input sequence of the UAV from the current time k to the end of the prediction time domain;

[0290] : The estimated value of the physical state at the i-th time step, including position and velocity; MPC uses the estimated value to plan control actions, and in the case of passive positioning, the estimated value represents the current system state;

[0291] : physical control input at the i-th time step;

[0292] : physical control input at the i-1th time step;

[0293] : The reference trajectory state at the i-th time step, indicating the target state that the system should track;

[0294] : The weighted quadratic norm of the state tracking error, Q is a positive definite weight matrix (state error weight), which assigns different weights to different state components;

[0295] : The weighted quadratic norm of the control input change, R is a positive definite weight matrix (control change weight), which is used to penalize sudden changes in the control input;

[0296] : End of time domain ( time steps) of the physical state;

[0297] : End of time domain ( The reference trajectory state (time steps) represents the target state that the system should track;

[0298] : The weighted quadratic norm of the terminal state error in the prediction time domain, P is the positive definite terminal weight matrix (terminal state weight);

[0299] : Weight coefficients of the three objectives of equilibrium state tracking, control smoothing and terminal state constraint;

[0300] : The prediction time domain length of MPC, which indicates the number of time steps that MPC predicts forward;

[0301] : The current time step of the drone;

[0302] : The time step index in the prediction domain (from k arrive k+H MPC ).

[0303] 3.2 Transformer Q (TQ) network learning positioning decision algorithm.

[0304] Transformer Q-Networks (TQ) are used to optimize localization decisions, enabling the drone to learn when active localization is most effective and learn to use active localization at key points in the trajectory.

[0305] TQ input status indication It consists of three components:

[0306] (1) represents the average belief / estimate of the patrol drones about the physical state of the system.

[0307] (2) is the variance of the belief, reflecting the uncertainty of the state estimate.

[0308] (3) is the remaining active positioning budget of the patrol UAV, indicating the remaining available active positioning times.

[0309] The output of TQ is a binary decision action , Indicates that the patrol drone performs active positioning to obtain the real status of the system. Indicates that the patrol UAV does not perform active positioning and estimates the system state through average particle filtering.

[0310] Transformer Q-Networks Architecture Design:

[0311] (1) State-action embedding design.

[0312] For continuous state , the embedding function is: ,in, s represents the continuous state vector, The weight matrix representing the state embedding; represents the embedding dimension (the dimension of the embedding vector), The number of dimensions representing the state (the dimension of the state space), Represents the bias vector of state embedding; GELU represents the Gaussian error linear unit activation function; LayerNorm represents the layer normalization function.

[0313] (2) State-action fusion.

[0314] Fuse state and action embeddings into a unified representation: ,in represents the fusion weight matrix dimension, Row 2 Column (state embedding Dimension + 2 action embeddings dimension), Indicates that the fusion bias vector belongs to dimensional real space, h Represents the representation vector after the state and action are embedded into each other. represents the action vector, represents the state embedding vector, represents the action embedding vector, represents vector concatenation, It represents the concatenation of the state embedding vector and the action embedding vector.

[0315] (3) Rotational Position Encoding (RoPE).

[0316] Apply rotational position encoding before attention computation:

[0317] ;

[0318] in, represents the rotation matrix applied to the query vector at position m over the dimension pair [2j,2j+1], represents the query vector at position m, Represents the query vector The value in the 2j dimension, Represents the query vector The value in the 2j+1 dimension, m represents the position index in the sequence, is the frequency parameter, and the calculation formula is , d represents the total number of embedding dimensions of the representation model, which is used to control the rotation frequency of different dimension pairs, j represents the index of the dimension pair, Indicates the rotation angle.

[0319] (4) Multi-head self-attention.

[0320] Multi-head self-attention divides the input into h heads and calculates:

[0321] ;

[0322] in, , Represents the output of multi-head self-attention, represents the number of attention heads, headi represents the i-th attention head, and Concat represents the connection operation, which splices the outputs of multiple heads together. Represents the output projection matrix, with dimensions , is the model dimension, represents the embedding representation of the input sequence, It represents the result of the input X after the query weight matrix transformation, i represents the index of the i-th attention head, ranging from 1 to h (total number of heads), represents the query weight matrix dedicated to the i-th attention head, It represents the key weight matrix dedicated to the i-th attention head. Represents the result after the input X is transformed by the key weight matrix. represents the value weight matrix dedicated to the i-th attention head, It represents the result of the input X after the value weight matrix transformation. Attention() represents the attention calculation function, which receives the query, key and value as input.

[0323] (5) Temporal Gated Transformer Block.

[0324] Each Transformer block contains three sub-layers:

[0325] a. Self-attention sub-layer:

[0326] ;

[0327] in, is the input tensor, which represents the original input features processed by the model or the output of the previous layer, represents the output tensor after processing by the self-attention sublayer, Represents the residual connection coefficient of the self-attention sublayer, which controls the weight of the residual information. LayerNorm() represents the layer normalization function. MultiHead() represents the multi-head attention mechanism, which allows the model to focus on different positions and different representation subspaces. Dropout() represents the random deactivation function, which prevents overfitting by randomly discarding some neurons.

[0328] b. Gated convolution sublayer:

[0329] Gated Linear Convolution Unit (GLU) calculation:

[0330] ;

[0331] in, represents the final output tensor after processing by the gated convolution sublayer, Represents the residual connection coefficient of the self-attention sub-layer, which controls the weight of the residual information. Represents the residual connection coefficient of the gated convolution sublayer, which controls the weight of the residual information. LayerNorm() represents the layer normalization function. Conv1d() represents the one-dimensional convolution operation. MultiHead() represents the multi-head attention mechanism, which allows the model to focus on different positions and different representation subspaces. Dropout() represents the random deactivation function, which prevents overfitting by randomly discarding some neurons. GLU is defined as The gated linear unit, and are the front and back halves of the input, is the element-wise multiplication operator, It is the sigmoid function that maps the input to a range between 0 and 1.

[0332] c. Feedforward network sublayer:

[0333] ;

[0334] in, represents the output tensor of the gated convolutional sublayer, which serves as the input of the feedforward network sublayer. Represents the final output tensor after the feedforward network sublayer processing, completing the three sublayer processing of the entire Transformer block, Represents the residual connection coefficient of the feedforward network sublayer, which controls the weight of the residual information. LayerNorm() is the layer normalization function. Represents the first weight matrix of the feedforward network, used for the first linear transformation, Represents the second weight matrix of the feedforward network, used for the second linear transformation, represents the first bias vector of the feedforward network, Represents the second bias vector of the feedforward network, and GELU() represents the Gaussian error linear unit activation function.

[0335] (6) Transformer Q network (TQ) output.

[0336] The Dueling architecture decomposes the Q value into a state value function V(s) and an advantage function A(s,a):

[0337] ;

[0338] in, is the state-action value function, which means that in state Take action The expected cumulative return value, is the state value function, which represents the state The intrinsic value of is the advantage function, which means that in state Take action The advantage value relative to the average action, Indicates that the status Take action Advantage value Indicates the current environment status. Indicates the currently selected action. represents any action in the action space, and |A| represents the size of the action space.

[0339] State value function: ;

[0340] in, is the state value function, Represents the output feature vector of the last position of the last layer of Transformer, The weight matrix representing the state value function, represents the transposed weight vector of the state-value function, Represents the first bias vector of the state value function, Represents the second bias vector of the state value function, and GELU() represents the Gaussian error linear unit activation function.

[0341] Advantage function: ;

[0342] in, is the advantage function, Represents the output feature vector of the last position of the last layer of Transformer, represents the weight matrix of the advantage function, represents the transposed weight vector of the advantage function, represents the first bias vector of the advantage function, Represents the second bias vector of the advantage function, and GELU() represents the Gaussian error linear unit activation function.

[0343] (7) Transformer Q network (TQ) training.

[0344] Meta-training of TQ, the loss function is as follows:

[0345] (4)

[0346] in, represents the loss function, represents the trajectory distribution of patrol drones, Indicates that from the distribution The specific reference trajectory sampled in represents the active positioning budget distribution of patrol drones, Indicates that from the distribution The specific budget value of sampling in represents the budget distribution associated with trajectory T, are the current network parameters, are the target network parameters, is the state at the current moment k (including physical state belief, variance and remaining budget), is the state at the next moment k+1, is the positioning action decision of the current moment k (0 or 1), is the positioning action decision at the current moment k, Indicates the actual tracking reference trajectory The rewards you get when represents the discount factor, Indicates the reference trajectory The state-action value predicted by the Q network with parameter θ at the current moment k is, Indicates the reference trajectory The target parameter is used at the next moment k+1 The state-action value predicted by the Q network, Indicates the reference trajectory TD target value under represents the expectation about the trajectory distribution T and the budget distribution B, Express expectations about states and actions, It means to select the action with the largest Q value among all possible actions at the next moment. Different trajectories and different budget combinations form the task space. Each specific task is a set Yes, different tasks are sampled in the task space to train the TD algorithm to improve the generalization ability of the trained Transformer Q.

[0347] 3.3 Transformer Q-Network-Model Prediction (TQMPC) Algorithm

[0348] For a given patrol UAV’s reference trajectory and active localization budget, the key component of TQMPC is the interaction between the two modules (TQ and MPC).

[0349] 3.3.1 Impact of TQ on MPC.

[0350] DQN is based on state Determine the optimal positioning action , this decision will affect the MPC belief state estimation and its subsequent control actions. Formula (5) is the state update equation.

[0351] (5)

[0352] in, : The estimated value of the physical state at the next moment k+1;

[0353] : The real physical state value at the next moment k+1;

[0354] : Information reliability attenuation factor, reflecting the attenuation of sensor information quality with state estimation error;

[0355] : Information attenuation basic coefficient, which controls the basic strength of attenuation and has a value range of [0,1];

[0356] : Information decay rate adjustment parameter, which controls the sensitivity of decay to state error;

[0357] : estimated value of the physical state at time k;

[0358] : the real physical state value at time k;

[0359] : The square of the Euclidean distance between the current estimated state and the true state;

[0360] : System dynamics models that predict the next state based on the current state, control inputs, and random disturbances;

[0361] : physical control input;

[0362] : random perturbation;

[0363] : The remaining positioning budget constraint at time k, which indicates the number of active positioning times currently available;

[0364] : Uncertainty growth function, which is related to the remaining positioning budget. The smaller the budget, the slower the uncertainty growth;

[0365] : The covariance matrix of the state estimate, representing the uncertainty of the state estimate;

[0366] :Locate the decision variables, when (Select active positioning) , the patrol drone now gets the real physical state , the state estimate is equal to the true state; when (Select passive positioning) , using the state transition function To make a prediction, the state estimate is not equal to the true state.

[0367] 3.3.2 Impact of MPC on TQ.

[0368] The deviation from the desired trajectory determined by MPC plays a crucial role in optimizing the TQ decision process. This relationship is formalized through a feedback mechanism where the performance of MPC provides a learning signal to TQ. This signal quantifies the control effectiveness of the MPC under the current positioning strategy. The trajectory deviation calculated by the MPC is used to improve the TQ's decision-making process. The trajectory tracking performance is used to guide the learning of the patrol UAV's positioning strategy, encouraging the TQ to prioritize positioning decisions that can reduce trajectory deviation.

[0369] (6)

[0370] in, : The reward signal provided by MPC to DQN is used to evaluate the effectiveness of the current positioning strategy;

[0371] : The estimated value of the physical state at time i, representing the physical quantities such as the position and speed of the drone;

[0372] : The reference trajectory state at time i, which represents the ideal state of the UAV;

[0373] : Control input at time i, representing the control command acting on the drone;

[0374] : The control input at time i+1 is used together with the control input at the previous time to calculate the control smoothness;

[0375] k: current time index;

[0376] H: prediction horizon length, indicating the number of steps MPC predicts into the future;

[0377] : The weight coefficient of trajectory tracking accuracy, which controls the importance of trajectory tracking error in the reward;

[0378] : The weight coefficient that controls smoothness and the importance of input changes in rewards;

[0379] : The weight coefficient of information content, which controls the importance of the information matrix in the reward;

[0380] : The cumulative state tracking error in the entire prediction time domain, Q is the state error weight matrix;

[0381] : The cumulative control change in the entire prediction time domain, R is the control change weight matrix;

[0382] : The trace of the information matrix, which measures the information content in the current state;

[0383] : Fisher information matrix, which represents the amount of information contained in the current state estimate;

[0384] : represents the time window or parameter related to the information matrix calculation;

[0385] : The estimated value of the physical state at time k, representing the physical quantities such as the position and speed of the drone.

[0386] As a weight parameter, it is used to adjust the strength of this feedback signal. Is the feedback signal of MPC performance to TQ, which is a reward scalar value. The minus sign in front of the formula ( ) indicates that this is a negative reward or penalty term. The larger the distance (the less accurate the trajectory tracking), the larger the absolute value of the negative reward.

[0387] The dynamic exchange between the TQ localization strategy and the MPC control output establishes a feedback loop. The total reward function of the TQ algorithm is shown in formula (7):

[0388] (7)

[0389] in, : The overall reward function of DQN, used to train the positioning decision network;

[0390] : Feedback reward obtained from MPC to evaluate the impact of localization strategy on trajectory tracking;

[0391] : The weight parameter that balances the MPC feedback and state estimation error, with a value range of [0,1];

[0392] : the real physical state value at time k;

[0393] : estimated value of the physical state at time k;

[0394] : The error between the current true state and the estimated state, using the Euclidean distance;

[0395] : The weight coefficient of the information gain term, which controls the proportion of long-term planning in the reward;

[0396] : Information gain function, which evaluates the potential value of the current positioning decision for future trajectory tracking;

[0397] : Total positioning budget, which indicates the maximum number of active positioning times allowed in the entire task;

[0398] : Remaining positioning budget, indicating the number of active positioning times currently available;

[0399] : The remaining trajectory length or remaining task complexity is used to adjust the information gain calculation;

[0400] : Large negative rewards when the budget is insufficient, forcing DQN to learn to abide by the budget constraints.

[0401] in, is the feedback control performance signal of MPC performance on TQ (Formula 6), is the actual physical state at time k, The state estimate at time k (average belief of the state), is the error (norm) between the true state and the estimated state considering the weights, It is a large negative reward when the active positioning budget is exceeded. Focus on trajectory tracking performance and state estimation accuracy. Through the combination of these two aspects, guide the TQ learning to learn adaptive active positioning decisions under the conditions of ensuring both the trajectory tracking performance of the patrol UAV and the accuracy of state estimation.

[0402] 4. Humanoid formation clustering formation control.

[0403] The basic formation of the humanoid formation of the patrol UAV consists of 1 leader and 2 followers. The leader realizes the random patrol path selection and tracking under resource constraints through the MEDRO-TQMPC algorithm. The followers form a basic humanoid formation by maintaining a certain distance from the leader through the reward function. On this basis, the formation is extended in multiples of three to realize the intelligent patrol path decision-making of 3 n UAV clusters.

[0404] 4.1 Design of the reward function for the basic humanoid formation.

[0405] By designing the reward function, the leader and follower UAVs maintain a certain distance, as Figure 5 shown. Set the distances D1 and D2 so that the followers finally reach the four positions of above, below, left, and right of the leader.

[0406] The leader is the center of the circular area. When the distance between the follower and the leader is less than D1, a negative reward will be obtained; when the distance between the follower and the leader is greater than D2, a negative reward will also be obtained; when the follower is in the circular area, a positive reward will be obtained for maintaining a suitable distance.

[0407] According to the above design, the reward function for the leader and follower to maintain a distance is expressed by Equation (8).

[0408] (8)

[0409] Among them, represents the reward given by the reward function to the follower; t represents the current time, represents the distance between the follower and the leader at time t, represents the distance between the follower and the leader at time t+1, D1 represents the lower limit threshold of the distance between the follower and the leader, D2 represents the upper limit threshold of the distance between the follower and the leader, D1<D2 is a positive constant, and the thresholds C1>C2>C3 represent the positive constant reward values obtained by the UAV in the interaction with the environment under different conditions.

[0410] 4.2 Design of the expansion of the humanoid formation.

[0411] The drone at the very front of the basic configuration of the humanoid formation of drones is the command drone, and the two drones connected to it are the follower drones. A reward function is set to reward the behavior of the followers maintaining a certain distance from the commander, so that the followers tend to stay on the arc of a circle with a certain distance as the radius around the commander. As Figure 6 shown, the light red area is set as the sphere of influence of each drone, and the red area is set as the single-step operation range of the drone, that is, the no-go area of the drone, so that while the followers maintain a certain distance from the commander, they can also maintain an appropriate distance from each other. At the same time, the radius of the arc is determined as the diagonal distance of the sphere of influence ( It does not directly appear in Equation 8, but is reflected in Figure 6 the radius of the arc in a, which is a pre-defined fixed parameter. Based on this fixed parameter, the values of D1 and D2 are determined), and the parameter is the ideal distance that the followers are expected to maintain from the leader in the design, that is, the radius of the arc; and are the actual distance measurement values, which change with time; D1 and D2 are the threshold boundaries set based on and satisfy the condition D1 < < D2, where D1 and D2 are the ceiling and floor integers of respectively.

[0412] Referring to Figure 6 , based on the three-drone formation, it is expanded into a nine-drone formation ( Figure 6 b), that is, a middle group composed of three three-drone groups. The group at the very front is the leading group, and the two groups connected to it are the following groups. The sphere of influence is determined as three diagonal distances, that is , D1 < < D2. Based on the above content, it is further expanded into a 27- ( Figure 6 c) drone formation, so that D1 and D2 in the reward function that makes the following groups tend to stay on the arc of a circle with a certain grid distance as the radius around the command group are determined as nine grid distances, that is, D1 < 9 < D2.

[0413] In summary, this embodiment includes the following technical solutions:

[0414] 1. A novel intelligent patrol path decision system and method for drone swarms are proposed: the multi-stage entropy-driven random matrix optimization - Transformer Q-network model predictive control (MEDRO-TQMPC), which simultaneously considers drone path selection, path tracking, and formation expansion under positioning constraints.

[0415] 2. A multi-stage entropy-driven random matrix optimization (MEDRO) algorithm was developed. The four stages are responsible for initialization and basic optimization, entropy maximization, diversity enhancement, and final fine-tuning. Entropy maximization is used as one of the core optimization goals. The matrix is ​​continuously improved in an iterative manner to meet the constraints of network topology, row sum to 1, and steady-state conditions.

[0416] 3. The developed Transformer Q network model predictive control (TQMPC) algorithm (and its related mathematical formulas) implements adaptive active positioning decisions for the UAV while satisfying positioning constraints through dynamic information exchange between the TQ active positioning decision module and the MPC trajectory tracking control module, ensuring tracking of the selected path.

[0417] 4. A special leader-follower mechanism: the leader integrates a more intelligent MEDRO-TQMPC algorithm, and the follower drone is only responsible for dynamic tracking and obstacle avoidance through the reward function. The number of drones is expanded based on 3, not an arbitrary number.

[0418] The beneficial effects of this embodiment include:

[0419] This embodiment integrates an intelligent drone patrol decision-making system and method. The paths generated by the Navigator Multi-stage Entropy-Driven Random Matrix Optimization (MEDRO) algorithm are highly randomized and more difficult to predict than those generated by existing algorithms. The Transformer Q Network Model Predictive Control (TQMPC) algorithm ensures that patrol drones track their selected paths within resource constraints through dynamic information exchange between the TQ active positioning decision module and the MPC trajectory tracking control module. This enables adaptive active positioning decisions within positioning constraints, resulting in higher tracking accuracy than existing pure MPC path tracking algorithms. A reward function is designed to implement a basic humanoid formation based on a navigator-follower system, and the formation is expanded based on a three-dimensional basis to increase the patrol's visual range.

[0420] Reference Figure 7 The embodiment of the present application further provides a device for determining a patrol path of a swarm of drones under resource constraints, which implements the above-mentioned method for determining a patrol path of a swarm of drones under resource constraints. The device includes:

[0421] A graph construction unit is used to discretize the actual physical locations to be patrolled to construct an undirected topological graph, where the nodes represent the locations to be patrolled and the edges between the nodes represent the flight paths of the drones;

[0422] A steady-state distribution generating unit is used to generate a target steady-state distribution of each patrol node according to the topological constraints of the constructed undirected topological graph, the importance of the node and the attack time;

[0423] a transfer matrix generation unit, configured to generate a plurality of state transfer matrices having the same target steady-state distribution but different transfer characteristics according to a multi-stage entropy-driven random matrix optimization algorithm, wherein each state transfer matrix represents a transfer probability between each node of the UAV cluster;

[0424] The path selection unit is used to initialize the position of the UAV according to the initial state transfer matrix and determine the path selected by the navigator;

[0425] A decision-making unit, configured to implement adaptive active positioning decisions based on the reference path selected by the navigator through dynamic information exchange between the TQ active positioning decision module and the MPC trajectory tracking control module of the navigator, thereby ensuring tracking of the selected path while satisfying positioning constraints;

[0426] A tracking unit, configured to select and track the path of the leader, with followers maintaining a set distance from the leader through a reward function to form a humanoid formation cluster of drones;

[0427] The formation control unit is used to automatically switch to the next transfer matrix when the initial transfer matrix reaches the transfer number threshold according to each of the state transfer matrices and human-like groupings, execute random patrol path selection and tracking, and implement unmanned cluster intelligent patrol path decision-making under resource constraints in a reciprocating cycle.

[0428] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0429] The present application also provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method of the present application. The electronic device can be any smart terminal, such as a tablet computer or an in-vehicle computer.

[0430] It can be understood that the contents of the above method embodiments are all applicable to the embodiments of the present device, the functions specifically implemented by the embodiments of the present device are the same as those of the method of the present application, and the beneficial effects achieved are also the same as those achieved by the method of the present application.

[0431] See also Figure 8 , Figure 8 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0432] The processor 801 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0433] The memory 802 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 802 and is called by the processor 801 to execute the methods of the embodiments of this application.

[0434] Input / output interface 803, used to implement information input and output;

[0435] Communication interface 804, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0436] Bus 805 , which transmits information between various components of the device (e.g., processor 801 , memory 802 , input / output interface 803 , and communication interface 804 );

[0437] The processor 801 , the memory 802 , the input / output interface 803 and the communication interface 804 are connected to each other in communication within the device via a bus 805 .

[0438] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method of the present application is implemented.

[0439] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiment, the functions specifically implemented by the present storage medium embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0440] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0441] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0442] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0443] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0444] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0445] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0446] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0447] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0448] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0449] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0450] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0451] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A method for UAV swarm patrol path decision-making under resource constraints, characterized by: The method comprises the following steps: The actual physical location to be patrolled is discretized to construct an undirected topological graph, where the nodes represent the locations to be patrolled and the edges between the nodes represent the flight paths of the drones. The target steady-state distribution of each patrol node is generated according to the topological constraints of the constructed undirected topological graph, the importance of the node and the attack time; Generate multiple state transfer matrices with the same target steady-state distribution but different transfer characteristics according to a multi-stage entropy-driven random matrix optimization algorithm, each state transfer matrix represents the transfer probability between each node of the drone cluster; Initialize the position of the UAV based on the initial state transfer matrix and determine the path chosen by the navigator; Based on the reference path selected by the navigator, the navigator's Transformer Q network model predictive control algorithm dynamically exchanges information between the TQ active positioning decision module and the MPC trajectory tracking control module to achieve adaptive active positioning decisions while satisfying positioning constraints, ensuring tracking of the selected path. According to the path selection and tracking of the leader, the followers maintain a set distance from the leader through a reward function to form a humanoid formation cluster of drones; According to each of the state transfer matrices and the human-like grouping cluster, when the initial transfer matrix reaches the transfer number threshold, it automatically switches to the next transfer matrix, executes random patrol path selection and tracking, and repeats the cycle to realize unmanned cluster intelligent patrol path decision under resource constraints.

2. The method for determining the patrol path of a swarm of unmanned aerial vehicles under resource constraints according to claim 1 is characterized in that: The method of generating a target steady-state distribution of each patrol node according to the topological constraints of the constructed undirected topological graph, the importance of the node, and the attack time comprises the following steps: Different nodes Different drones need to patrol different locations. Different nodes have different values. Attackers need different amounts of time to attack different nodes. Target nodes with high value and short attack times are selected as target nodes. Target nodes need to obtain more patrol resources, and patrol drones are more likely to visit target nodes. The expression of the target steady-state distribution is: ； ； in, , represents the i-th node, V represents the set of each of the nodes, Represents the target node The value of represents the attack time, Representative Node The importance weight parameter, Representative Node The importance weight parameter, n is the total number of nodes, Represents the target node The target steady-state distribution represents the probability distribution of the patroller at each target node after the Markov chain runs for a long time.

3. The method for determining the patrol path of a swarm of unmanned aerial vehicles under resource constraints according to claim 1 is characterized in that: The method of generating a plurality of state transfer matrices having the same target steady-state distribution but different transfer characteristics according to the multi-stage entropy-driven random matrix optimization algorithm, wherein each state transfer matrix represents the transfer probability between each node of the drone cluster, includes the following steps: Phase 1: Initialization and basic optimization: Generate a Q matrix that satisfies the topological constraints of the undirected topological graph using a bitangent mapping function; The self-loop probability is limited to a set value, and an iterative optimization method of adaptive learning rate with exponential decay and correction factor is applied to iteratively adjust to meet the steady-state equation; Phase 2: Entropy maximization: Calculate the maximum possible entropy, focus on optimizing rows with low entropy ratios, and redistribute off-diagonal probabilities while maintaining self-loop constraints; Simulate quantum superposition states and perform steady-state fine-tuning every few iterations to balance entropy optimization and steady-state constraints; Phase 3: Diversity Enhancement: Calculate the MAD between matrices to evaluate the differences; A GAN-like mechanism is introduced to identify high-probability and low-probability transitions for each generated matrix, analyze similarities in multiple dimensions, and identify the most critical differences for enhancement. The average usage pattern of the existing matrix is ​​analyzed to identify the critical path. The frequently used transfers are regarded as more important strategic decision points. The cascading differentiation between unrelated transfers is created through threshold trigger adjustment, random direction binary decision and probability redistribution mechanism. Stage 4: Final fine-tuning: Reapply constraints, balance between equilibrium and diversity, optimize multiple objective functions simultaneously, dynamically adjust entropy optimization and differentiation intensity, resolve potential conflicts between entropy maximization and diversity maximization, and find the best balance between entropy and diversity; When all preset constraints are met, the P matrix is ​​obtained as the candidate state transfer matrix.

4. The method for determining the patrol path of a swarm of unmanned aerial vehicles under resource constraints according to claim 1 is characterized in that: The method further includes determining a control action based on the target state transfer matrix and the reference trajectory, including the following steps: Determining a control action based on the target state transfer matrix and a reference trajectory using model predictive control; The objective function and constraints of the model predictive control are: ； in, : The control input sequence of the UAV from the current time k to the end of the prediction time domain; : The estimated value of the physical state at the i-th time step, including position and velocity; MPC uses the estimated value to plan control actions, and in the case of passive positioning, the estimated value represents the current system state; : physical control input at the i-th time step; : physical control input at the i-1th time step; : The reference trajectory state at the i-th time step, indicating the target state that the system should track; : weighted quadratic norm of the state tracking error, Q is the state error weight, which is used to assign different weights to different state components; : The weighted quadratic norm of the control input change, R is the control change weight, which is used to penalize the sudden change of the control input; : End of time domain The estimated value of the physical state at time steps; :End of time domain The reference trajectory state of the time step represents the target state that the system should track; : The weighted quadratic norm of the terminal state error in the prediction time domain, P is the terminal state weight; : Weight coefficients of the three objectives of equilibrium state tracking, control smoothing and terminal state constraint; : The prediction time domain length of MPC, which indicates the number of time steps that MPC predicts forward; : The current time step of the drone; :Prediction time domain from k arrive k+H MPC The time step index.

5. The method for determining the patrol path of a swarm of unmanned aerial vehicles under resource constraints according to claim 1 is characterized in that: Initializing the nodes of the drone cluster, the active positioning budget constraint, and the initial state transition matrix, and determining the path chosen by the leader, includes the following steps: The initial position of the drone and the active positioning budget constraint are input into Transformer Q. A random path is selected through the state transition matrix. Transformer Q is used to optimize the positioning decision and learn where to actively position on the selected path trajectory. The architecture of Transformer Q is as follows: State-action embedding design: For continuous state , the embedding function is: ,in, s represents the continuous state vector, The weight matrix representing the state embedding; represents the embedding dimension, Represents the state dimension, Represents the bias vector of state embedding; GELU represents the Gaussian error linear unit activation function; LayerNorm represents the layer normalization function; State-Action Fusion: Fuse state and action embeddings into a unified representation: ,in represents the fusion weight matrix dimension, Row 2 List, Indicates that the fusion bias vector belongs to dimensional real space, h Represents the representation vector after the state and action are embedded into each other. represents the action vector, represents the state embedding vector, represents the action embedding vector, represents vector concatenation, Indicates the concatenation of the state embedding vector and the action embedding vector; Rotational position encoding is applied before attention calculation, and a multi-head self-attention mechanism is adopted. Each temporal gated Transformer block includes a self-attention sublayer, a gated convolution sublayer, and a feedforward network sublayer; Meta-training Transformer Q, different trajectories and different budget combinations form a task space, each specific task is a set Yes, among them, represents a specific reference trajectory sampled from the trajectory distribution, represents the specific active positioning budget value sampled from the budget distribution, sampling different tasks in the task space to train the TD algorithm to improve the generalization ability of the trained Transformer Q; The state update equation is as follows: ； in, : The estimated value of the physical state at the next moment k+1; : The real physical state value at the next moment k+1; : Information reliability attenuation factor, reflecting the attenuation of sensor information quality with state estimation error; : Information attenuation basic coefficient, which controls the basic strength of attenuation and has a value range of [0,1]; : Information decay rate adjustment parameter, which controls the sensitivity of decay to state error; : estimated value of the physical state at time k; : the real physical state value at time k; : The square of the Euclidean distance between the current estimated state and the true state; : System dynamics models that predict the next state based on the current state, control inputs, and random disturbances; : physical control input; : random perturbation; : The remaining positioning budget constraint at time k, which represents the number of active positioning attempts remaining; : Uncertainty growth function, which is related to the remaining positioning budget. The smaller the budget, the slower the uncertainty growth; : The covariance matrix of the state estimate, representing the uncertainty of the state estimate; : Positioning decision variable, which is also a binary decision action output by the Transformer Q network , Indicates that the patrol drone performs active positioning to obtain the real status of the system. Indicates that the patrol UAV does not perform active positioning and estimates the system state through average particle filtering.

6. The method for determining the patrol path of a swarm of unmanned aerial vehicles under resource constraints according to claim 1 is characterized in that: For a given patrol UAV’s reference trajectory and active positioning budget, the key component of TQMPC is the interaction between TQ and MP, which includes the following steps: TQ is based on status Determine the optimal positioning action ,This decision will affect the MPC’s belief state estimation and its subsequent control actions; The trajectory deviation calculated by MPC is used to improve the TQ's decision-making process. The trajectory tracking performance is used to guide the learning of the patrol UAV's positioning strategy, encouraging the TQ to prioritize positioning decisions that can reduce trajectory deviation. The dynamic exchange between the TQ localization strategy and the MPC control output establishes a feedback loop. By combining the budget-exceeding constraint and the trajectory tracking performance reward, the TQ learning is guided to learn to make adaptive active localization decisions while ensuring both the trajectory tracking performance and the state estimation accuracy of the patrol UAV. The total reward function of the TQ algorithm is as follows: ； in, : The overall reward function of DQN, used to train the positioning decision network; : Feedback reward obtained from MPC to evaluate the impact of localization strategy on trajectory tracking; : The weight parameter that balances the MPC feedback and state estimation error, with a value range of [0,1]; : the real physical state value at time k; : estimated value of the physical state at time k; : The error between the current true state and the estimated state, using the Euclidean distance; : The weight coefficient of the information gain term, which controls the proportion of long-term planning in the reward; : Information gain function, which evaluates the potential value of the current positioning decision for future trajectory tracking; : Total positioning budget, which indicates the maximum number of active positioning times allowed in the entire task; : Remaining positioning budget, indicating the number of active positioning times currently available; : The remaining trajectory length or remaining task complexity is used to adjust the information gain calculation; : Large negative rewards when the budget is insufficient, forcing DQN to learn to abide by the budget constraints.

7. The method for determining a patrol path of a swarm of unmanned aerial vehicles under resource constraints according to claim 6 is characterized in that: The method further includes controlling each follower in the drone cluster to maintain a set distance from the corresponding leader to form a formation movement, including the following steps: Controlling each follower in the drone cluster to maintain a set distance from the corresponding leader according to a distance-maintaining reward function to form a formation movement; The reward function for maintaining distance is: ； in, Represents the reward given to the follower by the reward function; t represents the current moment, represents the distance between the follower and the leader at time t, represents the distance between the follower and the navigator at time t+1, D1 represents the lower limit threshold of the distance between the follower and the navigator, D2 represents the upper limit threshold of the distance between the follower and the navigator, D1 < D2 is a positive constant, and the thresholds C1>C2>C3 represent the positive constant reward values ​​obtained by the drone when interacting with the environment under different conditions.

8. A device for making decisions on patrol paths for drone swarms under resource constraints, characterized in that: The device comprises: A graph construction unit is used to discretize the actual physical locations to be patrolled to construct an undirected topological graph, where the nodes represent the locations to be patrolled and the edges between the nodes represent the flight paths of the drones; A steady-state distribution generating unit is used to generate a target steady-state distribution of each patrol node according to the topological constraints of the constructed undirected topological graph, the importance of the node and the attack time; a transfer matrix generation unit, configured to generate a plurality of state transfer matrices having the same target steady-state distribution but different transfer characteristics according to a multi-stage entropy-driven random matrix optimization algorithm, wherein each state transfer matrix represents a transfer probability between each node of the UAV cluster; The path selection unit is used to initialize the position of the UAV according to the initial state transfer matrix and determine the path selected by the navigator; A decision-making unit, configured to implement adaptive active positioning decisions based on the reference path selected by the navigator through dynamic information exchange between the TQ active positioning decision module and the MPC trajectory tracking control module of the navigator, thereby ensuring tracking of the selected path while satisfying positioning constraints; A tracking unit is used to select and track the path of the leader, and the followers maintain a set distance from the leader through a reward function to form a humanoid formation cluster of drones; The formation control unit is used to automatically switch to the next transfer matrix when the initial transfer matrix reaches the transfer number threshold according to each of the state transfer matrices and the human-like formation cluster, execute random patrol path selection and tracking, and realize unmanned cluster intelligent patrol path decision-making under resource constraints in a reciprocating cycle.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images