Unmanned aerial vehicle cluster patrol path decision-making method under resource constraint

Abstract

The invention discloses an unmanned aerial vehicle cluster patrol path decision-making method under resource constraint, and relates to the technical field of unmanned aerial vehicle path planning, and the method comprises the steps: carrying out the discretization of an actual to-be-patrolled physical position, and constructing an undirected topological graph; generating steady-state distribution of each patrol node according to topological constraints and importance degrees of the nodes; generating a plurality of transfer matrixes with the same steady-state distribution and different transfer characteristics according to a multi-stage entropy driving random matrix optimization algorithm; initializing the position of a navigator according to the transfer matrix and determining a selected path; according to the reference path of the navigator, the self-adaptive active positioning decision is realized under the positioning constraint to ensure the path tracking effect; according to path selection and tracking of the navigator, the follower and the navigator form a humanoid marshalling cluster through a reward function; and according to the multi-state transfer matrix and the humanoid marshalling, automatically switching to the next transfer matrix when a transfer frequency threshold value is reached, and realizing unmanned aerial vehicle cluster intelligent patrol path decision under resource constraint.

Description

Technical Field

[0001] This application relates to the technical field of UAV path planning, and particularly to a method for making decisions on the patrol paths of a UAV swarm under resource constraints. Background Art

[0002] The existing technology uses the Metropolis-Hastings algorithm to generate the state transition matrix of a Markov chain. It studies the trajectory planning of aircraft considering positioning error correction under multiple constraints. By constructing a grid map of the operation area of the unmanned device swarm according to the map information, the A* algorithm is used to complete path planning, and the Leader-Follower mode is used for formation control to form a swarm path.

[0003] Firstly, the existing technology is in a scattered state, and there is no connection between independent systems. Secondly, the state transition matrix of the Markov chain generated by the existing technology has a high self-loop characteristic, and it is easy for potential attackers to learn and predict the patrol path. Then, there are many calculations for the trajectory planning of aircraft under multiple constraints, but no algorithm has been found that specifically studies how to select and track paths for UAVs under positioning constraints. Finally, the existing Leader-Follower mode does not show how to select a randomized path and how to make formation path decisions under resource constraints. Summary of the Invention

[0004] The main purpose of the embodiments of this application is to propose a method for making decisions on the patrol paths of a UAV swarm under resource constraints, so as to realize the selection and tracking of paths under resource constraints.

[0005] To achieve the above object, on the one hand, an embodiment of this application proposes a method for making decisions on the patrol paths of a UAV swarm under resource constraints, and the method includes the following steps: Discretize the actual physical positions to be patrolled to construct an undirected topological graph, where nodes represent the positions to be patrolled, and the edges between nodes represent the flight paths of UAVs; Generate the target steady-state distribution of each patrol node according to the topological constraints of the constructed undirected topological graph, the importance of nodes, and the attack time; Generate multiple state transition matrices with the same target steady-state distribution but different transition characteristics according to the multi-stage entropy-driven stochastic matrix optimization algorithm, and each state transition matrix represents the transition probability of the UAV swarm between each node; Initialize the position of the UAV according to the initial state transition matrix and determine the path selected by the leader; According to the reference path selected by the leader, the Transformer Q-network model predictive control algorithm of the leader realizes adaptive active positioning decision-making by dynamically exchanging information between the TQ active positioning decision-making module and the MPC trajectory tracking control module under the condition of satisfying the positioning constraints, ensuring the tracking effect of the selected path; According to the path selection and tracking of the leader, the follower forms a humanoid formation cluster of unmanned aerial vehicles by maintaining a set distance from the leader through a reward function; According to each of the state transition matrices and the humanoid formation, when the initial transition matrix reaches the transition times threshold, it is automatically switched to the next transition matrix to execute the selection and tracking of a random patrol path, and how to repeat the cycle to realize the intelligent patrol path decision-making of the unmanned cluster under resource constraints.

[0006] In some embodiments, generating the target steady-state distribution of each patrol node according to the topological constraints of the constructed undirected topological graph, the importance degree of the nodes, and the attack time includes the following steps: Different nodes represent different positions where the unmanned aerial vehicles need to patrol. Different nodes have different values, and it takes different times for the attacker to attack different nodes. The high-value target nodes and the nodes with short attack times are used as target nodes. The target nodes need to obtain more patrol resources, and the probability of the patrol unmanned aerial vehicle accessing the target nodes is higher; The expression of the target steady-state distribution is: ; ; Among them, , represents the i-th node, V represents the set of all the nodes, represents the target node 's value, represents the attack time, represents the importance weight parameter of the node , represents the importance weight parameter of the node , n is the total number of nodes, represents the steady-state distribution of the target node . The target steady-state distribution represents the probability distribution of the patroller at each target node after the long-term operation of the Markov chain.

[0007] In some embodiments, generating multiple state transition matrices with the same target steady-state distribution but different transition characteristics according to the multi-stage entropy-driven stochastic matrix optimization algorithm, where each state transition matrix represents the transition probability between the nodes of the unmanned aerial vehicle cluster, includes the following steps: The first stage: Initialization and basic optimization: Generate a Q matrix that satisfies the topological constraints of the undirected topological graph using a double tangent mapping function; Limit the self-loop probability not to exceed a set value, and use an iterative optimization method with an exponentially decaying and corrected factor adaptive learning rate to iteratively adjust to satisfy the steady-state equation; Second stage: Entropy maximization: Calculate the maximum possible entropy, focus on optimizing the rows with low entropy ratios, and reallocate the off-diagonal probabilities while maintaining the self-loop constraint; Simulate the quantum superposition state, perform a steady-state fine-tuning every few iterations, and balance entropy optimization and steady-state constraints; Third stage: Diversity enhancement: Calculate the MAD between matrices for dissimilarity evaluation; Introduce a GAN-like mechanism to identify high-probability and low-probability transitions in each generated matrix, analyze the similarity in multiple dimensions, and find the most critical difference points for enhancement; Analyze the average usage pattern of existing matrices to identify critical paths. Transitions with high frequencies of use are regarded as more important strategic decision points, and cascade differences are created between irrelevant transitions through threshold-triggered adjustment, random-direction binary decision, and probability reallocation mechanisms; Fourth stage: Final fine-tuning: Reapply the balance of constraints, equilibrium, and diversity, synchronously optimize multiple objective functions, dynamically adjust the entropy optimization and differentiation intensity, resolve the potential conflict between entropy maximization and diversity maximization, and find the optimal balance point between entropy and diversity; When all preset constraint conditions are met, obtain the P matrix as the candidate state transition matrix.

[0008] In some embodiments, the method further includes determining control actions according to the target state transition matrix and the reference trajectory, including the following steps: Use model predictive control to determine control actions according to the target state transition matrix and the reference trajectory; The objective function and constraint conditions of the model predictive control are: ; Wherein, : The control input sequence of the unmanned aerial vehicle from the current time k to the end of the prediction horizon; : The estimated value of the physical state at the i-th time step, including position and velocity; MPC uses the estimated value to plan control actions, and in the case of passive positioning, the current system state is represented by the estimated value; : The physical control input at the i-th time step; : The physical control input at the (i - 1)-th time step; : The reference trajectory state at the i-th time step, representing the target state that the system should track; : The weighted quadratic norm of the state tracking error, where Q is the state error weight used to assign different weights to different state components; : The weighted quadratic norm of the control input variation, where R is the control variation weight used to penalize sudden changes in the control input; : The physical state estimate at the -th time step at the end of the time domain; : The reference trajectory state at the -th time step at the end of the time domain, representing the target state that the system should track; : The weighted quadratic norm of the state error at the end of the prediction horizon, where P is the terminal state weight; : The weight coefficient that balances the three objectives of state tracking, control smoothing, and terminal state constraint; : The prediction horizon length of the MPC, representing the number of time steps that the MPC predicts forward; : The current time step of the UAV; : From k to k + H MPC within the prediction horizon.

[0009] In some embodiments, initializing the nodes where the UAV swarm is located, the active positioning budget constraint, and the initial state transition matrix, and determining the path selected by the leader includes the following steps: Input the initial UAV position and the active positioning budget constraint into Transformer Q, and select a random path through the state transition matrix. Transformer Q is used to optimize the positioning decision and learn at which position on the selected path trajectory to perform active positioning; The architecture of Transformer Q is as follows: State-action embedding design: For the continuous state , the embedding function is: , where s represents the continuous state vector, represents the weight matrix of the state embedding; represents the embedding dimension, represents the state dimension, The bias vector representing the state embedding; GELU represents the Gaussian Error Linear Unit activation function; LayerNorm represents the layer normalization function; State-action fusion: Fuse the state and action embeddings into a unified representation: , where represents the dimension of the fusion weight matrix, row 2 column, indicates that the fusion bias vector belongs to the real number space of dimension, h represents the representation vector after fusing the state and action embeddings, represents the action vector, represents the state embedding vector, represents the action embedding vector, represents vector concatenation, represents concatenating the state embedding vector and the action embedding vector; Apply rotational position encoding before attention calculation, adopt the multi-head self-attention mechanism, and each temporal gated Transformer block includes a self-attention sublayer, a gated convolutional sublayer, and a feed-forward network sublayer; Perform meta-training on the Transformer Q. Different combinations of trajectories and budgets form the task space, and each specific task is a set pair, where, represents a specific reference trajectory sampled from the trajectory distribution, represents a specific active localization budget value sampled from the budget distribution. Sample different tasks in the task space to train the TD algorithm to improve the generalization ability of the trained Transformer Q; The state update equation is as follows: ; where, : The estimated physical state value at the next time step k + 1; : The true physical state value at the next time step k + 1; : The information reliability decay factor, reflecting the decay of the sensing information quality with the state estimation error; : The information decay base coefficient, controlling the basic intensity of the decay, with a value range of [0, 1]; : The information decay rate adjustment parameter, controlling the sensitivity of the decay to the state error; : The estimated physical state value at time step k; : the real physical state value at time k; : The square of the Euclidean distance between the current estimated state and the true state; : System dynamics models that predict the next state based on the current state, control inputs, and random disturbances; : physical control input at time k; : random disturbance at time k; : The remaining positioning budget constraint at time k, which represents the number of active positioning attempts remaining; : Uncertainty growth function, which is related to the remaining positioning budget. The smaller the budget, the slower the uncertainty growth; : The covariance matrix of the state estimate, representing the uncertainty of the state estimate; : Positioning decision variable, which is also a binary decision action output by the Transformer Q network , Indicates that the patrol drone performs active positioning to obtain the real status of the system. Indicates that the patrol UAV does not perform active positioning and estimates the system state through average particle filtering.

[0010] In some embodiments, for a given patrol UAV reference trajectory and active positioning budget, the key component of TQMPC is the interaction between TQ and MPC, including the following steps: TQ is based on status Determine the optimal positioning action ,This decision will affect the MPC’s belief state estimation and its subsequent control actions; The trajectory deviation calculated by MPC is used to improve the TQ's decision-making process. The trajectory tracking performance is used to guide the learning of the patrol UAV's positioning strategy, encouraging the TQ to prioritize positioning decisions that can reduce trajectory deviation. The dynamic exchange between the TQ localization strategy and the MPC control output establishes a feedback loop. By combining the budget-exceeding constraint and the trajectory tracking performance reward, the TQ learning is guided to learn to make adaptive active localization decisions while ensuring both the trajectory tracking performance and the state estimation accuracy of the patrol UAV. The total reward function of the TQ algorithm is as follows: ; in, : The overall reward function of DQN, used to train the positioning decision network; : The feedback reward obtained from the MPC, which evaluates the impact of the localization strategy on trajectory tracking; : The weight parameter that balances the MPC feedback and the state estimation error, with a value range of [0, 1]; : The true physical state value at time k; : The estimated value of the physical state at time k; : The error between the current true state and the estimated state, using the Euclidean distance; : The weight coefficient of the information gain term, which controls the proportion of long-term planning in the reward; : The information gain function, which evaluates the potential value of the current localization decision for future trajectory tracking; : The total localization budget, which represents the maximum number of active localizations allowed in the entire task; : The remaining localization budget, which represents the number of active localizations still available currently; : The remaining trajectory length or the remaining task complexity, which is used to adjust the information gain calculation; : A large negative reward when the budget is insufficient, which forces the DQN learning to comply with the budget constraint.

[0011] In some embodiments, the method further includes controlling each follower in the UAV cluster to maintain a set distance from the corresponding leader to form a formation movement, including the following steps: Controlling each follower in the UAV cluster to maintain a set distance from the corresponding leader to form a formation movement according to the distance-keeping reward function; The distance-keeping reward function is: ; Wherein, represents the reward given to the follower by the reward function; t represents the current time, represents the distance between the follower and the leader at time t, represents the distance between the follower and the leader at time t + 1, D1 represents the lower threshold of the distance between the follower and the leader, D2 represents the upper threshold of the distance between the follower and the leader, D1 < D2 is a positive constant, and the thresholds C1 > C2 > C3 represent the positive constant reward values obtained by the UAV in the interaction with the environment under different conditions.

[0012] To achieve the above object, on the other hand, an embodiment of the present application proposes a decision-making device for the patrol path of a UAV cluster under resource constraints, and the device includes: A graph construction unit, configured to discretize the actual physical positions to be patrolled to construct an undirected topological graph, where nodes represent the positions to be patrolled, and the edges between nodes represent the flight paths of UAVs; A steady-state distribution generation unit, configured to generate the target steady-state distribution of each patrol node according to the topological constraints of the constructed undirected topological graph, the importance of nodes, and the attack time; A transition matrix generation unit, configured to generate multiple state transition matrices with the same target steady-state distribution but different transition characteristics according to the multi-stage entropy-driven stochastic matrix optimization algorithm, and each state transition matrix represents the transition probability of the UAV cluster between each node; A path selection unit, configured to initialize the position of the UAV according to the initial state transition matrix and determine the path selected by the leader; A decision-making unit, configured to, according to the reference path selected by the leader, through the dynamic exchange of information between the TQ active positioning decision-making module and the MPC trajectory tracking control module of the leader's Transformer Q network model prediction control algorithm, achieve adaptive active positioning decision-making under the condition of satisfying the positioning constraints, and ensure the tracking effect of the selected path; A tracking unit, configured to, according to the path selection and tracking of the leader, the follower forms a UAV humanoid formation cluster by maintaining a set distance from the leader through a reward function; A formation control unit, configured to, according to each of the state transition matrices and the humanoid formation, automatically switch to the next state transition matrix when the initial transition matrix reaches the transition times threshold, and perform the selection and tracking of the random patrol path, and repeat the cycle to realize the intelligent patrol path decision-making of the unmanned cluster under resource constraints.

[0013] To achieve the above object, on the other hand, an embodiment of the present application proposes an electronic device, and the electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the above method is implemented.

[0014] To achieve the above object, on the other hand, an embodiment of the present application proposes a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above method is implemented.

[0015] The embodiments of the present application at least include the following beneficial effects: This application discretizes the actual physical positions to be patrolled to construct an undirected topological graph; generates the target steady-state distribution of each patrol node according to the topological constraints of the constructed undirected topological graph, the importance of the nodes, and the attack time; generates multiple transition matrices with the same steady-state distribution but different transition characteristics according to the multi-stage entropy-driven stochastic matrix optimization algorithm, and the generated path is more difficult to predict than that of the existing algorithm; initializes the position of the UAV according to the initial state transition matrix and determines the path selected by the leader; according to the reference path selected by the leader, the Transformer Q-network model prediction control algorithm of the leader realizes adaptive active positioning decision-making through the dynamic exchange of information between the TQ active positioning decision-making module and the MPC trajectory tracking control module under the condition of satisfying the positioning constraints, ensuring the tracking of the selected path, and the tracking effect is more accurate than that of the existing pure MPC path tracking algorithm; according to the path selection and tracking of the leader, the follower forms a UAV humanoid formation cluster by maintaining a set distance from the leader through a reward function, increasing the patrol visual range; according to each of the state transition matrices and the humanoid formation, when the initial transition matrix reaches the transition times threshold, it automatically switches to the next transition matrix, and performs the selection and tracking of the random patrol path, and repeats in this way to realize the intelligent patrol path decision-making of the unmanned cluster under resource constraints. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained without creative efforts based on these drawings.

[0017] Figure 1 It is a schematic flow chart of a method for making a patrol path decision for a UAV cluster under resource constraints provided by an embodiment of the present application; Figure 2 It is an example diagram of a Markov chain provided by an embodiment of the present application; Figure 3 It is a flow chart of the MEDRO algorithm provided by an embodiment of the present application; Figure 4 It is a flow chart of the TQMPC algorithm provided by an embodiment of the present application; Figure 5 It is a basic configuration of a humanoid formation provided by an embodiment of the present application; Figure 6 It is a schematic diagram of an extended formation of a humanoid formation provided by an embodiment of the present application; Figure 7 It is a schematic structural diagram of a device for making a patrol path decision for a UAV cluster under resource constraints provided by an embodiment of the present application; Figure 8A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0018] In order to make the objectives, technical solutions, and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0019] It can be understood that the terms "first", "second", etc. used in the present application may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if", "when" as used herein may be interpreted as "when...", "while...", or "in response to determining".

[0020] The terms "at least one", "a plurality of", "each", "any one", etc. used in the present application, at least one includes one, two, or more than two, a plurality of includes two or more than two, each refers to each of the corresponding plurality, and any one refers to any one of the plurality.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as understood by those skilled in the technical field to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0022] Before elaborating on the embodiments of the present application in detail, some related technologies involved in the embodiments of the present application are described first, as follows: Metropolis–Hastings algorithm: Metropolis–Hastings algorithm.

[0023] Transformer: A deep learning model construction.

[0024] TQ: Transformer Q-Networks, Transformer Q network.

[0025] MPC: Model Predictive Control, that is, Model Predictive Control.

[0026] MEDRO: Multi-stage Entropy-Driven Random-matrix Optimization, that is, Multi-stage Entropy-Driven Random-matrix Optimization.

[0027] TQMPC: Transformer Q-Networks Model Predictive Control, that is, Transformer Q Network Model Predictive Control.

[0028] Refer to Figure 1 , an embodiment of the present application provides a method for making a patrol path decision for an unmanned aerial vehicle cluster under resource constraints. This method may include, but is not limited to, S100 to S160, specifically as follows: S100: Discretize the actual physical positions to be patrolled to construct an undirected topological graph, where nodes represent the positions to be patrolled, and the edges between nodes represent the flight paths of the unmanned aerial vehicles; S110: Generate the target steady-state distribution of each patrol node according to the topological constraints of the constructed undirected topological graph, the importance of the nodes, and the attack time; S120: Generate multiple state transition matrices with the same target steady-state distribution but different transfer characteristics according to the multi-stage entropy-driven random matrix optimization algorithm. Each state transition matrix represents the transfer probability of the unmanned aerial vehicle cluster between each node; S130: Initialize the positions of the unmanned aerial vehicles according to the initial state transition matrix and determine the path selected by the leader; S140: According to the reference path selected by the leader, the Transformer Q-network model predictive control algorithm of the leader realizes adaptive active positioning decision-making through the dynamic exchange of information between the TQ active positioning decision-making module and the MPC trajectory tracking control module under the condition of satisfying the positioning constraints, and ensures the tracking effect of the selected path; S150: According to the path selection and tracking of the leader, the follower forms an anthropomorphic formation cluster of unmanned aerial vehicles by maintaining a set distance from the leader through a reward function; S160: According to each of the state transition matrices and the anthropomorphic formation, when the initial transition matrix reaches the transition times threshold, automatically switch to the next transition matrix, and perform the selection and tracking of the random patrol path, and repeat the cycle to realize the intelligent patrol path decision-making of the unmanned cluster under resource constraints.

[0029] Optionally, generating the target steady-state distribution of each patrol node according to the topological constraints of the constructed undirected topological graph, the importance of the nodes, and the attack time includes the following steps: Different nodes represent different positions that the drones need to patrol. Different nodes have different values, and it takes different times for an attacker to attack different nodes. The high-value target nodes and the nodes with short attack times are used as target nodes. The target nodes need to obtain more patrol resources, and the probability of the patrol drones accessing the target nodes is higher; The expression of the target steady-state distribution is as follows: ; ; where, , represents the i-th node, V represents the set of all the nodes, represents the target node value, represents the attack time, represents the node importance weight parameter, represents the node importance weight parameter, n is the total number of nodes, represents the target node steady-state distribution. The target steady-state distribution represents the probability distribution of the patroller at each target node after the long-term operation of the Markov chain.

[0030] Optionally, generating multiple state transition matrices with the same target steady-state distribution but different transition characteristics according to the multi-stage entropy-driven stochastic matrix optimization algorithm. Each state transition matrix represents the transition probability between the nodes of the drone cluster, including the following steps: The first stage: initialization and basic optimization: Use the double-tangent mapping function to generate a Q matrix that satisfies the topological constraints of the undirected topological graph; Limit the self-loop probability not to exceed the set value, and use the iterative optimization method with an exponential decay and modified factor adaptive learning rate to iteratively adjust to satisfy the steady-state equation; The second stage: entropy maximization: Calculate the maximum possible entropy, focus on optimizing the rows with low entropy ratios, and reallocate the non-diagonal probabilities while maintaining the self-loop constraint; Simulate the quantum superposition state, perform a steady-state fine-tuning every few iterations, and balance the entropy optimization and the steady-state constraint; The third stage: diversity enhancement: Calculate the MAD between matrices for dissimilarity evaluation; Introduce the GAN-like mechanism, identify the high-probability transitions and low-probability transitions of each generated matrix, analyze the similarity in multiple dimensions, and find the most critical difference points for enhancement; Analyze the average usage patterns of the existing matrix to identify the critical path. Transitions with high frequency usage are regarded as more important strategic decision points. Cascading differences are created among irrelevant transitions through threshold-triggered adjustment, binary decision in random directions, and probability reassignment mechanisms. Phase 4: Final fine-tuning: Reapply the balance of constraints, equilibrium, and diversity, synchronously optimize multiple objective functions, dynamically adjust the entropy optimization and differentiation intensity, resolve potential conflicts between entropy maximization and diversity maximization, and find the optimal balance point between entropy and diversity. When all preset constraint conditions are met, the P matrix is obtained as the candidate state transition matrix.

[0031] Optionally, the method further includes determining a control action according to the target state transition matrix and the reference trajectory, including the following steps: Use model predictive control to determine the control action according to the target state transition matrix and the reference trajectory. The objective function and constraint conditions of the model predictive control are: ; Wherein, : The control input sequence of the UAV from the current time k to the end of the prediction horizon; : The estimated value of the physical state at the i-th time step, including position and velocity; MPC uses the estimated value to plan the control action, and in the case of passive positioning, the current system state is represented by the estimated value. : The physical control input at the i-th time step; : The physical control input at the (i - 1)-th time step; : The reference trajectory state at the i-th time step, representing the target state that the system should track; : The weighted quadratic norm of the state tracking error, Q is the state error weight, used to assign different weights to different state components; : The weighted quadratic norm of the control input change, R is the control change weight, used to penalize the sudden change of the control input; : The estimated value of the physical state at the -th time step at the end of the time domain; : The reference trajectory state at the -th time step at the end of the time domain, representing the target state that the system should track; : The weighted quadratic norm of the state error at the end of the prediction horizon, where P is the terminal state weight; : The weight coefficients that balance the three objectives of state tracking, control smoothness, and terminal state constraints; : The prediction horizon length of the MPC, representing the number of time steps for which the MPC makes forward predictions; : The current time step of the UAV; : From k to k + H MPC within the prediction horizon.

[0032] Optionally, initialize the nodes where the UAV swarm is located, the active positioning budget constraint, and the initial state transition matrix, and determine the path selected by the leader, including the following steps: Input the initial UAV position and the active positioning budget constraint into Transformer Q, and select a random path through the state transition matrix. Transformer Q is used to optimize the positioning decision and learn where to perform active positioning on the selected path trajectory; The architecture of Transformer Q is as follows: State-action embedding design: For the continuous state , the embedding function is: , where s represents the continuous state vector, represents the weight matrix of the state embedding; represents the embedding dimension, represents the state dimension, represents the bias vector of the state embedding; GELU represents the Gaussian error linear unit activation function; LayerNorm represents the layer normalization function; State-action fusion: Fuse the state and action embeddings into a unified representation: , where represents the dimension of the fusion weight matrix, row 2 column, represents that the fusion bias vector belongs to the real number space of dimension, h represents the representation vector after fusing the state and action embeddings, represents the action vector, represents the state embedding vector, represents the action embedding vector, represents vector concatenation, Indicates the concatenation of the state embedding vector and the action embedding vector; Rotational position encoding is applied before attention calculation, and a multi-head self-attention mechanism is adopted. Each temporal gated Transformer block includes a self-attention sublayer, a gated convolution sublayer, and a feedforward network sublayer; Meta-training Transformer Q, different trajectories and different budget combinations form a task space, each specific task is a set Yes, among them, represents a specific reference trajectory sampled from the trajectory distribution, represents the specific active positioning budget value sampled from the budget distribution, sampling different tasks in the task space to train the TD algorithm to improve the generalization ability of the trained Transformer Q; The state update equation is as follows: ; in, : The estimated value of the physical state at the next moment k+1; : The real physical state value at the next moment k+1; : Information reliability attenuation factor, reflecting the attenuation of sensor information quality with state estimation error; : Information attenuation basic coefficient, which controls the basic strength of attenuation and has a value range of [0,1]; : Information decay rate adjustment parameter, which controls the sensitivity of decay to state error; : estimated value of the physical state at time k; : the real physical state value at time k; : The square of the Euclidean distance between the current estimated state and the true state; : System dynamics models that predict the next state based on the current state, control inputs, and random disturbances; : physical control input; : random perturbation; : The remaining positioning budget constraint at time k, which represents the number of active positioning attempts remaining; : Uncertainty growth function, which is related to the remaining positioning budget. The smaller the budget, the slower the uncertainty growth; : The covariance matrix of state estimation, representing the uncertainty of state estimation; : The positioning decision variable, which is also a binary decision action output by the Transformer Q network , indicates that the patrol UAV performs active positioning to obtain the true state of the system, indicates that the patrol UAV does not perform active positioning and estimates the system state through average particle filtering.

[0033] Optionally, for a given reference trajectory and active positioning budget of the patrol UAV, the key component of TQMPC is the interaction between two modules (TQ and MPC), including the following steps: TQ decides the optimal positioning action based on the state , and this decision affects the belief state estimation of MPC and its subsequent control actions; The trajectory deviation calculated by MPC is used to improve the decision-making process of TQ, guiding the learning of the patrol UAV positioning strategy through trajectory tracking performance, and motivating TQ to prefer positioning decisions that can reduce the trajectory deviation; The dynamic exchange between the TQ positioning strategy and the MPC control output establishes a feedback loop. By combining the budget constraint and the trajectory tracking performance reward, it guides TQ to learn to make adaptive active positioning decisions under the conditions of ensuring both the trajectory tracking performance of the patrol UAV and the accuracy of state estimation; The total reward function of the TQ algorithm is as follows: ; ; Among them, : The overall reward function of DQN, used to train the positioning decision network; : The feedback reward obtained from MPC, evaluating the impact of the positioning strategy on trajectory tracking; : The weight parameter balancing the MPC feedback and the state estimation error, with a value range of [0,1]; : The true physical state value at time k; : The estimated value of the physical state at time k; : The error between the current true state and the estimated state, using the Euclidean distance; : The weight coefficient of the information gain term, controlling the proportion of long-term planning in the reward; : Information gain function, which evaluates the potential value of the current positioning decision for future trajectory tracking; : Total positioning budget, which represents the maximum number of active positionings allowed in the entire mission; : Remaining positioning budget, which represents the number of active positionings still available currently; : Remaining trajectory length or remaining mission complexity, which is used to adjust the information gain calculation; : Large negative reward when the budget is insufficient, which forces DQN learning to comply with the budget constraint.

[0034] Optionally, the method further includes controlling each follower in the UAV cluster to maintain a set distance from the corresponding leader to form a formation movement, including the following steps: Controlling each follower in the UAV cluster to maintain a set distance from the corresponding leader to form a formation movement according to the distance-keeping reward function; The distance-keeping reward function is: ; Wherein, represents the reward given to the follower by the reward function; t represents the current moment, represents the distance between the follower and the leader at the t-th moment, represents the distance between the follower and the leader at the (t + 1)-th moment, D1 represents the lower threshold value of the distance between the follower and the leader, D2 represents the upper threshold value of the distance between the follower and the leader, D1 < D2 is a positive constant, and the thresholds C1 > C2 > C3 are positive constant reward values obtained by the UAV in the interaction with the environment under different conditions.

[0035] Next, the solution of the embodiment of the present application will be introduced and described in detail in combination with specific application examples.

[0036] This embodiment proposes a UAV swarm intelligent patrol path decision-making system and method: The leader realizes the selection and tracking of a stochastic path under resource constraints through the Multi-stage Entropy-Driven Random-matrix Optimization-Transformer Q Network Model Predictive Control (MEDRO-TQMPC) algorithm. The two algorithms are connected through a system environment module, and the system environment includes information such as a network topology map, steady-state conditions, and budget constraints. The leader's Multi-stage Entropy-Driven Random-matrix Optimization (MEDRO) algorithm dynamically adjusts and solves the stochastic path selection / planning problem with specific constraints through a transition matrix initial generation module and a transition matrix optimization and adjustment module. The leader's Transformer Q Network Model Predictive Control (TQMPC) algorithm dynamically exchanges information between a TQ active positioning decision-making module and an MPC trajectory tracking control module. The updated decision of TQ provides information for the control strategy of MPC, and the output result of MPC in turn optimizes the learning of TQ to ensure the tracking effect of the selected path under resource constraints. The follower forms a basic humanoid formation by maintaining a set distance from the leader through a reward function. On this basis, the formation is expanded in multiples of three to achieve intelligent patrol path decision-making for a cluster of 3 n ground unmanned systems.

[0037] The specific solution is as follows: 1. System modeling.

[0038] Discretize the physical environment according to the actual patrol situation of the UAVs and model it through an undirected topology map, where, represents the set of nodes (patrol positions). Each node in the undirected topology map represents a position in the real environment that needs to be patrolled and monitored by the UAVs, represents the set of edges, represents the probability of transferring from node to node and satisfies . and are not necessarily equal. represents the connection between node and , represents the path length between node and , represents the value of node . Attacker / drone attacking node The time required. Figure 2 Represents a Markov chain with five nodes and its corresponding transition matrix P, which is irreducible and non-periodic. The state transition matrix P is as follows: ; The transition probabilities between nodes are encoded in the transition matrix P. Each row of the matrix P represents a probability distribution (i.e., all values in each row add up to 1). The patrol drone starts from any initial node and then samples according to the probability distribution in the row corresponding to its current node to determine which area to patrol / inspect next.

[0039] 2. Multi-stage entropy-driven random matrix optimization (MEDRO) algorithm.

[0040] The Multi-stage Entropy Driven Random Matrix Optimization (MEDRO) algorithm is a four-stage combinatorial algorithm that efficiently solves the problem of generating random Markov transition matrices with specific constraints by generating steady-state distributions that meet special requirements based on a specific network topology.

[0041] 2.1 Generate the optimal steady-state distribution.

[0042] like Figure 2 The Markov chain shown, different nodes It represents the locations that different drones need to patrol. Different nodes have different values, and attackers need different amounts of time to attack different nodes. Therefore, target nodes (including high-value nodes and nodes with short attack times) need to obtain more patrol resources, and patrol drones have a higher probability of visiting these nodes. is a general steady-state distribution that represents the probability distribution of drones patrolling each node after the Markov chain runs for a long time. Therefore, it is necessary to design a steady-state distribution with specific optimization properties. Formula (1) shows how to calculate the steady-state distribution considering the target value and attack time.

[0043] (1) in, , , represents the i-th node, V represents the set of each of the nodes, Represents the target node The value of represents the attack time, Representative Node The importance weight parameter, Representative Node The importance weight parameter, n is the total number of nodes, Represents the steady-state distribution of the target nodes The target steady-state distribution represents the probability distribution of the patroller at each target node after the long-term operation of the Markov chain.

[0044] 2.2 The MEDRO algorithm generates the transition matrix.

[0045] To increase the prediction difficulty of the planned path of the patrol UAV, the MEDRO algorithm is used to generate multiple transition matrices P with the same steady-state distribution (Formula 1) but different transition characteristics, so as to implement a spatio-temporal decoupled random path selection strategy, as Figure 3 shown. Specifically, the multi-stage entropy-driven random matrix optimization (MEDRO) algorithm, with four stages responsible for initialization and basic optimization, entropy maximization, diversity enhancement, and final fine-tuning respectively, uses entropy maximization as one of the core optimization objectives and continuously improves the matrix through an iterative approach to meet the constraints of network topology, row sum equal to 1, and steady-state conditions.

[0046] The four stages are as follows: The first stage: Initialization and basic optimization.

[0047] 1. Random initialization: Use the double-tangent mapping function instead of linear probability assignment to generate a more complex random Q matrix that satisfies the topological constraints. The random initialization function is shown in formula (2), where represents the element in the i-th row and j-th column of the initial transition probability matrix Q, random() represents a function that generates random numbers in the interval [0,1], represents the adjacency matrix of the topological constraints, represents the element in the i-th row and k-th column of the adjacency matrix A, and n represents the total number of states.

[0048] (2) 2. Application of self-loop constraint: Limit the self-loop probability not to exceed the set value, , where represents the optimized self-loop probability (the probability of transitioning from state i to itself), represents the original self-loop probability (the probability of transitioning from state i to itself before optimization), The upper threshold of the self-loop probability. If the original self-loop probability , then keep the original value ; if the original self-loop probability , then keep the original value .

[0049] 3. Steady-state optimization: (1) An iterative optimization method with an adaptive learning rate, using exponential decay. When the error is large, a larger learning rate is used, and the learning rate is automatically reduced as the error decreases to avoid oscillations.

[0050] (2) Applying a correction factor: , where represents the original transition probability, the transition probability from state i to state j; represents the target steady-state probability, the steady-state probability of the desired state j; represents the currently estimated steady-state probability, the steady-state probability of state j under the current matrix; is the learning rate, iteratively adjusted to satisfy the steady-state equation , is the steady-state probability vector, represents the probability distribution when the Markov chain reaches the equilibrium state; the steady-state equation means that when the system reaches the steady state, the probability distribution no longer changes with time.

[0051] The second stage: Entropy maximization.

[0052] 1. Equalizing the distribution: (1) Calculating the maximum possible entropy: where represents the maximum possible entropy value, represents the number of feasible transitions from state i (the number of other states that can be transferred from state i).

[0053] (2) Moving towards a uniform distribution: , where represents the probability value of the system transferring from state i to state j in the random process, represents the average value of the transition probabilities of a certain row during the equalizing distribution process, represents the movement coefficient that controls the amplitude of the movement towards the uniform distribution.

[0054] 2. Adaptive entropy enhancement: (1) Focus on optimizing the rows with low entropy ratios, and reallocate the non-diagonal probabilities while maintaining the self-loop constraint.

[0055] (2) Simulate the quantum superposition state, allowing the matrix to evolve in multiple directions simultaneously and retaining the optimal results.

[0056] (3) Apply entropy enhancement only to the rows with entropy ratios lower than the target to avoid over-homogenization.

[0057] (4) Perform a steady-state fine-tuning every few iterations to balance entropy optimization and steady-state constraints.

[0058] The third stage: Diversity enhancement.

[0059] 1. Diversity assessment: Calculate the MAD (mean absolute difference) between matrices: ,in, It represents the average of the absolute value of the difference between the elements of matrix a and matrix b. represents the transition probability from state i to state j in the a-th matrix, represents the transition probability from state i to state j in the b-th matrix, K is the number of valid elements, which represents the total number of non-zero elements in the matrix involved in the calculation.

[0060] 2. Adversarial Adjustment: (1) Adopting innovative adversarial technology, a GAN-like mechanism is introduced, where the generator creates a new matrix and the discriminator evaluates predictability.

[0061] (2) Identify the high-probability and low-probability transitions of each generated matrix, deliberately reduce the high-probability transitions of the existing matrix in the new matrix, and improve the low-probability transition paths in the existing matrix.

[0062] (3) Analyze similarities in multiple dimensions, identify the most critical differences and enhance them, and make reverse adjustments to highly similar elements: if , further increase ; if , further reducing .

[0063] in, represents the transition probability from state i to state j in the a-th matrix, represents the transition probability from state i to state j in the bth matrix.

[0064] (4) Add a random factor random() to the adjustment to avoid generating completely predictable adversarial patterns and ensure that even adversarial reinforcement will not produce predictable anti-patterns.

[0065] 3. Critical path diversification: (1) History-based path importance assessment: Critical paths are identified by analyzing the average usage patterns of the existing matrix. Transfers with high frequency of use are considered to be more important strategic decision points.

[0066] (2) Threshold-triggered adjustment: Adjustment is triggered only when the current usage pattern is highly similar to the average pattern, ensuring that each critical path behaves differently in different matrices.

[0067] (3) Random directional binary decision: Randomly decide whether to increase or decrease the probability, maximize unpredictability, and adjust the range between 15% and 30%, which is enough to create a significant difference but not destroy the overall structure.

[0068] (4) Probability redistribution mechanism: When increasing / decreasing the probability of a certain transition, the probabilities of other transitions are correspondingly decreased / increased, maintaining the sum of total probabilities as 1 and creating cascading differences among unrelated transitions.

[0069] The fourth stage: Final fine-tuning.

[0070] 1. Reapply constraints: (1) Ensure that the self-loop probability is still within the limit range.

[0071] (2) Normalize each row so that its sum is 1.

[0072] (3) Re-optimize the steady-state distribution.

[0073] 2. Balance between equilibrium and diversity: Synchronously optimize multiple objective functions, dynamically adjust the entropy optimization and differentiation intensity, solve the potential conflict between entropy maximization and diversity maximization, and find the best balance point between entropy and diversity.

[0074] 3. Transformer Q-network model predictive control (TQMPC) algorithm.

[0075] To enable the UAV to better track the selected path, an algorithm combining deep reinforcement learning (Transformer Q-Networks, TQ) and model predictive control (MPC) is adopted. The updated decision of TQ provides information for the control strategy of MPC, and the output result of MPC in turn optimizes the learning of TQ to ensure the tracking effect of the UAV on the patrol trajectory. The TQMPC algorithm process is as Figure 4 shown.

[0076] 3.1 Model predictive control (MPC) trajectory tracking algorithm.

[0077] MPC realizes minimizing a predefined cost function by optimizing future control actions within a finite time range. Considering the UAV positioning constraints, the objective function and constraint conditions of MPC are shown in formula (3): (3) where : The control input sequence of the UAV from the current moment k to the end of the prediction horizon; : The physical state estimation value at the i-th time step, including position and velocity; MPC uses the estimation value to plan control actions, and represents the current system state through the estimation value in the case of passive positioning; : The physical control input at the i-th time step; : The physical control input at the (i - 1)-th time step; : The reference trajectory state at the i-th time step, representing the target state that the system should track; : The weighted quadratic norm of the state tracking error, where Q is a positive definite weight matrix (state error weight) that assigns different weights to different state components; : The weighted quadratic norm of the control input variation, where R is a positive definite weight matrix (control variation weight) used to penalize sudden changes in the control input; : The estimated physical state at the end of the time domain (the -th time step); : The reference trajectory state at the end of the time domain (the -th time step), representing the target state that the system should track; : The weighted quadratic norm of the state error at the end of the prediction horizon, where P is a positive definite terminal weight matrix (terminal state weight); : The weight coefficient that balances the three objectives of state tracking, control smoothness, and terminal state constraint; : The prediction horizon length of the MPC, representing the number of time steps that the MPC predicts forward; : The current time step of the UAV; : The time step index within the prediction horizon (ranging from k to k + H MPC ).

[0078] 3.2 Transformer Q (TQ) Network Learning Localization Decision Algorithm.

[0079] Transformer Q-Networks (TQ) are used to optimize the localization decision, enabling the UAV to learn when it is most effective to perform active localization and learn to use active localization at key points of the trajectory.

[0080] The input state representation of TQ consists of three components: (1) represents the average belief / estimate of the patrol UAV about the physical state of the system.

[0081] (2) is the variance of the belief, reflecting the uncertainty of the state estimate.

[0082] (3) is the remaining active positioning budget of the patrol UAV, indicating the remaining available number of active positioning times.

[0083] The output of TQ is a binary decision action , indicating that the patrol UAV performs active positioning to obtain the true state of the system, indicating that the patrol UAV does not perform active positioning and estimates the system state through average particle filtering.

[0084] Transformer Q-Networks architecture design: (1)State-action embedding design.

[0085] For continuous states , the embedding function is: , where s represents the continuous state vector, represents the weight matrix of state embedding; represents the embedding dimension (the dimension of the embedding vector), represents the number of dimensions of the state (the dimension of the state space), represents the bias vector of state embedding; GELU represents the Gaussian error linear unit activation function; LayerNorm represents the layer normalization function.

[0086] (2)State-action fusion.

[0087] Fuse the state and action embeddings into a unified representation: , where represents the dimension of the fusion weight matrix, row 2 column (state embedding dimension + 2 action embeddings dimension), represents that the fusion bias vector belongs to dimensional real space, h represents the representation vector after fusing the state and action embeddings, represents the action vector, represents the state embedding vector, represents the action embedding vector, represents vector concatenation, represents concatenating the state embedding vector and the action embedding vector.

[0088] (3)Rotary Position Encoding (RoPE).

[0089] Apply rotary position encoding before attention calculation: ; Among them, represents the rotation matrix applied to the query vector at position m on the dimension pair [2j, 2j + 1], represents the query vector at position m, represents the query vector value on the 2j dimension, represents the query vector value on the 2j + 1 dimension, m represents the position index in the sequence, is the frequency parameter, and the calculation formula is , d represents the total number of embedding dimensions of the model, used to control the rotation frequency of different dimension pairs, j represents the index of the dimension pair, represents the rotation angle.

[0090] (4)Multi-Head Self-Attention.

[0091] Multi-Head Self-Attention divides the input into h heads and calculates: ; Among them, , represents the output of multi-head self-attention, represents the number of attention heads, headi represents the i-th attention head, Concat represents the concatenation operation, which concatenates the outputs of multiple heads together, represents the output projection matrix, with dimensions , is the model dimension, represents the embedding representation of the input sequence, represents the result after the input X is transformed by the query (Query) weight matrix, i represents the index of the i-th attention head, ranging from 1 to h (total number of heads), represents the query weight matrix dedicated to the i-th attention head, represents the key (Key) weight matrix dedicated to the i-th attention head, represents the result after the input X is transformed by the key (Key) weight matrix, represents the value (Value) weight matrix dedicated to the i-th attention head, represents the result after the input X is transformed by the value (Value) weight matrix, Attention() represents the attention calculation function, which receives the query, key, and value as inputs.

[0092] (5)Temporal Gated Transformer Block.

[0093] Each Transformer block contains three sub-layers: a. Self-attention sub-layer: ; Among them, is the input tensor, representing the original input features processed by the model or the output of the previous layer. represents the output tensor after being processed by the self-attention sublayer. represents the residual connection coefficient of the self-attention sublayer, controlling the weight of the residual information. LayerNorm() represents the layer normalization function, MultiHead() represents the multi-head attention mechanism, allowing the model to focus on different positions and different representation subspaces, and Dropout() represents the random inactivation function, which prevents overfitting by randomly discarding some neurons.

[0094] b. Gated convolutional sublayer: The calculation of the gated linear convolutional unit (GLU): ; Among them, represents the final output tensor after being processed by the gated convolutional sublayer. represents the residual connection coefficient of the self-attention sublayer, controlling the weight of the residual information. represents the residual connection coefficient of the gated convolutional sublayer, controlling the weight of the residual information. LayerNorm() represents the layer normalization function, Conv1d() represents the one-dimensional convolutional operation, MultiHead() represents the multi-head attention mechanism, allowing the model to focus on different positions and different representation subspaces, Dropout() represents the random inactivation function, which prevents overfitting by randomly discarding some neurons, and GLU is the gated linear unit defined as and are the first and second halves of the input. is the element-wise multiplication operator. is the sigmoid function, mapping the input to a value between 0 and 1.

[0095] c. Feed-forward network sublayer: ; Among them, represents the output tensor of the gated convolutional sublayer, serving as the input to the feed-forward network sublayer. represents the final output tensor after being processed by the feed-forward network sublayer, completing the processing of the three sublayers of the entire Transformer block. represents the residual connection coefficient of the feed-forward network sublayer, controlling the weight of the residual information. LayerNorm() is the layer normalization function. represents the first weight matrix of the feed-forward network, used for the first linear transformation. represents the second weight matrix of the feed-forward network, used for the second linear transformation. represents the first bias vector of the feed-forward network, represents the second bias vector of the feed-forward network, and GELU() represents the Gaussian error linear unit activation function.

[0096] (6) Output of the Transformer Q network (TQ).

[0097] The Dueling architecture decomposes the Q-value into a state value function V(s) and an advantage function A(s,a): ; where, is the state-action value function, representing the expected cumulative return value of taking action in state , is the state value function, representing the intrinsic value of state , independent of which specific action is taken, is the advantage function, representing the advantage value of taking action relative to the average action in state , represents the advantage value of taking action in state represents the current environmental state, represents the currently selected action, represents any action in the action space, and |A| represents the size of the action space.

[0098] State value function: ; where, is the state value function, represents the output feature vector at the last position of the last layer of the Transformer, represents the weight matrix of the state value function, represents the transposed weight vector of the state value function, represents the first bias vector of the state value function, represents the second bias vector of the state value function, and GELU() represents the Gaussian error linear unit activation function.

[0099] Advantage function: ; where, is the advantage function, represents the output feature vector at the last position of the last layer of the Transformer, represents the weight matrix of the advantage function, represents the transposed weight vector of the advantage function, represents the first bias vector of the advantage function, represents the second bias vector of the advantage function, and GELU() represents the Gaussian error linear unit activation function.

[0100] (7) Training of the Transformer Q network (TQ).

[0101] Meta-training is performed on TQ, and the loss function is as follows: (4) where, represents the loss function, represents the trajectory distribution of the patrol drone, represents sampling from the distribution specific reference trajectory, represents the active positioning budget distribution of the patrol drone, represents sampling from the distribution specific budget value, represents the budget distribution related to the trajectory T, is the current network parameter, is the target network parameter, is the state at the current time k (including physical state belief, variance, and remaining budget), is the state at the next time k + 1, is the positioning action decision at the current time k (0 or 1), is the positioning action decision at the current time k, represents the actual tracking reference trajectory when the obtained reward, represents the discount factor, represents at the reference trajectory the state-action value predicted by the Q network using the parameter θ at the current time k, represents at the reference trajectory at the next time k + 1 using the target parameter the state-action value predicted by the Q network, represents at the reference trajectory the TD target value, represents the expectation with respect to the trajectory distribution T and the budget distribution B, represents the expectation with respect to the state and action, represents selecting the action with the largest Q value among all possible actions at the next time. Different trajectories and different budget combinations form the task space, and each specific task is a set pair. Different tasks are sampled on the task space to train the TD algorithm to improve the generalization ability of the trained Transformer Q.

[0102] 3.3 Transformer Q-network - Model Predictive (TQMPC) Algorithm.

[0103] For the reference trajectory and active localization budget of a given patrol drone, the key component of TQMPC is the interaction between two modules (TQ and MPC).

[0104] 3.3.1 Influence of TQ on MPC.

[0105] DQN determines the optimal localization action based on the state , and this decision affects the belief state estimation of MPC and its subsequent control actions. Equation (5) is the state update equation.

[0106] (5) where : Estimated value of the physical state at the next time step k + 1; : True physical state value at the next time step k + 1; : Information reliability decay factor, reflecting the decay of sensing information quality with the state estimation error; : Information decay base coefficient, controlling the basic intensity of decay, with a value range of [0, 1]; : Information decay rate adjustment parameter, controlling the sensitivity of decay to state error; : Estimated value of the physical state at time step k; : True physical state value at time step k; : Square of the Euclidean distance between the current estimated state and the true state; : System dynamics model, predicting the next state based on the current state, control input, and random perturbation; : Physical control input; : Random perturbation; : Remaining localization budget constraint at time step k, indicating the number of available active localization times currently; : Uncertainty growth function, related to the remaining localization budget, with less budget resulting in slower uncertainty growth; : Covariance matrix of state estimation, representing the uncertainty of state estimation; : The positioning decision variable. When (active positioning is selected), , the patrol UAV obtains the true physical state at this time , and the state estimation is equal to the true state; when (passive positioning is selected), , the state transition function is used for prediction, and the state estimation is not equal to the true state.

[0107] 3.3.2 Influence of MPC on TQ.

[0108] The expected trajectory deviation determined by MPC plays a crucial role in optimizing the TQ decision-making process. This relationship is formalized through a feedback mechanism, that is, the performance of MPC provides a learning signal for TQ , and this signal quantifies the control effectiveness of MPC under the current positioning strategy. The trajectory deviation calculated by MPC is used to improve the TQ decision-making process, guiding the learning of the patrol UAV positioning strategy through the trajectory tracking performance, and motivating TQ to preferentially select positioning decisions that can reduce the trajectory deviation.

[0109] (6) Among them, : The reward signal provided by MPC to DQN, used to evaluate the effect of the current positioning strategy; : The estimated value of the physical state at time i, representing physical quantities such as the position and speed of the UAV; : The reference trajectory state at time i, representing the ideal state where the UAV is expected to be; : The control input at time i, representing the control command acting on the UAV; : The control input at time i + 1, used together with the control input of the previous moment to calculate the control smoothness; k: The index of the current moment; H: The prediction horizon length, representing the number of steps that MPC predicts into the future; : The weight coefficient of the trajectory tracking accuracy, controlling the importance of the trajectory tracking error in the reward; : The weight coefficient of the control smoothness, controlling the importance of the control input change in the reward; : The weight coefficient of the information content, controlling the importance of the information matrix in the reward; : The cumulative state tracking error over the entire prediction horizon, where Q is the state error weight matrix; : The cumulative control variation over the entire prediction horizon, where R is the control variation weight matrix; : The trace of the information matrix, which measures the information content at the current state; : The Fisher information matrix, which represents the amount of information contained in the current state estimate; : Represents the time window or parameter related to the calculation of the information matrix; : The estimated value of the physical state at time k, representing physical quantities such as the position and velocity of the UAV.

[0110] As a weight parameter, it is used to adjust the strength of this feedback signal. is the feedback signal of the MPC performance to TQ, which is a reward scalar value. The negative sign in front of the formula ( ) indicates that this is a negative reward or penalty term. The larger the distance (the less accurate the trajectory tracking), the larger the absolute value of the negative reward.

[0111] The dynamic exchange between the TQ positioning strategy and the MPC control output establishes a feedback loop. The total reward function of the TQ algorithm is shown in formula (7): (7) Where, : The overall reward function of the DQN, which is used to train the positioning decision network; : The feedback reward obtained from the MPC, which evaluates the impact of the positioning strategy on trajectory tracking; : The weight parameter that balances the MPC feedback and the state estimation error, with a value range of [0, 1]; : The true physical state value at time k; : The estimated value of the physical state at time k; : The error between the current true state and the estimated state, using the Euclidean distance; : The weight coefficient of the information gain term, which controls the proportion of long-term planning in the reward; : The information gain function, which evaluates the potential value of the current positioning decision for future trajectory tracking; : Total positioning budget, representing the maximum number of active positionings allowed in the entire mission; : Remaining positioning budget, representing the number of active positionings still available currently; : Remaining trajectory length or remaining task complexity, used to adjust the information gain calculation; : Large negative reward when the budget is insufficient, forcing DQN learning to abide by the budget constraint.

[0112] Among them, is the feedback control performance signal of MPC performance to TQ (Formula 6), is the true physical state at time k, the state estimation (state average belief) at time k, is the error (norm) between the true state and the estimated state considering weights, is a large negative reward when exceeding the active positioning budget. Focus on the trajectory tracking performance, focus on the state estimation accuracy. Through the combination of these two, guide TQ to learn to make adaptive active positioning decisions under the conditions of ensuring both the trajectory tracking performance of the patrol UAV and the state estimation accuracy.

[0113] 4. Humanoid formation cluster formation control.

[0114] The basic formation of the patrol UAV humanoid formation consists of 1 leader and 2 followers. The leader realizes the selection and tracking of a random patrol path under resource constraints through the MEDRO-TQMPC algorithm. The followers form a basic humanoid formation by keeping a certain distance from the leader through the reward function. On this basis, the formation is expanded in multiples of three to realize the intelligent patrol path decision-making of a 3 n UAV cluster under resource constraints.

[0115] 4.1 Design of the basic humanoid formation reward function.

[0116] By designing the reward function, the leader and follower UAVs are kept at a certain distance, as Figure 5 shown. Set distances D1 and D2 so that the followers finally reach the four positions above, below, left, and right of the leader. The leader is the center of the circular area. When the distance between the follower and the leader is less than D1, a negative reward will be obtained; when the distance between the follower and the leader is greater than D2, a negative reward will also be obtained; when the follower is in the circular area, a positive reward will be obtained for maintaining a suitable distance.

[0117] According to the above design, the reward function for the leader and follower to keep a distance is expressed by Equation (8).

[0118] (8) Among them, represents the reward given by the reward function to the follower; t represents the current moment, represents the distance between the follower and the leader at moment t, represents the distance between the follower and the leader at moment t + 1, D1 represents the lower threshold of the distance between the follower and the leader, D2 represents the upper threshold of the distance between the follower and the leader, D1 < D2 is a positive constant, and the thresholds C1 > C2 > C3 represent the positive constant reward values obtained by the UAV in the interaction with the environment under different conditions.

[0119] 4.2 Extended design of humanoid formation.

[0120] For the basic configuration of the humanoid formation of UAVs, the UAV at the very front is the command UAV, and the two UAVs connected to it are the follower UAVs. A reward function is set to reward the behavior of the follower maintaining a certain distance from the commander, so that the follower tends to stay on the arc of a circle with a certain distance from the commander. As Figure 6 shown, the light red area is set as the sphere of influence of each UAV, and the red area is set as the single-step operation range of the UAV, that is, the no-fly zone of the UAV, so that while the follower maintains a certain distance from the commander, they can also maintain an appropriate distance from each other. At the same time, the radius of the arc is determined as the diagonal distance of the sphere of influence ( does not directly appear in formula 8, but is reflected in Figure 6 the radius of the arc in a, which is a predefined fixed parameter. Based on this fixed parameter, the values of D1 and D2 are determined), and the parameter is the ideal distance that the follower is expected to maintain from the leader in the design, that is, the radius of the arc; and are the actual distance measurement values, which will change with time; D1 and D2 are the threshold boundaries set based on , and they satisfy the condition D1 < < D2, where D1 and D2 are the upper and lower integer values of respectively.

[0121] Referring to Figure 6 , on the basis of a three-UAV formation, it is extended to a nine-UAV formation ( Figure 6 b), that is, a middle group composed of three three-UAV groups. The group at the very front is the leading group, and the two groups connected to it are the following groups. The sphere of influence is determined as three diagonal distances, that is, , D1 < < D2. Based on the above content, it is further extended to 27 ( Figure 6c) Aircraft formation, so that in the reward function where the following group tends to stay on the arc of a circle with a certain grid distance from the command group, D1 and D2 are determined to be nine grid distances, that is, D1 < 9 <D2.

[0122] In summary, this embodiment includes the following technical solutions: 1. A novel UAV swarm intelligent patrol path decision-making system and method are proposed: Multi-stage Entropy-driven Random Matrix Optimization - Transformer Q Network Model Predictive Control (MEDRO-TQMPC), which simultaneously considers UAV path selection, path tracking, and formation expansion under positioning constraints.

[0123] 2. The Multi-stage Entropy-driven Random Matrix Optimization (MEDRO) algorithm is developed. The four stages are responsible for initialization and basic optimization, entropy maximization, diversity enhancement, and final fine-tuning respectively. Using entropy maximization as one of the core optimization objectives, the matrix is continuously improved through an iterative approach to meet the constraints of network topology, row sum of 1, and steady-state conditions.

[0124] 3. The developed Transformer Q Network Model Predictive Control (TQMPC) algorithm (and its related mathematical formulas) realizes the adaptive active positioning decision of UAVs by dynamically exchanging information between the TQ active positioning decision module and the MPC trajectory tracking control module under the condition of meeting the positioning constraints, ensuring the tracking effect of the selected path.

[0125] 4. A special leader-follower mechanism. The leader integrates a more intelligent MEDRO-TQMPC algorithm. The follower UAVs are only responsible for dynamic tracking and obstacle avoidance through the reward function. The number of UAVs is expanded based on 3, rather than any number.

[0126] The beneficial effects of this embodiment include: This embodiment integrates a UAV intelligent patrol decision-making system and method. The path generated by the leader's multi-stage entropy-driven random matrix optimization (MEDRO) algorithm has highly randomized characteristics and is more difficult to predict than the paths generated by existing algorithms. The Transformer Q Network Model Predictive Control (TQMPC) algorithm ensures the tracking effect of the patrolling UAVs on the selected path under resource constraints by dynamically exchanging information between the TQ active positioning decision module and the MPC trajectory tracking control module, and realizes the adaptive active positioning decision of UAVs under the condition of meeting the positioning constraints. The tracking effect is more accurate than that of existing pure MPC path tracking algorithms. By designing the reward function, a humanoid formation basic configuration based on leader-follower is realized, and the formation is expanded based on 3 to increase the patrol visual range.

[0127] Refer to Figure 7, the embodiment of the present application further provides a decision-making device for the patrol path of an unmanned aerial vehicle (UAV) cluster under resource constraints, which realizes the above-mentioned decision-making method for the patrol path of an unmanned aerial vehicle cluster under resource constraints. The device includes: A graph construction unit, configured to discretize the actual physical positions to be patrolled to construct an undirected topological graph, where nodes represent the positions to be patrolled, and the edges between nodes represent the flight paths of UAVs; A steady-state distribution generation unit, configured to generate the target steady-state distribution of each patrol node according to the topological constraints of the constructed undirected topological graph, the importance degree of nodes, and the attack time; A transition matrix generation unit, configured to generate multiple state transition matrices with the same target steady-state distribution but different transition characteristics according to the multi-stage entropy-driven stochastic matrix optimization algorithm, and each state transition matrix represents the transition probability of the UAV cluster between each node; A path selection unit, configured to initialize the position of the UAV according to the initial state transition matrix and determine the path selected by the leader; A decision-making unit, configured to, according to the reference path selected by the leader, through the dynamic exchange of information between the TQ active positioning decision-making module and the MPC trajectory tracking control module of the leader's Transformer Q network model prediction control algorithm, realize adaptive active positioning decision-making under the condition of satisfying the positioning constraints, and ensure the tracking effect of the selected path; A tracking unit, configured to, according to the path selection and tracking of the leader, the follower forms a humanoid formation cluster of UAVs by maintaining a set distance from the leader through a reward function; A formation control unit, configured to, according to each of the state transition matrices and the humanoid formation, automatically switch to the next state transition matrix when the initial transition matrix reaches the transition times threshold, and perform the selection and tracking of the random patrol path, and repeat the above process to realize the intelligent patrol path decision-making of the unmanned cluster under resource constraints.

[0128] It can be understood that the content in the above method embodiment is applicable to the device embodiment of the present application. The functions specifically implemented by the device embodiment of the present application are the same as those in the above method embodiment, and the beneficial effects achieved are also the same as those in the above method embodiment.

[0129] The embodiment of the present application further provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the method of the embodiment of the present application is realized. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0130] It can be understood that the content in the above method embodiments is applicable to the device embodiments of the present application. The functions specifically implemented by the device embodiments of the present application are the same as those of the method of the present application, and the beneficial effects achieved are also the same as those of the method of the present application.

[0131] Please refer to Figure 8 , Figure 8 which schematically shows the hardware structure of an electronic device according to another embodiment. The electronic device includes: A processor 801, which can be implemented in ways such as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided by the embodiments of the present application; A memory 802, which can be implemented in forms such as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 can store an operating system and other application programs. When implementing the technical solutions provided by the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 802 and are called by the processor 801 to execute the methods of the embodiments of the present application; An input / output interface 803, which is used to implement information input and output; A communication interface 804, which is used to implement communication interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or through wireless means (such as mobile network, WIFI, Bluetooth, etc.); A bus 805, which transmits information between the various components of the device (such as the processor 801, the memory 802, the input / output interface 803, and the communication interface 804); Among them, the processor 801, the memory 802, the input / output interface 803, and the communication interface 804 are communicatively connected to each other inside the device through the bus 805.

[0132] The embodiments of the present application also provide a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method of the present application is implemented.

[0133] It can be understood that the content in the above method embodiments is applicable to this storage medium embodiment. The functions specifically implemented by this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those in the above method embodiments.

[0134] As a non-transitory computer-readable storage medium, a memory can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory can include high-speed random access memory, and can also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory optionally includes a memory remotely disposed relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0135] The embodiments described in the embodiments of the present application are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0136] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than those shown in the figures, or combine certain steps, or different steps.

[0137] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0138] Those of ordinary skill in the art can understand that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices can be implemented as software, firmware, hardware, and appropriate combinations thereof.

[0139] In the description of this application and the above-mentioned accompanying drawings, terms such as "first", "second", "third", "fourth", etc. (if any) are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of the present application described here can be implemented in an order different from those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that comprises a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0140] It should be understood that in this application, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects and indicates that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and both A and B exist at the same time. Here, A and B can be singular or plural. The character " / " generally means that the associated objects before and after are in an "or" relationship. "At least one (one) of the following" or its similar expression means any combination of these items, including any combination of single items (ones) or plural items (ones). For example, at least one (one) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0141] In several embodiments provided by this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the above-mentioned division of units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of devices or units can be in electrical, mechanical or other forms.

[0142] The units described above as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0143] In addition, in each embodiment of the present application, each functional unit can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0144] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in each embodiment of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs that can store programs.

[0145] The preferred embodiments of the embodiments of the present application have been described above with reference to the accompanying drawings. This does not limit the scope of the rights of the embodiments of the present application. Any modifications, equivalent replacements, and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the rights of the embodiments of the present application.

Claims

1. A decision-making method for the patrol path of an unmanned aerial vehicle (UAV) cluster under resource constraints, characterized in that, The method includes the following steps: Discretize the physical locations to be actually patrolled to construct an undirected topological graph, where the nodes represent the locations to be patrolled, and the edges between the nodes represent the flight paths of the drones; Generate the target steady-state distribution of each patrol node according to the topological constraints of the constructed undirected topological graph, the importance degree of the nodes, and the attack time; Generate multiple state transition matrices with the same target steady-state distribution but different transfer characteristics according to the multi-stage entropy-driven stochastic matrix optimization algorithm, and each state transition matrix represents the transfer probability of the drone cluster between each node; Initialize the positions of the drones according to the initial state transition matrix and determine the path selected by the leader; According to the reference path selected by the leader, the prediction control algorithm of the leader's Transformer Q network model realizes adaptive active positioning decision-making by dynamically exchanging information between the TQ active positioning decision-making module and the MPC trajectory tracking control module under the condition of satisfying the positioning constraints, and ensures the tracking effect of the selected path; According to the path selection and tracking of the leader, the followers form a humanoid formation cluster of drones by maintaining a set distance from the leader through a reward function; According to each of the state transition matrices and the humanoid formation, when the initial transition matrix reaches the transition times threshold, automatically switch to the next transition matrix, execute the selection and tracking of the random patrol path, and repeat the cycle to realize the intelligent patrol path decision-making of the unmanned cluster under resource constraints.

2. A method for making a decision on the patrol path of an unmanned aerial vehicle cluster under resource constraints according to claim 1, characterized in that, The generating the target steady-state distribution of each patrol node according to the topological constraints of the constructed undirected topological graph, the importance degree of the nodes, and the attack time includes the following steps: Different nodes Represent different positions that the drones need to patrol. Different nodes have different values. It takes different amounts of time for the attacker to attack different nodes. The high-value target nodes and the nodes with short attack times are taken as target nodes. The target nodes need to obtain more patrolling resources, and the probability that the patrolling drones visit the target nodes is higher; The expression of the target steady-state distribution is: ； ； Among them, , represents the i-th node, V represents the set of all the nodes, represents the target node value, represents the attack time, represents the node importance weight parameter, represents the node importance weight parameter, n is the total number of nodes, represents the target node steady-state distribution, the target steady-state distribution represents the probability distribution of the patroller at each target node after the Markov chain runs for a long time.

3. A method for making a patrol path decision for an unmanned aerial vehicle cluster under resource constraints according to claim 1, characterized in that The generating multiple state transition matrices with the same target steady-state distribution but different transfer characteristics according to the multi-stage entropy-driven stochastic matrix optimization algorithm, and each state transition matrix represents the transfer probability of the drone cluster between each node includes the following steps: The first stage: initialization and basic optimization: Generate a Q matrix that satisfies the topological constraints of the undirected topological graph using a double tangent mapping function; Limit the self-loop probability not to exceed the set value, and use an iterative optimization method with an adaptive learning rate of exponential decay and correction factor to iteratively adjust to satisfy the steady-state equation; The second stage: entropy maximization: Calculate the maximum possible entropy, focus on optimizing the rows with low entropy ratio, and reallocate the non-diagonal probabilities while maintaining the self-loop constraint; Simulate the quantum superposition state, perform a steady-state fine-tuning every few iterations to balance entropy optimization and steady-state constraints; The third stage: diversity enhancement: Calculate the MAD between matrices for dissimilarity evaluation; Introduce a GAN-like mechanism to identify the high-probability transfers and low-probability transfers of each generated matrix, analyze the similarity in multiple dimensions, and find the most critical difference points for enhancement; Analyze the average usage pattern of the existing matrices to identify the critical paths, and the transfers with high frequency of use are regarded as more important strategic decision points, and create cascading differences between unrelated transfers through a threshold-triggered adjustment, random direction binary decision, and probability reallocation mechanism; The fourth stage: final fine-tuning: Reapply the balance of constraints, equilibrium, and diversity, synchronously optimize multiple objective functions, dynamically adjust the entropy optimization and differentiation intensity, resolve the potential conflict between entropy maximization and diversity maximization, and find the optimal balance point between entropy and diversity; After all preset constraint conditions are satisfied, obtain the P matrix as the candidate state transition matrix.

4. A method for making a decision on the patrol path of an unmanned aerial vehicle cluster under resource constraints according to claim 1, characterized in that, The method further includes determining a control action according to the target state transition matrix and the reference trajectory, including the following steps: Use model predictive control to determine the control action according to the target state transition matrix and the reference trajectory; The objective function and constraint conditions of the model predictive control are: ； Among them, : The control input sequence of the UAV from the current moment k to the end of the prediction horizon; : The estimated physical state at the i-th time step, including position and velocity; the MPC uses the estimated value to plan control actions, and in the case of passive positioning, the current system state is represented by the estimated value; : The physical control input at the i-th time step; : The physical control input at the (i-1)-th time step; : The reference trajectory state at the i-th time step, representing the target state that the system should track; : The weighted quadratic norm of the state tracking error, where Q is the state error weight used to assign different weights to different state components; : The weighted quadratic norm of the control input variation, where R is the control variation weight used to penalize sudden changes in the control input; : Physical state estimate value at the th time step at the end of the time domain; : The reference trajectory state at the th time step at the end of the time domain, representing the target state that the system should track; : The weighted quadratic norm of the terminal state error of the prediction horizon, where P is the terminal state weight; : Weight coefficients for three objectives of equilibrium state tracking, control smoothing, and terminal state constraint; : The prediction horizon length of the MPC, which represents the number of time steps for the MPC to predict forward; : The current time step of the UAV; : Time step indices from k to k + H MPC within the prediction horizon.

5. A method for making a decision on the patrol path of an unmanned aerial vehicle cluster under resource constraints according to claim 1, characterized in that, Initialize the nodes where the unmanned aerial vehicle (UAV) cluster is located, the active positioning budget constraint, and the initial state transition matrix, and determine the path selected by the leader, including the following steps: Input the initial position of the UAV and the active positioning budget constraint into Transformer Q, select a random path through the state transition matrix, and Transformer Q is used to optimize the positioning decision and learn at which position on the selected path trajectory to perform active positioning; The architecture of Transformer Q is as follows: State-action embedding design: For continuous states , the embedding function is: , where s represents the continuous state vector, represents the weight matrix of state embedding; represents the embedding dimension, represents the state dimension, represents the bias vector of state embedding; GELU represents the Gaussian error linear unit activation function; LayerNorm represents the layer normalization function; State-action fusion: Fuse the state and action embeddings into a unified representation: , where represents the dimension of the fusion weight matrix, row 2 column, indicates that the fusion bias vector belongs to the real number space of dimension h represents the representation vector after fusing the state and action embeddings, represents the action vector, represents the state embedding vector, represents the action embedding vector, represents vector concatenation, indicates concatenating the state embedding vector and the action embedding vector; Apply rotational position encoding before attention calculation, and adopt the multi-head self-attention mechanism. Each temporal gated Transformer block includes a self-attention sublayer, a gated convolutional sublayer, and a feed-forward network sublayer; Perform meta-training on Transformer Q. Different combinations of trajectories and budgets form a task space, and each specific task is a set pair. Among them, represents a specific reference trajectory sampled from the trajectory distribution, represents a specific active localization budget value sampled from the budget distribution. Sample different tasks on the task space to train the TD algorithm to improve the generalization ability of the trained Transformer Q; The state update equation is shown as follows: ； Among them, : The estimated value of the physical state at the next moment k + 1; : The true physical state value at the next moment k+1; : Information reliability attenuation factor, which reflects the attenuation of the quality of sensing information with the state estimation error; : Information attenuation base coefficient, which controls the basic intensity of attenuation and ranges from [0, 1]; : Information attenuation rate adjustment parameter, which controls the sensitivity of attenuation to the state error; : Estimated value of the physical state at time k; : The true physical state value at time k; : The square of the Euclidean distance between the current estimated state and the true state; : A system dynamics model that predicts the next state based on the current state, control inputs, and random perturbations; : Physical control input; : Random perturbation; : The remaining positioning budget constraint at time k, indicating the current remaining number of active positioning attempts; : The uncertainty growth function, which is related to the remaining positioning budget. The less the budget is, the slower the uncertainty grows; : The covariance matrix of state estimation, representing the uncertainty of state estimation; : The positioning decision variable, which is also a binary decision action output by the Transformer Q network , indicates that the patrolling UAV performs active positioning to obtain the true state of the system, indicates that the patrolling UAV does not perform active positioning and estimates the system state through the average particle filter.

6. The method for making a UAV cluster patrol path decision under resource constraints according to claim 1, wherein For the reference trajectory and active positioning budget of a given patrol UAV, the key component of TQMPC is the interaction between TQ and MP, including the following steps: TQ Based on Status Determine the optimal positioning action , and this decision will affect the belief state estimation of MPC and its subsequent control actions; The trajectory deviation calculated by MPC is used to improve the decision-making process of TQ, guide the learning of the patrol UAV positioning strategy through the trajectory tracking performance, and encourage TQ to preferentially select positioning decisions that can reduce the trajectory deviation; The dynamic exchange between the TQ positioning strategy and the MPC control output establishes a feedback loop. By combining the rewards for exceeding the budget constraint and the trajectory tracking performance, guide TQ to learn to adaptively make active positioning decisions under the condition of ensuring both the trajectory tracking performance of the patrol UAV and the accuracy of state estimation; The total reward function of the TQ algorithm is shown as follows: ； Among them, : The overall reward function of DQN, which is used to train the positioning decision-making network; : The feedback reward obtained from MPC to evaluate the impact of the localization strategy on trajectory tracking; : The weight parameter that balances the MPC feedback and the state estimation error, with a value range of [0, 1]; : The true physical state value at time k; : The estimated value of the physical state at time k; : The error between the current true state and the estimated state, using the Euclidean distance; : The weight coefficient of the information gain term, which controls the proportion of the long-term plan in the reward; : Information gain function, which evaluates the potential value of the current localization decision for future trajectory tracking; : Total localization budget, representing the maximum number of active localizations allowed in the entire mission; : Remaining positioning budget, indicating the number of active positioning times still available currently; : The remaining trajectory length or remaining task complexity, used to adjust the information gain calculation; : Large negative rewards when the budget is insufficient, forcing DQN to learn to comply with budget constraints.

7. A method for making a decision on the patrol path of an unmanned aerial vehicle cluster under resource constraints according to claim 6, characterized in that The method further includes controlling each follower in the UAV cluster to maintain a set distance from the corresponding leader to form formation movement, including the following steps: Control each follower in the UAV cluster to maintain a set distance from the corresponding leader to form formation movement according to the reward function for maintaining distance; The reward function for maintaining distance is: ； Among them, represents the reward given by the reward function to the follower; t represents the current moment, represents the distance between the follower and the leader at time t, represents the distance between the follower and the leader at time t + 1, D1 represents the lower threshold of the distance between the follower and the leader, D2 represents the upper threshold of the distance between the follower and the leader, D1 < D2 is a positive constant, and the thresholds C1 > C2 > C3 represent the positive constant reward values obtained by the UAV in the interaction with the environment under different conditions.

8. An unmanned aerial vehicle cluster patrol path decision-making device under resource constraints, characterized in that, The device includes: A graph construction unit, configured to discretize the actual physical positions to be patrolled to construct an undirected topological graph, where nodes represent the positions to be patrolled, and the edges between nodes represent the UAV flight paths; A steady-state distribution generation unit, configured to generate the target steady-state distribution of each patrol node according to the topological constraints of the constructed undirected topological graph, the importance degree of the nodes, and the attack time; A transfer matrix generation unit, configured to generate multiple state transition matrices with the same target steady-state distribution but different transfer characteristics according to a multi-stage entropy-driven stochastic matrix optimization algorithm, where each state transition matrix represents the transfer probability of the UAV cluster between each node; A path selection unit, configured to initialize the position of the UAV according to an initial state transition matrix and determine the path selected by the leader; A decision-making unit, configured to, according to the reference path selected by the leader, through dynamic information exchange between the TQ active positioning decision-making module and the MPC trajectory tracking control module of the leader's Transformer Q network model prediction control algorithm, realize adaptive active positioning decision-making under the condition of satisfying the positioning constraints to ensure the tracking effect of the selected path; A tracking unit, configured to, according to the path selection and tracking of the leader, the follower forms a UAV anthropomorphic formation cluster by maintaining a set distance from the leader through a reward function; A formation control unit, configured to, according to each of the state transition matrices and the anthropomorphic formation, automatically switch to the next transfer matrix when the initial transfer matrix reaches the transfer times threshold, execute the selection and tracking of a random patrol path, and repeatedly cycle to realize the intelligent patrol path decision-making of the unmanned cluster under resource constraints.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the method described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the method described in any one of claims 1 to 7 is implemented.

Images