Intelligent distribution method, device, equipment and medium for unmanned aerial vehicle cluster resource distribution

Abstract

The invention relates to an intelligent distribution method and device for unmanned aerial vehicle cluster resource distribution, equipment and a medium. The method comprises the steps of obtaining basic information of communication networking and an unmanned aerial vehicle cluster, constructing a communication confrontation model to output position correlation parameters, then constructing a communication disturbance effect evaluation model to obtain a link suppression judgment threshold value, constructing a resource allocation optimization model in combination with hardware constraint and a double optimization target, and performing resource allocation. A Markov decision process model is built based on the above model, the model is trained to convergence through an MAPPO algorithm under a CTDE framework, an Actor network is configured to each unmanned aerial vehicle, and the unmanned aerial vehicles output a disturbance target selection result and a power distribution scheme according to local observation information. According to the method, disturbance efficiency maximization and resource consumption minimization are realized, the cluster cooperative disturbance capability and the resource utilization rate are improved, and resource allocation is more intelligent and accurate.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) swarm communication resource allocation technology, and in particular to an intelligent allocation method, apparatus, device and medium for UAV swarm resource allocation. Background Technology

[0002] With the rapid development of the low-altitude economy and the civilian drone industry, multi-drone collaborative operation technology is increasingly being applied in fields such as communication support, environmental monitoring, and emergency rescue. In civilian scenarios, drone swarms often need to perform signal modulation tasks on communication links in specific areas. For example, in emergency communication support, they may enhance the signal of faulty links or suppress communication links in specific airspaces to ensure the stability and security of legitimate communications.

[0003] Resource allocation is a core element in enabling drone swarms to perform signal control tasks. Individual drones have limited payload power and fixed flight time; therefore, rational resource allocation can maximize swarm operational efficiency and reduce energy consumption. As application scenarios expand, the complexity of collaborative scheduling of drone swarms increases exponentially, and traditional resource allocation strategies based on preset rules are no longer sufficient to meet the demands of dynamic environments.

[0004] Existing drone swarm resource allocation methods can be mainly divided into three categories: First, methods based on traditional optimization theory. These methods are prone to getting trapped in local optima when the type and quantity of resources increase, and the computational complexity increases exponentially. Second, swarm intelligence optimization algorithms. These methods have poor adaptability to dynamic environments, parameter adjustment depends on preset rules, and scalability is insufficient. Third, allocation strategies based on reinforcement learning. Some methods use decentralized training frameworks, resulting in low efficiency of multi-agent cooperation. Other methods directly transfer single-agent algorithms to multi-agent scenarios. When the number of drones increases, the input and output dimensions of the neural network increase sharply, making it difficult for the algorithm to converge.

[0005] In civilian multi-to-multi signal control scenarios, such as multi-drone collaborative support for urban emergency communication networks and collaborative management of multi-space communication links, higher demands are placed on the collaborative decision-making capabilities of drone swarms. Existing technologies cannot efficiently solve the challenges of resource allocation and collaborative scheduling in large-scale scenarios. Therefore, there is an urgent need for an intelligent allocation method that can adapt to large-scale swarms and balance collaborative efficiency with resource utilization to improve the operational performance of drone swarms in the civilian field. Summary of the Invention

[0006] Therefore, it is necessary to provide a method, device, equipment, and medium for intelligent allocation of drone swarm resources that can adapt to large-scale swarms and take into account both collaborative efficiency and resource utilization, in order to address the above-mentioned technical problems.

[0007] A method for intelligent allocation of resources in a drone swarm, the method comprising:

[0008] Acquire basic information about the communication network containing multiple target communication links within a three-dimensional target area, as well as basic information about the UAV swarm; Based on the aforementioned basic information, the mission area of ​​the UAV and the distribution range of the target communication links are determined by constructing a communication adversarial model, and the position association parameters between the UAV and the target receiver are output. Based on the line-of-sight channel model, the channel gain between the UAV and the target receiver is calculated by combining the position correlation parameters. The signal-to-interference ratio is not less than the preset suppression coefficient as the criterion for successful link control. A communication disturbance effect evaluation model is established, and the link suppression judgment threshold is obtained. Based on the constraints of the maximum transmit power of the UAV and the maximum number of disturbance links of a single UAV, combined with the link suppression judgment threshold, and with the optimization objective of maximizing disturbance effectiveness and minimizing resource consumption as the core, a UAV cluster resource allocation optimization model is constructed. Based on the aforementioned communication countermeasure model, communication disturbance effect evaluation model, and UAV swarm resource allocation optimization model, a Markov decision process model is constructed. Under the framework of centralized training and distributed execution, the Markov decision process model is trained based on the multi-agent proximal policy optimization algorithm until the reward function value converges, thus obtaining the trained policy model. The Actor network in the trained policy model is configured into each of the UAVs. Each UAV inputs its own local observation information into the Actor network to obtain the perturbation target selection result and power allocation scheme.

[0009] In one embodiment, the basic information of the communication network includes the transmitter location, receiver location, transmission power, center frequency, and importance index of each target communication link; The basic information of the drone swarm includes the location information of each drone, its maximum transmission power, and the maximum number of disturbance links per drone.

[0010] In one embodiment, the channel gain between the UAV and the target receiver is calculated based on a line-of-sight channel model, and is expressed as:

[0011] In the above formula, This indicates the fixed gain provided by the antenna and power amplifier. This represents the multipath fading gain that exhibits an exponential distribution. This represents path loss and satisfies the log-normal distribution characteristic. This represents the path loss index. This indicates the distance between the transmitter and receiver in the target communication link.

[0012] In one embodiment, the drone swarm resource allocation optimization model is expressed as:

[0013] In the above formula, This represents the signal-to-interference ratio, where... This represents the allocation factor for the drone's perturbation beam. Indicates drone For the receiver The power of the transmitted disturbance. Indicates the target receiver The suppression coefficient, This indicates the maximum transmit power of a single drone. This indicates the maximum number of target communication links that a single drone can simultaneously disrupt. , These represent the target communication link set and the drone set, respectively. This indicates the number of drones in the collection.

[0014] In one embodiment, the Markov decision process model includes a state space, an action space, and a reward function. The state space includes target link location information, UAV cluster location information, and historical resource allocation strategies. The action space includes UAV perturbation target selection and perturbation power allocation. The reward function correlates link suppression effect with resource consumption cost.

[0015] In one embodiment, during the training of the Markov decision process model: The global value function is calculated by a centralized Critic network based on global state information, which includes the location correlation parameters output by the communication adversarial model, the link suppression judgment threshold output by the communication disturbance effect evaluation model, and the constraints and optimization objectives output by the UAV swarm resource allocation optimization model. Based on the global value function, the policy advantage value of each UAV is estimated by combining the generalized advantage estimation method. At the same time, the parameter update range of each distributed Actor network is limited by the policy pruning mechanism to achieve iterative optimization.

[0016] In one embodiment, the convergence criterion for the reward function value is: within a preset number of consecutive training rounds, the mean fluctuation of the reward function is less than a preset threshold, and the mean fluctuation state of the reward function continues until the end of training.

[0017] A smart allocation device for unmanned aerial vehicle (UAV) swarm resources, the device comprising: The basic information acquisition module is used to acquire basic information about the communication network containing multiple target communication links within a three-dimensional target area, as well as basic information about the UAV cluster. The communication countermeasure model construction module is used to determine the mission area of ​​the UAV and the distribution range of the target communication link by constructing a communication countermeasure model based on the basic information, and output the position association parameters between the UAV and the target receiver. The communication disturbance effect evaluation model construction module is used to calculate the channel gain between the UAV and the target receiver based on the line-of-sight channel model and the location correlation parameters. The ratio of the target communication link disturbance signal to the effective signal is not less than a preset suppression coefficient as the criterion for successful link control. The module establishes a communication disturbance effect evaluation model and obtains the link suppression judgment threshold. The UAV swarm resource allocation optimization model construction module is used to construct a UAV swarm resource allocation optimization model based on the constraints of the maximum transmission power of the UAV, the maximum number of disturbance links of a single UAV, and the link suppression judgment threshold, with the core optimization objective of maximizing disturbance effectiveness and minimizing resource consumption. The Markov decision process model construction module is used to construct a Markov decision process model based on the communication adversarial model, the communication disturbance effect evaluation model, and the UAV swarm resource allocation optimization model. The model training module is used to train the Markov decision process model based on the multi-agent proximal policy optimization algorithm under the framework of centralized training and distributed execution until the reward function value converges, thus obtaining the trained policy model. The resource intelligent allocation module is used to configure the Actor network in the trained strategy model into each of the UAVs. Each UAV inputs its own local observation information into the Actor network to obtain the perturbation target selection result and power allocation scheme.

[0018] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in the above-described intelligent allocation method for unmanned aerial vehicle (UAV) swarm resources.

[0019] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described intelligent allocation method for unmanned aerial vehicle (UAV) swarm resources.

[0020] The aforementioned intelligent allocation method, device, equipment, and medium for UAV swarm resources construct a communication adversarial model based on basic information about the communication network and UAV swarm to determine the mission area of ​​the UAVs and the distribution range of target communication links. It outputs position correlation parameters between the UAVs and the target receiver. Based on a line-of-sight channel model, it calculates the channel gain between the UAVs and the target receiver using the position correlation parameters. A signal-to-interference-plus-suppression ratio (SINR) not less than a preset suppression coefficient is used as the criterion for successful link control. A communication disturbance effect evaluation model is established, and a link suppression judgment threshold is obtained. Based on constraints such as the maximum transmit power of the UAVs and the maximum number of disturbance links per UAV, combined with the link suppression judgment threshold, and... With the core optimization objective of maximizing perturbation effectiveness and minimizing resource consumption, a UAV swarm resource allocation optimization model is constructed. Based on the communication adversarial model, the communication perturbation effect evaluation model, and the UAV swarm resource allocation optimization model, a Markov decision process model is constructed. Under the framework of centralized training and distributed execution, the Markov decision process model is trained using a multi-agent proximal policy optimization algorithm until the reward function value converges, resulting in a trained policy model. The Actor network in the trained policy model is configured into each UAV, and each UAV inputs its local observation information into the Actor network to obtain the perturbation target selection result and power allocation scheme.

[0021] This method can accurately acquire basic information about communication networks and UAV swarms. By constructing a hierarchical communication adversarial model, a communication disturbance effect evaluation model, and a UAV swarm resource allocation optimization model, it clarifies the UAV operation boundaries, link suppression standards, and resource allocation constraints and optimization objectives. Combined with a Markov decision process model and a multi-agent near-end policy optimization algorithm under a centralized training and distributed execution framework, it achieves efficient training and convergence of intelligent allocation strategies. The trained Actor network is then configured to each UAV, enabling each UAV to independently output accurate disturbance target selection results and power allocation schemes based on its own local observation information. This effectively solves the problems of low efficiency, high resource consumption, and inaccurate decision-making in existing UAV swarm resource allocation. Under the premise of meeting UAV hardware constraints and link suppression requirements, it achieves the dual objectives of maximizing disturbance efficiency and minimizing resource consumption, improving the collaborative intelligent allocation capability and resource utilization efficiency of UAV swarms for multiple target communication links, and enhancing the intelligence and precision of intelligent resource allocation. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating an intelligent resource allocation method for a drone cluster in one embodiment. Figure 2 This is a schematic diagram of a drone swarm communication confrontation scenario in one embodiment; Figure 3This is a schematic diagram of a decentralized training and distributed execution architecture in one embodiment; Figure 4 This is a schematic diagram of a centralized training and distributed execution architecture in one embodiment; Figure 5 This is a schematic diagram of the multi-agent proximal policy optimization algorithm under a centralized training and distributed execution architecture in one embodiment. Figure 6 This is a diagram showing the comparison of average rewards for different algorithms during the training phase of an experiment. Figure 7 This is a diagram showing the comparison of average rewards for different algorithms during the testing phase of an experiment. Figure 8 This is a schematic diagram comparing the average perturbation performance of different algorithms during the testing phase of an experiment. Figure 9 This is a structural block diagram of an intelligent allocation device for drone cluster resource allocation in one embodiment; Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0024] In existing technologies, research on using reinforcement learning to solve many-to-many resource allocation problems remains relatively limited. Furthermore, most current resource allocation methods based on multi-agent reinforcement learning are essentially direct transfers and extensions of single-agent reinforcement learning algorithms, failing to fully adapt to the collaborative requirements of many-to-many resource allocation scenarios. When the number of agents increases, the large input and output dimensions of the neural network often make it difficult to converge to the optimal solution. In one embodiment, such as... Figure 1 As shown, an intelligent resource allocation method for unmanned aerial vehicle (UAV) swarms is provided, which specifically includes the following steps: Step S100: Obtain basic information on the communication network containing multiple target communication links within the three-dimensional target area, as well as basic information on the UAV cluster.

[0025] Step S110: Based on the basic information, the task area of ​​the UAV and the distribution range of the target communication link are determined by constructing a communication countermeasure model, and the position association parameters between the UAV and the target receiver are output.

[0026] Step S120: Based on the line-of-sight channel model, calculate the channel gain between the UAV and the target receiver by combining the position correlation parameters. Use the signal-to-interference ratio not less than the preset suppression coefficient as the criterion for successful link control. Establish a communication disturbance effect evaluation model and obtain the link suppression judgment threshold.

[0027] Step S130: Based on the constraints of the maximum transmit power of the UAV, the maximum number of disturbance links of a single UAV, and the link suppression judgment threshold, and with the optimization objective of maximizing disturbance effectiveness and minimizing resource consumption as the core, construct a UAV cluster resource allocation optimization model.

[0028] Step S140: Based on the communication adversarial model, the communication disturbance effect evaluation model, and the UAV swarm resource allocation optimization model, a Markov decision process model is constructed.

[0029] Step S150: Under the framework of centralized training and distributed execution, the Markov decision process model is trained based on the multi-agent proximal policy optimization algorithm until the reward function value converges, thus obtaining the trained policy model.

[0030] Step S160: Configure the Actor network in the trained policy model into each UAV. Each UAV inputs its own local observation information into the Actor network to obtain the perturbation target selection result and power allocation scheme.

[0031] In this method, the resource allocation optimization objective and the Markov decision process under this problem are modeled, and then the proposed CTDE-MAPPO algorithm is used to solve the collaborative resource allocation problem of UAV swarms. Experimental results show that, compared with existing algorithms such as SANER-PPO, DTDE-MAPPO, and Multi-Agent Soft Actor-Critic (MASAC), the proposed method can perform more stable and efficient policy learning in large-scale adversarial scenarios, and has a higher overall success rate in suppressing perturbations.

[0032] In this embodiment, a system model for the allocation of drone swarm resources is established, including a communication countermeasure model and a communication disturbance effect evaluation model. Then, for the resource allocation problem in the scenario of limited resources, a corresponding constraint combination optimization mathematical model is constructed.

[0033] In step S100, the basic information for the communication network includes the transmitter location, receiver location, transmission power, center frequency, and importance index of each target communication link. The basic information for the UAV swarm includes the location information of each UAV, its maximum transmission power, and the maximum number of disruptive links per UAV.

[0034] It should be noted here that, due to the complexity of actual application scenarios, and the relatively high difficulty for UAVs to obtain target parameter information, this method focuses on resource allocation strategies and does not involve analysis of reconnaissance technology. Therefore, it is assumed that the UAV has accurately obtained information such as target location, communication network transmitter transmission power and transmission frequency through reconnaissance means, and that the UAV has obtained the center frequency parameters of each target link by means of communication reconnaissance.

[0035] In step S110, based on the acquired basic information of the communication network and the basic information of the UAV cluster, a communication confrontation model adapted to the many-to-many communication confrontation scenario is constructed. Through this model, the mission area of ​​the UAV is delineated, the distribution range of the target communication links is clarified, and the position association parameters of the UAV and each target receiver are extracted and output, providing core spatial position basic data for subsequent channel gain calculation and disturbance effect evaluation.

[0036] Specifically, firstly, based on the geographical boundaries and mission requirements of the three-dimensional target area, and combined with the three-dimensional coordinates of the transmitters and receivers of each target communication link in the communication network, the overall distribution range of the M target communication links within the three-dimensional target area is determined. Simultaneously, considering the flight altitude restrictions and maneuver range constraints of each UAV in the UAV swarm, a dedicated mission area for each UAV is delineated to ensure that the coverage of the UAV mission area effectively aligns with the distribution range of the target communication links, meeting the spatial operation requirements for swarm coordinated disturbance. Secondly, the channel characteristics of the communication network are defined, setting each target communication link as an orthogonal channel with the same bandwidth to avoid frequency band overlap between links. Furthermore, based on the relative importance indicators of each target communication link obtained through reconnaissance, priority labeling of the target communication links is completed. Finally, the three-dimensional position coordinates of each UAV in the UAV swarm and the three-dimensional position coordinates of each target receiver are extracted. A one-to-one correspondence between each UAV and all target receivers is established through spatial coordinate calculations, forming and outputting the position association parameters between the UAVs and target receivers. These parameters include key information such as the spatial relative distance and azimuth angle between the UAVs and each target receiver, directly serving as the core input for subsequent calculations of channel gain based on the line-of-sight channel model.

[0037] In one embodiment, this method is implemented in a drone swarm communication network ( Figure 2 Communication warfare scenarios (taking surface naval formations as an example) Figure 2 As shown. This scenario considers a three-dimensional task region for adversarial communication, assuming that there are within this region... M Target communication links and N For a drone, the set of target communication links is represented as: Assuming the target communication link channels are orthogonal channels with the same bandwidth, the set of UAVs is... .

[0038] In this embodiment, the basic information for the communication network includes the transmitter location, receiver location, transmission power, center frequency, and importance index of each target communication link. The basic information for the UAV swarm includes the location information of each UAV, its maximum transmission power, and the maximum number of disruptive links per UAV.

[0039] Specifically, M The communication radios corresponding to each communication link are randomly and evenly distributed within the area, and the drones... ( The position of the target area is determined by coordinates. This indicates that the location of the target receiver is... The location of the target transmitter is The horizontal range of the task space is The vertical coordinate range is The flight altitude range of the drone is Then the first Drones in the target area D It can be represented as: (1) Furthermore, the relative importance indicators of each communication link were clarified: (2) In step S120, the drone flies at an altitude much higher than the ground obstacles. When the drone's altitude... h Meet the conditions (in Horizontal distance When the signal wavelength is λ, the probability of obstacle obstruction within the Fresnel zone approaches zero, allowing the direct wave to dominate signal propagation. The electromagnetic wave propagation path between the drone and the ground target receiver can be simplified to a line-of-sight channel model. i and target receiver j The channel gain between them is: (3) In formula (3), Defined as reference distance Channel gain at meters, Representing drones and target receiver The actual distance between them.

[0040] In practical communication scenarios, the attenuation effect of channel gain is dominated by both large-scale fading and small-scale fading. Large-scale fading refers to the gradual attenuation of signal strength during propagation in wireless communication systems due to factors such as transmission distance, obstacle obstruction, and environmental characteristics. Its main mechanisms stem from path loss and shadowing effects. Small-scale fading, on the other hand, is dominated by rapid fluctuations in signal phase and amplitude caused by multipath propagation, and its instantaneous channel gain follows a Rayleigh or Ricean distribution. Therefore, the transmitter... and receiver The channel gain between the UAV and the target receiver, calculated based on the line-of-sight channel model, is expressed as: (4) In formula (4), This indicates the fixed gain provided by the antenna and power amplifier. This represents the multipath fading gain that exhibits an exponential distribution. This represents path loss and satisfies the log-normal distribution characteristic. This represents the path loss index. This indicates the distance between the transmitter and receiver in the target communication link.

[0041] Furthermore, to quantitatively characterize the effect of UAV communication interference on the target receiver, it is necessary to assess whether the interference signal can effectively suppress the target signal. Therefore, the interference-to-signal ratio (ISR) and suppression coefficient are introduced as evaluation criteria for interference effect. The ISR is defined as the ratio of the interference signal power to the effective signal power, and the suppression coefficient is the minimum critical value of the ratio of interference power to signal power at the input of the receiving device under effective interference scenarios. Assuming that the total interference signal power received by the target receiver can be regarded as a linear superposition of all directions, its ISR can be expressed as: (5) In formula (5), This represents the allocation factor for the drone's perturbation beam. Indicates drone For the receiver Allocate the perturbation beam, that is, to Disturbance, a value of 0 indicates no disturbance. Indicates drone For the receiver The power of the transmitted disturbance. Indicates transmitter With receiver Communication power between them For transmitter and receiver Channel gain between. For drones and receiver Channel gain between.

[0042] When the interference-to-signal ratio of the communication link is higher than the corresponding suppression factor... K When this happens, it can be determined that the communication link is completely suppressed, i.e., the following conditions are met: (6) In formula (6), Indicates the target receiver The suppression coefficient.

[0043] The core problem this method aims to solve is how to achieve the optimal trade-off between disturbance effectiveness and energy efficiency under limited UAV swarm resources through joint optimization of disturbance target allocation and power control, thereby maximizing the swarm disturbance benefits. To ensure complete suppression of the communication network, the interference-to-signal ratio at the receiver end of each communication link must be greater than the suppression coefficient of that link, i.e.: (7) Furthermore, the sum of the perturbation power of a single UAV to different target links cannot exceed the maximum transmit power of that UAV, that is: (8) In formula (8), This indicates the maximum transmit power of a single drone.

[0044] Furthermore, the perturbation beam of a single drone is limited; a single drone can perturb a maximum of U links simultaneously, that is: (9) Furthermore, based on the obtained center frequency and relative importance index of each target link, the resource allocation efficiency function can be established as follows: (10) In formula (10), It represents a direct proportionality constant. Represents a symbolic function.

[0045] In step S130, with the hardware performance constraints of the UAV as the boundary and the link suppression judgment threshold as the performance benchmark, a UAV swarm resource allocation optimization model is constructed around the dual core optimization objectives of maximizing disturbance performance and minimizing resource consumption. This transforms the resource allocation problem of the UAV swarm into a constrained combinatorial optimization problem, providing quantitative optimization criteria and mathematical foundations for the subsequent construction of Markov decision process models and algorithm training.

[0046] Specifically, firstly, the link suppression judgment threshold output by the communication disturbance effect evaluation model is used as the core performance constraint to ensure that the model is built on the premise that the interference-to-signal ratio of each target communication link meets the suppression requirements, thus achieving effective suppression of the target communication links. Secondly, inherent hardware constraints of the UAV swarm are embedded, including the maximum transmit power constraint of a single UAV and the maximum number of disturbance links of a single UAV. This limits the sum of the disturbance power of a single UAV to each target communication link to not exceed its maximum transmit power, and the number of target communication links disturbed by a single UAV simultaneously to not exceed its maximum number of disturbance links, ensuring the model's practical engineering feasibility. The model is characterized by its current availability. Finally, a dual-core optimization objective is established: maximizing disturbance effectiveness and minimizing resource consumption. Maximizing disturbance effectiveness uses the relative importance of each target communication link as a weight, achieving priority suppression of high-priority links and maximizing the number of target communication links that meet the suppression requirements. Minimizing resource consumption uses the overall disturbance power consumption of the UAV swarm as a quantitative indicator, achieving efficient utilization of limited disturbance resources. The dual objectives are quantified and fused through an effectiveness function, ultimately constructing a UAV swarm resource allocation optimization model that combines effectiveness requirements, hardware constraints, and optimization objectives, clarifying the constraint boundaries and optimization direction of disturbance resource allocation.

[0047] In this embodiment, the drone swarm resource allocation optimization model is expressed as: (11) In formula (11), This represents the signal-to-interference ratio, where... This represents the allocation factor for the drone's perturbation beam. Indicates drone For the receiver The power of the transmitted disturbance. Indicates the target receiver The suppression coefficient, This indicates the maximum transmit power of a single drone. This indicates the maximum number of target communication links that a single drone can simultaneously disrupt. , These represent the target communication link set and the drone set, respectively. This indicates the number of drones in the collection.

[0048] In step S140, in the system model constructed above, the environmental state dynamically changes with the execution of UAV swarm actions, and the state evolution process at the next moment is determined only by the current environmental state and the swarm's disturbance behavior. Based on this model characteristic, this paper first constructs a Markov decision process for cooperative disturbance resource allocation, and then proposes a UAV swarm cooperative disturbance resource allocation method based on the CTDE-MAPPO algorithm. Here, CTDE stands for Centralized Training and Decentralized Execution, and MAPPO stands for Multi-Agent Proximal Policy Optimization.

[0049] In this embodiment, the Markov decision process model includes a state space, an action space, and a reward function. The state space includes target link location information, UAV cluster location information, and historical resource allocation strategies. The action space includes the selection of UAV disturbance targets and the allocation of disturbance power. The reward function relates the link suppression effect to the resource consumption cost.

[0050] Specifically, Markov Decision Process (MDP) models provide a mathematical framework for agents to learn optimal policies through interaction in stochastic environments. An MDP can be represented by a quintuple. Definition, including Representing the state space, Representing action space and This represents the reward function.

[0051] Furthermore, the state space S The set used to describe the possible states of the environment; in this method, the observed states of the agent include... Real-time target location information , Real-time drone cluster location information and Time-based resource allocation strategy The location information of the drone swarm and the target directly affects the perturbation channel gain. and communication channel gain For ease of calculation, the state space at the current moment... Setting by , and Composition, as a A matrix of M rows and M columns, i.e.: (12) Perturbation channel gain For one The matrix, i.e.: (13) In formula (13), Indicates drone Disturbance communication link The perturbation channel gain at that time. The communication channel gain is one. The vector, that is: (14) In formula (14), Indicates communication link Communication channel gain. Resource allocation results. It is The matrix represents the resource allocation result of the drone swarm.

[0052] Furthermore, the space of motion A The description of each UAV's actions includes the selection of the target link for the disturbance and the power of the disturbance transmitted, namely: (15) In formula (15), This represents the disturbance resources, including the allocation of disturbance targets and the allocation of disturbance power.

[0053] Furthermore, the reward function R This describes a scenario where multiple agents undertake a fully cooperative task in the allocation of resources within a drone swarm; therefore, all agents share the same reward function. The reward function is designed to achieve the goal of minimizing perturbation power while maintaining complete control over the communication link, i.e.: (16) In formula (16), and It is a proportionality coefficient used to represent the relative importance of complete suppression and resource consumption. . For a sign function, when At that time, the link is completely suppressed. Conversely, it is 0.

[0054] Generally, the system architecture of MARL can be summarized into three mainstream paradigms based on the different ways in which agents interact and collaborate during training and execution: Centralized Training and Centralized Execution (CTCE), Decentralized Training and Distributed Execution (DTDE), and Centralized Training and Distributed Execution (CTDE).

[0055] First, in the CTCE architecture, multi-agent training utilizes shared information, and during execution, actions are output using the same shared information. A direct implementation of CTCE is to transfer and extend single-agent reinforcement learning to multi-agent reinforcement learning. The CTCE training paradigm makes centralized decisions based on the global state during both training and execution, directly coordinating the actions of multiple agents. However, the dimensionality of the agent's decision network grows exponentially with the number of agents, resulting in poor scalability. DTDE, on the other hand, makes decisions based on the local observations of individual agents during both training and execution. Its distributed structure is simple to implement and highly scalable. However, during agent training, each agent treats other agents as part of the environment, amplifying non-stationarity during training and weakening overall learning performance and stability. The DTDE framework is as follows: Figure 3 As shown.

[0056] In the CTDE framework, training is performed using global information during the training phase, while during execution, each agent makes decisions based on its own local observations using an independent policy network. Centralized training allows agents to share information, facilitating better collaboration and improving policy quality and efficiency. Distributed execution, on the other hand, allows each agent to execute independently based on its own policy network, enhancing system scalability. This framework combines learning efficiency with execution feasibility, making it the mainstream framework for solving complex collaborative tasks. The CTDE framework in the CTDE-MAPPO algorithm is as follows: Figure 4 As shown.

[0057] Therefore, in step S150, the Markov decision process model is trained using the multi-agent proximal policy optimization (MAPPO) method within the CTDE framework, which is the CTDE-MAPPO method proposed in this paper. MAPPO is an on-policy algorithm that employs the classic Actor-Critic network architecture. Its core lies in using a policy update pruning mechanism, effectively overcoming the training oscillation problem caused by excessive update amplitude in traditional policy gradient methods, and effectively addressing the challenges of collaborative decision-making in complex dynamic environments. In the CTDE framework, each agent's Actor network inputs its own perceived environmental state information. Output action strategy, i.e. Centralized Critic networks utilize global states to calculate the global value function. The CTDE-MAPPO algorithm framework is as follows: Figure 5 As shown.

[0058] In this embodiment, when training the multi-agent model based on the CTDE-MAPPO algorithm: the centralized Critic network calculates the global value function based on global state information, which includes the location association parameters output by the communication adversarial model, the link suppression judgment threshold output by the communication disturbance effect evaluation model, and the constraints and optimization objectives output by the UAV swarm resource allocation optimization model. Then, based on the global value function, the policy advantage value of each UAV is estimated using the generalized advantage estimation method. Simultaneously, a policy pruning mechanism limits the parameter update amplitude of each distributed Actor network, achieving iterative optimization.

[0059] like Figure 5 As shown, relying on the Centralized Training and Distributed Execution (CTDE) framework and combining the Multi-Agent Proximal Policy Optimization (MAPPO) algorithm, the constructed Markov decision process model is iteratively trained until the reward function value satisfies the convergence condition. The trained policy model is then output, providing core algorithmic support for subsequent distributed perturbation decision-making for UAVs. The specific training process is as follows: First, the communication adversarial environment is initialized, and the environment state is normalized to improve training stability. Each UAV, acting as an agent, acquires its own local observation state (including target link position, its own position, and historical perturbation strategies) and inputs it into its respective Actor network. A joint action containing perturbation target selection and power allocation is generated through Gaussian distribution sampling. The action set is scaled and tensor reshaped by the constraint of the maximum number of perturbation links for a single UAV to filter effective perturbation beam allocation targets. This joint action is applied to the communication adversarial environment to calculate the corresponding reward value and the state at the next time step. The state at the next time step is also normalized, and the sample data such as "state, action, reward, next state, and action probability" are stored in the experience pool.

[0060] When the number of samples in the experience pool reaches the preset batch size, the batch samples are input into the centralized Critic network. Using global state information (including the location correlation parameters of the communication adversarial model, the link suppression judgment threshold of the communication disturbance effect evaluation model, and the constraints and optimization objectives of the resource allocation optimization model), the current state value and the state value at the next moment are calculated. Based on the generalized advantage estimation (GAE) method, combined with the above state values ​​and the reward value of environmental feedback, the policy advantage value of each agent's actions is accurately estimated, providing gradient guidance for the update of Actor network parameters.

[0061] During the Actor network update phase, the ratio of action probabilities between the old and new policies is calculated. The PPO-Clip objective function is used to limit the parameter update magnitude, preventing training oscillations due to over-updates. Simultaneously, an entropy regularization mechanism is introduced to enhance the agent's environment exploration capabilities. Policy parameters are iterated by minimizing the Actor loss function. During the Critic network update phase, the value estimation accuracy is optimized by minimizing the mean squared error. Critic network parameters are updated synchronously using gradient descent. The experience collection, value evaluation, and parameter optimization processes are repeatedly executed for continuous iterative training.

[0062] After each training round, the fluctuation of the reward function value is monitored. If the average fluctuation of the reward function is less than the preset threshold and remains so until the end of training, the model training is determined to be converged, the iteration is stopped, and the parameters of the Actor network and Critic network at this time are saved to form a trained policy model.

[0063] Specifically, the core algorithm process includes experience collection, advantage estimation, and network update, iterating repeatedly until the policy converges. In the experience collection phase, the environment is initialized. Normalization of the state yields To obtain the current local observation state Input the policy network for each agent To obtain the current set of perturbation strategies and the probability value of the action Then, the action set is scaled, and the perturbation beam allocation target is selected according to the perturbation beam constraint. The output tensor of the neural network is then reshaped to obtain... The reward is calculated by applying the current policy, i.e., the scaled action set, to the environment. and the next stage status and to After normalization, we get Then, the sample data obtained above is stored in the experience pool.

[0064] Furthermore, once the number of experiences in the experience pool reaches the set batch size, the stored batch sample data is input into the centralized Critic network to calculate the state value. and Perform generalized advantage estimation, advantage function A t The calculation is as follows: (17) When updating the policy network, i.e., the actor network, first calculate the ratio of action probabilities between the old and new policies: (18) The objective of the strategy after maximizing pruning is: (19) To encourage agents to explore the environment more broadly, an entropy regularization mechanism can be introduced into the loss function. The Actor loss function can then be expressed as: (20) In formula (20), Here, is the entropy coefficient, used to control the exploration intensity. When updating the Critic network, minimizing the mean square error of the Critic network is used; therefore, the Critic loss function can be expressed as: (twenty one) Finally, the parameters of the Actor and Critic networks are iteratively updated based on the gradient descent method.

[0065] In this embodiment, the condition for determining the convergence of the reward function value is: within a preset number of consecutive training rounds, the mean fluctuation of the reward function is less than a preset threshold, and the mean fluctuation state of the reward function continues until the end of training.

[0066] This paper also demonstrates the effectiveness of the proposed method through simulation experiments and comparative analysis. In the experiments, the proposed method is compared with the SANER-PPO, MASAC, and DTDE-MAPPO algorithms. Analysis and evaluation were conducted during the model training and testing phases to verify the effectiveness of the proposed method in policy learning in large-scale adversarial scenarios.

[0067] In setting the experimental parameters, a three-dimensional simulated adversarial scenario was constructed for the experiment: N drones were deployed, and the target communication network contained M receivers, with their horizontal positions randomly distributed in a 2km×2km area. The receiver altitude was set to 0km, and the drone altitude was limited to 0.8km. The distance is 1.2 km, and it follows a normal distribution with a mean of 1 km and a variance of 0.1 km. The environmental parameters for the allocation of drone swarm resources are shown in Table 1.

[0068] Table 1 Environmental Parameter Settings

[0069] This experiment considers a large-scale simulated adversarial scenario, setting the number of drones N=10 and the target communication links M=15. It is assumed that the relative importance of each communication link is known from prior electronic reconnaissance: W=[0.3157,0.4823,0.6391,0.5714,0.7286,0.2942,0.4538,0.6189,0.5473,0.7621,0.4835,0.6572,0.7734,0.4269,0.5916]. In this experiment, the individual drones in the drone swarm are homogeneous agents, therefore... Each Actor network has the same parameters, and the network parameter settings are shown in Table 2.

[0070] Table 2 Network Parameters

[0071] To verify the effectiveness of the proposed algorithm, a comparative analysis was conducted with three reinforcement learning algorithms in the experiment. The SANER-PPO algorithm is an improved PPO algorithm with the introduction of an enhancement mechanism, and it is a direct transfer and extension of the single-agent algorithm to multi-agent reinforcement learning. The MASAC algorithm is a maximum entropy reinforcement learning algorithm, belonging to the off-policy framework. The DTDE-MAPPO algorithm adopts the DTDE framework, and its remaining structure is consistent with the proposed CTDE-MAPPO method.

[0072] Experiments were conducted using the above parameter settings to obtain the reward function curves of the four algorithms during the training period. The results are as follows: Figure 6 As shown. Due to the inherent randomness of the environment and the randomness of the agent's exploration, the reward function value will fluctuate. In order to more clearly observe the long-term trend of policy performance, the training curve of the reward function was smoothed in the experiment, and the smoothing window was 440.

[0073] Depend on Figure 6It is evident that the CTDE-MAPPO algorithm stabilizes its policy output after 400,000 training steps, with the reward value slowly increasing thereafter, eventually converging to around 690. The DTDE-MAPPO algorithm converges to a local optimum after 200,000 training steps, with a final reward value of around 490. The SANER-PPO algorithm struggles to learn a stable policy in the early stages, resulting in a highly volatile reward function during training, ultimately also converging to a local optimum of around 490. The MASAC algorithm rapidly converges to a local optimum of around 490 in the early stages of training, but struggles to learn and explore better policies. In the DTDE-MAPPO algorithm training, each agent learns independently, and the observations of a single agent lack global state information (such as the actions or policies of other agents), making it difficult to learn complex perturbation-based cooperative policies, thus leading to training getting trapped in local optima. As an improved version of the single-agent PPO algorithm, the SANER-PPO algorithm integrates core mechanisms such as state normalization, advantage standardization, and entropy regularization to enhance its overall performance. However, it cannot fully adapt to the collaborative needs of many-to-many scenarios, exhibiting significant fluctuations in the reward function value during training and ultimately converging to a local optimum. The MASAC algorithm, a multi-agent reinforcement learning method based on the value maximum entropy framework, utilizes an off-policy framework to efficiently leverage historical data and achieve faster convergence. However, in the non-stationary environment of this task, the algorithm converges to a local optimum. This demonstrates that as the scale of the simulated adversarial scenario increases, the number of agents increases accordingly, and the dimensionality of the decision space also increases. The CTDE-MAPPO algorithm (the method proposed in this paper), with its CTDE framework, can obtain global state information through centralized training to guide policy updates while maintaining the decentralization of the execution phase, effectively reducing the decision dimensionality and thus driving the algorithm to converge to a higher reward level.

[0074] To further test the generalization ability of the CTDE-MAPPO algorithm, a test experiment with 80,000 training steps was conducted after the model training was completed. The average reward, complete suppression success rate, and average perturbation efficiency of different algorithms were compared in the test experiment. The reward function values ​​of different algorithms during the testing phase are as follows: Figure 7 As shown in the figure, during the testing phase, the CTDE-MAPPO algorithm had the highest average reward value and the most stable output cooperative perturbation strategy.

[0075] The suppression success rate is defined as the ratio of the number of communication links successfully perturbed by the cooperative perturbation strategy output in the test round to the total number of communication links. The criterion for successful perturbation is that the interference-to-signal ratio of the link is greater than the suppression coefficient. The suppression success rates of different algorithms are shown in Table 3. During the testing phase, the CTDE-MAPPO suppression success rate was 99.55%, which is higher than the other three algorithms, further verifying the excellent performance of the proposed method in completing large-scale multi-agent cooperative perturbation tasks.

[0076] Table 3. Suppression success rate of different algorithms

[0077] Table 4. Average Disturbance Power of UAV Swarms under Different Algorithms

[0078] Table 4 shows the average disturbance power consumption (in dBm) of the UAV swarm during the testing phase. Considering only the total disturbance power consumed by the UAV swarm, the CTDE-MAPPO algorithm is second only to the MASAC algorithm. However, in practical terms, both suppression success rate and disturbance power consumption need to be considered. Therefore, the performance of different algorithms can be measured using the resource allocation efficiency described earlier. The average allocation efficiency during the testing phase is as follows: Figure 8 As shown, the CTDE-MAPPO algorithm has the highest average perturbation efficiency, stabilizing at around 11.8. Although the DTDE-MAPPO and SANER-PPO algorithms consume less perturbation power than the proposed method, their average perturbation efficiency is significantly lower, verifying the superior performance of the CTDE-MAPPO algorithm in balancing swarm perturbation benefits and energy efficiency.

[0079] In the aforementioned intelligent resource allocation method for UAV swarms, a perturbation-based resource allocation method for UAV swarms is proposed, addressing core issues such as insufficient adaptation to collaborative requirements and convergence difficulties in large-scale scenarios under multi-communication adversarial scenarios. This method transforms the many-to-many perturbation resource allocation problem into a constrained combinatorial optimization problem, and establishes a Markov decision model for perturbation resource allocation accordingly. The algorithm's CTDE architecture ensures collaborative efficiency among multiple UAVs by guiding policy updates through a centralized Critic network, while employing a distributed Actor network to independently execute decisions, effectively reducing the increase in decision dimensionality caused by the increase in the number of agents.

[0080] Simulation results show that the CTDE-MAPPO algorithm has more stable convergence during the training phase compared to other comparative algorithms, with a final average reward value of around 690, significantly higher than the approximately 490 of the comparative algorithms. During the testing phase, the suppression success rate is as high as 99.55%, and the average perturbation efficiency is stable at around 11.8. While achieving efficient suppression of communication networks, it ensures the optimal balance between perturbation benefits and perturbation resource consumption, effectively solving the cooperation problem in multi-agent cooperative perturbation and improving the efficiency of policy learning and task execution performance in large-scale scenarios.

[0081] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but may be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0082] In one embodiment, such as Figure 9 As shown, an intelligent resource allocation device for unmanned aerial vehicle (UAV) swarms is provided, comprising: a basic information acquisition module 200, a communication adversarial model construction module 210, a communication disturbance effect evaluation model construction module 220, a UAV swarm resource allocation optimization model construction module 230, a Markov decision process model construction module 240, a model training module 250, and a resource intelligent allocation module 260, wherein: The basic information acquisition module 200 is used to acquire basic information about the communication network containing multiple target communication links within the three-dimensional target area, as well as basic information about the UAV cluster. The communication countermeasure model construction module 210 is used to determine the mission area of ​​the UAV and the distribution range of the target communication link by constructing a communication countermeasure model based on the basic information, and output the position association parameters between the UAV and the target receiver. The communication disturbance effect evaluation model construction module 220 is used to calculate the channel gain between the UAV and the target receiver based on the line-of-sight channel model and the location association parameters, and to establish a communication disturbance effect evaluation model with the signal-to-interference ratio not less than the preset suppression coefficient as the criterion for successful link control, and to obtain the link suppression judgment threshold. The UAV swarm resource allocation optimization model construction module 230 is used to construct a UAV swarm resource allocation optimization model based on the UAV's maximum transmission power constraint, the single UAV's maximum number of disturbance links constraint, the link suppression judgment threshold, and with the optimization objective of maximizing disturbance effectiveness and minimizing resource consumption as the core. Markov decision process model construction module 240 is used to construct a Markov decision process model based on the communication adversarial model, the communication disturbance effect evaluation model and the UAV swarm resource allocation optimization model. The model training module 250 is used to train the Markov decision process model based on the multi-agent proximal policy optimization algorithm under the centralized training and distributed execution framework until the reward function value converges, and the trained policy model is obtained. The resource intelligent allocation module 260 is used to configure the Actor network in the trained strategy model into each of the UAVs. Each UAV inputs its own local observation information into the Actor network to obtain the perturbation target selection result and power allocation scheme.

[0083] Specific limitations regarding the intelligent allocation device for UAV swarm resource allocation can be found in the limitations of the intelligent allocation method for UAV swarm resource allocation described above, and will not be repeated here. Each module in the aforementioned intelligent allocation device for UAV swarm resource allocation can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0084] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements an intelligent resource allocation method for unmanned aerial vehicle (UAV) swarms. The display screen can be an LCD screen or an e-ink display screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0085] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0086] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps: Acquire basic information about the communication network containing multiple target communication links within a three-dimensional target area, as well as basic information about the UAV swarm; Based on the aforementioned basic information, the mission area of ​​the UAV and the distribution range of the target communication links are determined by constructing a communication adversarial model, and the position association parameters between the UAV and the target receiver are output. Based on the line-of-sight channel model, the channel gain between the UAV and the target receiver is calculated by combining the position correlation parameters. The signal-to-interference ratio is not less than the preset suppression coefficient as the criterion for successful link control. A communication disturbance effect evaluation model is established, and the link suppression judgment threshold is obtained. Based on the constraints of the maximum transmit power of the UAV and the maximum number of disturbance links of a single UAV, combined with the link suppression judgment threshold, and with the optimization objective of maximizing disturbance effectiveness and minimizing resource consumption as the core, a UAV cluster resource allocation optimization model is constructed. Based on the aforementioned communication countermeasure model, communication disturbance effect evaluation model, and UAV swarm resource allocation optimization model, a Markov decision process model is constructed. Under the framework of centralized training and distributed execution, the Markov decision process model is trained based on the multi-agent proximal policy optimization algorithm until the reward function value converges, thus obtaining the trained policy model. The Actor network in the trained policy model is configured into each of the UAVs. Each UAV inputs its own local observation information into the Actor network to obtain the perturbation target selection result and power allocation scheme.

[0087] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Acquire basic information about the communication network containing multiple target communication links within a three-dimensional target area, as well as basic information about the UAV swarm; Based on the aforementioned basic information, the mission area of ​​the UAV and the distribution range of the target communication links are determined by constructing a communication adversarial model, and the position association parameters between the UAV and the target receiver are output. Based on the line-of-sight channel model, the channel gain between the UAV and the target receiver is calculated by combining the position correlation parameters. The signal-to-interference ratio is not less than the preset suppression coefficient as the criterion for successful link control. A communication disturbance effect evaluation model is established, and the link suppression judgment threshold is obtained. Based on the constraints of the maximum transmit power of the UAV and the maximum number of disturbance links of a single UAV, combined with the link suppression judgment threshold, and with the optimization objective of maximizing disturbance effectiveness and minimizing resource consumption as the core, a UAV cluster resource allocation optimization model is constructed. Based on the aforementioned communication countermeasure model, communication disturbance effect evaluation model, and UAV swarm resource allocation optimization model, a Markov decision process model is constructed. Under the framework of centralized training and distributed execution, the Markov decision process model is trained based on the multi-agent proximal policy optimization algorithm until the reward function value converges, thus obtaining the trained policy model. The Actor network in the trained policy model is configured into each of the UAVs. Each UAV inputs its own local observation information into the Actor network to obtain the perturbation target selection result and power allocation scheme.

[0088] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0089] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0090] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for intelligent allocation of resources in a drone swarm, characterized in that, The method includes: Acquire basic information about the communication network containing multiple target communication links within a three-dimensional target area, as well as basic information about the UAV swarm; Based on the aforementioned basic information, the mission area of ​​the UAV and the distribution range of the target communication links are determined by constructing a communication adversarial model, and the position association parameters between the UAV and the target receiver are output. Based on the line-of-sight channel model, the channel gain between the UAV and the target receiver is calculated by combining the position correlation parameters. The signal-to-interference ratio is not less than the preset suppression coefficient as the criterion for successful link control. A communication disturbance effect evaluation model is established, and the link suppression judgment threshold is obtained. Based on the constraints of the maximum transmit power of the UAV and the maximum number of disturbance links of a single UAV, combined with the link suppression judgment threshold, and with the optimization objective of maximizing disturbance effectiveness and minimizing resource consumption as the core, a UAV cluster resource allocation optimization model is constructed. Based on the aforementioned communication countermeasure model, communication disturbance effect evaluation model, and UAV swarm resource allocation optimization model, a Markov decision process model is constructed. Under the framework of centralized training and distributed execution, the Markov decision process model is trained based on the multi-agent proximal policy optimization algorithm until the reward function value converges, thus obtaining the trained policy model. The Actor network in the trained policy model is configured into each of the UAVs. Each UAV inputs its own local observation information into the Actor network to obtain the perturbation target selection result and power allocation scheme.

2. The intelligent allocation method for unmanned aerial vehicle (UAV) swarm resources according to claim 1, characterized in that, The basic information of the communication network includes the transmitter location, receiver location, transmission power, center frequency, and importance index of each target communication link; The basic information of the drone swarm includes the location information of each drone, its maximum transmission power, and the maximum number of disturbance links per drone.

3. The intelligent allocation method for unmanned aerial vehicle (UAV) swarm resources according to claim 2, characterized in that, The channel gain between the UAV and the target receiver is calculated based on the line-of-sight channel model and expressed as follows: In the above formula, This indicates the fixed gain provided by the antenna and power amplifier. This represents the multipath fading gain that exhibits an exponential distribution. This represents path loss and satisfies the log-normal distribution characteristic. This represents the path loss index. This indicates the distance between the transmitter and receiver in the target communication link.

4. The intelligent allocation method for unmanned aerial vehicle (UAV) swarm resources according to claim 3, characterized in that, The drone cluster resource allocation optimization model is expressed as follows: In the above formula, This represents the signal-to-interference ratio, where... This represents the allocation factor for the drone's perturbation beam. Indicates drone For the receiver The power of the transmitted disturbance. Indicates the target receiver The suppression coefficient, This indicates the maximum transmit power of a single drone. This indicates the maximum number of target communication links that a single drone can simultaneously disrupt. , These represent the target communication link set and the drone set, respectively. This indicates the number of drones in the collection.

5. The intelligent allocation method for unmanned aerial vehicle (UAV) swarm resources according to any one of claims 1-4, characterized in that, The Markov decision process model includes a state space, an action space, and a reward function. The state space includes target link location information, UAV cluster location information, and historical resource allocation strategies. The action space includes UAV perturbation target selection and perturbation power allocation. The reward function correlates link suppression effectiveness with resource consumption costs.

6. The intelligent allocation method for unmanned aerial vehicle (UAV) swarm resources according to claim 5, characterized in that, When training the Markov decision process model: The global value function is calculated by a centralized Critic network based on global state information, which includes the location correlation parameters output by the communication adversarial model, the link suppression judgment threshold output by the communication disturbance effect evaluation model, and the constraints and optimization objectives output by the UAV swarm resource allocation optimization model. Based on the global value function, the policy advantage value of each UAV is estimated by combining the generalized advantage estimation method. At the same time, the parameter update range of each distributed Actor network is limited by the policy pruning mechanism to achieve iterative optimization.

7. The intelligent allocation method for unmanned aerial vehicle (UAV) swarm resources according to claim 6, characterized in that, The convergence criterion for the reward function value is: within a preset number of consecutive training rounds, the mean fluctuation of the reward function is less than a preset threshold, and the mean fluctuation state of the reward function continues until the end of training.

8. A smart allocation device for unmanned aerial vehicle (UAV) swarm resources, characterized in that, The device includes: The basic information acquisition module is used to acquire basic information about the communication network containing multiple target communication links within the three-dimensional target area, as well as basic information about the UAV cluster; The communication countermeasure model construction module is used to determine the mission area of ​​the UAV and the distribution range of the target communication link by constructing a communication countermeasure model based on the basic information, and output the position association parameters between the UAV and the target receiver. The communication disturbance effect evaluation model construction module is used to calculate the channel gain between the UAV and the target receiver based on the line-of-sight channel model and the location correlation parameters. The ratio of the target communication link disturbance signal to the effective signal is not less than a preset suppression coefficient as the criterion for successful link control. The module establishes a communication disturbance effect evaluation model and obtains the link suppression judgment threshold. The UAV swarm resource allocation optimization model construction module is used to construct a UAV swarm resource allocation optimization model based on the constraints of the maximum transmission power of the UAV, the maximum number of disturbance links of a single UAV, and the link suppression judgment threshold, with the core optimization objective of maximizing disturbance effectiveness and minimizing resource consumption. The Markov decision process model construction module is used to construct a Markov decision process model based on the communication adversarial model, the communication disturbance effect evaluation model, and the UAV swarm resource allocation optimization model. The model training module is used to train the Markov decision process model based on the multi-agent proximal policy optimization algorithm under the framework of centralized training and distributed execution until the reward function value converges, thus obtaining the trained policy model. The resource intelligent allocation module is used to configure the Actor network in the trained strategy model into each of the UAVs. Each UAV inputs its own local observation information into the Actor network to obtain the perturbation target selection result and power allocation scheme.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images