Cooperative interference unmanned aerial vehicle auxiliary communication method and system facing disaster area wireless network

Abstract

The invention discloses a cooperative interference unmanned aerial vehicle auxiliary communication method and system for a disaster area wireless network, and the method comprises the steps: respectively defining unmanned aerial vehicles as relay unmanned aerial vehicles and cooperative interference unmanned aerial vehicles through introducing a role distribution mechanism of the unmanned aerial vehicles; the relay unmanned aerial vehicle is responsible for enhancing a communication link between the ground base station and the user; the cooperative interference unmanned aerial vehicle actively sends an interference signal to prevent illegal information acquisition of the eavesdropping unmanned aerial vehicle; the flight path of the unmanned aerial vehicle is optimized through precise modeling based on three-dimensional Cartesian coordinates and a design power distribution and flight path joint optimization method, and the unmanned aerial vehicle can reasonably adjust the position in a dynamic environment of a disaster area so as to improve channel conditions and further enhance communication reliability and safety; a Markov decision process is used for modeling a trajectory and power optimization problem of the unmanned aerial vehicle, and a multi-agent reinforcement learning algorithm is used for solving, so that the overall performance of the system and the communication security are improved, and an efficient and reliable solution is provided for emergency communication in a disaster area.

Description

Background Art

[0002] Drones can be quickly deployed in natural disasters or humanitarian crises to provide temporary network connections and help restore communications. As natural disasters occur frequently today, drones are playing an increasingly important role in assisting the establishment of wireless LANs in disaster areas. Drones can help provide network coverage and solve communication problems in areas where traditional infrastructure is damaged and ground mobile base stations cannot cover. In the process of wireless communication in disaster areas, drones can act as a relay. The ground mobile base station first sends the signal to the relay drone, which then transmits it to the ground users in need through the relay drone. At the same time, the flexibility of the relay drone can be used to better provide communication services for areas that are not covered by ground mobile base stations.

[0003] However, in the process of transmitting secret information, there may be an eavesdropping drone that eavesdrops on the secret signals transmitted by the ground mobile base station and the relay drone, thus damaging the communication security of the wireless network in the disaster area. Therefore, in order to improve the communication security performance of the wireless network in the disaster area, we select an idle relay drone and convert it into a cooperative jamming drone. By sending jamming signals to the eavesdropping drone, the quality of the eavesdropping drone's eavesdropping signal is reduced. At the same time, the cooperative jamming drone is ordered to follow the eavesdropping drone as much as possible to facilitate the transmission of jamming signals, thereby improving the communication security performance of the wireless network in the disaster area.

[0004] Deep reinforcement learning combines deep learning and reinforcement learning, and is mainly used to solve complex decision-making problems. Deep learning extracts features of high-dimensional data through multi-layer neural networks, while reinforcement learning optimizes decision-making strategies through reward signals based on the interaction between the agent and the environment. In recent years, deep reinforcement learning has demonstrated powerful capabilities in games, robot control, autonomous driving and other fields, especially in dealing with large-scale state spaces and complex strategies, significantly improving learning efficiency and decision-making quality.

[0005] Currently, there is no prior art involving a collaborative jamming drone-assisted secure communication method for wireless networks in disaster areas. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a collaborative interference drone-assisted communication method and system for disaster area wireless networks in response to the deficiencies in the above-mentioned prior art, which is used to solve the security communication technology problem of an eavesdropping drone in the disaster area wireless network.

[0007] The present invention adopts the following technical solutions:

[0008] The cooperative jamming drone-assisted communication method for disaster area wireless networks includes the following steps:

[0009] Construct a disaster area wireless network assisted by cooperative jamming drones in the presence of eavesdropping drones, use a three-dimensional Cartesian coordinate system to represent mobile ground base stations in the disaster area wireless network, build a drone mobility model, and the location coordinates of ground users, relay drones, cooperative jamming drones, and eavesdropping drones. Obtain the distances from the ground mobile base station to the relay drone, the cooperative jamming drone, and the eavesdropping drone, and obtain the channel energy gain between the ground mobile base station, the relay drone, the cooperative jamming drone, and the eavesdropping drone.

[0010] Constrain the transmission power of ground mobile base stations, relay drones and cooperative jamming drones within the corresponding range of average transmission power and maximum transmission power, determine the safety rate from mobile ground base stations to relay drones and from relay drones to ground users at time t, design and optimize the flight trajectory and transmission power of relay drones and cooperative jamming drones to maximize the minimum safety rate of ground users, and design joint optimization of power allocation and flight trajectory;

[0011] According to the flight trajectory, transmission power and interaction of the drone with the environment, a Markov decision process is constructed. The wireless network in the disaster area assisted by drones is used as the environment. Relay drones, cooperative jamming drones and eavesdropping drones are used as agents to construct a multi-objective optimization problem based on the Markov decision process. A multi-agent reinforcement learning algorithm is introduced based on the multi-agent deep deterministic gradient algorithm to solve the multi-objective optimization problem based on the Markov decision process. i The strategy of agent i in the joint trajectory design and power allocation algorithm is determined, and the optimal strategy communication is achieved by modifying the parameters of the evaluation network.

[0012] Preferably, there are M drones and N ground users in the disaster area wireless network. and represents the role represented by the drone at time point t;

[0013] when When the UAV acts as a relay to assist the communication between the ground mobile base station and the ground user, it is represented by m r ;

[0014] when When the UAV acts as a cooperative jammer to prevent the eavesdropping UAV from eavesdropping, it is represented by m j ;

[0015] The constraints are as follows:

[0016]

[0017] in, The UAV acts as a relay to communicate with ground user n at time intervals. For ground users, For drone collection, The UAV acts as a cooperative jammer to interfere with the eavesdropping UAV at time intervals.

[0018] Preferably, in the drone mobility model, the constraints of the drone during flight are summarized as follows:

[0019]

[0020] Among them, d min The minimum safe distance to prevent collision between drones, ∥·∥ is the Euclidean distance, For drones a The plane coordinates at the initial moment (0 moment) of the flight duration, For drones a The plane coordinates at the final moment of the flight duration (T moment), where T is the flight duration, To calculate the plane coordinates of the relay UAV and the cooperative jammer UAV at time intervals, is the plane coordinates of the relay UAV and the cooperative jammer UAV at the time interval, m a and m a ′ are two different drones in the drone set, For drone collection, For the preset drone m a Takeoff initial plane coordinates, For the preset drone m a Final landing plane coordinates.

[0021] Preferably, the channel energy gain between the ground mobile base station to the relay drone, the cooperative jamming drone and the eavesdropping drone and h s,e [t] is represented by:

[0022]

[0023] Among them, β0 is the channel energy gain parameter at a reference distance of 1 meter, is the -2 power of the distance from the ground base station to the relay drone at the time interval, is the plane coordinate of the relay UAV at the time interval, q s is the plane coordinate of the mobile ground base station, H is the minimum allowable flight altitude of the UAV;

[0024] The channel energy gain from the relay UAV and the cooperative jammer UAV to the ground user is expressed as:

[0025]

[0026] in, is the -2 power of the distance from the relay drone to the ground user, is the -2 power of the distance from the cooperative jamming drone to the ground user, q n is the plane coordinate of the ground user;

[0027] The channel energy gain from the relay drone, cooperative jammer drone to the eavesdropping drone is expressed as:

[0028]

[0029] in, is the -2 power of the distance from the relay drone to the eavesdropping drone at the time interval, is the -2 power of the distance from the cooperative jamming drone to the eavesdropping drone at the time interval, q e [t] is the plane coordinate of the eavesdropping drone at the time interval, The plane coordinates of the cooperatively jamming UAVs at time intervals.

[0030] Preferably, each drone can only work as a relay drone or a collaborative jammer drone within a time interval, with the following constraints:

[0031]

[0032] in, is the time interval at which the UAV acts as a relay to communicate with ground user n, N is the maximum number of ground users, is the interference of the UAV as a cooperative jammer UAV to the eavesdropping UAV at the time interval, and t is the time interval.

[0033] Preferably, the problem of maximizing the minimum safety rate can be expressed as:

[0034]

[0035] in, is the minimum safety rate for relaying UAVs to ground users, is the plane coordinate of the relay drone, is the plane coordinates of the cooperative jamming drone, is the transmission power of the relay UAV, is the transmission power of the cooperative jammer UAV, (C1)-(C5) and (C10) represent the role assignment constraint rules of the UAV, (C6)-(C9) represent the movement trajectory constraints of the UAV, and (C11)-(C14) represent the transmission power constraints of the UAV and the mobile ground base station.

[0036] Preferably, at time t, the reachable safety rate of the mobile ground base station to the relay drone and the eavesdropping drone is and R s,e [t] is represented by:

[0037]

[0038] in, and is the white Gaussian noise at the relay drone and the eavesdropping drone, p s [t] is the transmission power of the mobile ground base station during the time interval, is the channel energy gain from the mobile ground base station to the relay drone at time interval, is the transmission power of the cooperative jammer drone at the time interval, is the channel energy gain from the relay UAV to the cooperative jammer UAV at the time interval, h s,e [t] is the channel energy gain from the mobile ground base station to the eavesdropping drone at the time interval, is the channel energy gain from the cooperative jamming UAV to the eavesdropping UAV at the time interval;

[0039] Achievable safety rate from relay drones to ground users and eavesdropping drones and It is expressed as:

[0040]

[0041] in, is the white Gaussian noise at the ground user;

[0042] At time t, the safety rate from the mobile ground base station to the relay drone and from the relay drone to the ground user is and Respectively expressed as:

[0043]

[0044] Where [x] + where represents the maximum value of x and 0.

[0045] Preferably, constructing the Markov decision process is specifically as follows:

[0046] The optimization problem of the flight trajectory and transmission power of the UAV is represented by a five-tuple Markov decision process {I, S, A, R, R}, where I represents a finite set of agents, S represents a finite set of states of the agents, A represents the action set of the agents, T represents the transition probability of the agent states, and R represents the reward function.

[0047] Based on the multi-agent deep deterministic gradient algorithm, a multi-agent reinforcement learning algorithm is introduced to solve the optimization problem of P1. Each agent includes an actor network, a critic network, an actor target network and a critic target network. In the multi-agent deep deterministic gradient algorithm, the actor network only uses local information for operation, the critic network uses global information for improvement, and each agent considers the influence of other agents when making decisions. i Determine the strategy of agent i in the joint trajectory design and power allocation algorithm; by modifying the parameters of the evaluation network and Enable optimal strategic communications.

[0048] Preferably, π i Determine the strategy of agent i in the joint trajectory design and power allocation algorithm by modifying the parameters of the evaluation network and The best strategy is:

[0049] Operating experience(s) t ,a t ,r t ,s t+1 ) is obtained through the interaction between the agent and the environment and is retained in the experience replay pool D; during training, the parameters of the evaluation network are updated by extracting small batches of samples ∈ from the experience replay buffer D; by minimizing the loss function, the critic network modifies the evaluation network parameters Maximize the policy objective function, as the network parameters are evaluated and Modification, gradually adjust the parameter μ′ i and Q′ i .

[0050] In a second aspect, an embodiment of the present invention provides a cooperative jamming drone-assisted communication system for a disaster area wireless network, including:

[0051] Parameter module, constructs a disaster area wireless network assisted by cooperative jamming drones in the presence of eavesdropping drones, uses a three-dimensional Cartesian coordinate system to represent mobile ground base stations in the disaster area wireless network, builds a drone mobile model, and the location coordinates of ground users, relay drones, cooperative jamming drones, and eavesdropping drones are obtained. The distances from the ground mobile base station to the relay drone, the cooperative jamming drone, and the eavesdropping drone are obtained, and the channel energy gains between the ground mobile base station, the relay drone, the cooperative jamming drone, and the eavesdropping drone are obtained;

[0052] The optimization module constrains the transmission power of the ground mobile base station, relay drone and cooperative jamming drone within the corresponding average transmission power and maximum transmission power range, determines the safety rate from the mobile ground base station to the relay drone and from the relay drone to the ground user at time t, designs and optimizes the flight trajectory and transmission power of the relay drone and the cooperative jamming drone to maximize the minimum safety rate of the ground user, and designs the joint optimization of power allocation and flight trajectory;

[0053] The solution module constructs a Markov decision process based on the flight trajectory, transmission power and interaction of the drone with the environment. The wireless network in the disaster area assisted by the drone is used as the environment, and the relay drone, cooperative jamming drone and eavesdropping drone are used as agents to construct a multi-objective optimization problem based on the Markov decision process. A multi-agent reinforcement learning algorithm is introduced based on the multi-agent deep deterministic gradient algorithm to solve the multi-objective optimization problem based on the Markov decision process. i The strategy of agent i in the joint trajectory design and power allocation algorithm is determined, and the optimal strategy communication is achieved by modifying the parameters of the evaluation network.

[0054] In a third aspect, a computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned collaborative interference drone-assisted communication method for disaster-affected wireless networks when executing the computer program.

[0055] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned collaborative interference drone-assisted communication method for disaster-affected wireless networks.

[0056] In a fifth aspect, a chip includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned collaborative interference drone-assisted communication method for disaster-affected wireless networks are implemented.

[0057] In a sixth aspect, an embodiment of the present invention provides an electronic device, including a computer program, which, when executed by the electronic device, implements the steps of the above-mentioned collaborative interference drone-assisted communication method for disaster-affected wireless networks.

[0058] Compared with the prior art, the present invention has at least the following beneficial effects:

[0059] A cooperative jamming UAV-assisted communication method for disaster area wireless networks. The purpose of the cooperative jamming UAV-assisted communication method for disaster area wireless networks is to achieve efficient and secure communication in complex disaster area environments through the cooperative jamming and relay functions of UAVs. Its core benefit is to enhance the flexibility, reliability and security of the communication system in the disaster area, while achieving optimal communication performance under limited resource conditions.

[0060] First, by introducing the role allocation mechanism of drones, drones are defined as relay drones and cooperative jammer drones respectively; relay drones are responsible for enhancing the communication link between ground base stations and users, reducing path loss and improving signal strength; cooperative jammer drones actively send jamming signals to prevent eavesdropping drones from illegally obtaining information; through precise modeling based on three-dimensional Cartesian coordinates, the distance between drones and channel energy gain can be dynamically calculated, providing the necessary parameter support for optimizing communication link performance. This dynamic role allocation mechanism can not only efficiently utilize system resources, but also flexibly respond to communication and security needs in the changing environment of disaster areas.

[0061] Secondly, a method for joint optimization of power allocation and flight trajectory was designed to maximize the minimum safety rate of ground users. By incorporating the transmission power and trajectory optimization of relay drones and cooperative jamming drones into constraints, the user communication security is maximized to prevent the communication rate of any user from being lower than the safety critical value. At the same time, this optimization process can also effectively reduce unnecessary energy consumption and maximize resource utilization efficiency. By optimizing the flight trajectory of the drone, the drone can reasonably adjust its position in the dynamic environment of the disaster area to improve channel conditions and further enhance communication reliability and security.

[0062] Finally, the trajectory and power optimization problem of the UAV is modeled using the Markov decision process and solved by a multi-agent reinforcement learning algorithm. The UAV is regarded as multiple agents. The algorithm introduces the collaboration mechanism of the actor network and the critic network, so that each agent combines the global information optimization strategy based on local information to ensure the global optimality of the system. The algorithm not only takes into account the collaborative relationship between the UAVs, but also dynamically adjusts the strategy to adapt to the complex environment of the disaster area, and finally improves the overall performance and communication security of the system. The method steps of the present invention significantly improve the adaptability and security of the wireless network in the disaster area through multi-level and multi-objective optimization design. On the one hand, the system resource utilization efficiency is improved by the dynamic allocation and optimization of the UAV roles; on the other hand, the communication performance and security requirements of the system are effectively balanced through power and trajectory optimization and multi-agent reinforcement learning algorithms, providing an efficient and reliable solution for emergency communications in disaster areas.

[0063] Furthermore, focusing on the dynamics and complexity of wireless networks in disaster areas, an accurate channel energy gain model is constructed to comprehensively describe the channel characteristics and interaction relationships between ground base stations, relay drones, cooperative jamming drones, and eavesdropping drones. Its purpose is to provide a theoretical basis for the communication security optimization of the system and support dynamic adaptation and functional coordination of multiple nodes and multiple links. Through this modeling method, the communication reliability and security of the disaster area network can be effectively improved, laying the foundation for subsequent trajectory optimization and power allocation.

[0064] Furthermore, the setting of channel energy gain directly affects the quality, speed, reliability of data transmission and the overall performance of the network. Through the reasonable configuration and management of channel energy gain, the signal quality can be maximized, the adaptability of the network can be enhanced, the reliability of communication can be improved, and the overall efficiency of the system can be improved. These contents can not only provide theoretical support for the design and optimization of communication systems, but also provide technical guidance for solving practical communication problems.

[0065] Furthermore, drones can dynamically switch roles. They can act as relay drones to help mobile ground base stations communicate with ground users, or they can act as collaborative jammers to suppress the eavesdropping behavior of eavesdropping drones, thereby ensuring the communication security of the system and improving the communication security rate from mobile ground base stations to ground users.

[0066] Furthermore, the reasonable constraint and optimization design of transmission power can maximize the communication security rate while suppressing the impact of eavesdropping, thereby improving the communication security of wireless networks in disaster areas. By introducing the calculation of the achievable security rate, combined with the channel energy gain between ground mobile base stations, relay drones, cooperative jamming drones and eavesdropping drones, a mathematical model is established to quantify the security performance of the communication process, and on this basis, an optimization algorithm is designed to maximize the minimum security rate of the network.

[0067] Furthermore, the setting of the achievable security rate is a key concept, which involves the optimal security level that can be achieved under given channel conditions, system resources and security requirements. The setting of the achievable security rate is not only related to the extent to which the system can resist various attacks and security threats, but also affects the system performance, data transmission efficiency and protection of user privacy. The setting of the achievable security rate is not only the key to evaluating the security of the system, but also can provide a theoretical basis for the selection of security strategies in the design, optimization and actual deployment of communication systems.

[0068] Furthermore, the multi-agent reinforcement learning algorithm is used to optimize the flight trajectory and transmission power of the drone, thereby achieving efficient collaboration and interference management in the disaster area wireless network. The core of this method is to use the agent to gradually adjust the flight trajectory and power allocation of the drone through experience replay and strategy optimization during the interaction with the environment to maximize the system's safety rate and communication performance.

[0069] It can be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0070] To summarize, the present invention combines cooperative jamming UAVs, relay UAVs and ground mobile base stations, and uses reinforcement learning algorithms to optimize the trajectory and power distribution of UAVs, and proposes a cooperative jamming UAV-assisted communication method for disaster area wireless networks. The core advantage lies in its comprehensiveness and adaptability, and it can dynamically adjust communication strategies in complex environments, thereby significantly improving the overall communication quality and security of the system.

[0071] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0073] Figure 1 A model diagram of a wireless network in a disaster area assisted by a drone in the present invention;

[0074] Figure 2 It is a flow chart of a multi-objective optimization algorithm based on a Markov decision process in the present invention;

[0075] Figure 3 The trajectory simulation diagram of the relay drone and the cooperative jammer drone at different iteration numbers;

[0076] Figure 4 The simulation diagram of the relationship between the maximum transmission power of the relay drone and the system safety rate under different iteration numbers;

[0077] Figure 5 The simulation diagram shows the cumulative flight distance of the relay UAV and the cooperative jammer UAV at different iteration numbers.

[0078] Figure 6The energy consumption simulation diagram of the relay UAV and the cooperative jammer UAV at different iteration numbers;

[0079] Figure 7 The simulation diagram of the relationship between the transmission power and flight duration of the cooperative jamming UAV at different iteration numbers;

[0080] Figure 8 Simulation plots of average safety rate and average convergence times between the power allocation algorithm and the deep deterministic gradient algorithm for joint trajectory design;

[0081] Fig. 9 A schematic diagram of a computer device provided by an embodiment of the present invention;

[0082] Fig.10 The present invention is a block diagram of an electronic device provided according to an embodiment. DETAILED DESCRIPTION

[0083] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0084] In the description of the present invention, it should be understood that the terms “include” and “comprises” indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0085] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0086] It should be further understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present invention generally indicates that the associated objects are in an "or" relationship.

[0087] It should be understood that, although the terms first, second, third, etc. may be used to describe preset ranges, etc. in the embodiments of the present invention, these preset ranges should not be limited to these terms. These terms are only used to distinguish preset ranges from each other. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0088] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0089] Various structural schematic diagrams of the embodiments disclosed in the present invention are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may additionally design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0090] The present invention provides a cooperative interference UAV-assisted communication method for disaster area wireless networks. First, by constructing a communication model between UAVs and ground users and calculating the channel energy gain based on a three-dimensional Cartesian coordinate system, an accurate environmental modeling is effectively provided for subsequent optimization, ensuring the communication quality between different nodes; secondly, under the premise of transmission power constraints, the flight trajectory and power distribution of relay UAVs and cooperative interference UAVs are optimized by maximizing the minimum safety rate, thereby ensuring the efficiency and anti-interference ability of the communication network. This strategy is particularly suitable for disaster area environments and can effectively cope with complex and dynamically changing communication needs; finally, multi-agent reinforcement learning is introduced, and the flight path and power distribution of the UAV are dynamically adjusted through the experience replay mechanism and strategy optimization, realizing adaptive optimization under different environmental conditions. This method not only improves the stability and reliability of communication, but also enables the system to cope with sudden signal interference, ensuring the efficient operation of the communication network in the disaster area.

[0091] The present invention provides a cooperative interference drone-assisted communication method for disaster area wireless networks, comprising the following steps:

[0092] S1. Model design;

[0093] S101, system model;

[0094] See also Figure 1 The present invention considers a disaster area wireless network assisted by cooperative jamming drones in the presence of eavesdropping drones. The relay drone acts as a relay to assist the wireless communication from the mobile ground base station (S) to the ground user. In order to improve the security performance of the system, the idle relay drones closest to the eavesdropping drone are selected and converted into cooperative jamming drones. The quality of the eavesdropping signal of the eavesdropping drone is reduced by sending jamming signals to the eavesdropping drone.

[0095] There are M drones and N ground users in the disaster area wireless network. Binary matrix and Indicates the role represented by the drone at time point t.

[0096] when When , the UAV acts as a relay to assist the communication between the ground mobile base station and the ground user, which is expressed as

[0097] when When , the UAV acts as a cooperative jammer to prevent the eavesdropping UAV from eavesdropping, which is expressed as

[0098] In a time interval, for each ground user, there is at most one relay UAV assisting in communication, and for each relay UAV, it serves at most one ground user, so there are the following constraints:

[0099]

[0100] Similarly, a cooperative jammer drone can only jam one eavesdropper drone in a time interval:

[0101]

[0102] It is worth noting that in order to facilitate the solution of the problem, only a single ground (n) user in the disaster area wireless network is studied. Therefore, correspondingly, only a group of relay drones and cooperative jammer drones are studied, denoted as m a ,a∈{r,j}.

[0103] S102, drone movement model;

[0104] The limited flight time T0 of the drone is divided into T = T0 / Δt time intervals, where Δt is small enough that the position of the drone can be considered fixed in a single time interval. To represent a collection of time intervals.

[0105] A three-dimensional Cartesian coordinate system is used to represent the location coordinates of mobile ground base stations, ground users, relay drones, cooperative jamming drones, and eavesdropping drones in the wireless network in the disaster area.

[0106] represents the plane coordinates of the relay UAV and the cooperative jammer UAV at the time interval;

[0107] q e [t] = [x e (t),y e (t)] T Represents the plane coordinates of the eavesdropping drone at the time interval.

[0108] The speed and acceleration of the relay UAV and the cooperative jammer UAV at the time interval are

[0109] According to the Taylor expansion formula, the relationship between the plane coordinates, velocity and acceleration of the relay UAV and the cooperative jammer UAV is:

[0110]

[0111] Similarly, the speed and acceleration of the eavesdropping drone at the time interval are

[0112] The maximum speed constraint of the drone is v max , so the maximum flight distance of the drone in a time interval is d max =v max Δt. Since the drone needs to refuel at the specified starting point and end point, the starting point and end point are set to

[0113] Generally speaking, the movement trajectories of ground mobile base stations and ground users are much smaller than those of drones, so their plane coordinates can be regarded as stationary, represented by q s =[x s ,y s ] T ,q n =[x n ,y n ] T .

[0114] If a relay drone is set up, the collaborative jamming drone and the eavesdropping drone can obtain each other's instantaneous position coordinates through radar or camera.

[0115] The constraints of the drone during flight are summarized as follows:

[0116]

[0117] Among them, dmin represents the minimum safety distance to prevent collision between drones, and ∥·∥ represents the Euclidean distance.

[0118] S103, communication model;

[0119] Set up relay drones, cooperative jamming drones and eavesdropping drones to fly at a fixed altitude In order to achieve the best communication performance, the relay drone, cooperative jammer drone and eavesdropping drone fly at the minimum operable altitude H.

[0120] Therefore, the three-dimensional coordinates of the relay drone, cooperative jamming drone, and eavesdropping drone over time are The corresponding plane coordinates are

[0121] Therefore, the distances from the ground mobile base station to the relay drone, the cooperative jamming drone, and the eavesdropping drone are:

[0122]

[0123] Similarly, at time t, the distances from the relay drone and the cooperative jammer drone to the ground user and the eavesdropping drone are expressed as:

[0124]

[0125] In addition, the distance between the relay UAV and the cooperative jammer UAV is expressed as:

[0126]

[0127] The line-of-sight communication model is adopted between the UAV and the ground user and between the ground user and the UAV. Therefore, according to the free space path loss model, the channel energy gain between the ground mobile base station to the relay UAV, the cooperative jamming UAV and the eavesdropping UAV is expressed as:

[0128]

[0129] Indicates the channel energy gain at a reference distance of 1m.

[0130] Similarly, the channel energy gain from the relay UAV and the cooperative jammer UAV to the ground user is expressed as:

[0131]

[0132] Since the relay drone, cooperative jamming drone and eavesdropping drone are on the same horizontal plane, the channel energy gain from the relay drone, cooperative jamming drone to the eavesdropping drone is expressed as:

[0133]

[0134] S104, UAV role conversion.

[0135] Due to the half-duplex operation mode, relay drones and cooperative jammer drones can be used as relay drones or cooperative jammer drones through dynamic role conversion. Each drone can only work as a relay drone or a cooperative jammer drone in a time interval, with the following constraints:

[0136]

[0137] In order to improve the secure communication performance of wireless networks in disaster areas, the relay drone closest to the eavesdropping drone is selected and converted into a collaborative jamming drone, thereby transmitting jamming signals to the eavesdropping drone.

[0138] S2, research question formulation;

[0139] S201, performance indicators;

[0140] Transmission power p of ground mobile base station, relay drone and cooperative jammer drone s [t] and It should be constrained within the range of the corresponding average transmission power and maximum transmission power. The constraint is expressed as:

[0141]

[0142] At time t, the achievable safety rate from the mobile ground base station to the relay drone and the eavesdropping drone is expressed as:

[0143]

[0144] in, and represents the white Gaussian noise at the relay drone and the eavesdropping drone.

[0145] Similarly, the achievable safety rate from the relay drone to the ground user and the eavesdropping drone is expressed as:

[0146]

[0147] in, represents white Gaussian noise at the ground user.

[0148] Therefore, at time t, the safety rates from the mobile ground base station to the relay drone and from the relay drone to the ground user are expressed as:

[0149]

[0150] Where [x]+ where represents the maximum value of x and 0.

[0151] S202. Research problem expression.

[0152] This paper studies the worst maximum security rate problem in wireless networks in disaster areas. In order to evaluate the worst security performance from the perspective of ground users, it is assumed that the eavesdropping drone approaches the ground users in the closest way. The minimum security rate of ground users is maximized by designing and optimizing the flight trajectory and transmission power of the relay drone and the cooperative jamming drone.

[0153] The problem of maximizing the minimum safety rate is expressed as:

[0154]

[0155] st(C1)-(C14)

[0156] Among them, (C1)-(C5) and (C10) represent the role assignment constraint rules of the UAV, (C6)-(C9) represent the movement trajectory constraints of the UAV, and (C11)-(C14) represent the transmission power constraints of the UAV and the mobile ground base station.

[0157] S3. Multi-objective optimization algorithm based on multi-agent reinforcement learning.

[0158] S301, Multi-objective optimization problem based on Markov decision process;

[0159] According to the flight trajectory, transmission power and interaction of the UAV with the environment, a Markov decision process is constructed. The optimization problem of the flight trajectory and transmission power of the UAV is represented by a five-tuple Markov decision process {I, S, A, T, R}.

[0160] Among them, I represents a finite set of agents, S represents a finite set of states of agents, A represents an action set of agents, T represents the transition probability of the agent state, and R represents a reward function.

[0161] Considering the overall performance of the wireless network in the disaster area assisted by the drone, the wireless network in the disaster area assisted by the drone is regarded as the environment, and the relay drone, cooperative jamming drone and eavesdropping drone are regarded as intelligent agents. The detailed five-tuple Markov decision process is as follows:

[0162] (1) State space: The state spaces of the relay drone, cooperative jamming drone, and eavesdropping drone are defined as:

[0163] s r (t) = {{sv} r (t),{sq} r (t),{sp} r(t)}

[0164] s j (t) = {{sv} j (t),{sq} j (t),{sp} j (t)}

[0165] s e (t) = {{sv} e (t),{sq} e (t)}

[0166] Among them, {sv} i (t),{sq} i (t) represents the speed and position of the UAV at time t, i∈{m,j,e}, {sp} k (t) represents the transmission power of the UAV, k∈{m,j}; suppose the relay UAV and the cooperative jammer UAV observe each other’s position coordinates.

[0167] (2) Action space: The action space of the relay drone and the cooperative jammer drone includes the flight speed {v} k (t) and transmission power {p} k (t); In addition, the action space of the eavesdropping drone includes the flight speed {v} e (t);

[0168] (3) State transition probability: The state transition probability indicates that at time t, when the agent selects an action to execute, it will change from the current state s t Transition to next state s t+1 probability;

[0169] For all s t ∈S,a t ∈A, satisfying the following constraints:

[0170]

[0171] (4) Reward function: In the UAV-assisted wireless disaster area wireless network, the goal of the optimization problem P1 is to maximize the minimum safety rate, and the training purpose of the agent is to maximize the cumulative reward function.

[0172] The instantaneous reward function of each agent at time t is expressed as:

[0173] r r (t) = R mr,n [t],

[0174] r j (t) = R mr,n [t],

[0175] r e (t) = R mr,e [t].

[0177] S302, trajectory and power optimization algorithm based on multi-agent deep deterministic gradient algorithm.

[0178] A multi-agent reinforcement learning algorithm based on the multi-agent deep deterministic gradient algorithm is introduced to solve the optimization problem of P1. The multi-agent deep deterministic gradient algorithm is an improvement on the actor-critic and deep deterministic policy gradient algorithms. It adopts the operation mode of centralized training and distributed execution. Each agent includes an actor network, a critic network, an actor-target network, and a critic-target network. In the multi-agent deep deterministic gradient algorithm, the actor network only uses local information for operation, the critic network uses global information for improvement, and each agent considers the influence of other agents when making decisions.

[0179] π j Determine the strategy of agent i in the joint trajectory design and power allocation algorithm. By modifying the parameters of the evaluation network and The optimal strategy is achieved. In the process, the parameters of the network are evaluated and Modifications are made in continuous time.

[0180] Specifically, operational experience (s t ,a t ,r t ,s t+1 ) is obtained through the interaction between the agent and the environment and is retained in the experience replay pool D.

[0181] During training, the parameters of the evaluation network are updated by extracting mini-batches of samples ∈ from the experience replay buffer D. By minimizing the loss function, the critic network modifies the evaluation network parameters The formula of the loss function can be expressed as follows:

[0182]

[0183] Among them, Q′ i (·) represents the state-action value function of the target network. In order to modify the parameters θ of the critic network i μ , we need to maximize the policy objective function. The expression of the policy objective function can be expressed as follows:

[0184]

[0185] Among them, μi (·) represents the deterministic strategy π i The actor evaluation network function.

[0186] Instead of setting the parameter μ′ i and Q′ i Directly transfer to the target network, but with the evaluation network parameters and The modifications are adjusted step by step as follows:

[0187]

[0188] Among them, λ x <<1,λ z <<1.

[0189] Algorithm 1 represents the training process of the above-mentioned multi-agent deep deterministic gradient algorithm to solve the joint trajectory design and power allocation problem.

[0190]

[0191] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as systems, methods or program products. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "platforms".

[0192] In another embodiment of the present invention, a collaborative interference drone-assisted communication system for disaster-area wireless networks is provided. The system can be used to implement the above-mentioned collaborative interference drone-assisted communication method for disaster-area wireless networks. Specifically, the collaborative interference drone-assisted communication system for disaster-area wireless networks includes a parameter module, an optimization module and a solution module.

[0193] Among them, the parameter module constructs a disaster area wireless network assisted by a cooperative jamming drone with an eavesdropping drone, uses a three-dimensional Cartesian coordinate system to represent the mobile ground base station in the disaster area wireless network, builds a drone mobile model, and the position coordinates of ground users, relay drones, cooperative jamming drones, and eavesdropping drones are obtained. The distance between the ground mobile base station and the relay drone, the cooperative jamming drone and the eavesdropping drone is obtained, and the channel energy gain between the ground mobile base station, the relay drone, the cooperative jamming drone, and the eavesdropping drone is obtained;

[0194] The optimization module constrains the transmission power of the ground mobile base station, relay drone and cooperative jamming drone within the corresponding average transmission power and maximum transmission power range, determines the safety rate from the mobile ground base station to the relay drone and from the relay drone to the ground user at time t, designs and optimizes the flight trajectory and transmission power of the relay drone and the cooperative jamming drone to maximize the minimum safety rate of the ground user, and designs the joint optimization of power allocation and flight trajectory;

[0195] The solution module constructs a Markov decision process based on the flight trajectory, transmission power and interaction of the drone with the environment. The wireless network in the disaster area assisted by the drone is used as the environment, and the relay drone, cooperative jamming drone and eavesdropping drone are used as agents to construct a multi-objective optimization problem based on the Markov decision process. A multi-agent reinforcement learning algorithm is introduced based on the multi-agent deep deterministic gradient algorithm to solve the multi-objective optimization problem based on the Markov decision process. i The strategy of agent i in the joint trajectory design and power allocation algorithm is determined, and the optimal strategy communication is achieved by modifying the parameters of the evaluation network.

[0196] In another embodiment of the present invention, a terminal device is provided, which includes a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, graphics processors (GPU), tensor processors (TPU), digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of the cooperative interference drone assisted communication method for disaster area wireless networks, including:

[0197] A wireless network in the disaster area assisted by cooperative jamming drones in the presence of eavesdropping drones is constructed. A three-dimensional Cartesian coordinate system is used to represent the mobile ground base stations in the wireless network in the disaster area. A drone movement model is constructed, and the position coordinates of ground users, relay drones, cooperative jamming drones and eavesdropping drones are constructed. The distances from the ground mobile base station to the relay drone, the cooperative jamming drone and the eavesdropping drone are obtained, and the channel energy gains between the ground mobile base station, the relay drone, the cooperative jamming drone and the eavesdropping drone are obtained; the transmission power of the ground mobile base station, the relay drone and the cooperative jamming drone is constrained within the corresponding range of the average transmission power and the maximum transmission power, and the time t when the mobile ground base station arrives is determined. Relay UAV, safety rate of relay UAV to ground users, design and optimize the flight trajectory and transmission power of relay UAV and cooperative jamming UAV to maximize the minimum safety rate of ground users, design power allocation and flight trajectory joint optimization; construct Markov decision process according to the flight trajectory, transmission power and interaction with the environment of UAV, take UAV-assisted disaster area wireless network as the environment, take relay UAV, cooperative jamming UAV and eavesdropping UAV as intelligent agents, construct multi-objective optimization problem based on Markov decision process, introduce a multi-agent reinforcement learning algorithm based on multi-agent deep deterministic gradient algorithm to solve the multi-objective optimization problem based on Markov decision process; transform π i The strategy of agent i in the joint trajectory design and power allocation algorithm is determined, and the optimal strategy communication is achieved by modifying the parameters of the evaluation network.

[0198] See also Fig. 9 , the terminal device is a computer device, and the computer device 60 of this embodiment includes: a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61. When the computer program 63 is executed by the processor 61, the cooperative interference drone-assisted communication method for disaster-affected wireless networks in the embodiment is implemented. To avoid repetition, it is not described one by one here. Alternatively, when the computer program 63 is executed by the processor 61, the functions of each model / unit in the cooperative interference drone-assisted communication system for disaster-affected wireless networks in the embodiment are implemented. To avoid repetition, it is not described one by one here.

[0199] The computer device 60 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The computer device 60 may include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art will appreciate that Fig. 9 This is only an example of the computer device 60 and does not constitute a limitation of the computer device 60. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the computer device may also include input and output devices, network access devices, buses, etc.

[0200] The processor 61 may be a central processing unit (CPU), or other general-purpose processors, graphics processing units (GPU), tensor processing units (TPU), digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0201] The memory 62 may be an internal storage unit of the computer device 60, such as a hard disk or memory of the computer device 60. The memory 62 may also be an external storage device of the computer device 60, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the computer device 60.

[0202] Furthermore, the memory 62 may include both an internal storage unit of the computer device 60 and an external storage device. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 may also be used to temporarily store data that has been output or is to be output.

[0203] See also Fig.10 The terminal device is an electronic device 600, which is in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0204] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention described in the above method section of this specification. For example, the processing unit 610 can perform the following steps: Figure 2 Follow the steps shown in .

[0205] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0206] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0207] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0208] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.

[0209] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a terminal device for storing programs and data. It is understandable that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and the extended storage medium supported by the terminal device, and can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that more specific examples (non-exhaustive list) of the computer-readable storage medium here include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0210] Computer readable storage media also include data signals propagated in baseband or as part of a carrier wave, which carry readable program codes. Such propagated data signals can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium can also be any readable medium other than a readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the readable storage medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0211] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0212] One or more instructions stored in a computer-readable storage medium may be loaded and executed by a processor to implement the corresponding steps of the cooperative interference drone-assisted communication method for disaster area wireless networks in the above embodiment; one or more instructions in the computer-readable storage medium are loaded and executed by the processor as follows:

[0213] A wireless network in the disaster area assisted by cooperative jamming drones in the presence of eavesdropping drones is constructed. A three-dimensional Cartesian coordinate system is used to represent the mobile ground base stations in the wireless network in the disaster area. A drone movement model is constructed, and the position coordinates of ground users, relay drones, cooperative jamming drones and eavesdropping drones are constructed. The distances from the ground mobile base station to the relay drone, the cooperative jamming drone and the eavesdropping drone are obtained, and the channel energy gains between the ground mobile base station, the relay drone, the cooperative jamming drone and the eavesdropping drone are obtained; the transmission power of the ground mobile base station, the relay drone and the cooperative jamming drone is constrained within the corresponding range of the average transmission power and the maximum transmission power, and the time t when the mobile ground base station arrives is determined. Relay UAV, safety rate of relay UAV to ground users, design and optimize the flight trajectory and transmission power of relay UAV and cooperative jamming UAV to maximize the minimum safety rate of ground users, design power allocation and flight trajectory joint optimization; construct Markov decision process according to the flight trajectory, transmission power and interaction with the environment of UAV, take UAV-assisted disaster area wireless network as the environment, take relay UAV, cooperative jamming UAV and eavesdropping UAV as intelligent agents, construct multi-objective optimization problem based on Markov decision process, introduce a multi-agent reinforcement learning algorithm based on multi-agent deep deterministic gradient algorithm to solve the multi-objective optimization problem based on Markov decision process; transform π i The strategy of agent i in the joint trajectory design and power allocation algorithm is determined, and the optimal strategy communication is achieved by modifying the parameters of the evaluation network.

[0214] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings here can usually be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0215] Simulation Results

[0216] Simulation parameter table

[0217]

[0218] See also Figure 3 , shows the trajectory routes of the relay drone and the cooperative jammer drone in each iteration, and the scenario is set that the eavesdropping drone approaches the ground user by the shortest route to eavesdrop. Figure 3 It shows that in the process of signal transmission to the ground user, the relay drone will first move away from the ground user due to the presence of the eavesdropping drone, and then move closer to the ground user due to the need for signal transmission. After the information transmission process is completed, the relay drone will move away from the ground user again and return to the set end point. In the process of jamming signal transmission, the jamming drone first flies in the direction of the ground user to interfere with the eavesdropping drone. When the distance to the ground user is close, in order to reduce the interference to the ground user's signal, the jamming drone will move away from the ground user and then fly back to the set end point. This dynamic trajectory adjustment process shows that the drone will flexibly adjust the flight path according to environmental changes and mission requirements when performing tasks to achieve the best signal transmission and jamming effect. This method not only ensures the security of signal transmission, but also minimizes the impact on ground users, reflecting the effectiveness of the joint design trajectory and power allocation strategy.

[0219] See also Figure 4 , showing the scheme of selecting cooperative drones as jammers in different iterations relative to the maximum transmission power The security performance of the confidentiality rate. First, we observed that the confidentiality rate showed an upward trend when the maximum transmission power of the relay drone increased. This is because higher transmission power enhances the signal strength, thereby improving the reception quality of ground users and ultimately improving the confidentiality rate. In addition, the confidentiality rate also shows an upward trend as the number of iterations of the reinforcement learning algorithm increases. This is because the solution obtained is continuously optimized as the data is continuously trained. Figure 4 As shown in the figure, the improvement in security performance is more obvious in the initial iteration, while the gain in subsequent iterations gradually decreases. This shows that in the early stages of the algorithm, the system learns and adjusts quickly, which can significantly improve the confidentiality rate. As the training progresses, the system is close to the optimal strategy, so the performance improvement decreases. This phenomenon shows that when designing a cooperative jamming scheme for drones, reasonable transmission power selection and continuous reinforcement learning training are key factors in improving security.

[0220] See also Figure 5 , showing the statistical results of the cumulative moving distance of the relay drone and the cooperative jammer drone at different iteration times. Figure 5It can be seen that as the number of iterations increases, the cumulative moving distance of the relay drone and the cooperative jamming drone gradually decreases. This shows that as the number of iterations increases, the cumulative moving distance of the two drones becomes shorter, allowing them to get closer to ground users when transmitting relay and jamming signals. Reducing the cumulative moving distance of the relay drone and the cooperative jamming drone can improve the energy efficiency of the drone. This means that by optimizing the flight trajectory and continuously improving the strategy, the drone can achieve more efficient signal transmission with lower energy consumption while completing the mission. This finding emphasizes the importance of iterative learning and trajectory optimization in drone collaborative tasks, which helps to improve the overall performance and operational efficiency of drones.

[0221] See also Figure 6 , the energy usage of relay drones and cooperative jamming drones at different iteration numbers was evaluated. It can be clearly seen that with the increase in the number of iterations, the energy consumption of relay drones and cooperative jamming drones shows a downward trend. This is because the flight trajectories of relay drones and cooperative jamming drones gradually approach ground users through iterative training of data by reinforcement learning algorithms, thereby reducing the cumulative moving distance of the two drones and thus reducing the energy consumption during flight. In addition, cooperative jamming drones need to send jamming signals to eavesdropping drones. Therefore, the energy consumption of cooperative jamming drones is significantly higher than that of relay drones.

[0222] See also Figure 7 , showing the variation curve of interference power when cooperative jammers interfere with eavesdropping drones in different iterations. Figure 7 As shown in the figure, during the interference operation of the cooperative interference UAV, the interference power first shows an upward trend, then gradually decreases to the minimum value, then continues to rise, and finally falls back to the initial value. This is because in the process of transmitting the interference signal, the UAV gradually approaches the ground user at the beginning. In order to minimize the interference to the ground user, the interference power decreases; and when the UAV is far away from the ground user, the interference power increases. In addition, as the number of iterations increases, the flight path of the cooperative interference UAV is continuously optimized, and the distance to the ground user is getting closer and closer, so the interference power decreases accordingly.

[0223] See also Figure 8 , demonstrating the excellent performance of the method of the present invention. Figure 8In the paper, the average confidentiality and convergence speed of the proposed multi-agent deep deterministic gradient algorithm and the deep deterministic policy gradient algorithm are compared. The learning rate of the deep deterministic policy gradient algorithm is 0.001 and the γ value is 0.95. The deep deterministic policy gradient algorithm is a deep reinforcement learning algorithm that adopts an actor-critic framework, in which the agents trained using the deep deterministic policy gradient algorithm only collect their own observation data during the training process. The simulation results show that the proposed joint trajectory design and power allocation algorithm outperforms the baseline algorithm in terms of security performance and convergence speed.

[0224] In summary, the present invention provides a cooperative jamming drone-assisted communication method and system for disaster area wireless networks, and studies the problem of secure wireless communication in wireless disaster area wireless networks with eavesdropping drones. From the perspective of ground users, we explored the problem of maximizing the worst-case confidentiality rate under the assumption that the eavesdropping drone approaches the ground user at the shortest distance. We enhance the security performance of wireless disaster area wireless networks by optimizing the flight trajectory and transmission power of relay drones and cooperative jamming drones. Due to the flexibility and complexity of drone flight paths, we use a joint trajectory design and power allocation algorithm to design and train the flight trajectory and transmission energy of relay drones and cooperative jamming drones, thereby solving the problem of secure wireless communication in wireless disaster area wireless networks. Simulation results show that the algorithm proposed by us has better performance than the reference algorithm.

[0225] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0226] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0227] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0228] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0229] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0230] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0231] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0232] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0233] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0234] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0235] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A cooperative jamming UAV-assisted communication method for disaster area wireless networks, characterized in that: The following steps are involved: Construct a disaster area wireless network assisted by cooperative jamming drones in the presence of eavesdropping drones, use a three-dimensional Cartesian coordinate system to represent mobile ground base stations in the disaster area wireless network, build a drone mobility model, and the location coordinates of ground users, relay drones, cooperative jamming drones, and eavesdropping drones. Obtain the distances from the ground mobile base station to the relay drone, the cooperative jamming drone, and the eavesdropping drone, and obtain the channel energy gain between the ground mobile base station, the relay drone, the cooperative jamming drone, and the eavesdropping drone. Constrain the transmission power of ground mobile base stations, relay drones and cooperative jamming drones within the corresponding range of average transmission power and maximum transmission power, determine the safety rate from mobile ground base stations to relay drones and from relay drones to ground users at time t, design and optimize the flight trajectory and transmission power of relay drones and cooperative jamming drones to maximize the minimum safety rate of ground users, and design joint optimization of power allocation and flight trajectory; According to the flight trajectory, transmission power and interaction of the drone with the environment, a Markov decision process is constructed. The wireless network in the disaster area assisted by drones is used as the environment. Relay drones, cooperative jamming drones and eavesdropping drones are used as agents to construct a multi-objective optimization problem based on the Markov decision process. A multi-agent reinforcement learning algorithm is introduced based on the multi-agent deep deterministic gradient algorithm to solve the multi-objective optimization problem based on the Markov decision process. i The strategy of agent i in the joint trajectory design and power allocation algorithm is determined, and the optimal strategy communication is achieved by modifying the parameters of the evaluation network.

2. The cooperative jamming drone-assisted communication method for disaster area wireless networks according to claim 1 is characterized in that: There are M drones and N ground users in the disaster area wireless network. and represents the role represented by the drone at time point t; when When the UAV acts as a relay to assist the communication between the ground mobile base station and the ground user, it is represented by m r ; when When the UAV acts as a cooperative jammer to prevent the eavesdropping UAV from eavesdropping, it is represented by m j ; The constraints are as follows: in, The UAV acts as a relay to communicate with ground user n at time intervals. For ground users, For drone collection, The UAV acts as a cooperative jammer to interfere with the eavesdropping UAV at time intervals.

3. The cooperative jamming drone-assisted communication method for disaster area wireless networks according to claim 2 is characterized in that: In the UAV mobility model, the constraints of the UAV during flight are summarized as follows: Among them, d min The minimum safe distance to prevent collision between drones, ∥·∥ is the Euclidean distance, For drones a The plane coordinates at the initial moment (0 moment) of the flight duration, For drones a The plane coordinates at the final moment of the flight duration (T moment), where T is the flight duration, To calculate the plane coordinates of the relay UAV and the cooperative jammer UAV at time intervals, is the plane coordinates of the relay UAV and the cooperative jammer UAV at the time interval, m a and m a ′ For two different drones in the drone collection, For drone collection, For the preset drone m a Takeoff initial plane coordinates, For the preset drone m a Final landing plane coordinates.

4. The cooperative jamming drone-assisted communication method for disaster area wireless networks according to claim 2 is characterized in that: Channel energy gain between ground mobile base station to relay drone, cooperative jamming drone and eavesdropping drone and h s,e [t] is represented by: Among them, β0 is the channel energy gain parameter at a reference distance of 1 meter, is the -2 power of the distance from the ground base station to the relay drone at the time interval, is the plane coordinate of the relay UAV at the time interval, q s is the plane coordinate of the mobile ground base station, H is the minimum allowable flight altitude of the UAV; The channel energy gain from the relay UAV and the cooperative jammer UAV to the ground user is expressed as: in, is the -2 power of the distance from the relay drone to the ground user, is the -2 power of the distance from the cooperative jamming drone to the ground user, q n is the plane coordinate of the ground user; The channel energy gain from the relay drone, cooperative jammer drone to the eavesdropping drone is expressed as: in, is the -2 power of the distance from the relay drone to the eavesdropping drone at the time interval, is the -2 power of the distance from the cooperative jamming drone to the eavesdropping drone at the time interval, q e [t] is the plane coordinate of the eavesdropping drone at the time interval, The plane coordinates of the cooperatively jamming UAVs at time intervals.

5. The cooperative jamming drone-assisted communication method for disaster area wireless networks according to claim 1 is characterized in that: Each drone can only work as a relay drone or a collaborative jammer drone within a time interval, with the following constraints: in, is the time interval at which the UAV acts as a relay to communicate with ground user n, N is the maximum number of ground users, is the interference of the UAV as a cooperative jammer UAV to the eavesdropping UAV at the time interval, and t is the time interval.

6. The cooperative jamming drone-assisted communication method for disaster area wireless networks according to claim 1 is characterized in that: The problem of maximizing the minimum safety rate can be expressed as: P1: st(C1)-(C14) in, is the minimum safety rate for relaying UAVs to ground users, is the plane coordinate of the relay drone, is the plane coordinates of the cooperative jamming drone, is the transmission power of the relay UAV, is the transmission power of the cooperative jammer UAV, (C1)-(C5) and (C10) represent the role assignment constraint rules of the UAV, (C6)-(C9) represent the movement trajectory constraints of the UAV, and (C11)-(C14) represent the transmission power constraints of the UAV and the mobile ground base station.

7. The cooperative jamming drone-assisted communication method for disaster area wireless networks according to claim 6 is characterized in that: At time t, the reachable safety rate of the mobile ground base station to the relay drone and the eavesdropping drone and It is expressed as: in, and is the white Gaussian noise at the relay drone and the eavesdropping drone, p s [t] is the transmission power of the mobile ground base station during the time interval, is the channel energy gain from the mobile ground base station to the relay drone at time interval, is the transmission power of the cooperative jammer drone at the time interval, is the channel energy gain from the relay UAV to the cooperative jammer UAV at the time interval, h s,e [t] is the channel energy gain from the mobile ground base station to the eavesdropping drone at the time interval, is the channel energy gain from the cooperative jamming UAV to the eavesdropping UAV at the time interval; Achievable safety rate from relay drones to ground users and eavesdropping drones and It is expressed as: in, is the white Gaussian noise at the ground user; At time t, the safety rate from the mobile ground base station to the relay drone and from the relay drone to the ground user is and Respectively expressed as: Where [x] + where represents the maximum value of x and 0.

8. The cooperative jamming drone-assisted communication method for disaster area wireless networks according to claim 1 is characterized in that: The Markov decision process is constructed as follows: The optimization problem of the flight trajectory and transmission power of the UAV is represented by a five-tuple Markov decision process {I, S, A, T, R}, where I represents a finite set of agents, S represents a finite set of states of the agents, A represents an action set of the agents, T represents the transition probability of the agent states, and R represents a reward function. Based on the multi-agent deep deterministic gradient algorithm, a multi-agent reinforcement learning algorithm is introduced to solve the optimization problem of P1. Each agent includes an actor network, a critic network, an actor target network and a critic target network. In the multi-agent deep deterministic gradient algorithm, the actor network only uses local information for operation, the critic network uses global information for improvement, and each agent considers the influence of other agents when making decisions. i Determine the strategy of agent i in the joint trajectory design and power allocation algorithm; by modifying the parameters of the evaluation network and Enable optimal strategic communications.

9. The cooperative jamming drone-assisted communication method for disaster area wireless networks according to claim 8 is characterized in that: π i Determine the strategy of agent i in the joint trajectory design and power allocation algorithm by modifying the parameters of the evaluation network and The best strategy is: Operating experience(s) t ,a t ,r t ,s t+1 ) is obtained through the interaction between the agent and the environment and is retained in the experience replay pool D; during training, the parameters of the evaluation network are updated by extracting small batches of samples ∈ from the experience replay buffer D; by minimizing the loss function, the critic network modifies the evaluation network parameters Maximize the policy objective function, as the network parameters are evaluated and Modification, gradually adjust the parameter μ′ i and Q′ i .

10. A cooperative jamming drone-assisted communication system for disaster area wireless networks, characterized in that: include: Parameter module, constructs a disaster area wireless network assisted by cooperative jamming drones in the presence of eavesdropping drones, uses a three-dimensional Cartesian coordinate system to represent mobile ground base stations in the disaster area wireless network, builds a drone mobile model, and the location coordinates of ground users, relay drones, cooperative jamming drones, and eavesdropping drones are obtained. The distances from the ground mobile base station to the relay drone, the cooperative jamming drone, and the eavesdropping drone are obtained, and the channel energy gains between the ground mobile base station, the relay drone, the cooperative jamming drone, and the eavesdropping drone are obtained; The optimization module constrains the transmission power of the ground mobile base station, relay drone and cooperative jamming drone within the corresponding average transmission power and maximum transmission power range, determines the safety rate from the mobile ground base station to the relay drone and from the relay drone to the ground user at time t, designs and optimizes the flight trajectory and transmission power of the relay drone and the cooperative jamming drone to maximize the minimum safety rate of the ground user, and designs the joint optimization of power allocation and flight trajectory; The solution module constructs a Markov decision process based on the flight trajectory, transmission power and interaction of the drone with the environment. The wireless network in the disaster area assisted by the drone is used as the environment, and the relay drone, cooperative jamming drone and eavesdropping drone are used as agents to construct a multi-objective optimization problem based on the Markov decision process. A multi-agent reinforcement learning algorithm is introduced based on the multi-agent deep deterministic gradient algorithm to solve the multi-objective optimization problem based on the Markov decision process. i The strategy of agent i in the joint trajectory design and power allocation algorithm is determined, and the optimal strategy communication is achieved by modifying the parameters of the evaluation network.

Images