Unmanned aerial vehicle dynamic trajectory optimization method and system for emergency communication scenarios

Abstract

The application discloses an unmanned aerial vehicle dynamic trajectory optimization method and system for an emergency communication scene, and relates to the unmanned aerial vehicle communication field.The technical solution points are as follows: a service sensing model of the emergency communication scene is constructed based on a named data network, wherein the service sensing model refers to dynamically adjusting the unmanned aerial vehicle trajectory to provide data transmission services for emergency services sent by users; an optimization problem is established to maximize the coverage rate of the unmanned aerial vehicle to the user end and the weighted service response rate, and to minimize the average energy consumption of the unmanned aerial vehicle, and the optimization problem is expressed as a partially observable Markov decision process model; a group of unmanned aerial vehicle agents are established, and a multi-agent deep deterministic policy gradient algorithm is used to train the group of unmanned aerial vehicle agents by using the utility function output by the partially observable Markov decision process model; and the local state data is decided according to the trained group of unmanned aerial vehicle agents, so as to adjust the flight trajectory of the unmanned aerial vehicle and the service type.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of unmanned aerial vehicle communication, in particular, to an unmanned aerial vehicle dynamic trajectory optimization method and system for emergency communication scenarios. BACKGROUND

[0002] At present, common traditional emergency communication methods include land mobile radio systems, satellite communication and vehicle-mounted emergency communication systems. Unmanned aerial vehicles can be used as mobile air base stations to provide emergency communication services for ground terminals in battlefields, disaster sites, crowded roads, blind areas and rural areas.

[0003] Unmanned aerial vehicle-assisted emergency communication networks have problems such as fast network topology changes and highly dynamic hotspot content. Traditional IP networks characterized by host address-based addressing and end-to-end connection have certain defects in timeliness, mobility and security. Therefore, future network paradigms represented by named data networks, the information-centered concept, and features such as content name-based routing and ubiquitous network caching will be more suitable for unmanned aerial vehicle-assisted emergency communication networks for business awareness.

[0004] In addition, the main problems to be solved for unmanned aerial vehicle-assisted ground communication are how to deploy unmanned aerial vehicles and how to dynamically adjust the trajectories of unmanned aerial vehicles according to ground requirements. In existing research, it is common to study 3D deployment and trajectory optimization of unmanned aerial vehicles with energy consumption, throughput, delay, coverage range and other optimization objectives, and most of them ignore the influence of business demand differences and emergency levels in emergency communication scenarios on optimization objectives. That is, less consideration is given to the influence of business demand differences and emergency levels in emergency communication scenarios on optimization objectives, which cannot meet the needs of emergency services in a timely manner and lacks practical application ability.

[0005] Therefore, how to perceive business and quickly respond to emergency tasks according to business differences is a problem that needs to be solved in emergency communication networks. SUMMARY

[0006] The purpose of the present application is to provide an unmanned aerial vehicle dynamic trajectory optimization method and system for emergency communication scenarios, which can significantly improve the response rate of emergency services in emergency communication scenarios, effectively reduce the average energy consumption of unmanned aerial vehicles, and improve the emergency communication capabilities of unmanned aerial vehicles in emergency communication scenarios.

[0007] The first aspect of the present application provides an unmanned aerial vehicle dynamic trajectory optimization method for emergency communication scenarios, which comprises:

[0008] A service-aware model for emergency communication scenarios is built based on named data networking. The service-aware model dynamically adjusts drone trajectories to provide data transmission services for emergency services sent by users. The emergency communication scenario refers to an emergency communication network established by a collection of drones, a collection of rescue personnel equipment, and a collection of ordinary user equipment. Drones, rescue personnel equipment, and ordinary user equipment all use named data networking protocols for data transmission within the emergency communication network.

[0009] Establish an optimization problem to maximize the coverage rate and weighted service response rate of drones to user terminals and minimize the average energy consumption of drones, and express the optimization problem as a partially observable Markov decision process model;

[0010] Establishing a drone swarm agent, and training the drone swarm agent using a multi-agent deep deterministic policy gradient algorithm with a utility function output by the partially observable Markov decision process model;

[0011] Based on the trained drone swarm agent, decisions are made on local status data to adjust the drone's flight trajectory and service category.

[0012] In one implementation, before building a service perception model of an emergency communication system based on a named data network, the method further includes: adding a Push data active transmission mechanism to extend the protocol of the named data network so that after the content is published, the content producer can directly push it to the user end.

[0013] In one implementation, a service awareness model for emergency communication scenarios is constructed based on named data networking, including:

[0014] Add request information to the interest packet and data packet, where the request information includes the requester identity and service category. The requester identity includes ordinary user equipment or rescue personnel equipment, and the service category includes emergency service, real-time service, or non-real-time service.

[0015] Different weight coefficients are assigned to emergency services, real-time services and non-real-time services, namely λ1, λ2 and λ3, and the conditions λ1>>λ2>>λ3 are met. The conditions λ1>>λ2>>λ3 indicate that services with high urgency are processed first.

[0016] In one implementation, an emergency service queue is added to the named data network, and interest packet information of three different urgency levels is stored in emergency service queues of different priorities respectively. The emergency service queue storing interest packet information of different urgency levels is added to the pending interest table so that the drone node responds to requests with higher urgency first.

[0017] In an implementation, when the rescue personnel device and / or the ordinary user device initiates a request to the UAV, the UAV will broadcast an interest packet in the emergency communication scenario. After the routing node in the emergency communication scenario receives the interest packet, the specific processing procedure of the business-aware model is as follows:

[0018] S1.1, query whether a copy of a data packet corresponding to the interest packet has been cached in a local content repository of the routing node. If the query hits, return to the requester and discard the corresponding interest packet; otherwise, jump to S1.2 and S1.3.

[0019] S1.2, query whether an entry of the interest packet has been recorded in a pending interest table of the routing node. If the query hits, record the source information and the forwarding interface of the interest packet and wait for the corresponding data packet; otherwise, jump to S1.3.

[0020] S1.3, query whether one or more next interfaces for forwarding the interest packet have been recorded in a forwarding information base of the routing node. If the query hits, forward the interest packet to the corresponding interface; otherwise, jump to S1.4.

[0021] S1.4, broadcast the interest packet to all routing nodes connected to all adjacent interfaces of the routing node in the same way.

[0022] Wherein, the interest packet refers to uplink data, and the data packet refers to downlink data.

[0023] In an implementation, when the request sent by the rescue personnel device and / or the ordinary user device is responded, the rescue personnel device and / or the ordinary user device will broadcast a data packet in the emergency communication scenario. After the routing node in the emergency communication network receives the data packet, the specific processing procedure of the business-aware model is as follows:

[0024] S2.1, query whether an interest packet corresponding to the data packet is waiting for a response in a pending interest table of the routing node. If the query hits, record the source address information in the interest packet, delete the corresponding record in the pending interest table, and jump to S2.3; otherwise, jump to S2.2.

[0025] S2.2, query whether there is forwarding information corresponding to the data packet in a forwarding information base of the routing node. If the query hits, select the shortest path to forward the data packet to all routing nodes needing the data packet according to the forwarding information base and the recorded source address information, and jump to S2.3; otherwise, discard the data packet.

[0026] S2.3, judge whether the data packet needs to be cached according to a least frequently used cache strategy. If the data packet needs to be cached, cache it locally; otherwise, do not cache the data packet.

[0027] In an implementation, the expression of the utility function is as follows: Among them, I t represents the coverage rate at time t, J t represents the weighted service response rate at time t, K t represents the average energy consumption at time t, γ i , γ j and γ k Represent the discount factors corresponding to coverage, weighted service response rate and average energy consumption respectively, and τ represents the current time;

[0028] Among them, the discount factor satisfies the following constraints: γ i ≥0,γ j ≥0,γ k ≥0&γ i +γ j +γ k =1.

[0029] In one implementation, a drone swarm agent is established, and a state space, action space, and reward function of the drone swarm agent are established; and the optimization problem is expressed as a partially observable Markov decision process model, including:

[0030] Initialize the experience replay pool;

[0031] At the beginning of each moment t, the drone swarm agent i follows the current strategy network π i , current status s i ={o u ,o d ,o i} and noise, select and perform an action a i,t , a i,t ~π i (·|s i,t ), at the same time, the drone swarm agent i establishes a connection with the rescuer equipment and / or ordinary user equipment that initiates the emergency request and responds to the emergency service;

[0032] After establishing a communication link between the UAV swarm agent i and the corresponding target device, the UAV swarm agent i will send the current state s i,t Transition to the second state s i,t+1 , and obtain the reward data R i,t , R i,t is the reward data obtained by drone swarm agent i at time t;

[0033] After the state transfer and reward data acquisition are completed, the experience replay pool records the state transfer (x t ,x t+1 ,a t ,r t ), x t+1 is the state data set of the agent at time t+1.

[0034] In one implementation, the drone swarm agent is trained using a multi-agent deep deterministic policy gradient algorithm with elements of the partially observable Markov decision process model output, including:

[0035] At the end of time t, the deep deterministic policy gradient algorithm is used to update the policy network of agent i: the target value for: in, is a function with a delay parameter θ′ i The target policy network set, j represents the randomly sampled state transition number, γ represents the discount rate, is the target value network of drone i; They are drones 1,…,N w Corresponding action data; represents the reward function of drone i; N w The number of drone devices;

[0036] At the end of time t, by minimizing the loss To update the value network of drone swarm agent i; in, Represents the set of all policy networks; is the value network of drone swarm agent i;

[0037] At the end of the decision time t, the deep deterministic policy gradient algorithm is used Update the weights of the policy network of agent i:

[0038] Update the weights θ′ of the target network i :θ′ i ←βθ i +(1-β)θ′ i ;β is the learning rate;

[0039] Repeat the updating steps of the policy network and the value network until the policy network π of the drone swarm agent i is i and value network convergence.

[0040] In a second aspect, the present application provides a UAV dynamic trajectory optimization system for emergency communication scenarios, the system comprising:

[0041] A service awareness model establishment module is used to build a service awareness model for emergency communication scenarios based on named data networking. The service awareness model refers to dynamically adjusting the trajectory of drones to provide data transmission services for emergency services sent by users. The emergency communication scenario refers to an emergency communication network established by a collection of drones, a collection of rescue personnel equipment, and a collection of ordinary user equipment. Drones, rescue personnel equipment, and ordinary user equipment all use the named data networking protocol for data transmission within the emergency communication network.

[0042] An optimization problem establishment module is used to establish an optimization problem that maximizes the coverage rate and weighted service response rate of the drone to the user terminal and minimizes the average energy consumption of the drone, and expresses the optimization problem as a partially observable Markov decision process model;

[0043] A training module, configured to establish a drone swarm agent, and train the drone swarm agent using a multi-agent deep deterministic policy gradient algorithm with a utility function output by the partially observable Markov decision process model;

[0044] The trajectory optimization module is used to make decisions on local state data based on the trained drone swarm agent to adjust the drone's flight trajectory and service category.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] The present invention provides a method and system for optimizing the dynamic trajectory of unmanned aerial vehicles (UAVs) for emergency communication scenarios. First, by constructing a service-aware model for emergency communication scenarios based on the Named Data Network (NDN), the UAV trajectory is dynamically adjusted to provide timely services for emergency services. Simultaneously, an optimization problem is established to maximize the UAV's user coverage, the system's weighted service response rate, and minimize the system's average energy consumption. Second, the optimization problem is formulated as a partially observable Markov decision process, mapping the UAV's motion into an action space while observing the surrounding environment and recording the state space. Then, based on a multi-agent deep deterministic policy gradient, the data corresponding to the elements of the partially observable Markov decision process model are used to centrally train the established UAV swarm agents until convergence. Finally, the UAV swarm agents use the collected local state data to make decisions and timely adjust the UAV's trajectory to serve the emergency services they serve. As a result, the present invention can significantly improve the response rate of emergency services issued in emergency communication scenarios, effectively reduce the average energy consumption of UAVs, and enhance the emergency communication capabilities in emergency communication scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0048] Figure 1 A schematic diagram of a flow chart of a method for optimizing the dynamic trajectory of a drone for emergency communication scenarios provided by an embodiment of the present application is shown;

[0049] Figure 2 The following is a schematic diagram showing the architecture of an emergency communication scenario provided by an embodiment of the present application;

[0050] Figure 3 The NDN packet structure with request information and zone information provided by an embodiment of the present application is shown;

[0051] Figure 4 A flowchart of the network protocol processing of the Named Data Network (NDN) with a Push data active transmission mechanism, service awareness capability, and asymmetric routing function provided by an embodiment of the present application is shown;

[0052] Figure 5 A structural diagram of a UAV dynamic trajectory optimization system for emergency communication scenarios provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0053] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with examples and drawings. The exemplary embodiments of the present invention and their descriptions are only used to explain the present invention and are not intended to limit the present invention.

[0054] It should be noted that the terms "include" or "may include" used in various embodiments of the present application indicate the presence of the claimed function, operation or element, and do not limit the addition of one or more functions, operations or elements. In addition, as used in various embodiments of the present application, the terms "include", "have" and their cognates are only intended to indicate specific features, numbers, steps, operations, elements, components or combinations of the foregoing items, and should not be understood as first excluding the presence of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing items or the possibility of adding one or more features, numbers, steps, operations, elements, components or combinations of the foregoing items.

[0055] In the description of the present invention, “plurality” means two or more, unless otherwise clearly defined.

[0056] Please refer to Figure 1 , Figure 1 The following is a flow chart showing a method for optimizing the dynamic trajectory of a drone for emergency communication scenarios provided by an embodiment of the present application. Figure 1 As shown, the method includes:

[0057] S110, construct a service perception model for emergency communication scenarios based on named data networking, where the service perception model refers to dynamically adjusting the trajectory of drones to provide data transmission services for emergency services sent by users, where the emergency communication scenario refers to an emergency communication network established by a collection of drones, a collection of rescue personnel equipment, and a collection of ordinary user equipment, and drones, rescue personnel equipment, and ordinary user equipment all use the named data networking protocol for data transmission within the emergency communication network.

[0058] In an embodiment of the present disclosure, before building a service perception model of an emergency communication system based on a named data network, the method further includes: adding a Push data active transmission mechanism to extend the protocol of the named data network so that after the content is published, the content producer can directly push it to the user end.

[0059] Specifically, the present invention adds a "Push" data transmission method. Push technology generally refers to a mechanism in which the server actively sends data to the client. Users subscribe to unpublished content in advance. When the content is published, the content producer can directly push it to the user without sending a request.

[0060] In some embodiments, a service awareness model for an emergency communication scenario is constructed based on a named data network, including:

[0061] Add request information to the interest packet and data packet, where the request information includes the requester identity and service category. The requester identity includes ordinary user equipment or rescue personnel equipment, and the service category includes emergency service, real-time service, or non-real-time service.

[0062] Different weight coefficients are assigned to emergency services, real-time services and non-real-time services, namely λ1, λ2 and λ3, and the condition λ1>>λ2>>λ3 is met. The condition λ1>>λ2>>λ3 means that services with high urgency are processed first.

[0063] Specifically, in order to better perform business perception, the present invention adds request information in the interest packet and data packet, such as Figure 3 As shown, the weight coefficients of each service are λ1, λ2 and λ3 respectively, and the condition λ1>>λ2>>λ3 is satisfied, indicating that the service with higher urgency is processed first.

[0064] When three types of services are simultaneously available on a node, emergency services should be prioritized, followed by real-time services, and finally non-real-time services. Therefore, a new data structure—the emergency service queue—is added to the named data network. The specific process is described in the following embodiment.

[0065] In some embodiments, an emergency service queue is added to the named data network, and interest packet information of three different urgency levels is stored in emergency service queues of different priorities respectively. The emergency service queue storing interest packet information of different urgency levels is added to the pending interest table so that the drone node responds to requests with higher urgency first.

[0066] like Figure 2 As shown, the UAV emergency communication system includes a collection of UAVs Rescue equipment collection and common user device collections Among them, N w 、N d and N u They represent the number of devices, namely drones, rescue personnel equipment, and ordinary user devices, and all of the above devices support the improved NDN protocol, but ordinary user devices do not have caching capabilities.

[0067] The emergency communication system can handle both active push notifications for emergency services and passive requests for standard services. ① When an emergency message is sent, the drone detects the user information subscribing to the content through the network and pushes the content directly to the subscriber immediately, eliminating the need for a request. ② When a user initiates a request, nearby drones are prioritized based on distance, and the request interest packet is broadcast to the emergency communication network for processing according to the NDN protocol. Assuming the system operates on a time-slot basis, the timeline is divided into T time slots of duration Δt, where t∈{1,2,3,…,T} is the time slot index.

[0068] The UAV location movement model, wireless communication model, content model, and energy consumption model are established through the service perception model based on the NDN protocol. The details are as follows:

[0069] 1. Build a drone mobility model

[0070] The position coordinates of the drone can be expressed as pos w ={x w ,y w ,z w}, where is the height of the drone relative to the sea level. Similarly, the coordinates of rescuers and ordinary users can be expressed as pos d ={x d ,y d ,z d} and pos u ={x u ,y u ,z uThe user's movement can be ignored relative to the drone's movement speed, so this paper only considers the drone's movement model. The definition of drone movement is to establish a spherical coordinate system with the right direction and the horizontal direction as the reference direction, and set the reference direction to v θ =0° and v φ =0°, moving speed is v w , the maximum speed is Then the UAV mobility model can be expressed as:

[0071]

[0072] In addition, in order to avoid UAV collision, the distance between any two UAVs cannot be less than the minimum safety distance, which can be expressed as: d w,w′ =||pos w -pos w′ ||≥d min (2).

[0073] 2. Establishing a Wireless Communication Model

[0074] The present invention considers the line-of-sight and non-line-of-sight propagation models of the transmission link between the UAV and the user. The link adopts millimeter wave transmission and uses the standard logarithmic fading model. Its path loss can be expressed as:

[0075] Among them, d w,u =||pos w -pos u || represents the distance between drone w and user u, and Represent the path loss index of line-of-sight and non-line-of-sight links, and Denote the fading random variables of line-of-sight and non-line-of-sight links respectively. The probability of line-of-sight connection can be calculated as follows:

[0076] Among them, κ and ζ are fixed coefficients that depend on the surrounding environment, is the elevation angle between user u and drone w. Therefore, the average loss between the drone and the user can be expressed as:

[0077] Based on the path loss, the signal-to-noise ratio (SINR) can be expressed as: Among them, p w,u represents the transmission power allocated by UAV w to user u, δ 2 represents the environmental noise. Then, the transmission rate between drone w and user u can be obtained as: Rw,u = B w,u log2(1 + SINR w,u )(7), where B w,u represents the channel bandwidth between the UAV w and the user u. Similarly, R w,w′ and R w,d represent the transmission rates between the UAV-UAV and the UAV-rescuer, respectively.

[0078] In addition, due to the limited coverage of the UAV, the movement of the UAV and the users causes some users to enter or leave the coverage of the UAV. In order to cope with such network topology changes, a wireless communication coverage model needs to be designed to determine the association between the UAV and the users. In order to guarantee the quality of service, the minimum threshold power p min that the user receives the signal power needs to satisfy the condition: p u = p w,u - PL w,u ≥ p min (8).

[0079] This is equivalent to the path loss needing to be within a certain threshold range, and when the path loss exceeds this range, the link is interrupted. This threshold corresponds to a coverage area that the UAV can serve, and users located in this area can establish a communication link with the UAV. The coverage of the UAV can overlap. If the user u is within the service range of the UAV w, it is represented by μ w,u = 1, otherwise μ w,u = 0. Similarly, if the rescuer d is within the service range of the UAV w, it is represented by μ w,d = 1, otherwise μ w,d = 0.

[0080] III. Establishing a content model

[0081] In the NDN-based emergency communication network, the UAV nodes and the rescue device nodes have in-network caching capabilities, and the cache status of the nodes can be obtained in real time. In addition, the name of each content is unique, and the request information of the user can be obtained. Therefore, the content model mainly considers the content caching model and the content request model.

[0082] For the content caching model, it is assumed that the content library is where F is the number of contents contained in the content library, D f represents the size of content f, and and represent the available cache space size of the UAV w and the rescue device d, respectively, and and respectively represent the cache status of the UAV w and the rescue device d for the content f, if the UAV w or the rescue device d caches the content f, the corresponding c w,f or c d,f is 1, otherwise 0. Then the remaining cache space of the UAV w and the rescue device d is:

[0083]

[0084] and needs to satisfy the condition:

[0085]

[0086] For the content request model, in the UAV emergency communication network, if the ordinary user u initiates a request for the content f, the available r u,f = 1, otherwise r u,f = 0. In addition, in combination with the emergency degree of user request, the content request of the ordinary user u can be expressed as Similarly, the content request of the rescue personnel d can be expressed as

[0087] Four, establish energy consumption model

[0088] In the emergency communication network, the UAV needs to move in the emergency area to provide timely communication service, so the energy consumption model needs to consider the power consumption and communication energy consumption of the UAV.

[0089] For power consumption, the energy consumption of this part can be divided into hovering energy consumption and moving energy consumption, which can be expressed as:

[0090] Among them, m w , r w and n w are the weight, radius and number of the propeller of the UAV, g and p respectively represent the gravitational constant and air density, v w is the speed of the UAV moving, is the maximum moving speed of the UAV w, and respectively represent the hardware power consumption when the UAV w moves at full speed and stops at a certain position. Therefore, the power consumption of the UAV to make a trajectory adjustment and complete data transmission content can be expressed as: Among them, is the time of the UAV moving, defined as the ratio of the moving distance of the UAV w and the speed size v w ; T fIt is the time when the drone hovers to transmit data, and is positioned as the content size D f and the transmission rate R w,u The ratio of:

[0091]

[0092] Regarding communication energy consumption, from formula 3-6, we can see that the transmission power of the drone to transmit data is p w,u , then the energy consumed by the drone to transmit content is:

[0093] Assume that the total energy of the drone at takeoff is equal to E w , Indicates the current remaining energy of the drone. represents the remaining energy of the UAV after performing k trajectory adjustments and completing the data transmission task, then:

[0094]

[0095] And the conditions are met:

[0096] S120, establishing an optimization problem to maximize the coverage rate and weighted service response rate of the drone to the user terminal and minimize the average energy consumption of the drone, and expressing the optimization problem as a partially observable Markov decision process model.

[0097] In the embodiments of this disclosure, for the four models described above, in emergency communication scenarios, user requests vary greatly and the degree of service urgency varies greatly. To respond to emergency tasks in a timely manner, this invention establishes a service-aware model based on NDN and dynamically adjusts the flight trajectory of drones to serve emergency tasks, maximizing the drone's user coverage and the system's weighted service response rate while minimizing the system's average energy consumption. Therefore, the objective function of the optimization problem can be composed of the following three parts:

[0098] 1. System coverage: As shown in Formula 3-8, a communication link can only be established when the path loss between drone w and user u is within a certain threshold, that is, drone w covers user u. The coverage ratio of the drone swarm can then be expressed as:

[0099] Here, “∪” represents an “or” operation, and ΔI can be used to represent the change in coverage rate caused by each adjustment of the drone’s position.

[0100] 2. System Weighted Service Response Rate: Since the goal of this study is to respond to more urgent requests promptly by dynamically adjusting drone trajectories, different weights are assigned to different services to maximize the system weighted service response rate. This is the ratio of the weighted service volume of all requests initiated by ground users that can be promptly responded to by drones to the weighted service volume of all requests, which can be expressed as: Among them, ζ is the gain coefficient requested by the rescuer, and ΔJ can be used to represent the change in the system weighted service response rate brought about by each adjustment of the drone position.

[0101] 3. System average energy consumption: Since drones need to move to provide emergency communication services, but their energy is limited, this study hopes to save energy as much as possible to reduce the system average energy consumption and provide longer service in the emergency communication area. It can be expressed as: ΔK can be used to represent the change in the average energy consumption of the system caused by each adjustment of the drone's position.

[0102] In summary, the optimization problem can be expressed as:

[0103] The constraint condition st includes formulas (2), (8), (10) and (16) provided in the above embodiment.

[0104] Where T represents the total number of times the drone adjusts its trajectory, and t represents the tth time the drone adjusts its trajectory. t 、J t and K t They represent the coverage rate, system weighted service response rate and system average energy consumption after the trajectory is adjusted for the first time, γ i , γ j and γ k The following relationship must be satisfied: i ≥0,γ j ≥0,γ k ≥0andγ i +γ j +γ k =1(21).

[0105] S130, establishing a drone swarm agent, and using a multi-agent deep deterministic policy gradient algorithm to train the drone swarm agent with the utility function output by the partially observable Markov decision process model.

[0106] In an embodiment of the present disclosure, the drone emergency communication system includes a drone collection Rescue equipment collection and common user device collections Among them, N w 、Nd and N u Represents the number of devices, and all of the above devices support the data naming network protocol, but ordinary user devices do not have the cache capability. This embodiment describes the three-dimensional motion of the drone in a spherical coordinate system, with the right and horizontal directions as reference directions. The user's request and the drone's status s={o u ,o d ,o w}, and use this to decide the direction and speed of movement a w ={v w ,v θ ,v φ}, expect to gain a larger reward function R τ Since the goal of each agent is to collaboratively maximize the utility function composed of coverage, system weighted traffic volume, and system average energy consumption, each agent has the same utility function at the same decision moment. The expression of the utility function is: Among them, I t represents the coverage rate at time t, J t represents the weighted service response rate at time t, K t represents the average energy consumption at time t, γ i , γ j and γ k Represent the discount factors corresponding to coverage, weighted service response rate, and average energy consumption, respectively. τ represents the current moment. i, j, and k are only used as identifiers and have no practical meaning.

[0107] Among them, the discount factor satisfies the following constraints: γ i ≥0,γ j ≥0,γ k ≥0&γ i +γ j +γ k =1.

[0108] represents the state set of all UAV agents, represents the action set of the agent, Represents the reward set of the agent.

[0109] Specifically, the partially observable Markov decision process model is as follows:

[0110] Initialize the experience replay pool;

[0111] At the beginning of each moment t, the drone swarm agent i follows the current strategy network π i , current status s i ={o u ,o d ,o i} and noise, select and perform an action a i,t , a i,t ~π i (·|s i,t ), at the same time, the drone swarm agent i establishes a connection with the rescuer equipment and / or ordinary user equipment that initiates the emergency request and responds to the emergency service;

[0112] After establishing a communication link between the UAV swarm agent i and the corresponding target device, the UAV swarm agent i will send the current state s i,t Transition to the second state s i,t+1 , and obtain the reward data R i,t , R i,t is the reward data obtained by drone swarm agent i at time t;

[0113] After the state transfer and reward data acquisition are completed, the experience replay pool records the state transfer (x t ,x t+1 ,a t ,r t ), x t+1 is the state data set of the agent at time t+1.

[0114] The drone swarm agent is trained using a multi-agent deep deterministic policy gradient algorithm with elements of the partially observable Markov decision process model output, including:

[0115] At the end of time t, the deep deterministic policy gradient algorithm is used to update the policy network of agent i: the target value for: in, is a function with a delay parameter θ′ i The target policy network set, j represents the randomly sampled state transition number, γ represents the discount rate, is the target value network of drone i; They are drones 1,…,N w Corresponding action data; represents the reward function of drone i; N w The number of drone devices;

[0116] At the end of time t, by minimizing the loss To update the value network of drone swarm agent i; in, Represents the set of all policy networks; is the value network of drone swarm agent i;

[0117] At the end of the decision time t, the deep deterministic policy gradient algorithm is used Update the weights of the policy network of agent i:

[0118] Update the weights θ′ of the target network i :θ′ i ←βθ i +(1-β)θ′ i ;β is the learning rate;

[0119] Repeat the updating steps of the policy network and the value network until the policy network π of the drone swarm agent i is i and value network convergence.

[0120] The above training process can be described as follows: First, for each UAV agent i, a strategy network π is initialized. i and value network The AC frame composed of θ i The policy network π is the weight i Able to generate differentiable samples through Gumbel-Softmax distribution, Represents the state set of all agents.

[0121] Then, for each UAV agent i, the target policy network π′ corresponding to the policy network and value network is initialized i and target value network

[0122] Finally, the algorithm initializes an experience replay pool Transfer with storage state (x t ,x t+1 ,a t ,r t ),in, represents the action set of the UAV agent, Represents the reward set of the agent. When the size of the experience replay pool reaches a certain scale, a small batch is randomly sampled from it to update the policy network and value network. After the initialization is completed, the agent training process begins. At each drone trajectory decision time t, the drone agent i collects the user request status information o u ={pos u ,r u} and rescuers request status information o d ={pos d ,r d}, and then perceive the status information of the drone Then according to the current status information s i ={o u ,o d,o i}、Current policy network π i and noise, select action a i,t Then, the UAV agent i establishes a connection with the user who initiated the emergency request and responds to the service, and updates its own state s i,t to s i,t+1 , obtain the reward R of this decision from the value network i,t After collecting the state transition (x t ,x t+1 ,a t ,r t ) and store it in the experience replay pool Then randomly sample a mini-batch (x j ,x j+1 ,a j ,r j At the end of the drone trajectory decision time t, each drone agent i updates the value network by minimizing the loss and the policy network using the policy gradient method. Finally, the updated value network and policy network are used to update the target network, and training is repeated until convergence.

[0123] S140, making decisions based on the trained drone swarm agent on the local state data to adjust the drone's flight trajectory and the service category.

[0124] In the embodiments disclosed herein, the decision-making process based on local state data (such as the current flight angle and flight parameters of the drones) is based on a trained drone swarm agent, which is a well-known technique in the art. Therefore, this embodiment will not be described in detail.

[0125] By combining the methods described in the above embodiments, by dynamically adjusting the trajectory of drones to provide timely services for emergency services, the response rate of emergency services can be significantly improved, the average energy consumption of the system can be effectively reduced, and the emergency communication capabilities can be improved.

[0126] The following is the data transmission process when the routing node in the emergency communication scenario receives the interest packet, such as Figure 4 As shown, in some embodiments, when a rescuer device and / or a general user device initiates a request to a drone, the drone broadcasts an interest packet in the emergency communication scenario. After the routing node in the emergency communication scenario receives the interest packet, the specific processing process of the service perception model is as follows:

[0127] S1.1. Query the routing node's local content repository to see if a copy of the Data Packet corresponding to the Interest Packet has been cached. If the query hits, return it to the requester and discard the corresponding Interest Packet; otherwise, jump to S1.2 and S1.3;

[0128] S1.2. Check whether the pending interest table of the routing node has an entry for the interest packet. If the query hits, record the source information and forwarding interface of the interest packet and wait for the corresponding data packet; otherwise, jump to S1.3;

[0129] S1.3. Query the forwarding information base of the routing node to see if one or more next interfaces for forwarding the Interest packet have been recorded. If the query hits, forward the Interest packet to the corresponding interface; otherwise, jump to S1.4;

[0130] S1.4. Broadcast the Interest packet to all routing nodes connected to the adjacent interfaces of the routing node in the same manner;

[0131] Among them, interest packets refer to uplink data, and data packets refer to downlink data.

[0132] The following is the data transmission process when the routing node in the emergency communication scenario receives the data packet, Figure 4 As shown, in some embodiments, when a request sent by a rescuer device and / or a common user device is responded to, the rescuer device and / or the common user device broadcasts a data packet in the emergency communication scenario. After a routing node in the emergency communication network receives the data packet, the specific processing process of the service perception model is as follows:

[0133] S2.1. Query the pending interest table of the routing node for the interest packet corresponding to the data packet waiting for a response. If the query hits, record the source address information in the interest packet, delete the corresponding record in the pending interest table, and jump to S2.3; otherwise, jump to S2.2;

[0134] S2.2. Query the forwarding information base of the routing node to see if there is forwarding information corresponding to the data packet. If the query hits, select the shortest path based on the forwarding information base and the recorded source address information to forward the data packet to all routing nodes that need the data packet, and jump to S2.3. Otherwise, discard the data packet.

[0135] S2.3. Determine whether the data packet is cached in the routing node according to the least used cache strategy. If caching is required, cache it locally; otherwise, do not cache the data packet.

[0136] Please refer to Figure 5 , Figure 5 The following is a schematic diagram showing the structure of a UAV dynamic trajectory optimization system for emergency communication scenarios provided by an embodiment of the present application. Figure 5 As shown, the system includes:

[0137] A service awareness model establishment module 510 is used to establish a service awareness model for emergency communication scenarios based on named data networking, wherein the service awareness model refers to dynamically adjusting the trajectory of drones to provide data transmission services for emergency services sent by users, wherein the emergency communication scenario refers to an emergency communication network established by a collection of drones, a collection of rescue personnel equipment, and a collection of ordinary user equipment, wherein the drones, rescue personnel equipment, and ordinary user equipment all use the named data networking protocol for data transmission within the emergency communication network;

[0138] An optimization problem establishment module 520 is used to establish an optimization problem to maximize the coverage rate and weighted service response rate of the drone to the user terminal and minimize the average energy consumption of the drone, and to express the optimization problem as a partially observable Markov decision process model;

[0139] A training module 530 is configured to establish a drone swarm agent and train the drone swarm agent using a multi-agent deep deterministic policy gradient algorithm with the utility function output by the partially observable Markov decision process model;

[0140] The trajectory optimization module 540 is used to make decisions on local state data based on the trained drone swarm agent to adjust the flight trajectory of the drone and the business category of the service.

[0141] In an embodiment of the present application, a drone dynamic trajectory optimization system for emergency communication scenarios is provided. First, by constructing a service perception model for emergency communication scenarios based on the named data network (NDN), the drone trajectory is dynamically adjusted to provide services for emergency services in a timely manner. At the same time, an optimization problem is established to maximize the drone's coverage of users, the system's weighted service response rate, and minimize the system's average energy consumption. Secondly, the above optimization problem is expressed as a partially observable Markov decision process, mapping the drone's motion to the action space, and observing the surrounding environment to record the state space. Then, based on the multi-agent deep deterministic policy gradient, the data corresponding to the elements of the partially observable Markov decision process model are received to centrally train the established drone swarm agent until the agent converges; finally, the drone swarm agent uses the collected local state data to make decisions and adjust the drone trajectory in a timely manner to serve the emergency services. As a result, the present invention can significantly improve the response rate of emergency services issued in emergency communication scenarios, effectively reduce the average energy consumption of the drone system, and enhance the emergency communication capabilities in emergency communication scenarios.

[0142] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for optimizing the dynamic trajectory of a UAV for emergency communication scenarios, characterized in that: Methods include: Add a push data active transmission mechanism to extend the named data network protocol, so that when content is published, it can be pushed directly to the user end by the content producer; A service-aware model for emergency communication scenarios is built based on named data networking. The service-aware model dynamically adjusts drone trajectories to provide data transmission services for emergency services sent by users. The emergency communication scenario refers to an emergency communication network established by a collection of drones, a collection of rescue personnel equipment, and a collection of ordinary user equipment. Drones, rescue personnel equipment, and ordinary user equipment all use named data networking protocols for data transmission within the emergency communication network. An optimization problem is established to maximize the coverage rate and weighted service response rate of drones to user terminals and minimize the average energy consumption of drones. The optimization problem is expressed as a partially observable Markov decision process model. The coverage rate is expressed as: Among them, ∪ represents the OR operation, N u Indicates the number of common user devices, N w represents the number of drone devices, u represents ordinary user devices, w represents drones, μ w,u Indicates whether the general user device is within the service range of the drone; The expression of weighted service response rate is: Among them, N d represents the number of equipment for rescue personnel, f F Indicates the Fth content, F is the number of contents in the content library, D f Indicates the size of the content f, c w,f Indicates that drone w has cached content f, c d,f Indicates that the rescuer device d has cached content f, r u,f Indicates the urgency of sending content by ordinary user equipment u, r d,f Indicates the urgency of the content sent by the rescuer's device d, Indicates the weight coefficient corresponding to the urgency of the content f sent by ordinary user equipment u, The weight coefficient corresponding to the urgency of the content f sent by the rescuer's device d, μ w,d Indicates whether the rescuer's device d is within the service range of the UAV w, and ζ represents the gain coefficient requested by the rescuer; The expression of the average energy consumption of the UAV is: in, represents the energy consumed by the drone to transmit the content, It represents the power consumption of the drone to make a trajectory adjustment and complete the data transmission. w Indicates the total energy of the drone when it takes off; Establishing a drone swarm agent, and training the drone swarm agent using a multi-agent deep deterministic policy gradient algorithm with a utility function output by the partially observable Markov decision process model; Based on the trained drone swarm agent, decisions are made on local status data to adjust the drone's flight trajectory and service category.

2. The method according to claim 1, characterized in that Building a service-aware model for emergency communication scenarios based on named data networking, including: Add request information to the interest packet and data packet, where the request information includes the requester identity and service category. The requester identity includes ordinary user equipment or rescue personnel equipment, and the service category includes emergency service, real-time service, or non-real-time service. Different weight coefficients are assigned to emergency services, real-time services and non-real-time services, namely λ1, λ2 and λ3, and the condition λ1>>λ2>>λ3 is met. The condition λ1>>λ2>>λ3 means that services with high urgency are processed first.

3. The method according to claim 2, characterized in that An emergency service queue is added to the named data network, and the interest packet information of three different urgency levels is stored in emergency service queues of different priorities respectively. The emergency service queues storing the interest packet information of different urgency levels are added to the pending interest table so that the drone node can respond to requests with high urgency first.

4. The method according to claim 3, characterized in that When the rescuer device and / or ordinary user device initiates a request to the drone, the drone will broadcast an interest packet in the emergency communication scenario. After the routing node in the emergency communication scenario receives the interest packet, the specific processing process of the service perception model is as follows: S1.

1. Query the routing node's local content repository to see if a copy of the Data Packet corresponding to the Interest Packet has been cached. If the query hits, return it to the requester and discard the corresponding Interest Packet; otherwise, jump to S1.2 and S1.3; S1.

2. Check whether the pending interest table of the routing node has an entry for the interest packet. If the query hits, record the source information and forwarding interface of the interest packet and wait for the corresponding data packet; otherwise, jump to S1.3; S1.

3. Query the forwarding information base of the routing node to see if one or more next interfaces for forwarding the Interest packet have been recorded. If the query hits, forward the Interest packet to the corresponding interface; otherwise, jump to S1.4; S1.

4. Broadcast the Interest packet to all routing nodes connected to the adjacent interfaces of the routing node in the same manner; Among them, interest packets refer to uplink data, and data packets refer to downlink data.

5. The method according to claim 4, characterized in that When the request sent by the rescuer device and / or ordinary user device is responded to, the rescuer device and / or ordinary user device will broadcast the data packet in the emergency communication scenario. After the routing node in the emergency communication network receives the data packet, the specific processing process of the service perception model is as follows: S2.

1. Query the pending interest table of the routing node for the interest packet corresponding to the data packet waiting for a response. If the query hits, record the source address information in the interest packet, delete the corresponding record in the pending interest table, and jump to S2.

3. Otherwise, jump to S2.2; S2.

2. Query the forwarding information base of the routing node to see if there is forwarding information corresponding to the data packet. If the query hits, select the shortest path based on the forwarding information base and the recorded source address information to forward the data packet to all routing nodes that need the data packet, and jump to S2.

3. Otherwise, discard the data packet. S2.

3. Determine whether the data packet is cached in the routing node according to the least used cache strategy. If caching is required, cache it locally; otherwise, do not cache the data packet.

6. The method according to claim 1, characterized in that The expression of the utility function is: Among them, I t represents the coverage rate at time t, J t represents the weighted service response rate at time t, K t represents the average energy consumption at time t, γ i , γ j and γ k Represent the discount factors corresponding to coverage, weighted service response rate and average energy consumption respectively, and τ represents the current time; Among them, the discount factor satisfies the following constraints: γ i ≥0,γ j ≥0,γ k ≥0&γ i +γ j +γ k =1.

7. The method according to claim 6, characterized in that Establish a drone swarm agent, and establish the state space, action space, and reward function of the drone swarm agent; express the optimization problem as a partially observable Markov decision process model, including: Initialize the experience replay pool; At the beginning of each moment t, the drone swarm agent i follows the current strategy network π i , current status s i ={o u ,o d ,o i } and noise, select and perform an action a i,t , a i,t ~π i (·|s i,t ), at the same time, the drone swarm agent i establishes a connection with the rescuer equipment and / or ordinary user equipment that initiates the emergency request and responds to the emergency service; After establishing a communication link between the UAV swarm agent i and the corresponding target device, the UAV swarm agent i will send the current state s i,t Transition to the second state s i,t+1 , and obtain the reward data R i,t , R i,t is the reward data obtained by drone swarm agent i at time t; After the state transfer and reward data acquisition are completed, the experience replay pool records the state transfer (x t ,x t+1 ,a t ,r t ), x t+1 is the state data set of the agent at time t+1.

8. The method according to claim 7, characterized in that The drone swarm agent is trained using a multi-agent deep deterministic policy gradient algorithm with elements of the partially observable Markov decision process model output, including: At the end of time t, the deep deterministic policy gradient algorithm is used to update the policy network of agent i: the target value for: in, is a function with a delay parameter θ i ′ is the target policy network set, j represents the randomly sampled state transition number, γ represents the discount rate, is the target value network of drone i; They are drones 1,…,N w Corresponding action data; represents the reward function of drone i; N w The number of drone devices; At the end of time t, by minimizing the loss To update the value network of drone swarm agent i; in, Represents the set of all policy networks; is the value network of drone swarm agent i; At the end of the decision time t, the deep deterministic policy gradient algorithm is used Update the weights of the policy network of agent i: Update the weights θ of the target network i ′:θ i ′←βθ i +(1-β)θ i ′; β is the learning rate; Repeat the updating steps of the policy network and the value network until the policy network π of the drone swarm agent i is i and value network convergence.

9. A UAV dynamic trajectory optimization system for emergency communication scenarios, characterized in that the system include: A service awareness model establishment module is used to add a push data active transmission mechanism to expand the named data network protocol, so that when content is published, the content producer can directly push it to the user end; a service awareness model for emergency communication scenarios is built based on the named data network, where the service awareness model refers to dynamically adjusting the trajectory of drones to provide data transmission services for emergency services sent by users. The emergency communication scenario refers to an emergency communication network established by a collection of drones, a collection of rescue personnel equipment, and a collection of ordinary user equipment. The drones, rescue personnel equipment, and ordinary user equipment all use the named data network protocol for data transmission within the emergency communication network; The optimization problem establishment module is used to establish an optimization problem to maximize the coverage rate and weighted service response rate of the drone to the user terminal and minimize the average energy consumption of the drone, and express the optimization problem as a partially observable Markov decision process model; wherein, the expression of coverage rate is: Among them, ∪ represents the OR operation, N u Indicates the number of common user devices, N w represents the number of drone devices, u represents ordinary user devices, w represents drones, μ w,u Indicates whether the general user device is within the service range of the drone; The expression of weighted service response rate is: Among them, N d represents the number of equipment for rescue personnel, f F Indicates the Fth content, F is the number of contents in the content library, D f Indicates the size of the content f, c w,f Indicates that drone w has cached content f, c d,f Indicates that the rescuer device d has cached content f, r u,f Indicates the urgency of sending content by ordinary user equipment u, r d,f Indicates the urgency of the content sent by the rescuer's device d, Indicates the weight coefficient corresponding to the urgency of the content f sent by ordinary user equipment u, The weight coefficient corresponding to the urgency of the content f sent by the rescuer's device d, μ w,d Indicates whether the rescuer's device d is within the service range of the UAV w, and ζ represents the gain coefficient requested by the rescuer; The expression of the average energy consumption of the UAV is: in, represents the energy consumed by the drone to transmit the content, It represents the power consumption of the drone to make a trajectory adjustment and complete the data transmission. w Indicates the total energy of the drone when it takes off; A training module, configured to establish a drone swarm agent, and train the drone swarm agent using a multi-agent deep deterministic policy gradient algorithm with a utility function output by the partially observable Markov decision process model; The trajectory optimization module is used to make decisions on local state data based on the trained drone swarm agent to adjust the drone's flight trajectory and service category.

Images