Safety cooperative control method and system for unmanned aerial vehicle to realize mobile crowd perception

Abstract

The invention discloses a security cooperative control method and system for realizing mobile crowd perception by unmanned aerial vehicles. The method comprises the steps of setting a random action exploration process, selecting initial actions of multiple unmanned aerial vehicles, calculating rewards based on a mobile crowd perception model of the multiple unmanned aerial vehicles, establishing an experience pool and updating states of the multiple unmanned aerial vehicles. Extracting a sample from the experience pool, calculating an expected income of multi-unmanned aerial vehicle data collection, and updating a reviewer network according to the expected income; constructing a language representation ERNIE model based on knowledge fusion enhancement and a Stackelberg game model, performing gradient descent through a gradient calculated by the Stackelberg game model, and solving a result of minimum gradient descent; and updating the target strategy network, adding different disturbances, observing fluctuation of reward values under different disturbances, if a convergence condition is met, storing the current strategy, otherwise, returning to extract samples from the experience pool again. According to the method, the safety and robustness problems caused by unmanned aerial vehicle cooperative control deviation can be relieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicle dispatching, and in particular relates to a method and system for safely coordinating control of unmanned aerial vehicles to achieve mobile crowd perception. Background Art

[0002] Mobile Crowd Sensing (MCS) based on smart devices has become an attractive paradigm. With the development of 5G and above technologies, real-time applications of unmanned aerial vehicles have become possible by using drones as aerial base stations (BSs) to move and collect data from multiple users. Existing drone scheduling algorithms all achieve global optimal solutions based on the precise scheduling of drones. In actual situations, due to the influence of various environmental factors, it is often difficult to achieve precise scheduling of drones, and even individual drones will have large deviations under malicious attacks. Studies have shown that attackers can destroy the accuracy of scheduling decisions by modifying or injecting erroneous data, causing deviations in drone scheduling, thereby affecting the work of drones in the next stage. In this case, a robust control method needs to be designed to mitigate the impact of scheduling deviations.

[0003] Existing solutions based on deep reinforcement learning do not take into account the deviation of drone control and the attacker's attack on the drone's trajectory when scheduling drones, which will lead to a decrease in algorithm efficiency and even threaten flight safety. For example, in the patent application "Unmanned Aerial Vehicle Autonomous Guidance Control Method Based on DDPG" with publication number CN110806756B, a drone autonomous guidance control based on the DDPG (Deep deterministic policy gradient) algorithm is proposed, focusing on using deep learning to improve the autonomy and task execution efficiency of drones. For another example, the patent application "UAV Swarm Intelligence Perception Scheduling Method, System, Equipment and Medium" with publication number CN115729258A proposes to model the drone scheduling problem as a constrained cooperative Markov game, and solve the optimal scheduling strategy by designing state, action, reward and loss functions, so that the drone can maximize the perception benefit under the constraints of limited resources (such as battery charging budget). However, these methods do not consider that attackers will destroy the accuracy of scheduling decisions by modifying or injecting erroneous data, thereby causing deviations in drone scheduling. The patent application with publication number CN116931543A, "A safe flight control method, system, device and medium for multi-UAV data collection", proposes several key improvements to MADDPG (Multi-Agent Deep Deterministic Policy Gradient, multi-agent reinforcement learning algorithm), aiming to improve the security and robustness of multi-UAV data collection, especially the protection against physical perception inconsistency (PSI) and location attacks. However, this method does not have sufficient protection measures when facing complex environmental changes or malicious attacks. Summary of the invention

[0004] The purpose of the present invention is to address the problems in the above-mentioned prior art and provide a method and system for safe collaborative control of drones to achieve mobile crowd perception, thereby alleviating the safety and robustness problems caused by the collaborative control deviation of drones.

[0005] In order to achieve the above object, the present invention has the following technical solutions:

[0006] In a first aspect, a method for safe collaborative control of a drone to achieve mobile crowd perception is provided, comprising:

[0007] Construct a multi-UAV mobile crowd perception model;

[0008] Initialize the states of multiple drones, set a random action exploration process, select the initial actions of multiple drones, and initialize the critic network and policy network;

[0009] Calculate rewards based on the multi-drone mobile crowd perception model, build an experience pool, and update the multi-drone status;

[0010] Draw samples from the experience pool, calculate the expected return of multi-drone data collection, and update the critic network based on the expected return;

[0011] Construct the ERNIE model of language representation enhanced by knowledge fusion and the Stackelberg game model, perform gradient descent through the gradient calculated by the Stackelberg game model, and solve the result of minimizing the gradient descent;

[0012] Update target policy network;

[0013] After the target policy network is updated, different disturbances are added to observe the fluctuations of the reward value under different disturbances. If the convergence conditions are met, the current policy is saved, otherwise it returns to re-sample from the experience pool.

[0014] As a preferred solution, the multi-UAV mobile crowd perception model includes a system model and a threat model. The system model includes a multi-UAV flight model, a communication model and an energy consumption model; the multi-UAV flight model defines two attributes of multi-UAV flight angle and speed; the communication model includes the distance from multiple UAVs to ground users, channel gain, signal-to-noise ratio and unloading rate; the energy consumption model includes communication power, parasitic power, induction power and delay; and the threat model is a modeling of attackers.

[0015] As a preferred solution, the construction process of the multi-UAV flight model includes:

[0016] Let U{U|U=1,2,…,U}, M{M|M=1,2,…,M} represent the UAV and mobile user in the 3D target area respectively, and set their coordinates as In addition, there is a height h b high-rise building B{B|B=1,2,…,B}, when h b ≥h u When the drone is in the same building, it needs to avoid the corresponding building;

[0017] In each time slot [t, t+1), each drone u moves at a speed In a certain direction Moving τ time, v max is the maximum speed of the drone; each mobile user m starts from Move to And collect data separately, given the smart devices equipped with the expected sampling frequency of

[0018] The construction process of the communication model includes:

[0019] The path loss PL model for line-of-sight LoS and non-line-of-sight NLoS links is as follows:

[0020]

[0021] Where α LoS , β LoS , α NLoS , β NLoS are the environmental parameters on the floating intercept and slope, is the three-dimensional distance between mobile user m and drone u;

[0022] When a user uploads data to a drone, surrounding buildings and other users may block the LoS transmission path. The LoS probability for the period [t, t+1) is calculated as follows:

[0023]

[0024] In the formula, is the Euclidean distance between user m and drone u, where drone u is at height h u , and each user is modeled as having an average height h uscr and average diameter g user Cylinder; h device is the height of the user carrying the smart device; λ is the density of the blocker; it is assumed that the density of mobile users follows a Poisson distribution with parameter λ and the mobile user carrying the smart device is located at h device Height, the average path loss is:

[0025]

[0026] Let the maximum coupling loss MCL be the maximum loss that the system can tolerate and still operate at the conducted power level;

[0027] The construction process of the energy consumption model includes:

[0028] Constructing user residual data and Aoi model includes the following steps:

[0029] At time [t, t+1), user m tries to upload all remaining data to the nearest drone u. If PL t (u, m) is tolerable, indicating that the upload is successful, and the remaining data of user m Updated by the following expression:

[0030]

[0031] In the formula, G Tx and G Rxare the gains of the Tx and Rx antennas respectively. The final data collection amount for each user is:

[0032]

[0033] The update expression for user Aoi is:

[0034]

[0035] The energy consumption of the drone in each time slot is:

[0036]

[0037] Where c1, c2, c3 are constants that depend on the weight of the drone, rotor, blades, and air density; v tip and are the tip speed and average speed of the rotor respectively; is the flight speed of UAV u at time t; τ is the moving time;

[0038] When building threat models, attackers can undermine the accuracy of scheduling decisions by modifying or injecting erroneous data.

[0039] As a preferred solution, the steps of initializing the states of multiple drones, setting a random action exploration process, selecting the initial actions of multiple drones, and initializing the critic network and the policy network are as follows: a = φ(o) + X k Select the drone action, where a is the drone action, φ is the deterministic policy in the multi-agent deep deterministic policy gradient MADDPG algorithm, and X k is the random action exploration process of the UAV at time slot k.

[0040] As a preferred solution, in the steps of calculating rewards based on the multi-UAV mobile crowd perception model, establishing an experience pool, and updating the multi-UAV status, Calculate the reward, where:

[0041] Indicates the penalty when the drone u hits an obstacle or runs out of energy; C collect With C AoI is a constant; M is the number of users;

[0042] The experience pool contains the old state, action, reward and new state. (s, r, a, s′) is written into the experience pool F. The experience pool F contains (s′, s, a1, ..., a u , ..., a U ), all drone experiences were recorded;

[0043] The old state s is replaced by the new state s′ for the next time slot.

[0044] As a preferred solution, the steps of extracting samples from the experience pool, calculating the expected benefits of multi-UAV data collection, and updating the critic network according to the expected benefits include:

[0045] The expected benefit gradient of multi-UAV data collection is calculated as follows:

[0046]

[0047] Where o is the observation state of the UAV, w contains the observation results of all UAVs, {θ1,...,θ U} is the strategy set of multiple drones, φ u is the strategy of drone u under the deterministic strategy, A φ (w,a1,...,a u ) is the centralized action value function under the deterministic strategy φ, which contains all the drone actions and state information; a is the action given by the current strategy, and F is the experience pool;

[0048] The specific way to update the critic network is:

[0049] L(θ)=E F [(A φ (w,a1,...,a U )-b 2 )]

[0050] b=r+ζA φ′ (w′, a′1, ..., a′ U )| a′＝φ′ (o)

[0051] Where ζ represents the discount factor, 0<ζ<1; L(θ) is the loss function, b is the target value of the advantage function, and r is the reward value; w′ is the next state after executing the action in the current state; a′=φ′(o) represents the optimal action output by the policy network in the next state.

[0052] As a preferred solution, the language representation ERNIE model based on knowledge fusion enhancement is constructed as follows:

[0053]

[0054] The Stackelberg game model is constructed as follows:

[0055]

[0056] U θis an update operation based on gradient ascent, which is used to gradually maximize the divergence between the perturbation and the original observation, where Represents operator combination The details are as follows:

[0057]

[0058] The Stackelberg game model updates the gradient as follows:

[0059]

[0060] according to Update the target network;

[0061] Where o k is the observation value of the kth agent; θ represents the parameter of the strategy; δ K (o, θ) is the perturbation value generated by θ after K steps of updating; is the strategy of the kth agent; D is a distance function used to measure the difference between the outputs of two strategies; α, η are learning rates.

[0062] As a preferred solution, when adding different disturbances after the target policy network is updated, the disturbance values ​​are set to 0, 0.1, 0.2, and 0.3 respectively; the reward value fluctuations under different action disturbances are observed, and if the reward value fluctuation is less than 0.1, convergence is satisfied.

[0063] In the second aspect, a safety collaborative control system for a drone to realize mobile crowd perception is provided, including:

[0064] Perception model construction module, used to construct multi-UAV mobile crowd perception model;

[0065] The initialization module is used to initialize the states of multiple drones, set a random action exploration process, select the initial actions of multiple drones, and initialize the critic network and the policy network;

[0066] The experience pool establishment module is used to calculate rewards based on the multi-UAV mobile crowd perception model, establish an experience pool, and update the status of multiple UAVs;

[0067] An expected return calculation module, which is used to draw samples from the experience pool, calculate the expected return of multi-drone data collection, and update the critic network based on the expected return;

[0068] The gradient descent solution module is used to build the language representation ERNIE model and Stackelberg game model based on knowledge fusion enhancement, perform gradient descent through the gradient calculated by the Stackelberg game model, and solve the result of minimizing the gradient descent;

[0069] A target policy network update module, used for updating the target policy network;

[0070] The strategy screening module adds different disturbances after the target strategy network is updated, observes the fluctuation of the reward value under different disturbances, and saves the current strategy if the convergence conditions are met, otherwise returns to re-sample from the experience pool.

[0071] In a third aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, a safe collaborative control method for the drone to achieve mobile crowd perception is implemented.

[0072] Compared with the prior art, the present invention has at least the following beneficial effects:

[0073] In crowd sensing technology, when realizing the problem of mobile crowd sensing by deploying drones, the existing policy function based on deep reinforcement learning algorithm learning is heavily dependent on the precise scheduling of drones during the calculation process. In the real environment, the scheduling of drones will inevitably have various deviations, which will lead to the destruction of the reinforcement learning algorithm, thereby affecting the scheduling control of the entire system and even threatening flight safety. The present invention sets a random action exploration process, improves the deterministic strategy in the multi-agent deep deterministic policy gradient (MADDPG) algorithm, and constructs the language representation ERNIE model based on knowledge fusion enhancement and the Stackelberg game model. The gradient calculated by the Stackelberg game model is used for gradient descent to ensure that a relatively smooth policy function can still be obtained in the presence of scheduling deviations, thereby reducing the impact of scheduling deviations, so that the control method of the present invention can maintain stability and robustness in an uncertain environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0075] Figure 1 Flowchart of a method for safe collaborative control of a drone to achieve mobile crowd perception according to an embodiment of the present invention;

[0076] Figure 2 Schematic diagram of an application scenario of a method for safe collaborative control of a drone to achieve mobile crowd perception according to an embodiment of the present invention. DETAILED DESCRIPTION

[0077] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, ordinary technicians in this field can also obtain other embodiments without making creative work.

[0078] See also Figure 1 , 2 The embodiment of the present invention proposes a safe collaborative control method for UAVs to achieve mobile crowd perception, which is used to alleviate the safety and robustness problems caused by the collaborative control deviation of UAVs, and mainly includes the following steps:

[0079] S1. Construct a multi-UAV mobile crowd perception model;

[0080] S2, initialize the state of multiple drones, set a random action exploration process, select the initial actions of multiple drones, and initialize the critic network and policy network;

[0081] S3, calculate rewards based on the multi-UAV mobile crowd perception model, establish an experience pool, and update the multi-UAV status;

[0082] S4, draw samples from the experience pool, calculate the expected benefits of multi-drone data collection, and update the critic network based on the expected benefits;

[0083] S5. Construct the ERNIE model of language representation based on knowledge fusion enhancement and the Stackelberg game model, perform gradient descent through the gradient calculated by the Stackelberg game model, and solve the result of minimizing the gradient descent;

[0084] S6, update the target policy network;

[0085] S7. After the target policy network is updated, add different disturbances and observe the fluctuation of the reward value under different disturbances. If the convergence conditions are met, save the current policy, otherwise return to re-draw samples from the experience pool.

[0086] In a possible implementation, the multi-UAV mobile crowd perception model described in step S1 includes a system model and a threat model. The system model includes a multi-UAV flight model, a communication model, and an energy consumption model, including two roles: a UAV cluster and a ground user. The multi-UAV flight model defines two attributes of multi-UAV flight angle and speed. The communication model includes the distance from multiple UAVs to ground users, channel gain, signal-to-noise ratio, and unloading rate. The energy consumption model includes communication power, parasitic power, induction power, and delay. The threat model is a modeling of attackers.

[0087] Furthermore, the construction process of the multi-UAV flight model includes:

[0088] Let U{U|U=1,2,…,U}, M{M|M=1,2,…,M} represent the UAV and mobile user in the 3D target area respectively, and set their coordinates as In addition, there is a height h b high-rise building B{B|B=1,2,…,B}, when h b ≥h u When the drone is in the same building, it needs to avoid the corresponding building;

[0089] In each time slot [t, t+1), each drone u moves at a speed In a certain direction Moving τ time, v max is the maximum speed of the drone; each mobile user m starts from Move to And collect data separately, given the smart devices equipped with the expected sampling frequency of

[0090] The construction process of the communication model includes:

[0091] The path loss PL model for line-of-sight LoS and non-line-of-sight NLoS links is as follows:

[0092]

[0093] Where α LoS , β LoS , α NLoS , β NLoS are the environmental parameters on the floating intercept and slope, is the three-dimensional distance between mobile user m and drone u;

[0094] When a user uploads data to a drone, surrounding buildings and other users may block the LoS transmission path. The LoS probability for the period [t, t+1) is calculated as follows:

[0095]

[0096] In the formula, is the Euclidean distance between user m and drone u, where drone u is at height h u , and each user is modeled as having an average height h uscr and average diameter g user Cylinder; h device is the height of the user carrying the smart device; λ is the density of the blocker; it is assumed that the density of mobile users follows a Poisson distribution with parameter λ and the mobile user carrying the smart device is located at h device Height, so the average path loss is:

[0097]

[0098] Let the maximum coupling loss MCL be the maximum loss that the system can tolerate and still operate at the conducted power level.

[0099] The construction process of the energy consumption model includes:

[0100] Constructing user residual data and Aoi model includes the following steps:

[0101] At time [t, t+1), user m tries to upload all remaining data to the nearest drone u. If PL t (u, m) is tolerable, indicating that the upload is successful, and the remaining data of user m Updated by the following expression:

[0102]

[0103] In the formula, G Tx and G Rx are the gains of the Tx and Rx antennas respectively. The final data collection amount for each user is:

[0104]

[0105] The update expression for user Aoi is:

[0106]

[0107] The energy consumption of the drone in each time slot is:

[0108]

[0109] Where c1, c2, c3 are constants that depend on the weight of the drone, rotor, blades, and air density; v tip and are the tip speed and average speed of the rotor respectively; is the flight speed of UAV u at time t; τ is the moving time.

[0110] When building threat models, attackers can undermine the accuracy of scheduling decisions by modifying or injecting erroneous data.

[0111] In a possible implementation, step S2 is based on a=φ(o)+X k Select the drone action, where a is the drone action, φ is the deterministic policy in the multi-agent deep deterministic policy gradient MADDPG algorithm, and X k is the random action exploration process of the UAV at time slot k.

[0112] In a possible implementation, step S3 is performed by Calculate the reward, where: Indicates the penalty when the drone u hits an obstacle or runs out of energy; C collect With C AoI is a constant; M is the number of users.

[0113] The experience pool contains the old state, action, reward and new state. (s, r, a, s′) is written into the experience pool F. The experience pool F contains (s′, s, a1, ..., a u , ..., a U ), all drone experiences were recorded;

[0114] The old state s is replaced by the new state s′ for the next time slot.

[0115] In a possible implementation, step S4 calculates the expected benefit gradient of multi-UAV data collection according to the following formula:

[0116]

[0117] Where o is the observation state of the UAV, w contains the observation results of all UAVs, {θ1,...,θ U} is the strategy set of multiple drones, φ u is the strategy of drone u under the deterministic strategy, A φ (w,a1,...,a u ) is the centralized action value function under the deterministic strategy φ, which contains all drone actions and state information; a is the action given by the current strategy, and F is the experience pool.

[0118] The specific way to update the critic network is:

[0119] L(θ)=E F [A φ (w,a1,...,a U )-b 2 )]

[0120] b=r+ζA φ′ (w′, a′1, ..., a′ U )| a′＝φ′ (o)

[0121] Where ζ represents the discount factor, 0<ζ<1; L(θ) is the loss function, b is the target value of the advantage function, and r is the reward value; w′ is the next state after executing the action in the current state; a′=φ′(o) represents the optimal action output by the policy network in the next state.

[0122] In a possible implementation manner, the language representation ERNIE model based on knowledge fusion enhancement in step S5 is constructed as follows:

[0123]

[0124] The Stackelberg game model is constructed as follows:

[0125]

[0126] U θ is an update operation based on gradient ascent, which is used to gradually maximize the divergence between the perturbation and the original observation, where Represents operator combination The details are as follows:

[0127]

[0128] The Stackelberg game model updates the gradient as follows:

[0129]

[0130] according to Update the target network;

[0131] Where o k is the observation value of the kth agent; θ represents the parameter of the strategy; δ K (o, θ) is the perturbation value generated by θ after K steps of updating; is the strategy of the kth agent; D is a distance function used to measure the difference between the outputs of two strategies; α, η are learning rates.

[0132] In a possible implementation, step S6 is based on Update the target network.

[0133] In one possible implementation, in step S7, when different disturbances are added after the target policy network is updated, the disturbance values ​​are set to 0, 0.1, 0.2, and 0.3, respectively; the reward value fluctuations under different action disturbances are observed, and if the reward value fluctuation is less than 0.1, convergence is satisfied, and the current policy is saved, otherwise, return to step S4 to re-extract samples from the experience pool.

[0134] The safe collaborative control method for unmanned aerial vehicles to achieve mobile crowd perception in the embodiment of the present invention considers for the first time the scheduling deviation problem of multiple unmanned aerial vehicles to achieve mobile crowd perception based on deep reinforcement learning, constructs a unmanned aerial vehicle flight model and a communication control model, and improves the policy update of the MADDPG algorithm so that it can obtain a smooth policy function and reduce the impact of scheduling deviation. By applying the method of the present invention to the existing unmanned aerial vehicle system, environmental related parameters are designed in the unmanned aerial vehicle flight model and the communication control model, including the three-dimensional coordinates of the unmanned aerial vehicle, the flight speed, the flight angle, the link loss and some related evaluation indicators to calculate the reward value. In the process of improving the MADDPG algorithm, by adding a regularizer in the policy search By using an improved policy update mechanism, the algorithm can still obtain a relatively smooth policy function in the presence of scheduling deviations, thereby reducing the impact of scheduling deviations and enabling the algorithm to remain stable and robust in an uncertain environment. The practical application verification proves the applicability and robustness of the method in the existing UAV system, ensuring that it can operate effectively in various complex environments.

[0135] Another embodiment of the present invention further provides a safety collaborative control system for a drone to realize mobile crowd perception, comprising:

[0136] The perception model construction module is used to construct a multi-UAV mobile crowd perception model;

[0137] The initialization module is used to initialize the states of multiple drones, set a random action exploration process, select the initial actions of multiple drones, and initialize the critic network and the policy network;

[0138] The experience pool establishment module is used to calculate rewards based on the multi-UAV mobile crowd perception model, establish an experience pool, and update the status of multiple UAVs;

[0139] An expected return calculation module, which is used to draw samples from the experience pool, calculate the expected return of multi-drone data collection, and update the critic network based on the expected return;

[0140] The gradient descent solution module is used to build the language representation ERNIE model and Stackelberg game model based on knowledge fusion enhancement, perform gradient descent through the gradient calculated by the Stackelberg game model, and solve the result of minimizing the gradient descent;

[0141] A target policy network update module, used for updating the target policy network;

[0142] The strategy screening module adds different disturbances after the target strategy network is updated, observes the fluctuation of the reward value under different disturbances, and saves the current strategy if the convergence conditions are met, otherwise returns to re-sample from the experience pool.

[0143] To address the problem of UAV scheduling deviation, the present invention, based on the existing MADDPG algorithm, alleviates the problem caused by UAV scheduling deviation by adding an adversarial regularizer to the original algorithm strategy search and minimizing it, thereby effectively improving the safety and robustness of multi-UAV collaborative control.

[0144] Another embodiment of the present invention further provides an electronic device, comprising:

[0145] A memory storing at least one instruction; and a processor executing the instruction stored in the memory to implement the safety collaborative control method for the drone to achieve mobile crowd perception.

[0146] Another embodiment of the present invention further proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for implementing the safe collaborative control of the drone to achieve mobile crowd perception.

[0147] Exemplarily, the instructions stored in the memory may be divided into one or more modules / units, which are stored in a computer-readable storage medium and executed by the processor to complete the safety collaborative control method for realizing mobile crowd sensing of a drone according to the present invention. The one or more modules / units may be a series of computer-readable instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the server.

[0148] The electronic device may be a computing device such as a smart phone, a notebook, a PDA, and a cloud server. The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the electronic device may also include more or fewer components, or a combination of certain components, or different components, for example, the electronic device may also include an input / output device, a network access device, a bus, etc.

[0149] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0150] The memory may be an internal storage unit of the server, such as a hard disk or memory of the server. The memory may also be an external storage device of the server, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the server. Furthermore, the memory may include both an internal storage unit of the server and an external storage device. The memory is used to store the computer-readable instructions and other programs and data required by the server. The memory may also be used to temporarily store data that has been output or is to be output.

[0151] It should be noted that the information interaction, execution process and other contents between the above-mentioned module units are based on the same concept as the method embodiment. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0152] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0153] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera device / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0154] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0155] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for safe collaborative control of unmanned aerial vehicles to achieve mobile crowd perception, characterized in that: include: Construct a multi-UAV mobile crowd perception model; Initialize the states of multiple drones, set a random action exploration process, select the initial actions of multiple drones, and initialize the critic network and policy network; Calculate rewards based on the multi-drone mobile crowd perception model, build an experience pool, and update the multi-drone status; Draw samples from the experience pool, calculate the expected return of multi-drone data collection, and update the critic network based on the expected return; Construct the ERNIE model of language representation enhanced by knowledge fusion and the Stackelberg game model, perform gradient descent through the gradient calculated by the Stackelberg game model, and solve the result of minimizing the gradient descent; Update target policy network; After the target policy network is updated, different disturbances are added to observe the fluctuations of the reward value under different disturbances. If the convergence conditions are met, the current policy is saved, otherwise it returns to re-sample from the experience pool.

2. According to claim 1, the method for safe collaborative control of unmanned aerial vehicles to achieve mobile crowd perception is characterized in that: The multi-UAV mobile crowd perception model includes a system model and a threat model. The system model includes a multi-UAV flight model, a communication model and an energy consumption model; the multi-UAV flight model defines two attributes of multi-UAV flight angle and speed; the communication model includes the distance from multiple UAVs to ground users, channel gain, signal-to-noise ratio and unloading rate; the energy consumption model includes communication power, parasitic power, induction power and delay; the threat model is the modeling of attackers.

3. The method for safe collaborative control of unmanned aerial vehicles to achieve mobile crowd perception according to claim 2 is characterized in that: The construction process of the multi-UAV flight model includes: Let U{U|U=1,2,…,U}, M{M|M=1,2,…,M} represent the UAV and mobile user in the 3D target area respectively, and set their coordinates as In addition, there is a height h b high-rise building B{B|B=1,2,…,B}, when h b ≥h u When the drone is in the same building, it needs to avoid the corresponding building; In each time slot [t, t+1), each UAV u moves at a speed In a certain direction Moving τ time, v max is the maximum speed of the drone; each mobile user m starts from Move to And collect data separately, given the smart devices equipped with the expected sampling frequency of The construction process of the communication model includes: The path loss PL model for line-of-sight LoS and non-line-of-sight NLoS links is as follows: Where α LoS , β LoS , α NLoS , β NIoS are the environmental parameters on the floating intercept and slope, is the three-dimensional distance between mobile user m and drone u; When a user uploads data to a drone, surrounding buildings and other users may block the LoS transmission path. The LoS probability for the period [t, t+1) is calculated as follows: In the formula, is the Euclidean distance between user m and drone u, where drone u is located at height hu and each user is modeled as having an average height h user and average diameter g user Cylinder; h device is the height of the user carrying the smart device; λ is the density of the blocker; assuming that the density of mobile users follows a Poisson distribution with parameter λ and the mobile user carrying the smart device is located at h dwvice Height, the average path loss is: Let the maximum coupling loss MCL be the maximum loss that the system can tolerate and still operate at the conducted power level; The construction process of the energy consumption model includes: Constructing user residual data and Aoi model includes the following steps: At time [t, t+1), user m tries to upload all remaining data to the nearest drone u. If PL t (u, m) is tolerable, indicating that the upload is successful, and the remaining data of user m Updated by the following expression: In the formula, G Tx and G Rx are the gains of the Tx and Rx antennas respectively. The final data collection amount for each user is: The update expression for user Aoi is: The energy consumption of the drone in each time slot is: Where c1, c2, c3 are constants that depend on the weight of the drone, rotor, blades, and air density; v tip and are the tip speed and average speed of the rotor respectively; is the flight speed of UAV u at time t; τ is the moving time; When building threat models, attackers can undermine the accuracy of scheduling decisions by modifying or injecting erroneous data.

4. The method for safe collaborative control of unmanned aerial vehicles to achieve mobile crowd perception according to claim 1 is characterized in that: In the steps of initializing the states of multiple drones, setting a random action exploration process, selecting the initial actions of multiple drones, and initializing the critic network and the policy network, according to a=φ(o)+X k Select the drone action, where a is the drone action, φ is the deterministic policy in the multi-agent deep deterministic policy gradient MADDPG algorithm, and X k is the random action exploration process of the UAV at time slot k.

5. According to claim 1, the method for safe collaborative control of unmanned aerial vehicles to achieve mobile crowd perception is characterized in that: In the steps of calculating rewards based on the multi-UAV mobile crowd perception model, establishing an experience pool, and updating the multi-UAV status, Calculate the reward, where: Indicates the penalty when the drone u hits an obstacle or runs out of energy; C collect With C AoI is a constant; M is the number of users; The experience pool contains the old state, action, reward and new state. (s, r, a, s′) is written into the experience pool F. The experience pool F contains (s′, s, a1, ..., a u , ..., a U ), all drone experiences were recorded; The old state s is replaced by the new state s′ for the next time slot.

6. The method for safe collaborative control of unmanned aerial vehicles to achieve mobile crowd perception according to claim 1 is characterized in that: The steps of extracting samples from the experience pool, calculating the expected return of multi-drone data collection, and updating the critic network according to the expected return include: The expected benefit gradient of multi-UAV data collection is calculated as follows: Where o is the observation state of the UAV, w contains the observation results of all UAVs, {θ1,...,θ U } is the strategy set of multiple drones, φ u is the strategy of drone u under the deterministic strategy, A φ (w,a1,...,a u ) is the centralized action value function under the deterministic strategy φ, which contains all the drone actions and state information; a is the action given by the current strategy, and F is the experience pool; The specific way to update the critic network is: L(θ)=E F [(A φ (w, a1,..., a U )-b 2 )] b=r+ζA φ′ (w′,a′1,...,a′ U )| a′＝φ′(o) Where ζ represents the discount factor, 0<ζ<1; L(θ) is the loss function, b is the target value of the advantage function, r is the reward value; w′ is the next state after executing the action in the current state; a′=φ′(o) represents the optimal action output by the policy network in the next state.

7. The method for safe collaborative control of unmanned aerial vehicles to achieve mobile crowd perception according to claim 1 is characterized in that: The ERNIE model of language representation based on knowledge fusion enhancement is constructed as follows: The Stackelberg game model is constructed as follows: U θ is an update operation based on gradient ascent, which is used to gradually maximize the divergence between the perturbation and the original observation, where represents the operator combination The details are as follows: The Stackelberg game model updates the gradient as follows: according to Update the target network; Where o k is the observation value of the kth agent; θ represents the parameter of the strategy; δ K (o, θ) is the perturbation value generated by θ after K steps of updating; is the strategy of the kth agent; D is a distance function used to measure the difference between the outputs of two strategies; α, η are learning rates.

8. The method for safe collaborative control of unmanned aerial vehicles to achieve mobile crowd perception according to claim 1 is characterized in that: When adding different disturbances after the target policy network is updated, the disturbance values ​​are set to 0, 0.1, 0.2, and 0.3 respectively; the reward value fluctuations under different action disturbances are observed, and if the reward value fluctuation is less than 0.1, convergence is satisfied.

9. A safety collaborative control system for drones to achieve mobile crowd perception, characterized in that: include: The perception model construction module is used to construct a multi-UAV mobile crowd perception model; The initialization module is used to initialize the states of multiple drones, set a random action exploration process, select the initial actions of multiple drones, and initialize the critic network and the policy network; The experience pool establishment module is used to calculate rewards based on the multi-UAV mobile crowd perception model, establish an experience pool, and update the status of multiple UAVs; An expected return calculation module, which is used to draw samples from the experience pool, calculate the expected return of multi-drone data collection, and update the critic network based on the expected return; The gradient descent solution module is used to build the language representation ERNIE model and Stackelberg game model based on knowledge fusion enhancement, perform gradient descent through the gradient calculated by the Stackelberg game model, and solve the result of minimizing the gradient descent; A target policy network update module, used for updating the target policy network; The strategy screening module adds different disturbances after the target strategy network is updated, observes the fluctuation of the reward value under different disturbances, and saves the current strategy if the convergence conditions are met, otherwise returns to re-sample from the experience pool.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for safe collaborative control of a drone for mobile crowd perception as described in any one of claims 1 to 8 is implemented.

Images