Unmanned aerial vehicle swarm perception method for urban monitoring data delay guarantee
By constructing a trajectory evaluation model and an asynchronous backtracking algorithm to optimize the perception strategy of UAV swarms, the problems of task value evaluation and cooperation mode in urban monitoring data latency assurance tasks were solved, achieving efficient data latency assurance and decision accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING INST OF TECH
- Filing Date
- 2022-09-19
- Publication Date
- 2026-08-04
AI Technical Summary
In urban surveillance data latency assurance tasks, drone swarms face challenges such as difficulty in evaluating mission value, limited sensing range, and unclear cooperation models, leading to data latency exceeding thresholds and affecting data value and decision-making accuracy.
By constructing a trajectory evaluation model and combining asynchronous backtracking algorithm and deep reinforcement learning, the perception strategy of UAV swarms is optimized, the search space is reduced, and the decision-making accuracy and cooperation efficiency are improved. Historical sequence modeling and division of labor evaluation model are used to optimize UAV trajectory planning and data acquisition.
It effectively reduces the probability of data latency exceeding the threshold, improves data collection rate and decision accuracy, and is suitable for data latency assurance tasks in complex urban environments.
Smart Images
Figure CN115617472B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of mobile swarm perception, specifically relating to a drone swarm perception method for ensuring latency of urban surveillance data. Background Technology
[0002] In recent years, China has carried out a large number of video surveillance projects to improve and strengthen the three-dimensional social security prevention and control system. Since video surveillance relies on the Internet for data transmission, data cannot be effectively acquired under some emergency conditions (such as communication interruptions caused by natural disasters). Mobile crowd sensing technology, as an efficient data acquisition mode, has been widely used in smart cities, disaster relief and other fields, in which human participants in data collection have played a leading role.
[0003] Unlike traditional mobile swarm sensing centered on human participants, swarm sensing centered on mobile unmanned terminals, especially drones, can provide high-quality, wide-area, and low-latency data acquisition services in dangerous or inaccessible areas (such as disaster ruins and uninhabited areas). In a smart city, various types of urban monitoring points are distributed, such as surveillance cameras, Wi-Fi routers, and sensors in factories. By deploying swarms of drones equipped with Wi-Fi / 5G receivers or high-precision sensors, monitoring data can be collected efficiently and flexibly from these monitoring points in complex environments. It is worth noting that many swarm sensing tasks in real-world scenarios are highly time-sensitive, and there exists a latency threshold representing the maximum data acquisition delay. Once this threshold is exceeded, the value of the data rapidly decreases (e.g., during the critical 72 hours of disaster relief).
[0004] However, in real-world scenarios, drone swarms, acting as mission executors, have very limited energy carrying capacity. Therefore, drone swarms need to find a behavior pattern that maximizes data value while minimizing energy consumption. Considering mobile swarm sensing scenarios aimed at urban data monitoring, the latency assurance technology centered on drone swarms faces the following challenges:
[0005] Technical Challenge 1: Difficulty in evaluating task value. In existing mobile group perception tasks, most tasks are independent, and the value of completing a task can often be directly given. In urban data monitoring tasks, task value is given by data latency. However, the relationship between data, as the main carrier for evaluating task value, and the main objective is unclear. Specifically, due to the large trajectory space, it is difficult to find the trajectory corresponding to the minimum latency. Therefore, it is necessary to build a trajectory evaluation model to achieve the goal of minimizing latency.
[0006] Technical Challenge 2: Limited Perception Range. In the field of mobile swarm perception, the perception capabilities of drone swarms are often limited. In cities, the presence of numerous buildings further reduces the perception capabilities of drone swarms, and perception capability directly determines the accuracy of decision-making. By modeling historical sequences and remembering past mobile experiences, drone swarms can better understand the structure of the urban environment and the relative positions of drones. With historical sequence modeling, each drone can avoid buildings and other drones, which can both prevent accidents and ensure the accuracy of decision-making.
[0007] Technical Challenge 3: UAV Swarm Cooperation Mode. Data in urban environments is often highly time-sensitive, with varying amounts of data in each piece of data, and the distribution of data in the city can be extremely uneven. Therefore, compared to other mobile swarm sensing tasks, the task of ensuring latency for urban monitoring data requires not only that the UAV swarm explores the distribution of data, but also that, after exploration, a clear cooperation mode is found to ensure that all monitored data is within the latency threshold as much as possible; otherwise, the value of acquiring the data will be greatly reduced.
[0008] In view of the technical problems mentioned above in the existing technology, the present invention provides a drone swarm perception method for ensuring latency of urban monitoring data. Summary of the Invention
[0009] This invention proposes a drone swarm perception method for ensuring latency in urban surveillance data.
[0010] The present invention adopts the following technical solution:
[0011] A drone swarm perception method for ensuring latency in urban surveillance data includes:
[0012] Step 1: The main process of the scheduling center creates a data latency and trajectory pool, and initializes the perception strategy;
[0013] Step 2: The dispatch center starts several sub-processes to simulate the generation and latency update of urban monitoring data;
[0014] Step 3: The drone swarm collects simulated trajectories based on the current perception strategy and uploads them to the data latency and trajectory pool;
[0015] Step 4: The dispatch center collects information on the drone swarm and urban monitoring points, evaluates the simulated trajectory, and saves the evaluation results to the data latency and trajectory pool.
[0016] Step 5: The scheduling center optimizes the perception strategy using an asynchronous backtracking algorithm based on the data latency and the data in the trajectory pool, and then synchronizes the optimized perception strategy to each subprocess.
[0017] Step 6: Repeat steps 2-5 until the drone swarm perception strategy no longer changes, thus obtaining the optimal perception strategy;
[0018] Step 7: Based on the optimal perception strategy, the dispatch center sends the optimal trajectory instruction to the drone swarm, dynamically scheduling the drone swarm to ensure the latency of urban monitoring data.
[0019] Further, step 1 includes:
[0020] An empty data latency and trajectory pool is created on the main process of the scheduling center.
[0021] Further, step 1 includes:
[0022] Initialize a baseline drone swarm perception strategy.
[0023] Further, step 1 includes:
[0024] Several subprocesses are established, the drone swarm perception strategy in the subprocesses is synchronized, and the simulation environment parameters in each subprocess are initialized. The simulation environment parameters include: number of drones, amount of data generated by urban monitoring points, data transmission signal-to-noise ratio threshold, and data latency threshold.
[0025] Further, step 2 includes:
[0026] Step 21: The scheduling center starts several sub-processes, and the sub-processes begin simulating a new round of data latency guarantee tasks;
[0027] Step 22: Initialize the data volume of any city monitoring point. Data latency κ = 0. Each city monitoring point maintains a data queue to store data. At a certain time step t in a round, city monitoring point p will dynamically generate data of amount Δ according to the environment and store it in the data queue. The data latency κ is updated to tm(t), where m(t) is the waiting time of the data at the front of the data queue. When the data latency exceeds a specified threshold, the data latency will increase and be updated to tm(t) + ∈; where ∈ represents the additional penalty for data latency after exceeding the threshold.
[0028] Step 23: Each subprocess simulates the changes in data latency between the drone swarm trajectory and the city monitoring points in its respective simulation environment in an asynchronous manner. When a certain round of data latency assurance task detects that the drone swarm has collided with an obstacle or run out of energy, the subprocess immediately terminates this round of data latency assurance task and reinitializes its own simulation environment parameters.
[0029] Furthermore, step 3 includes:
[0030] Step 31: Each drone observes the surrounding conditions. After transformation by the mapping operator MLP(·), it becomes And sequentially process the historical sequence according to the first calculation model and the second calculation model. The modeling is performed, and the first and second computational models are as follows:
[0031]
[0032]
[0033] In equations (1) and (2) above, u represents the UAV number, L represents the length of the historical sequence considered, t represents the current time, Gate(··) is the control module, and MHA(·) is the multi-head attention module. The UAV generates a perception strategy based on the historical sequence modeling results. Used for simulating trajectory collection;
[0034] Step 32: Each drone perceives the strategy from the current subprocess. Mid-sampled action Perform movement and data acquisition actions Move to the location of the urban monitoring point in the current environment where sensing data needs to be collected, and collect the remaining data in the data queue of the urban monitoring point; the child process will then collect the data. Data generation time This movement and data acquisition action and current child process awareness strategy The data is sent to the latency and trajectory pool, where i represents the i-th data at the front of the data queue, and c and g are used as subscripts to distinguish the data generation and collection time. The time spent by the drone to collect data is represented by c, and the time spent generating data is represented by g.
[0035] Furthermore, in step 32, the movement and data acquisition actions are performed. The time consumed is The remaining data in the data queue from the city monitoring points will consume in a short time. After the data collection is complete, the remaining data volume at city monitoring point p becomes in It is the set of indexes of data collected by drone u from urban monitoring point p within time step t, satisfying the time constraint.
[0036] Furthermore, step 4 includes:
[0037] Step 41: The scheduling center uses a third calculation model to evaluate latency assurance based on data latency and trajectory pool data. The third calculation model is as follows:
[0038]
[0039] Here, ENV serves as a comment marker, indicating the reward set for the environment section. It is the set of indexes of data collected by drone u from urban monitoring point p within time step t. Penalties for monitoring data latency exceeding a latency threshold;
[0040] Step 42: The drone swarm sends the current drone location to the dispatch center. The dispatch center evaluates regional coverage based on data latency and trajectory pool using the fourth calculation model, which is as follows:
[0041]
[0042] UAV is used as a annotation marker to indicate the observation and reward of the unmanned aerial vehicle (UAV) portion. Represents the data latency and the relationship between the current location of the drone and the trajectory pool. The set of K locations with the smallest Euclidean distance dist(·,·), where K is 10;
[0043] Step 43: The drone swarm sends the observed urban monitoring point information to the dispatch center. Points of Interest (POIs) are used as annotation markers to indicate observations and rewards at urban monitoring points. The dispatch center evaluates regional division of labor based on the fifth calculation model, which is as follows:
[0044]
[0045] in, It is the predictor f p The probability that a drone is responsible for a given area is determined based on the information output from observed urban surveillance points.
[0046] Step 44: Based on the results of the third, fourth, and fifth calculation models, the dispatch center uses the sixth calculation model to perform a total trajectory evaluation and then submits the total trajectory evaluation. drone proximity status The data latency and trajectory pool are saved, and the sixth calculation model is as follows:
[0047]
[0048] in, and To calculate according to the fourth and fifth calculation models respectively, It is the regional exploration coefficient, calculated according to the seventh calculation model by the discriminator f. d and fixed discriminator model f d The average output difference is calculated; the seventh calculation model is as follows:
[0049]
[0050] Step 45: The dispatch center, based on the observed information from urban monitoring points... Proximity to drones According to the eighth calculation model, the average mean squared error and the maximum information entropy are used as loss functions, and the stochastic gradient descent algorithm is used to update the discriminator f. d and predictor f p The eighth calculation model is as follows:
[0051]
[0052] Where OH and CE are the one-hot encoding and cross-entropy function, respectively.
[0053] Furthermore, step 5 includes:
[0054] Step 51: When the data latency and the simulated trajectory data in the trajectory pool meet the update requirements of the first perception strategy, sample out batch simulated trajectory data from the data latency and trajectory pool.
[0055] Step 52: Based on the batch simulation trajectory data, the scheduling center calculates the value estimation function using the asynchronous backtracking algorithm according to the ninth calculation model. The ninth calculation model is as follows:
[0056]
[0057] in, The original value assessment model for the central main process, where γ is the discount factor for the value estimation function. As an importance sampling weighting factor, The single-step TD error is represented by 'act', where 'act' is a subscript indicating the strategy of the central master process, and 'tr' represents the value estimation function of the central master process.
[0058] Step 53: Based on the calculation results of the ninth calculation model, the scheduling center updates the original value assessment model using the method of minimizing the mean square error, with the objective function... As shown in the tenth computational model:
[0059]
[0060] Step 54: The scheduling center adopts the policy gradient algorithm to optimize the perception policy, where the objective function of the policy gradient is... As shown in the eleventh calculation model:
[0061]
[0062] in, The calculation results are from the ninth calculation model. This represents the probability distribution of the current perception strategy;
[0063] Step 55: The scheduling center synchronizes the optimized perception strategy to each subprocess.
[0064] Furthermore, step 7 includes:
[0065] Step 71: The main program of the dispatch center initializes the environment and generates motion trajectories for the drone swarm based on the optimal perception strategy output after step 6 is completed. The trajectory is the optimal trajectory that minimizes the latency of urban monitoring data.
[0066] Step 72: In the scenario of drone swarm perception for urban monitoring data, the dispatch center continuously sends control commands to the drone swarm based on the calculated optimal trajectory, dispatches the drone swarm to move, dynamically ensures the data acquisition latency within the perception area, and sends the perception results back to the dispatch center.
[0067] Compared with the prior art, the superior effects of the present invention are as follows:
[0068] 1. The UAV swarm perception method for ensuring latency of urban monitoring data described in this invention effectively reduces human involvement by mobilizing UAV swarms. Compared with traditional mobile swarm perception scenarios, UAVs, as the leading swarm perception, require only a small amount of human resources and can acquire a large amount of data. This method is suitable for complex urban environments that require a large amount of data for learning. Furthermore, the fact that UAV swarms can fly in the air and move at a speed much greater than that of humans makes them very suitable for participating in time-sensitive tasks. Therefore, compared with traditional assurance tasks, UAVs can be used to participate in urban monitoring data latency assurance tasks to improve the satisfaction of task completion.
[0069] 2. The UAV swarm perception method for ensuring latency in urban monitoring data described in this invention effectively assists in minimum latency search by constructing a trajectory evaluation model. The trajectory evaluation model includes two parts: regional coverage evaluation and regional division of labor evaluation. The regional coverage evaluation model determines whether the current location has been visited by calculating the distance between the current location and previously visited locations. Since achieving global minimum latency requires the UAV swarm to cover as many areas as possible, the model considering regional coverage can effectively reduce the search space. The regional division of labor evaluation model considers the relationships between UAVs more deeply, enabling each UAV to consider only areas not considered by other UAVs, further narrowing the trajectory range to be searched. Compared with the direct search of traditional models, the trajectory evaluation model significantly reduces the time to find the minimum latency trajectory, overcoming the problems of computational difficulty and high time consumption.
[0070] 3. The UAV swarm perception method for ensuring latency of urban monitoring data described in this invention effectively enables UAVs to learn the urban built environment by modeling time series data, reducing the probability of UAV swarms colliding with buildings. Since the simulation is forced to stop after the UAV swarm collides with an obstacle, effectively reducing collisions can increase search speed and decision accuracy. To this end, the method uses a control module and an attention module to model historical experience, allowing UAVs to refer to past states when making decisions. Compared with traditional models, it can explore better local minimum latency trajectories and has a very good effect on environments with concentrated data distribution and dense local buildings in the city.
[0071] 4. The UAV swarm perception method for ensuring latency of urban monitoring data described in this invention, by adding an exploration mechanism, enables the UAV swarm to learn a reasonable division of labor pattern. Specifically, the division of labor evaluation model attempts to learn a model that maps states to the distribution of UAV serial numbers, and trains the model by minimizing the information entropy of this distribution, so that the model learns that one state corresponds to one UAV. Then, the model uses the obtained information entropy as an exploration reward to encourage UAVs to complete their assigned work areas, thereby achieving the purpose of division of labor and cooperation. Attached Figure Description
[0072] Figure 1 This is a schematic diagram illustrating a specific scenario in which a drone swarm is used to provide latency assurance for urban surveillance data in an embodiment of the present invention.
[0073] Figure 2 This is a schematic diagram of the drone swarm scheduling algorithm based on deep reinforcement learning in an embodiment of the present invention;
[0074] Figure 3 This is a schematic diagram illustrating the impact of the number of drones on the total data latency in an embodiment of the present invention;
[0075] Figure 4 This is a schematic diagram illustrating the impact of the number of drones on the data latency threshold violation rate in an embodiment of the present invention;
[0076] Figure 5 This is a schematic diagram illustrating the impact of the number of drones on the data acquisition rate in an embodiment of the present invention;
[0077] Figure 6 A schematic diagram illustrating the impact of the number of drones on energy consumption rate in an embodiment of the present invention;
[0078] Figure 7 This is a schematic diagram illustrating the impact of the latency threshold on the total data latency in an embodiment of the present invention;
[0079] Figure 8 This is a schematic diagram illustrating the impact of the latency threshold on the data latency threshold violation rate in an embodiment of the present invention;
[0080] Figure 9 A schematic diagram illustrating the impact of the latency threshold on the data acquisition rate in an embodiment of the present invention;
[0081] Figure 10 This is a schematic diagram illustrating the impact of the delay threshold on energy consumption rate in an embodiment of the present invention;
[0082] Figure 11 This is a schematic diagram illustrating the combined impact of the number of drones and the latency threshold on the total data latency in an embodiment of the present invention. Detailed Implementation
[0083] To better understand the above-mentioned objectives, features and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.
[0084] Example
[0085] like Figure 1 As shown, the UAV swarm perception method for ensuring latency in urban surveillance data includes:
[0086] Step 1: The main process of the scheduling center creates the data latency and trajectory pool and initializes the perception strategy;
[0087] Step 2: The dispatch center starts multiple sub-processes to simulate the generation and latency update of urban monitoring data;
[0088] Step 3: The drone swarm collects simulated trajectories based on the current perception strategy and uploads them to the data latency and trajectory pool;
[0089] Step 4: The dispatch center collects information from the drone swarm and urban monitoring points to evaluate the simulated trajectory, and saves the evaluation results to the data latency and trajectory pool;
[0090] Step 5: The scheduling center optimizes the perception strategy based on data latency and trajectory pool using an asynchronous backtracking algorithm, and synchronizes the optimized perception strategy to each subprocess.
[0091] Step 6: Repeat steps 2, 3, 4, and 5 until the drone swarm perception strategy no longer changes;
[0092] Step 7: Based on the optimal perception strategy, the dispatch center sends the optimal trajectory instruction to the drone swarm to dynamically schedule the drone swarm and ensure the latency of urban monitoring data.
[0093] Specifically, step 1 includes:
[0094] An empty data latency and trajectory pool is established on the main process of the scheduling center for a drone swarm perception scenario oriented towards urban monitoring data, and a baseline drone swarm perception strategy is initialized. Then, multiple sub-processes are established, the drone swarm perception strategies in the sub-processes are synchronized, and the simulation environment parameters in each sub-process are initialized. The simulation environment parameters include: number of drones, amount of data generated by urban monitoring points, data transmission signal-to-noise ratio threshold, and data latency threshold. The method establishes a drone swarm perception scenario oriented towards ensuring the latency of urban monitoring data, deploying U drones as executors of data latency assurance. In the scenario, there are P urban monitoring points that continuously generate data and some building obstacles that drones should avoid.
[0095] Specifically, step 2 includes:
[0096] Step 21: The scheduling center starts multiple sub-processes, which begin simulating a new round of data latency assurance tasks. The entire data latency assurance task is divided into T equal discrete time steps, each with the same duration τ. The drone swarm divides each time step into two parts: movement time... and collection time During the movement time, each drone moves at a fixed velocity μ in the direction. distance traveled The duration of the movement is The drone u still has [remaining time step] Data collection takes place over a period of time. Considering practical deployment applications, the communication model between the drone swarm and the urban monitoring point jointly considers the effects of line-of-sight (LoS) and non-line-of-sight (NLoS). The probability of line-of-sight transmission between drone u and urban monitoring point p is:
[0097]
[0098] Where α1 and α2 are constants related to the scene environment, and θ u，p Let represent the elevation angle between the drone u and the urban monitoring point p. Furthermore, considering the large-scale fading of the scene channel, the channel fading between the drone u and the urban monitoring point p can be calculated by the following formula:
[0099]
[0100] In the above formula, PL LoS and PLN LoS Let represent the additional channel attenuation under the two transmission conditions, respectively. Assuming the signal transmission power of the UAV swarm and the ambient noise power are constant, the signal-to-noise ratio of this communication can be expressed as: Considering actual transmission loss, when the communication signal-to-noise ratio is less than a certain threshold... If the communication fails, the drone is deemed unable to decode the data and obtain valid information. The data transmission rate is calculated based on Shannon's theorem. The drone uses a polling method to collect data from the 10 nearest urban monitoring points in sequence.
[0101] Step 22: Initialize the data volume of any city monitoring point. Data latency κ = 0. Each city monitoring point maintains a first-in-first-out (FIFO) data queue to store data. At a certain time step t in a round, city monitoring point p will dynamically generate data of amount Δ according to the environment and store it in the data queue. The data latency κ is updated to tm(t), where m(t) is the waiting time of the data at the front of the data queue. When the data latency exceeds a specified threshold, the data latency will increase rapidly and be updated to tm(t) + ∈, where ∈ represents the additional penalty for data latency after exceeding the threshold.
[0102] Step 23: Each subprocess simulates the changes in data latency between the drone swarm trajectory and the city monitoring points in its respective simulation environment in an asynchronous manner. When a certain round of data latency assurance task detects that the drone swarm has collided with an obstacle or run out of energy, the subprocess immediately terminates this round of data latency assurance task and reinitializes its own simulation environment parameters.
[0103] Specifically, step 3 includes:
[0104] Step 31: Each drone observes the surrounding conditions. After transformation by the mapping operator MLP(·), it becomes And sequentially process the historical sequence according to the first calculation model and the second calculation model. The modeling is performed, and the first and second computational models are as follows:
[0105]
[0106]
[0107] In the above formula, u represents the UAV number, L represents the length of the historical sequence considered, t represents the current time, Gate(··) is the control module, and MHA(·) is the multi-head attention module. The UAV generates a perception strategy based on the historical sequence modeling results. Used for simulating trajectory collection;
[0108] Step 32: Each drone starts from the current perception strategy. Mid-sampled action Consumption Time-based movement and data acquisition actions That is, move to the location of the urban monitoring point in the current environment where sensing data needs to be collected, and within the remaining time... Collect the remaining data from the data queues of these urban monitoring points. After collection, the amount of remaining data in urban monitoring point p becomes... in It is the set of indexes of data collected by drone u from urban monitoring point p within time step t, and it must satisfy time constraints. The child process will collect data time. Data generation time This drone operation and current perception strategy Data transmission latency and trajectory pool.
[0109] Specifically, step 4 includes:
[0110] Step 41: The scheduling center uses a third calculation model to evaluate latency assurance based on data latency and the trajectory pool. The third calculation model is as follows:
[0111]
[0112] Here, ENV serves as a comment marker, indicating the reward set for the environment section. This indicates the degree of latency optimization for urban surveillance data. Penalties for monitoring data latency exceeding a latency threshold;
[0113] Step 42: The drone swarm sends the current drone location to the dispatch center. The dispatch center evaluates regional coverage based on data latency and trajectory pool using the fourth calculation model, which is as follows:
[0114]
[0115] in, Represents the data latency and the relationship between the current location of the drone and the trajectory pool. The set of K locations with the smallest Euclidean distance dist(·,·);
[0116] Step 43: The drone swarm sends the observed urban monitoring point information to the dispatch center. The dispatch center evaluates the regional division of labor based on the fifth calculation model, which is as follows:
[0117]
[0118] in, It is the predictor f p The probability that a drone is responsible for a given area is determined based on the information output from observed urban surveillance points.
[0119] Step 44: Based on the results of the third, fourth, and fifth calculation models, the dispatch center performs a total trajectory evaluation according to the sixth calculation model, and then submits the total trajectory evaluation... drone proximity status The data latency and trajectory pool are saved, and the sixth calculation model is as follows:
[0120]
[0121] in, It is the regional exploration coefficient, calculated according to the seventh calculation model by the discriminator f. d and fixed discriminator model f d The average output difference is calculated; the seventh calculation model is as follows:
[0122]
[0123] Step 45: The dispatch center, based on the observed information from urban monitoring points... Proximity to drones According to the eighth calculation model, the average mean squared error and the maximization of information entropy are used as the loss functions.
[0124] Update the discriminant f using the stochastic gradient descent algorithm. d and predictor f p The eighth calculation model is as follows:
[0125]
[0126] Where OH and CE are the one-hot encoding and cross-entropy function, respectively.
[0127] Specifically, step 5 includes:
[0128] Step 51: When the data latency and the simulated trajectory data in the trajectory pool meet the update requirements of the first perception strategy, sample out batch simulated trajectory data from the data latency and trajectory pool.
[0129] Step 52: Based on the batch simulation trajectory data, the scheduling center calculates the value estimation function using the asynchronous backtracking algorithm according to the ninth calculation model. The ninth calculation model is as follows:
[0130]
[0131] in, This is the original valuation model, where γ is the discount factor for the valuation function. As an importance sampling weighting factor, This refers to the single-step TD error;
[0132] Step 53: Based on the calculation results of the ninth calculation model, the scheduling center updates the original value assessment model using the method of minimizing the mean square error, with the objective function... As shown in the tenth computational model:
[0133]
[0134] Here, tr represents the central master process, and This represents the value estimation function of the central master process;
[0135] Step 54: The scheduling center adopts the policy gradient algorithm to optimize the perception policy, where the objective function of the policy gradient is... As shown in the eleventh calculation model:
[0136]
[0137] in, The calculation results are from the ninth calculation model. This represents the probability distribution of the current perception strategy;
[0138] Step 55: The scheduling center synchronizes the optimized perception strategy to each subprocess.
[0139] Specifically, step 7 includes:
[0140] Step 71: The main program of the dispatch center initializes the environment and generates the action trajectory for the drone swarm based on the optimal perception strategy output after step 6 is completed. This trajectory is the optimal trajectory that can minimize the latency of urban monitoring data.
[0141] Step 72: In the scenario of drone swarm perception for urban monitoring data, the dispatch center continuously sends control commands to the drone swarm based on the calculated optimal trajectory, dispatches the drone swarm to move, dynamically ensures the data acquisition latency within the perception area, and sends the perception results back to the dispatch center.
[0142] In the system test of the method described in this embodiment of the invention, a scenario for ensuring the latency of urban monitoring data was constructed using real taxi trajectories from Beijing. Baidu Maps was used to mark the building obstacles that the drones needed to avoid. Specifically, the 20% of points with the most visits were selected as urban monitoring points, and a day in the dataset was mapped to 240 time steps, each lasting 20 seconds. If a vehicle passed through within each time step, it was considered as generating urban monitoring data. The amount of monitoring data generated, Δ, was 1Mb. Referring to the performance report of industrial drones, the drone's mission altitude was set to 100 meters, its flight speed μ to 20 m / s, its initial energy reserve to 719.2 KJ, and the signal transmission power of the drone swarm to be constant. Scene noise power The data transmission signal-to-noise ratio threshold is
[0143] In the implementation of the algorithm, the number of asynchronous child processes was set to 8, and the historical sequence modeling length was set to 20. The set has a capacity of 10, the discount factor γ of the value estimation function is 0.99, and the specific network structure of the algorithm is as follows: Figure 2 As shown.
[0144] To demonstrate the effectiveness of the method in ensuring latency for urban surveillance data, a comprehensive system test was conducted on the method described in this embodiment. The specific evaluation criteria are the following four system metrics when the entire latency assurance task for urban surveillance data is completed:
[0145] 1. Total data latency (κ): The average latency of all city surveillance data;
[0146] 2. Data latency threshold violation rate (χ): The proportion of time during which data latency exceeds a given threshold out of the total time;
[0147] 3. Data Acquisition Rate (ζ): The proportion of the total data collected by the drone swarm to the total data generated by urban monitoring points;
[0148] 4. Energy consumption rate (ξ): The proportion of energy consumed by the drone swarm in collecting data.
[0149] In the following tests, the number of drones U and the latency threshold Λ in the scene were changed in turn for comparative testing and the results were analyzed.
[0150] The test results of this algorithm are evaluated in detail below, and compared with the following six benchmark algorithms:
[0151] 1. HATRPO: Employs a central multi-agent deep reinforcement learning method, and is currently the best algorithm for central multi-agent learning.
[0152] 2. IMPALA: A deep reinforcement learning method that uses asynchronous computing mechanisms, it is currently the best algorithm for asynchronous deep reinforcement learning.
[0153] 3. DRL-freshMCS: It adopts the deep reinforcement learning algorithm framework of IQN and is currently the best method to solve the problem of minimizing data latency in UAV swarm perception using deep reinforcement learning algorithms.
[0154] 4. DDPG-ESWA: A deep reinforcement learning framework that uses the DDPG algorithm, it is another method for solving minimum data latency.
[0155] 5. Shortest Path: A genetic algorithm is used to find the shortest path to all city monitoring points, and the drone swarm traverses and visits all city monitoring points.
[0156] 6. Random: Each drone adopts a random strategy for movement and data collection.
[0157] Two sets of simulation tests were then conducted, with the number of drones U and the latency threshold Λ in the scenario as independent variables, and the above evaluation indicators as dependent variables, namely total data latency (κ), data latency threshold violation rate (χ), data acquisition rate (ζ), and energy consumption rate (ξ).
[0158] like Figure 3-6 As shown in the figures, this set of figures illustrates the impact of the number of drones on data latency assurance tasks. In this set of experiments, the latency threshold Λ = 100 time steps, and the number of drones U = 1 to U = 10 were changed sequentially. As the number of drones increased, both the total data latency κ and the data latency threshold violation rate χ decreased significantly. This is because multiple drones share tasks and learn a more efficient cooperation mode, reducing duplicate data collection and invalid movement. However, the increase in the number of drones also poses a greater challenge to solution space exploration. The DRL-freshMCS method based on ∈-greedy exploration and the DDPG-EWSA method based on OU noise are far from sufficient in data latency assurance scenarios. When exploring using random strategies, the IMPALA method outperforms the previous two, but it lacks modeling of historical sequences and still suffers from severe training instability. HATRPO outperforms all other comparative methods, but still lags behind the method described in this embodiment. This is because HATRPO adopts a centralized training method, with the drone swarm sharing a global evaluation function. However, this ignores the "harmful" behavior of individual drones, leading to a decline in the overall task completion quality. Thanks to the time modeling of historical sequences by the first and second computational models, and the comprehensive evaluation of latency protection, area exploration, and division of labor by the sixth computational model, the method described in this embodiment ultimately ensures lower total data latency, lower latency threshold violation rate, and higher data acquisition rate.
[0159] like Figure 7-10As shown in the figures, this set of figures illustrates the impact of latency thresholds on data latency assurance tasks. In this set of experiments, the number of drones was fixed at U = 3. As the latency threshold Λ gradually increased from 40 to 120, both the latency threshold violation rate χ and the total data latency κ decreased significantly. This is because as the latency threshold increases, the constraints on the latency assurance task become more relaxed. Therefore, with a fixed number of drones, all algorithms can better complete the latency assurance task. However, the method described in this embodiment outperforms other algorithms under various latency conditions. The core reason is that IMPALA does not model the time series, while HATRPO and DDPG-EWSA do not have control modules for the first and second computational models for time series modeling. This method can better integrate time series with large state spaces, allowing UAV swarms to better understand the distribution of environmental data and discover data-dense areas. It is suitable for complex urban environments. Therefore, the method described in this embodiment effectively improved the χ and κ of the UAV swarm in the experimental environment. In addition, although the optimization target of the method described in this embodiment is data latency, the data acquisition rate ζ is also significantly higher than other algorithms. This is because the fourth calculation model in the trajectory evaluation model further enhances the exploration of the environment by the UAV swarm in the early stage of learning. This early understanding of the environment enables the UAV swarm to have better work division in the early stage of learning, and thus rationally access all data aggregation areas, laying the foundation for later local trajectory optimization, and finally significantly improving the data acquisition rate.
[0160] like Figure 11 As shown in the figure, the comparison of latency threshold Λ and total data latency κ under different numbers of U drones using the method described in this embodiment is illustrated. The experiment shows that under low latency threshold conditions, the more drones there are, the lower the total data latency, and the difference in total data latency caused by the number of drones is greater. This proves that the algorithm of this patent can effectively allocate the work of the drone swarm under low latency threshold conditions, and is therefore suitable for tasks with low latency thresholds, such as disaster relief and discovery.
[0161] This invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the invention as claimed. The scope of protection of this invention is defined by the appended claims.
Claims
1. A drone swarm perception method for ensuring latency in urban surveillance data, comprising: Step 1: The main process of the scheduling center creates a data latency and trajectory pool, and initializes the perception strategy; Step 2: The dispatch center starts several sub-processes to simulate the generation and latency update of urban monitoring data; Step 3: The drone swarm collects simulated trajectories based on the current perception strategy and uploads them to the data latency and trajectory pool; Step 4: The dispatch center collects information on the drone swarm and urban monitoring points, evaluates the simulated trajectory, and saves the evaluation results to the data latency and trajectory pool. Step 41: The scheduling center uses a third calculation model to evaluate latency assurance based on data latency and trajectory pool data. The third calculation model is as follows: ， in, It is a drone At time step From the city's monitoring points The index set of data collected in the middle, This is a penalty for monitoring data latency exceeding a latency threshold. This represents the actual generation time of the i-th piece of information that needs to be collected at location p, based on the currently observed urban monitoring point information. This indicates the actual generation time of the (i-1)th piece of information that needs to be collected at position p; Step 42, the UAV group sends the current UAV position to the dispatch center The dispatch center evaluates the regional coverage based on the data delay and the trajectory pool according to a fourth calculation model, and the fourth calculation model is as follows: ， wherein, representative data latency and the Euclidean distance of the current UAV position in the trajectory pool smallest a set of positions, is 10; Step 43, the UAV group sends the observed city monitoring point information to the dispatch center , The dispatch center evaluates the regional division according to a fifth calculation model, and the fifth calculation model is as follows: ， wherein, is a predictor The drone responsible for the area probability according to the observed city monitoring point information output; Step 44, according to the results of the third, fourth and fifth calculation models, the scheduling center adopts the sixth calculation model to perform a total trajectory evaluation, and the total trajectory evaluation is sent to the UAV UAV adjacent state Save to data delay and trajectory pool, the sixth calculation model is as follows: ， wherein, is a region exploration coefficient, computed according to a seventh computation model from the discriminator and the average output difference of the fixed discriminator model ; the seventh computation model is as follows: ， Step 45, the dispatching center adjusts the monitoring point according to the observed city monitoring point information and the proximity state of the unmanned aerial vehicle , according to the eighth calculation model, taking the average mean square error and maximizing the information entropy as the loss function, using the stochastic gradient descent algorithm to update the discriminator and the predictor , the eighth calculation model is as follows: ， wherein, and are one-hot encoding and cross-entropy function, respectively; Step 5: Based on data latency and trajectory pool data, the scheduling center optimizes the perception strategy using an asynchronous backtracking algorithm and synchronizes the optimized perception strategy to each subprocess. Step 51: When the data latency and the simulated trajectory data in the trajectory pool meet the update requirements of the first perception strategy, sample out batch simulated trajectory data from the data latency and trajectory pool. Step 52: Based on the batch simulation trajectory data, the scheduling center calculates the value estimation function using the asynchronous backtracking algorithm according to the ninth calculation model. The ninth calculation model is as follows: ， in, The original value assessment model for the central main process, The discount factor for the value estimation function. As an importance sampling weighting factor, single step Error; where act is a subscript. Indicates the strategy of the central master process; This represents the value estimation function of the central master process; Step 53, according to the calculation result of the ninth calculation model, the dispatching center updates the original value evaluation model by using the method of minimizing the mean square error, and the objective function As shown in the tenth calculation model: ， Step 54, the scheduling center takes a policy gradient algorithm to optimize the perception strategy, wherein the objective function of the policy gradient is As shown in the eleventh calculation model: ， wherein, is the result of a ninth calculation model, is the probability distribution of the current perception strategy; Step 55: The scheduling center synchronizes the optimized perception strategy to each subprocess. Step 6: Repeat steps 2-5 until the drone swarm perception strategy no longer changes, thus obtaining the optimal perception strategy; Step 7: Based on the optimal perception strategy, the dispatch center sends the optimal trajectory instruction to the drone swarm, dynamically scheduling the drone swarm to ensure the latency of urban monitoring data. 2.The method of claim 1, wherein, Step 1 includes: An empty data latency and trajectory pool is created on the main process of the scheduling center. 3.The method of claim 2, wherein, Step 1 includes: And initialize a baseline drone swarm perception strategy.
4. The method of claim 3, wherein, Step 1 includes: Several subprocesses are established, the drone swarm perception strategy in the subprocesses is synchronized, and the simulation environment parameters in each subprocess are initialized. The simulation environment parameters include: number of drones, amount of data generated by urban monitoring points, data transmission signal-to-noise ratio threshold, and data latency threshold.
5. The method of claim 1, wherein, Step 2 includes: Step 21: The scheduling center starts several sub-processes, and the sub-processes begin simulating a new round of data latency guarantee tasks; Step 22: Initialize the data volume of any city monitoring point. Data latency Each city monitoring point maintains a data queue to store data, at a certain time step in the round. City monitoring points The amount of data generated dynamically based on the environment is The data is then stored in a data queue, and the data latency is... Updated to ,in This is the waiting time for the data at the very front of the data queue. When the data latency exceeds a specified threshold, the data latency will increase and be updated. ;in This indicates an additional penalty for data latency when the threshold is exceeded; Step 23: Each subprocess simulates the changes in data latency between the drone swarm trajectory and the city monitoring points in its respective simulation environment in an asynchronous manner. When a certain round of data latency assurance task detects that the drone swarm has collided with an obstacle or run out of energy, the subprocess immediately terminates this round of data latency assurance task and reinitializes its own simulation environment parameters.
6. The method of claim 1, wherein, Step 3 includes: Step 31, each drone observes a neighborhood state , through a mapping operator is transformed into , and sequentially models the history sequence according to a first and a second computational model, the first and the second computational model being as follows: ， ， In the above formula (1), (2), u represents the number of the unmanned aerial vehicle, L represents the length of the historical sequence considered, and t represents the current time, is a control module, is a multi-head attention module, and the unmanned aerial vehicle generates a perception strategy according to a modeling result of the historical sequence for simulating trajectory collection; Step 32: Each drone perceives the strategy from the current subprocess. Mid-sampled action Perform movement and data acquisition actions The process moves to the location of the urban monitoring point in the current environment where sensing data needs to be collected, and collects the remaining data in the data queue of the urban monitoring point; the child process then collects the data. Data generation time This movement and data acquisition action and current child process awareness strategy Send data .
7. The method of claim 1, wherein, Step 7 includes: Step 71: The main program of the dispatch center initializes the environment and generates motion trajectories for the drone swarm based on the optimal perception strategy output after step 6 is completed. The trajectory is the optimal trajectory that minimizes the latency of urban monitoring data. Step 72, in the UAV swarm perception scene facing urban monitoring data, the scheduling center continuously sends control instructions to the UAV swarm according to the calculated optimal trajectory, schedules the UAV swarm to move, dynamically guarantees the data acquisition time delay in the perception area, and sends the perception result back to the scheduling center.