A resource allocation method and device for unmanned aerial vehicle spatiotemporal crowdsourcing

Abstract

The present invention relates to the field of communication technology, and in particular to a resource allocation method and device for unmanned aerial vehicle spatiotemporal crowdsourcing. The method comprises: mapping the three-dimensional service area of ​​unmanned aerial vehicles and IoT devices to two dimensions, obtaining the coordinate range and time data set of the two-dimensional service area; collecting IoT data in each time period based on the time data set of the unmanned aerial vehicle; presetting a no-fly zone for unmanned aerial vehicles, calculating the boundary constraints and no-fly zone constraints of the unmanned aerial vehicle; presetting multiple optimization objectives for weighted balance; expressing the optimization problem as a Markov decision process; and using a double-delayed deep deterministic policy gradient method to allocate the flight action of the unmanned aerial vehicle and the power of the IoT. The present invention deploys an advanced reinforcement learning algorithm, namely TD3. It uses a deep neural network to approximate the policy and value functions based on the actor-critic architecture, so it performs well in high performance and low time complexity, and verifies the effectiveness of the proposed algorithm.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a resource allocation method and device for unmanned aerial vehicle spatiotemporal crowdsourcing. Background Art

[0002] Spatiotemporal crowdsourcing uses the potential of the crowd to complete tasks that must be completed at a specific place and time. Today, drones (Unmanned Aerial Vehicles, UAVs) are equipped with various sensors such as cameras, radars, and lidars. This invention witnesses the great potential of building smart cities by analyzing sensor data collected by drones, including time series, videos, and point clouds. At the same time, advanced vehicle communication technologies, such as 5G New Radio (NR) Vehicle-to-Everything (V2X), enable these data to be shared between drones and various Internet of Things (IoT) devices with low latency and high reliability. In addition, the ubiquity and mobility of drones provide conditions for their wide and flexible distribution at different places and times. The above aspects inspire drone-supported spatiotemporal crowdsourcing, in which drones collaborate to complete various tasks in urban environments, such as air quality monitoring, illegal parking detection, and emergency awareness.

[0003] Existing UAV spatiotemporal crowdsourcing works mainly focus on three indicators: energy consumption, time sensitivity, and fairness.

[0004] To reduce the total energy consumption of the system, including the energy consumption of the drone and IoT devices, some researchers take it as an optimization goal, while others limit it by setting energy constraints for the drone. In addition, some researchers indirectly reduce energy consumption by minimizing the number of drones. To ensure time sensitivity, researchers usually aim to minimize the total mission completion time as an optimization goal or require the drone to complete data collection within a specific time requirement. Fairness refers to the equal opportunity for each IoT device's data to be collected. To ensure fairness, some researchers maximize the total amount of collected data or the total number of IoT devices collected, while others impose constraints to ensure that the data of each IoT device is collected at least once during the entire mission or that the data of all IoT devices is collected within a specific time requirement. In addition, some authors directly consider equalizing the opportunity for each IoT device to be collected during the drone flight.

[0005] In addition, in cities, there are many areas where flying drones is prohibited to protect air traffic (e.g., airports, nuclear power plants, refineries, and national administrative areas), which are called no-fly zones or restricted areas. Some researchers have considered constraints on the location of drones to ensure that drones do not enter no-fly zones throughout their flight routes to avoid these restricted areas.

[0006] Researchers may consider the above metrics separately or simultaneously. While most existing works consider single objective optimization, a few works consider both time sensitivity and fairness for resource allocation. More specifically, some authors combine the above objectives or constraints for joint optimization, while other authors introduce a new metric to measure the average freshness of IoT device data, called Age of Information (AoI). Minimizing the average AoI means ensuring that the data of each IoT device will be collected in a timely manner, and is therefore well suited to ensure fairness.

[0007] However, there are still some limitations in existing works as follows: First, those works that consider both time sensitivity and fairness mainly ignore the no-fly zones in smart cities, which are an indispensable condition for influencing variable decisions. Although no-fly zones are considered in some existing works, they are hardly adapted to drone-enabled spatiotemporal crowdsourcing considering the above three key characteristics. In addition, most simulation experiments are only conducted using synthetic data, and few are evaluated in real-world scenarios with actual urban buildings and environmental data. Summary of the invention

[0008] In order to solve the technical problem that most simulation experiments in the prior art are conducted only using synthetic data, rarely evaluated in real-world scenarios of actual urban buildings and environmental data, and ignoring the no-fly zones in smart cities, the embodiment of the present invention provides a resource allocation method and device for drone spatiotemporal crowdsourcing. The technical solution is as follows:

[0009] On the one hand, a resource allocation method for drone spatiotemporal crowdsourcing is provided, the method comprising:

[0010] S1. Obtain a three-dimensional service area including drones and IoT devices, map the three-dimensional service area to two dimensions, and obtain the coordinate range of the two-dimensional service area;

[0011] S2. Obtain the time data set of the drone in the two-dimensional service area, and collect IoT data in each time period based on the time data set of the drone; preset the drone no-fly zone, and calculate the boundary constraints and no-fly zone constraints of the drone based on the time data set and IoT data of the drone;

[0012] S3. Preset multiple optimization objectives and perform weighted balance on the multiple optimization objectives; obtain the optimization problem based on the boundary constraints and no-fly zone constraints of the UAV;

[0013] S4. Formulate the optimization problem as a Markov decision process;

[0014] S5. Based on the Markov decision process, a double-delayed deep deterministic policy gradient method is used to allocate the flight actions of the drone and the power of the IoT.

[0015] Optionally, in S2, a time data set of the drone in the two-dimensional service area is obtained, and IoT data in each time period is collected based on the time data set of the drone, including:

[0016] The entire drone service time is divided into N time periods to obtain the time data set of the drones in the two-dimensional service area;

[0017] In each time period, the drone will fly to the corresponding location planned in a certain time period; obtain the location set and initial location of IoT and drone in a certain time period;

[0018] At the beginning of a certain time period, IoT devices generate a corresponding amount of data waiting to be collected by drones.

[0019] Optionally, in S2, a no-fly zone for drones is preset, and boundary constraints and no-fly zone constraints of the drones are calculated based on the time data set and IoT data of the drones, including:

[0020] Preset drone no-fly zones B J , the no-fly zone can be of any shape;

[0021] Based on the drone's time data set and IoT data, the drone's boundary constraints are shown in the following formula (1):

[0022]

[0023] Among them, x represents the x-axis coordinate; y represents the y-axis coordinate, U represents the drone position, k represents the drone number, and n represents the time point number. represents the x-axis coordinate of the k-th UAV in the n-th time period, represents the y-axis coordinate of the kth UAV in the nth time period, R represents the range of the two-dimensional service area; z represents the z-axis coordinate, z min Indicates the lowest position of the drone, represents the z-axis coordinate of the k-th drone in the n-th time period, z max Indicates the highest position of the drone;

[0024] The no-fly zone constraints for drones are shown in the following formula (2):

[0025]

[0026] in, express.

[0027] Optionally, in S3, multiple optimization objectives are preset and weighted balance is performed on the multiple optimization objectives; based on the boundary constraints and no-fly zone constraints of the UAV, an optimization problem is obtained, including:

[0028] Preset multiple optimization targets, including: average AoI of all drones during service time, energy loss of IoT during service time, and energy loss of drones during service time;

[0029] Perform weighted balance on multiple optimization objectives, and the optimization variables are the drone position U and IoT transmission power P;

[0030] According to the following formula (3), the optimization objectives are weighted and balanced to obtain the optimization problem:

[0031]

[0032] Among them, P1 represents the optimization problem, represents the average AoI of all drones during the service time; I represents IoT data; It represents the energy consumption of IoT during service time; The energy loss of the drone during the service time; κ1 represents the weight of IoT energy; κ2 represents the weight of drone energy; C1 and C2 are the boundary constraints of the drone; C3 is the no-fly zone constraint of the drone; C4 represents the limitation of the transmission power of the IoT device; C5 represents the limitation of the maximum flight distance of the drone in the x-axis direction within a time period; C6 represents the limitation of the maximum flight distance of the drone in the y-axis direction within a time period; C7 represents the limitation of the maximum flight distance of the drone in the z-axis direction within a time period.

[0033] Optionally, in S4, the optimization problem is expressed as a Markov decision process, including:

[0034] Defining Markov Decision Process in, represents the state space, represents the action space, represents the state transition probability, Indicates reward;

[0035] Based on the optimization problem, calculate t n The state space function at time Action Space Function State transfer function And the reward function

[0036] Minimizing the optimization problem, the reward usually takes into account the average AoI, the energy loss of the drone and IoT, and the objective function is shown in the following formula (4):

[0037]

[0038] Among them, O n represents the objective function of the optimization problem; M represents the number of IoT devices; m represents the serial number of IoT devices; represents the energy consumption of the mth IoT device in the nth time period; It represents the energy consumption of the kth UAV in the nth time period.

[0039] Optionally, in S5, based on the Markov decision process, a double-delayed deep deterministic policy gradient method is used to allocate the flight action of the drone and the power of the IoT, including:

[0040] Obtain the Markov decision process; construct the TD3 algorithm, which includes: a policy network with parameter θ The two parameters are and Value Network and A target policy network and two target q-value networks and

[0041] Using the TD3 algorithm, the expected return of performing a specific action in a given state is constructed to construct an action-value function, a target policy network, and two target value networks as slowly updated versions of the policy network and the evaluation network;

[0042] The flight actions of the drone and the power of IoT are allocated based on the TD3 algorithm.

[0043] Optionally, use the TD3 algorithm to construct the action-value function, the target policy network, and two target value networks as slowly updated versions of the policy network and the evaluation network, with the expected reward of performing a specific action in a given state, including:

[0044] Use the TD3 algorithm to interact with the environment and obtain past state transition information Among them, the TD3 algorithm includes multiple neural networks;

[0045] Transfer the past state information Store into the preset replay buffer;

[0046] A batch of state transition information is randomly selected from the replay buffer for experience replay, and the network parameters of multiple neural networks in the TD3 algorithm are updated.

[0047] Optionally, updating network parameters of multiple neural networks in the TD3 algorithm includes:

[0048] The neural network in the TD3 algorithm includes: a policy network, two q-value networks, a target policy network and two target q-value networks;

[0049] Update the two Q-value networks by constructing a minimization loss function;

[0050] Update the policy network by maximizing the expected value of the reward;

[0051] The target policy network and the two target q-value networks are updated through soft updating.

[0052] On the other hand, a resource allocation device for drone spatiotemporal crowdsourcing is provided, and the device is applied to a resource allocation method for drone spatiotemporal crowdsourcing, and the device includes:

[0053] The data acquisition module is used to obtain the three-dimensional service area including drones and IoT devices, map the three-dimensional service area to two dimensions, and obtain the coordinate range of the two-dimensional service area;

[0054] The constraint module is used to obtain the time data set of the drone in the two-dimensional service area, and collect IoT data in each time period based on the time data set of the drone; preset the drone no-fly zone, and calculate the boundary constraints and no-fly zone constraints of the drone based on the time data set and IoT data of the drone;

[0055] The optimization module is used to preset multiple optimization objectives and perform weighted balance on multiple optimization objectives; based on the boundary constraints and no-fly zone constraints of the UAV, the optimization problem is obtained;

[0056] Markov decision module, used to formulate the optimization problem as a Markov decision process;

[0057] The resource allocation module is used to allocate the flight actions of drones and the power of IoT based on the Markov decision process and the double-delayed deep deterministic policy gradient method.

[0058] On the other hand, a resource allocation device for drone spatiotemporal crowdsourcing is provided, and the resource allocation device for drone spatiotemporal crowdsourcing comprises: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the resource allocation methods for drone spatiotemporal crowdsourcing described above is implemented.

[0059] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned resource allocation methods for drone spatiotemporal crowdsourcing.

[0060] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0061] The method provided by the present invention takes into account the characteristics of the system and the urban environment in the smart city. The present invention formalizes the crowdsourcing resource allocation problem based on UAV as a multi-objective optimization problem, aiming to minimize the average AoI and total energy consumption at the same time, while being constrained by time sensitivity, no-fly zones, regional boundaries, drone dynamics and transmission range. Energy consumption includes the part generated by IoT device data transmission and the part generated during the flight of the drone. Considering the inherent complexity and time complexity of the problem itself, the present invention formalizes it into a Markov Decision Process (MDP) and uses Twin Delayed Deep Deterministic Policy Gradient (TD3) to solve it.

[0062] The deployment of TD3 brings two major benefits when solving problems. First, TD3 focuses on decision making on time series and is able to find the optimal strategy in a multi-step decision process, so it is suitable for long-term optimal solution. In addition, since TD3 constantly interacts with the environment, it is able to adapt to dynamic changes and uncertainties in complex scenes or environments. In order to improve the convergence speed in such a complex environment, compared with other TD3 algorithms, it has high stability, can converge quickly, and is particularly suitable for solving problems in continuous action spaces. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0064] Figure 1 It is a flow chart of a resource allocation method for drone spatiotemporal crowdsourcing provided by an embodiment of the present invention;

[0065] Figure 2 is a network deployment diagram provided by an embodiment of the present invention;

[0066] Figure 3 is a network architecture diagram provided by an embodiment of the present invention;

[0067] Figure 4 is an initial scene graph provided by an embodiment of the present invention;

[0068] Figure 5 is a graph showing the relationship between rewards and algebraic changes provided by an embodiment of the present invention;

[0069] Figure 6 is a graph showing the relationship between average AoI and algebraic changes provided by an embodiment of the present invention;

[0070] Figure 7 It is a block diagram of a resource allocation device for spatiotemporal crowdsourcing of drones provided by an embodiment of the present invention;

[0071] Figure 8 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0072] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0073] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0074] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0075] In the embodiments of the present invention, sometimes a subscript such as W1 may be expressed in a non-subscript form such as W1. When the difference is not emphasized, the meanings to be expressed are consistent.

[0076] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0077] The embodiment of the present invention provides a method for allocating resources for drone spatiotemporal crowdsourcing, which can be implemented by a drone spatiotemporal crowdsourcing resource allocation device, which can be a terminal or a server. Figure 1The flowchart of the resource allocation method of drone spatiotemporal crowdsourcing is shown in the figure. The processing flow of the method may include the following steps:

[0078] S1. Obtain a three-dimensional service area including drones and IoT devices, map the three-dimensional service area to two dimensions, and obtain a coordinate range of the two-dimensional service area.

[0079] S2. Obtain the time data set of the drones in the two-dimensional service area, and collect IoT data in each time period based on the time data set of the drones; preset a no-fly zone for the drones, and calculate the boundary constraints and no-fly zone constraints of the drones based on the time data set and IoT data of the drones.

[0080] In a feasible implementation, in S2, a time data set of the drone in the two-dimensional service area is obtained, and IoT data in each time period is collected based on the time data set of the drone, including:

[0081] The entire drone service time is divided into N time periods to obtain the time data set of the drones in the two-dimensional service area;

[0082] In each time period, the drone will fly to the corresponding location planned in a certain time period; obtain the location set and initial location of IoT and drone in a certain time period;

[0083] At the beginning of a certain time period, IoT devices generate a corresponding amount of data waiting to be collected by drones.

[0084] In a feasible implementation, in S2, a drone no-fly zone is preset, and the boundary constraints and no-fly zone constraints of the drone are calculated based on the drone's time data set and IoT data, including:

[0085] Preset drone no-fly zones B J , the no-fly zone can be of any shape;

[0086] Based on the drone's time data set and IoT data, the drone's boundary constraints are shown in the following formula (1):

[0087]

[0088] Among them, x represents the x-axis coordinate; y represents the y-axis coordinate, U represents the drone position, k represents the drone number, and n represents the time point number. represents the x-axis coordinate of the k-th UAV in the n-th time period, represents the y-axis coordinate of the kth UAV in the nth time period, R represents the range of the two-dimensional service area; z represents the z-axis coordinate, z min Indicates the lowest position of the drone, represents the z-axis coordinate of the k-th drone in the n-th time period, z max Indicates the highest position of the drone;

[0089] The no-fly zone constraints for drones are shown in the following formula (2):

[0090]

[0091] in, express.

[0092] In a feasible implementation, consider a three-dimensional service area, where both the drone and the IoT device are located. Map the service area to two dimensions, and then the two-dimensional service area is called R, and its range is R = {(x, y) | x min ≤x≤x max ,y min ≤y≤y max}. Let the drone and IoT sets be U = {U1, U2, ..., U K} and I={I1,I2,…,I M The entire drone service time is divided into N time periods, and the time set is T = {t1, t2, ..., t N}. In each time period, the drone will fly to the location planned for that time period, and then hover over the location to collect IoT data. The network deployment diagram is as follows Figure 2 shown.

[0093] In a feasible implementation, let and Represents IoT and drone in time period t n The set of positions where A T represents the conjugate transpose of the matrix. The initial position of the drone at the service time is In time period t n At the beginning, the amount of data generated by IoT devices is S m，n Waiting for the drone to collect. Regardless of whether the data was collected in the previous time period, at t n There are J no-fly zones in the area, denoted as B1, B2, ..., B J The no-fly zone can be of any shape, and the no-fly zone is within the three-dimensional service area. The drone must be within the service area R, and the height of the drone must meet the requirements, and the drone cannot fly into the no-fly zone. Therefore, the present invention can obtain the boundary constraints and no-fly zone constraints as follows:

[0094]

[0095]

[0096] The goal of this invention is to ensure the timeliness of the collected data and achieve satisfactory energy consumption of drones and IoT devices. First, this invention introduces the concept of measuring the freshness of the data collected by drones from IoT devices, called AoI. Let α m，k，n Represents an associated variable, indicating that at t n UAV k Whether to collect IoTI m If data is collected, then α m，k，n =1; otherwise, α m，k，n = 0. Assume that an IoT device can only be collected by one drone, while a drone can collect information from multiple IoT devices, and the drone device can only collect information within the maximum communication range allowed. At this time, in the time period t n I m The AoI can be defined as

[0097]

[0098] Where min(a, b) represents the minimum value of a and b. m，n Indicates that at t n inner I m The upload delay is τ, which is the maximum tolerable upload time. n inner I m If not uploaded, then T m，n Not considered. Note that the lower the AoI, the higher the m The fresher the data is. n inner I m The upload delay is I m The time to transmit data to the drone is calculated as

[0099]

[0100] Among them, R m，n Indicates I m and The data rate between

[0101]

[0102]

[0103] Among them, max represents the maximum value function, B s Indicates the bandwidth corresponding to the transmission, p m，n Indicates I m In t n The transmission power within Indicates that at t n Mainland air channel, that is, Im Towards The channel for transmitting data.

[0104] The channel variable can be calculated as

[0105]

[0106] Where g0 represents the channel power gain at a correlation distance of 1 m, and G0 represents the antenna gain. m arrive The three-dimensional Euclidean distance can be expressed as

[0107]

[0108] Where ||a||2 represents the L2 normal form of vector a. Since the present invention considers implementing orthogonal frequency division multiple access for IoT devices that transmit information to the same drone, the interference between devices that send signals to the same drone can be ignored. Interference term I m，n Including only IoT devices that send data to other drones, it can be calculated as

[0109]

[0110] Next, the present invention introduces the energy consumption of IoT and drones. n inner I m The energy consumption of transmitting data is

[0111]

[0112] When the drone moves from position Fly to location When the flight energy loss is

[0113]

[0114] Where E0 represents the energy loss per unit distance.

[0115] S3. Preset multiple optimization objectives and perform weighted balance on the multiple optimization objectives; obtain the optimization problem based on the boundary constraints and no-fly zone constraints of the UAV;

[0116] In a feasible implementation, in S3, multiple optimization objectives are preset and weighted balance is performed on the multiple optimization objectives; based on the boundary constraints and no-fly zone constraints of the UAV, an optimization problem is obtained, including:

[0117] Preset multiple optimization targets, including: average AoI of all drones during service time, energy loss of IoT during service time, and energy loss of drones during service time;

[0118] The first optimization goal is the average AoI of all drones during the service time, which is calculated as

[0119]

[0120] The second and third optimization objectives are the energy loss of IoT and drone during service time, which are calculated as

[0121]

[0122]

[0123] Let U={U1, U2,…,U N} represents the set of drone positions, where Indicates that at t n The corresponding set when . Let P = {P1, P2, ..., P N} represents the transmission power set of IoT devices, where P n ={p 1，n , p 2，n ,…,p M，n} indicates that it is in t n The corresponding subset of time.

[0124] In a feasible implementation, multiple optimization objectives are weighted and balanced, and the optimization variables are the drone position U and the IoT transmission power P;

[0125] According to the following formula (3), the optimization objectives are weighted and balanced to obtain the optimization problem:

[0126]

[0127] Among them, P1 represents the optimization problem, represents the average AoI of all drones during the service time; I represents IoT data; It represents the energy consumption of IoT during service time; The energy loss of the drone during the service time; κ1 represents the weight of IoT energy; κ2 represents the weight of drone energy; C1 and C2 are the boundary constraints of the drone; C3 is the no-fly zone constraint of the drone; C4 represents the limitation of the transmission power of the IoT device; C5 represents the limitation of the maximum flight distance of the drone in the x-axis direction within a time period; C6 represents the limitation of the maximum flight distance of the drone in the y-axis direction within a time period; C7 represents the limitation of the maximum flight distance of the drone in the z-axis direction within a time period.

[0128] S4. Formulate the optimization problem as a Markov decision process;

[0129] In a feasible implementation, in S4, the optimization problem is expressed as a Markov decision process, including:

[0130] Defining Markov Decision Process in, represents the state space, represents the action space, represents the state transition probability, Indicates reward;

[0131] Based on the optimization problem, calculate t n The state space function at time Action Space Function State transfer function And the reward function

[0132] Minimizing the optimization problem, the reward usually takes into account the average AoI, the energy loss of the drone and IoT, and the objective function is shown in the following formula (4):

[0133]

[0134] Among them, O n represents the objective function of the optimization problem; M represents the number of IoT devices; m represents the serial number of IoT devices; represents the energy consumption of the mth IoT device in the nth time period; It represents the energy consumption of the kth UAV in the nth time period.

[0135] In a feasible implementation, the present invention proposes a method based on double-delayed deep deterministic policy gradient to solve the above complex resource allocation problem of drone spatiotemporal crowdsourcing. In order to use this method, first, the problem is reformulated as a Markov decision process (MDP) to determine the flight action of the drone and the power allocation of the IoT. This problem can be defined as a four-tuple The four elements represent the state space, action space, state transition probability and reward respectively.

[0136] t n The state space at time includes the positions of all drones, the positions of all IoT devices, the transmission power, AoI and the amount of data generated in real time, and the associated variables of IoT and drones, which are expressed as

[0137]

[0138] t n The action space at includes the three-dimensional moving distance of all drones and the transmission power change of IoT devices, which can be expressed as

[0139]

[0140] In order to satisfy constraints C4-C7, the present invention sets the maximum and minimum values ​​of the action space, namely

[0141]

[0142]

[0143] The state transition function is Execute an action Then change to the next state The probability of, which reflects the basic framework of the interaction between the agent and the environment, is defined as Where P(A|B) represents the conditional probability function.

[0144] When the agent is in a given state Take specific actions The reward is an evaluation of the effectiveness of the feedback from the environment. The agent adjusts its strategy by maximizing its reward. n Intrinsic rewards are defined as:

[0145]

[0146] Among them, ω and c are the proportional coefficient and constant coefficient respectively. By adjusting these two coefficients, the convergence of the algorithm can be accelerated. To minimize the optimization problem P1, the reward usually considers t n The average AoI within the target area, the energy consumption of drones and IoT, and the objective function are expressed as:

[0147]

[0148] In addition, in order to satisfy the constraints in problem P1, the present invention adds a penalty function, which is expressed as:

[0149]

[0150] In one possible implementation, the reward In the example, the On in front is a reward item with an indicator, and the O n Yes n The reward is calculated based on the three indicators of average AoI within the vehicle, and energy consumption of drones and loT. Usually, rewards need to take these indicators into consideration, because the goal of reinforcement learning is to maximize the reward function, and the previous optimization problem is to minimize these indicators. Therefore, in reinforcement learning, defining rewards as the weighted sum of these indicators can ultimately minimize the optimization problem. Since the penalty function H is set for the reward n , so the reward is determined based on the reward item with the indicator and the reward penalty item together.

[0151] S5. Based on the Markov decision process, a double-delayed deep deterministic policy gradient method is used to allocate the flight actions of the drone and the power of the IoT.

[0152] In a feasible implementation, in S5, based on the Markov decision process, a double-delayed deep deterministic policy gradient method is used to allocate the flight action of the drone and the power of the IoT, including:

[0153] Obtain the Markov decision process; construct the TD3 algorithm, which includes: a policy network with parameter θ The two parameters are and Value Network and A target policy network and two target q-value networks and

[0154] Using the TD3 algorithm, the expected return of performing a specific action in a given state is constructed to construct an action-value function, a target policy network, and two target value networks as slowly updated versions of the policy network and the evaluation network;

[0155] The flight actions of the drone and the power of IoT are allocated based on the TD3 algorithm.

[0156] In one possible implementation, the TD3 algorithm is used to construct the action-value function, the target policy network and two target value networks as slowly updated versions of the policy network and the evaluation network, with the expected return of performing a specific action in a given state, including:

[0157] Use the TD3 algorithm to interact with the environment and obtain past state transition information Among them, the TD3 algorithm includes multiple neural networks;

[0158] Transfer the past state information Store into the preset replay buffer;

[0159] A batch of state transition information is randomly selected from the replay buffer for experience replay, and the network parameters of multiple neural networks in the TD3 algorithm are updated.

[0160] In a feasible implementation, the present invention proposes a method based on double-delayed deep deterministic policy gradient to solve the above Markov decision process. The TD3 algorithm aims to maximize the expected return and solves the resource allocation problem of spatiotemporal crowdsourcing under drones. In order to achieve the above optimization, 6 neural networks are considered in the TD3 algorithm. In the TD3 algorithm, the deterministic policy network outputs the action that maximizes the expected return. The two value networks estimate the expected return of performing a specific action in a given state, that is, the action-value function. In addition, a target policy network and two target q-value networks serve as slowly updated versions of the policy network and the evaluation network to enhance the stability of the training process.

[0161] In a feasible implementation, updating network parameters of multiple neural networks in the TD3 algorithm includes:

[0162] The neural network in the TD3 algorithm includes: a policy network, two q-value networks, a target policy network and two target q-value networks;

[0163] Update the two Q-value networks by constructing a minimization loss function;

[0164] Update the policy network by maximizing the expected value of the reward;

[0165] Update the target policy network and the target q-value network;

[0166] The update process is as follows:

[0167] (1) Update the q-value network and

[0168] The two networks are updated by minimizing the following loss function:

[0169]

[0170] Where E[·] represents the expected function and y represents the target value, which is calculated as:

[0171]

[0172] Wherein, γ represents the discount factor, and the present invention has

[0173] (2) Update the strategy network

[0174] The policy network can be updated by maximizing the expected value of the reward, defined as:

[0175]

[0176] (3) Update the target strategy network and a target q-value network and

[0177] The network is updated to:

[0178] F4: π θ′ ←ξπ θ +(1-ξ)π θ′ ,

[0179]

[0180] Among them, ξ represents the soft update parameter that controls the rate at which the target network parameters are updated toward the existing network parameters.

[0181] The TD3 algorithm updates the above network parameters alternately to maximize the expected return. The network architecture of TD3 is as follows Figure 3 The pseudo code of TD3 algorithm is shown in Algorithm 1.

[0182] Table 1 Resource allocation method for UAV spatiotemporal crowdsourcing based on double-delayed deep deterministic policy gradient

[0183]

[0184] In a feasible implementation, in order to verify the effectiveness and correctness of the TD3 algorithm proposed in this paper, the present invention considers a practical scenario with a drone and multiple IoT devices. The range of the scenario is [-500 meters, 500 meters] × [-500 meters, 500 meters]. The service time of the drone is divided into N = 100 time periods to support the spatiotemporal crowdsourcing of drones to collect information. In order to avoid collision with ground obstacles, the flight altitude of the drone is set between [100 meters, 200 meters]. There are M = 20 IoT devices in the area. The initial position of the IoT device is randomly generated in the area, such as Figure 4 In addition, the present invention also randomly generates the initial position of the drone that meets the boundary constraints and the no-fly zone constraints. The initial position of the drone is (400 meters, 0 meters, 150 meters), as shown in Figure 4 In this scenario, the present invention considers a rectangular no-fly zone, such as Figure 4 The pink cube in the figure is shown. The center of the cube is located at (0m, 0m) on the XOY plane, and the length, width and height are 200m, 200m and 180m respectively. The other parameters are set to m, κ1=10 3 , c=1.3,B s =1Hz, τ=1sg0=1G0=2.2846E0=0.1 p min =0.01W, p max =0·1W, h=20, ξ=0.01.

[0185] Note that each update generation takes place over the entire UAV service time. Figure 5 The graph shows the relationship between the rewards of the three algorithms and the algebraic changes. In addition to the TD3 algorithm given in this patent, there is another advanced and mature reinforcement learning method called Deep Deterministic Policy Gradient (DDPG). In addition, there is also a random method for randomly searching the action space. Figure 5 It can be seen that with the increase of generations, the reward of TD3 increases significantly, which shows that the proposed TD3 algorithm is accurate and effective, and can solve the resource allocation problem of drone spatiotemporal crowdsourcing in this paper. In addition, with the increase of generations, although the algorithm reward of DDPG increases slightly, the increase is much smaller than that of TD3. This shows that the effect of TD3 is better than that of DDPG. This is because TD3 uses two independent value networks to estimate the value of actions and takes the smaller of the two estimated values ​​when updating the policy. This method can reduce the problem of overestimation of values, thereby improving the stability and performance of learning. In addition, TD3 adopts a conservative strategy in the update frequency of the policy network, that is, the update frequency of the policy network is less than the update frequency of the value network. This delayed update can ensure that the policy network is updated on the basis of a relatively stable value function, thereby reducing the fluctuation of policy performance caused by inaccurate value estimation. Moreover, the effects of TD3 and DDPG algorithms are better than those of random methods.

[0186] Figure 6 The average AoI changes with the number of generations. Since one of the main optimization goals of this paper is the average AoI, it can reflect the freshness of the data, so it is very important to judge the effect of the average AoI after the update. Figure 6 As can be seen from the figure, as the number of generations increases, the average AoI of the TD3 algorithm decreases significantly, which shows its effectiveness and correctness. In addition, the performance of the TD3 algorithm is better than that of DDPG, and they are both better than random methods. This conclusion is consistent with the above conclusion.

[0187] In the embodiment of the present invention, mathematical modeling and MDP are used to model the spatiotemporal crowdsourcing resource allocation problem supported by multiple drones. The present invention jointly minimizes fairness and total energy consumption among IoT devices while considering the constraints of time sensitivity, no-fly zones, drone dynamics, regional boundaries, and transmission range.

[0188] In order to solve the complex multi-step decision-making process with continuous action space, the present invention deploys an advanced reinforcement learning algorithm, namely TD3. It uses deep neural networks to approximate the policy and value functions based on the actor-critic architecture, so it excels in high performance and low time complexity.

[0189] The present invention verifies the effectiveness of the proposed algorithm. The results show that TD3 converges faster and has better performance than another mature reinforcement learning algorithm DDPG.

[0190] Figure 7 The block diagram of a resource allocation device for drone spatiotemporal crowdsourcing according to an exemplary embodiment is shown, and the device is used in a resource allocation method for drone spatiotemporal crowdsourcing. Figure 7 The device includes a data acquisition module 310, an initial constraint module 320, an optimization module 330, a Markov decision module 340, and a resource allocation module 350. Among them:

[0191] The data acquisition module 310 is used to acquire the three-dimensional service area including the drone and the IoT device, map the three-dimensional service area to two dimensions, and obtain the coordinate range of the two-dimensional service area;

[0192] The constraint module 320 is used to obtain a time data set of the drone in the two-dimensional service area, and collect IoT data in each time period based on the time data set of the drone; preset a no-fly zone for the drone, and calculate the boundary constraints and no-fly zone constraints of the drone based on the time data set and IoT data of the drone;

[0193] The optimization module 330 is used to preset multiple optimization objectives and perform weighted balance on the multiple optimization objectives; based on the boundary constraints and no-fly zone constraints of the UAV, an optimization problem is obtained;

[0194] A Markov decision module 340 for formulating the optimization problem as a Markov decision process;

[0195] The resource allocation module 350 is used to allocate the flight actions of the drone and the power of the IoT based on the Markov decision process and using the double-delayed deep deterministic policy gradient method.

[0196] Optionally, the constraint module 320 is used to divide the entire drone service time into N time periods to obtain a time data set of the drones in the two-dimensional service area;

[0197] In each time period, the drone will fly to the corresponding location planned in a certain time period; obtain the location set and initial location of IoT and drone in a certain time period;

[0198] At the beginning of a certain time period, IoT devices generate a corresponding amount of data waiting to be collected by drones.

[0199] Optionally, the constraint module 320 is used to

[0200] Preset drone no-fly zones B J , the no-fly zone can be of any shape;

[0201] Based on the drone's time data set and IoT data, the drone's boundary constraints are shown in the following formula (1):

[0202]

[0203] Among them, x represents the x-axis coordinate; y represents the y-axis coordinate, U represents the drone position, k represents the drone number, and n represents the time point number. represents the x-axis coordinate of the k-th UAV in the n-th time period, represents the y-axis coordinate of the kth UAV in the nth time period, R represents the range of the two-dimensional service area; z represents the z-axis coordinate, z min Indicates the lowest position of the drone, represents the z-axis coordinate of the k-th drone in the n-th time period, z max Indicates the highest position of the drone;

[0204] The no-fly zone constraints for drones are shown in the following formula (2):

[0205]

[0206] in, express.

[0207] Optionally, the optimization module 330 is used to preset multiple optimization targets, the multiple optimization targets including: the average AoI of all drones during the service time, the energy loss of IoT during the service time, and the energy loss of drones during the service time;

[0208] Perform weighted balance on multiple optimization objectives, and the optimization variables are the drone position U and IoT transmission power P;

[0209] According to the following formula (3), the optimization objectives are weighted and balanced to obtain the optimization problem:

[0210]

[0211] Among them, P1 represents the optimization problem, represents the average AoI of all drones during the service time; I represents IoT data; It represents the energy consumption of IoT during service time; The energy loss of the drone during the service time; κ1 represents the weight of IoT energy; κ2 represents the weight of drone energy; C1 and C2 are the boundary constraints of the drone; C3 is the no-fly zone constraint of the drone; C4 represents the limitation of the transmission power of the IoT device; C5 represents the limitation of the maximum flight distance of the drone in the x-axis direction within a time period; C6 represents the limitation of the maximum flight distance of the drone in the sparse axis direction within a time period; C7 represents the limitation of the maximum flight distance of the drone in the z-axis direction within a time period.

[0212] Optionally, a Markov decision module 340 is used to define a Markov decision process in, represents the state space, represents the action space, P represents the state transition probability, Indicates reward;

[0213] Based on the optimization problem, calculate t n The state space function at time Action Space Function State transfer function And the reward function

[0214] Minimizing the optimization problem, the reward usually takes into account the average AoI, the energy loss of the drone and IoT, and the objective function is shown in the following formula (4):

[0215]

[0216] Among them, O n represents the objective function of the optimization problem; M represents the number of IoT devices; m represents the serial number of IoT devices; represents the energy consumption of the mth IoT device in the nth time period; It represents the energy consumption of the kth UAV in the nth time period.

[0217] Optionally, the resource allocation module 350 is used to obtain a Markov decision process; construct a TD3 algorithm, the TD3 algorithm includes: a policy network with a parameter θ The two parameters are and Value Network and A target policy network and two target q-value networks and

[0218] Using the TD3 algorithm, the expected return of performing a specific action in a given state is constructed to construct an action-value function, a target policy network, and two target value networks as slowly updated versions of the policy network and the evaluation network;

[0219] The flight actions of the drone and the power of IoT are allocated based on the TD3 algorithm.

[0220] Optionally, use the TD3 algorithm to construct the action-value function, the target policy network, and two target value networks as slowly updated versions of the policy network and the evaluation network, with the expected reward of performing a specific action in a given state, including:

[0221] Use the TD3 algorithm to interact with the environment and obtain past state transition information Among them, the TD3 algorithm includes multiple neural networks;

[0222] Transfer the past state information Store into the preset replay buffer;

[0223] A batch of state transition information is randomly selected from the replay buffer for experience replay, and the network parameters of multiple neural networks in the TD3 algorithm are updated.

[0224] Optionally, updating network parameters of multiple neural networks in the TD3 algorithm includes:

[0225] The neural network in the TD3 algorithm includes: a policy network, two q-value networks, a target policy network and two target q-value networks;

[0226] Update the two Q-value networks by constructing a minimization loss function;

[0227] Update the policy network by maximizing the expected value of the reward;

[0228] The target policy network and the two target q-value networks are updated through soft updating.

[0229] In the embodiment of the present invention, mathematical modeling and MDP are used to model the spatiotemporal crowdsourcing resource allocation problem supported by multiple drones. The present invention jointly minimizes fairness and total energy consumption among IoT devices while considering the constraints of time sensitivity, no-fly zones, drone dynamics, regional boundaries, and transmission range.

[0230] In order to solve the complex multi-step decision-making process with continuous action space, the present invention deploys an advanced reinforcement learning algorithm, namely TD3. It uses deep neural networks to approximate the policy and value functions based on the actor-critic architecture, so it excels in high performance and low time complexity.

[0231] The present invention verifies the effectiveness of the proposed algorithm. The results show that TD3 converges faster and has better performance than another mature reinforcement learning algorithm DDPG.

[0232] Figure 8 is a schematic diagram of a resource allocation device for drone spatiotemporal crowdsourcing provided by an embodiment of the present invention, such as Figure 8 As shown, the resource allocation device for drone spatiotemporal crowdsourcing may include the above Figure 7 The resource allocation device for drone spatiotemporal crowdsourcing shown. Optionally, the resource allocation device 410 for drone spatiotemporal crowdsourcing may include a first processor 2001.

[0233] Optionally, the resource allocation device 410 for drone spatiotemporal crowdsourcing may also include a memory 2002 and a transceiver 2003 .

[0234] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0235] Combine the following Figure 8 The components of the resource allocation device 410 for drone spatiotemporal crowdsourcing are specifically introduced as follows:

[0236] The first processor 2001 is the control center of the resource allocation device 410 for drone spatiotemporal crowdsourcing, and can be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement an embodiment of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs).

[0237] Optionally, the first processor 2001 can perform various functions of the resource allocation device 410 for drone spatiotemporal crowdsourcing by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002.

[0238] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 8 CPU0 and CPU1 are shown in FIG.

[0239] In a specific implementation, as an embodiment, the resource allocation device 410 for drone spatiotemporal crowdsourcing may also include multiple processors, such as Figure 8 The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0240] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0241] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently, and may be accessed through the interface circuit ( Figure 8 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0242] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0243] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 8 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0244] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A resource allocation method for drone spatiotemporal crowdsourcing, characterized in that: The method comprises: S1. Obtain a three-dimensional service area including drones and IoT devices, map the three-dimensional service area to two dimensions, and obtain a coordinate range of the two-dimensional service area; S2. Obtain a time data set of the drone in the two-dimensional service area, and collect IoT data in each time period based on the time data set of the drone; preset a no-fly zone for the drone, and calculate the boundary constraints and no-fly zone constraints of the drone based on the time data set of the drone and the IoT data; S3, presetting multiple optimization objectives, performing weighted balance on the multiple optimization objectives; obtaining an optimization problem based on the boundary constraints and no-fly zone constraints of the UAV; In S3, multiple optimization objectives are preset and weighted balance is performed on the multiple optimization objectives; based on the boundary constraints and no-fly zone constraints of the UAV, an optimization problem is obtained, including: Preset multiple optimization goals, the multiple optimization goals include: average AoI of all drones during the service time, energy loss of IoT during the service time, and energy loss of drones during the service time; The multiple optimization objectives are weighted and balanced, and the optimization variables are the drone position U and the IoT transmission power P; According to the following formula (3), the optimization objective is weighted and balanced to obtain the optimization problem: Among them, P1 represents the optimization problem, represents the average AoI of all drones during the service time; I represents IoT data; It represents the energy consumption of IoT during service time; The energy loss of the drone during the service time; κ1 represents the weight of IoT energy; κ2 represents the weight of drone energy; C1 and C2 are the boundary constraints of the drone; C3 is the no-fly zone constraint of the drone; C4 represents the limit on the transmission power of the IoT device; C5 represents the limit on the maximum flight distance of the drone in the x-axis direction within a time period; C6 represents the limit on the maximum flight distance of the drone in the y-axis direction within a time period; C7 represents the limit on the maximum flight distance of the drone in the z-axis direction within a time period; S4, expressing the optimization problem as a Markov decision process; S5. Based on the Markov decision process, a double-delayed deep deterministic policy gradient method TD3 is used to allocate the flight actions of the drone and the power of the IoT; In S5, based on the Markov decision process, a double-delayed deep deterministic policy gradient method is used to allocate the flight action of the drone and the power of the IoT, including: Obtain the Markov decision process; construct a TD3 algorithm, the TD3 algorithm includes: a policy network with a parameter θ The two parameters are and Value Network and A target policy network and two target q-value networks and Using the TD3 algorithm, the expected return of performing a specific action in a given state is constructed to construct an action-value function, a target policy network, and two target value networks as slowly updated versions of the policy network and the evaluation network; The flight action of the drone and the power of IoT are allocated based on the TD3 algorithm.

2. The resource allocation method for drone spatiotemporal crowdsourcing according to claim 1 is characterized in that: In S2, a time data set of a drone in a two-dimensional service area is obtained, and IoT data in each time period is collected based on the time data set of the drone, including: The entire drone service time is divided into N time periods to obtain the time data set of the drones in the two-dimensional service area; In each time period, the drone will fly to the corresponding location planned in a certain time period; obtain the location set and initial location of IoT and drone in a certain time period; At the beginning of a certain time period, IoT devices generate a corresponding amount of data waiting to be collected by drones.

3. The resource allocation method for drone spatiotemporal crowdsourcing according to claim 2 is characterized in that: In S2, a drone no-fly zone is preset, and a boundary constraint and a no-fly zone constraint of the drone are calculated based on the time data set of the drone and the IoT data, including: Preset drone no-fly zones B J , the no-fly zone is of any shape; Based on the time data set of the drone and the IoT data, the boundary constraint of the drone is as shown in the following formula (1): Among them, x represents the x-axis coordinate; y represents the y-axis coordinate, U represents the drone position, k represents the drone number, and n represents the time point number. represents the x-axis coordinate of the k-th UAV in the n-th time period, represents the y-axis coordinate of the kth UAV in the nth time period, R represents the range of the two-dimensional service area; z represents the z-axis coordinate, z min Indicates the lowest position of the drone, represents the z-axis coordinate of the k-th drone in the n-th time period, z max Indicates the highest position of the drone; The no-fly zone constraints for drones are shown in the following formula (2): in, express.

4. The resource allocation method for drone spatiotemporal crowdsourcing according to claim 3 is characterized in that: In S4, the optimization problem is expressed as a Markov decision process, including: Defining Markov Decision Process in, represents the state space, represents the action space, represents the state transition probability, Indicates reward; Based on the optimization problem, calculate t n The state space function at time Action Space Function State transfer function And the reward function To minimize the optimization problem, the reward usually takes into account the average AoI, the energy loss of the drone and IoT, and the objective function is shown in the following formula (4): Among them, O n represents the objective function of the optimization problem; M represents the number of IoT devices; m represents the serial number of IoT devices; represents the energy consumption of the mth IoT device in the nth time period; It represents the energy consumption of the kth UAV in the nth time period.

5. The resource allocation method for drone spatiotemporal crowdsourcing according to claim 4 is characterized in that: The TD3 algorithm is used to construct the action-value function, the target policy network and two target value networks as slowly updated versions of the policy network and the evaluation network, based on the expected return of performing a specific action in a given state, including: Adopt TD3 algorithm to interact drones and IoT devices with the environment and obtain past state transition information Wherein, the TD3 algorithm includes multiple neural networks; Repeat the interaction between the drone and the IoT device and the environment to obtain multiple past state transition information, and then Store into the preset replay buffer; A batch of state transition information is randomly extracted from the replay buffer for experience replay, and network parameters of multiple neural networks in the TD3 algorithm are updated.

6. The resource allocation method for drone spatiotemporal crowdsourcing according to claim 5 is characterized in that: The updating of network parameters of multiple neural networks in the TD3 algorithm includes: The neural network in the TD3 algorithm includes: a policy network, two q-value networks, a target policy network and two target q-value networks; Update the two q-value networks by constructing a minimization loss function; updating the policy network by maximizing the expected value of the reward; The target policy network and the two target q-value networks are updated by soft updating.

7. A resource allocation device for drone spatiotemporal crowdsourcing, the resource allocation device for drone spatiotemporal crowdsourcing is used to implement the resource allocation method for drone spatiotemporal crowdsourcing as claimed in any one of claims 1 to 6, characterized in that: The device comprises: A data acquisition module is used to acquire a three-dimensional service area including drones and IoT devices, map the three-dimensional service area to two dimensions, and obtain a coordinate range of the two-dimensional service area; A constraint module is used to obtain a time data set of a drone in a two-dimensional service area, and collect IoT data in each time period based on the time data set of the drone; preset a no-fly zone for the drone, and calculate a boundary constraint and a no-fly zone constraint for the drone based on the time data set of the drone and the IoT data; An optimization module, used to preset multiple optimization objectives and perform weighted balance on the multiple optimization objectives; and obtain an optimization problem based on the boundary constraints and no-fly zone constraints of the UAV; A Markov decision module, used for expressing the optimization problem as a Markov decision process; The resource allocation module is used to allocate the flight actions of the drone and the power of the IoT based on the Markov decision process and using a double-delayed deep deterministic policy gradient method.

8. A resource allocation device for drone spatiotemporal crowdsourcing, characterized in that: The resource allocation device of the drone spatiotemporal crowdsourcing includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 6 is implemented.

Images