Heterogeneous unmanned aerial vehicle network task unloading method based on service cache

Abstract

The invention discloses a heterogeneous unmanned aerial vehicle network task unloading method based on service cache, and belongs to the technical field of communication. The heterogeneous unmanned aerial vehicle network task offloading method for introducing an enhanced unmanned aerial vehicle into a lightweight unmanned aerial vehicle network is provided for solving the problems that the lightweight unmanned aerial vehicle network is difficult to provide sufficient cache resources, service cache configuration is likely to be unbalanced, and user service experience is poor. According to the method, a deep reinforcement learning algorithm is adopted, a cruise trajectory of a heterogeneous unmanned aerial vehicle group is dynamically planned according to position distribution and task unloading requests of ground users, a service caching decision is constructed according to lightweight unmanned aerial vehicle caching capacity constraints and service popularity distribution of associated users, and the service caching decision is determined according to the task unloading requests. And a task unloading and resource allocation decision is constructed according to the airborne computing resources and the service cache state, so that the service cache efficiency of the unmanned aerial vehicle group is improved, the task unloading completion rate is maximized, and the unit task unloading time delay of the system is reduced.

Description

Technical Field

[0001] The present invention belongs to the field of communication technology, and in particular relates to a heterogeneous unmanned aerial vehicle network task offloading method based on service cache. Background Art

[0002] Due to their high maneuverability and flexibility, drones have become an important part of future networks. By deploying low-altitude drone edge networks in areas with weak ground communication facilities, good communication and edge computing services can be provided to ground users.

[0003] Most of the existing research on unmanned aerial vehicle network task offloading focuses only on the scheduling of offloaded tasks and the allocation of computing resources, and rarely considers the serious contradiction between the diversity of service types and the limitations of onboard cache resources. Existing low-altitude edge networks usually use homogeneous drone networks, and lightweight drone networks based on network cost considerations are usually difficult to equip with sufficient cache resources, which may lead to uneven distribution of service caches, thus affecting the user's service experience.

[0004] To solve the above problems, the present invention proposes a heterogeneous UAV network task offloading method by introducing an enhanced UAV into a lightweight UAV network. The method uses a double-delay deep deterministic policy gradient algorithm [1] to establish a deep reinforcement learning model. By adaptively planning the cruise trajectory of the UAV swarm, dynamically planning the onboard service cache and computing resource allocation, the service cache efficiency and computing resource utilization of the UAV swarm are maximized, thereby effectively improving the task completion rate and reducing the system unit task offloading delay.

[0005] [1]F.Scott,VHHerke,M.David.Addressing Function Approximation Errorin Actor-Critic Methods[C].Proceedings of the 35th International Conference on Machine Learning,Stockholm,Sweden,2018:2587-2601. Summary of the invention

[0006] The present invention aims to solve the above problems of the prior art. A heterogeneous UAV network task offloading method based on service cache is proposed. The technical solution of the present invention is as follows:

[0007] A heterogeneous UAV network task offloading method based on service cache comprises the following steps:

[0008] 101. A double-delay deep deterministic policy gradient algorithm is used to construct a deep reinforcement learning model, including a drone swarm action strategy generator, a drone swarm state-action value generator, a service cache update decision generator, a task offloading and resource allocation decision generator, an experience sample storage area, and a random sample set. The cruise working time of the drone swarm is divided into K equal-length time slots. Assuming that the system state remains unchanged in any time slot k, the system state vector [s k ,a k ,r k ,s k+1 ], where s k is the state vector of the k-time slot drone swarm, a k is the k-time slot UAV group action vector, r k is the k-time slot system reward function, s k+1 is the state vector of the drone swarm in time slot k+1, and k is initialized to 0;

[0009] 102. If k<K, let k=k+1, the service cache update decision generator of the enhanced drone is based on the state vector s k and service popularity distribution, updating the service cache configuration decision Λ of the k-slot lightweight drone swarm k , jump to step 103; otherwise, jump to step 106;

[0010] 103. According to the task request of the ground user in time slot k, the state vector s of the drone swarm k and serving cache configuration decisions Λ k , update the task offloading decision Ω of k time slot through the task offloading and resource allocation decision generator k and computing resource allocation decision Φ k ;

[0011] 104. Obtain s through the drone swarm action strategy generator k The action vector of the drone group in state a k , according to {Λ k ,Ω k ,Φ k}Execute the task offloading in k time slots and obtain the drone group action reward r k , according to s k with a k Calculate and obtain the k+1 time slot drone swarm state vector s k+1 , the system state vector [s k ,a k ,r k ,s k+1 ] Added experience sample storage area;

[0012] 105. Randomly sample from the experience sample storage area to obtain a random sample set, and import the random sample set into the drone swarm action strategy generator and the drone swarm state-action value generator for training, and jump to step 102;

[0013] 106. The patrol work is completed.

[0014] Furthermore, in step 101, the system state vector [s k ,a k ,r k ,s k+1 ], where the state vector s of the drone swarm in time slot k is k , UAV swarm motion vector a k 、System reward function r k , drone swarms in s k Execute action a in state k After that, the state vector s of the drone swarm in time slot k+1 is k+1 , as shown in formulas (1), (2), (3), and (4) respectively:

[0015]

[0016]

[0017]

[0018]

[0019] In formula (1), represents the three-dimensional coordinates of UAV n in time slot k, represents the set of all drones, including enhanced drones and all lightweight drones; in formula (2), is the horizontal flight angle of UAV n in time slot k, is the horizontal flight distance of UAV n in time slot k, is the vertical flight distance of drone n in time slot k; in formula (3), ε is the drone action reward weight factor, where 0≤ε≤1, ψ k represents the task completion rate of the k-time slot system, represents the normalized system unit task delay of k time slots, and the calculation method is shown in formulas (5) and (6):

[0020]

[0021]

[0022] In formula (5), if the task of user m is offloaded within the task tolerance delay, then ψ m=1, otherwise, ψ m =0,M k represents the set of users who request task offloading in k time slots; in formula (6), represents the completion delay of unit task offloading of user m in time slot k, and the calculation method is shown in formula (7):

[0023]

[0024] In formula (7), T m,k is the task offloading completion delay of user m in time slot k, and the transmission delay from the task offloading of user m in time slot k to the execution of drone n Computational delay Composition, d m,k Request task data volume for user m in time slot k.

[0025] Further, in step 102, the service cache update decision generator of the enhanced UAV updates the service cache configuration decision Λ of the k-time slot lightweight UAV group. k The method is as follows:

[0026] 1) Let the set of lightweight drones be N, the set of service functions be S, and initialize the temporary set N′=N,S′=S;

[0027] 2) If Take out the first element n in N′ and jump to step 3), otherwise, jump to step 6);

[0028] 3) Based on the local popularity n of each service type s in the coverage area of ​​lightweight drone n s , calculate the cache value w of service type s on lightweight UAV n n,s , according to w n,s The value of arranging the elements in the set S′ in descending order;

[0029] 4) If Take out the first element s in S′ and jump to step 5); otherwise, let S′=S and jump to step 2);

[0030] 5) If UAV n has cached service s, jump to step 4), otherwise, the enhanced UAV caches the service type s according to the cache capacity constraint of the lightweight UAV n and the cache value w n,s Priority, using the “least frequently used LFU” algorithm, update the service cache for lightweight drone n, and jump to step 4);

[0031] 6) Output k-time slot service cache configuration decision Λ k .

[0032] Furthermore, the cache value w of the service type s on the lightweight drone n in step 3) is n,s The calculation method is shown in formula (8):

[0033]

[0034] In formula (8), i s Indicates the amount of cached data of service s, n s represents the local popularity of service type s in the coverage area of ​​lightweight UAV n.

[0035] Furthermore, in step 103, the task offloading decision Ω of the k time slot is updated by the task offloading and resource allocation decision generator. k and computing resource allocation decision Φ k The method is as follows:

[0036] 7) Let the set of users who request task offloading in time slot k be M k , initialize the task offloading decision of k time slots in, The binary decision variable for offloading tasks from user m to drone n in time slot k, and computing the resource allocation decision in, For the computing resources allocated to user m by drone n in time slot k, let the temporary set M = M k ;

[0037] 8) Calculate the task priority p based on the task request of each user m in the set M m,k , and according to p m,k The elements in the value pair set M are arranged in descending order, and the decision Λ is made based on the cache configuration of the drone service k And the task type requested by each user m in the set M, establish a candidate unloading drone set N for each user m m,k ;

[0038] 9) If Take out the first element m and jump to step 10), otherwise, jump to step 12);

[0039] 10) If Compute user m offloads tasks to N m,k The transmission delay of candidate drone n according to The value pair set N m,k Arrange the elements in ascending order, take out the first element n, and calculate the minimum computing resources that drone n needs to allocate for user m Jump to step 11), otherwise, jump to step 9);

[0040] 11) If the computing resources available on drone n Update the computational resource allocation decision Θ k , let user m make task offloading decision Jump to step 9), otherwise, jump to step 10);

[0041] 12) For drone collection For each drone n in To M k Each The user, Update the computing resource allocation decision Φ k , jump to step 13), otherwise, jump to step 13);

[0042] 13) Output k-time slot task offloading decision Ω k and computing resource allocation decision Φ k .

[0043] Furthermore, the task priority p requested by each user m in step 8) m,k The calculation method of is shown in formula (9):

[0044]

[0045] In formula (9), is the maximum tolerable delay of a task requested by user m in time slot k.

[0046] Furthermore, in step 10), user m offloads the task to N m,k The transmission delay of candidate drone n and the minimum computing resources that drone n needs to allocate to user m The calculation method of is shown in formulas (10) and (11):

[0047]

[0048]

[0049] In formula (10), is the association state between user m and drone n in time slot k. If user m is within the coverage of drone n, then otherwise, is the offloading relationship between user m and drone n in time slot k. If the task of user m is offloaded to drone n, then otherwise, That is, the associated drone acts as a relay to forward the task to the unloading drone for processing; represents the transmission rate from user m to associated drone n in time slot k, represents the transmission rate of the k-time slot associated UAV n and the unloaded UAV n′. In formula (11), c m,k represents the task complexity requested by user m in time slot k; and The calculation methods are shown in formulas (12) and (13) respectively:

[0050]

[0051]

[0052] In formula (12), is the channel bandwidth of user m, is the communication channel gain from user m to drone n in time slot k, n0 is the noise power, is the transmission power of user m; in formula (13), represents the channel bandwidth of UAV n, is the communication channel gain from UAV n to UAV n′ in time slot k, is the transmission power of UAV n.

[0053] Further, in step 104, according to s k with a k Calculate to obtain k+1 The methods include:

[0054] Will Import the UAV swarm action strategy generator to obtain the k-time slot UAV swarm action vector Get it again in,

[0055]

[0056] The advantages and beneficial effects of the present invention are as follows:

[0057] The present invention discloses a heterogeneous unmanned aerial vehicle network task offloading method based on service cache. Most of the existing research on unloading of unmanned aerial vehicle network tasks only focuses on the scheduling of unloaded tasks and the allocation of computing resources, and rarely considers the serious contradiction between the diversity of service types and the limitations of onboard cache resources. Existing low-altitude edge networks usually adopt homogeneous unmanned aerial vehicle networking, and lightweight unmanned aerial vehicle networks based on network cost considerations are usually difficult to equip with sufficient cache resources, which may lead to uneven distribution of service caches, thereby affecting the user's service experience. In view of the above problems, the present invention proposes a heterogeneous unmanned aerial vehicle network task offloading method that introduces an enhanced unmanned aerial vehicle into a lightweight unmanned aerial vehicle network. The method uses a double-delay deep deterministic policy gradient algorithm to establish a deep reinforcement learning model, adaptively plans the cruise trajectory of the unmanned aerial vehicle group according to the location distribution and task offloading request of ground users, constructs service cache decisions according to the cache capacity constraints of lightweight unmanned aerial vehicles and the local popularity distribution of various types of services in the coverage area, and constructs task offloading and resource allocation decisions according to onboard computing resources and service cache status, thereby maximizing the service cache efficiency of the unmanned aerial vehicle group, effectively improving the task completion rate and reducing the system unit task offloading delay. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 The present invention provides a preferred embodiment of a flow chart of a heterogeneous UAV network task offloading method based on service cache. DETAILED DESCRIPTION

[0059] The following will describe the technical solutions in the embodiments of the present invention in detail in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention.

[0060] The technical solution of the present invention to solve the above technical problems is:

[0061] The concepts and models involved in the present invention are as follows:

[0062] The present invention assumes that the low-altitude network is composed of multiple drones, each drone carries an edge server, and each server can cache a certain number of different types of services. Assuming that ground users do not participate in task calculations due to limited computing power, if the drone server caches the service type of the task requested by the associated user and has sufficient computing and communication resources, the ground user can offload the task to the associated drone for execution. Otherwise, the associated drone forwards the task to other drones that cache the corresponding service for execution. The cruising working time of the drone group is divided into K equal-length time slots, and the system state remains unchanged in any kth (∈K) time slot. Let M k represents the set of users who request task offloading in k time slots, represents the set of all drones, including enhanced drones and all lightweight drones. The service type set is S = {s}, where enhanced drones can store all services, while each lightweight drone can only store part of the services in the set S due to limited storage space.

[0063] The technical solution of the present invention is described as follows:

[0064] 1. Network state vector, action vector, and reward function for ground-air environment

[0065] The state vector s of the drone swarm in k time slots k , UAV swarm motion vector a k 、System reward function r k , drone swarms in s k Execute action a in state k After that, the state vector s of the drone swarm in time slot k+1 is k+1 , as shown in formulas (1), (2), (3), and (4) respectively:

[0066]

[0067]

[0068]

[0069]

[0070] In formula (1), represents the three-dimensional coordinates of UAV n in time slot k, represents the set of all drones, including enhanced drones and all lightweight drones; in formula (2), is the horizontal flight angle of UAV n in time slot k, is the horizontal flight distance of UAV n in time slot k, is the vertical flight distance of drone n in time slot k; in formula (3), ε is the drone action reward weight factor, where 0≤ε≤1, ψ k represents the task completion rate of the k-time slot system, represents the normalized system unit task delay of k time slots, and the calculation method is shown in formulas (5) and (6):

[0071]

[0072]

[0073] In formula (5), if the task of user m is offloaded within the task tolerance delay, then ψ m =1, otherwise, ψ m=0,M k represents the set of users who request task offloading in k time slots; in formula (6), represents the completion delay of unit task offloading of user m in time slot k, and the calculation method is shown in formula (7):

[0074]

[0075] In formula (7), T m,k is the task offloading completion delay of user m in time slot k, and the transmission delay from the task offloading of user m in time slot k to the execution of drone n Computational delay Composition, d m,k The amount of task data requested by user m in time slot k;

[0076] 2. Calculate the cache value w of service type s on lightweight drone n n,s Methods

[0077] Cache value w of service type s on lightweight UAV n n,s The calculation method is shown in formula (8):

[0078]

[0079] In formula (8), i s Indicates the amount of cached data of service s, n s represents the local popularity of service type s in the coverage area of ​​lightweight UAV n;

[0080] 3. Calculate the task priority p requested by each user m m,k Methods

[0081] The priority p of each task requested by user m m,k The calculation method is shown in formula (9):

[0082]

[0083] In formula (9), is the maximum tolerable delay of a task requested by user m in time slot k;

[0084] 4. Calculate the task offloaded by user m to N m,k The transmission delay of candidate drone n Methods

[0085] User m offloads the task to N m,k The transmission delay of candidate drone n The calculation method is shown in formula (10):

[0086]

[0087] In formula (10), is the association state between user m and drone n in time slot k. If user m is within the coverage of drone n, then otherwise, is the offloading relationship between user m and drone n in time slot k. If the task of user m is offloaded to drone n, then otherwise, That is, the associated drone acts as a relay to forward the task to the unloading drone for processing; represents the transmission rate from user m to associated drone n in time slot k, represents the transmission rate of the k-time slot associated UAV n and the unloaded UAV n′, and The calculation methods are shown in formulas (11) and (12) respectively:

[0088]

[0089]

[0090] In formula (11), is the channel bandwidth of user m, is the communication channel gain from user m to drone n in time slot k, n0 is the noise power, is the transmission power of user m; in formula (12), represents the channel bandwidth of UAV n, is the communication channel gain from UAV n to UAV n′ in time slot k, is the transmission power of UAV n;

[0091] 5. Calculate the minimum computing resources that drone n needs to allocate to user m Methods

[0092] The minimum computing resources that drone n needs to allocate to user m The calculation method is shown in formula (13):

[0093]

[0094] In formula (13), c m,k represents the task complexity requested by user m in time slot k;

[0095] 6. According to s k with a k Calculate to obtain k+1 Methods

[0096] Will Import the UAV swarm action strategy generator to obtain the k-time slot UAV swarm action vector Get it again in,

[0097] 7. Sub-algorithm 1: Enhanced UAV service cache update decision generator updates the service cache configuration decision Λ of the k-time slot lightweight UAV swarm k

[0098] Step 1: Let the set of lightweight drones be N, the set of service functions be S, and initialize the temporary set N′=N,S′=S;

[0099] Step 2: If Take out the first element n in N′ and jump to step 3, otherwise, jump to step 6;

[0100] Step 3: Based on the local popularity n of each service type s in the coverage area of ​​lightweight drone n s , calculate the cache value w of service type s on lightweight UAV n n,s , according to w n,s The value of arranging the elements in the set S′ in descending order;

[0101] Step 4: If Take out the first element s in S′ and jump to step 5. Otherwise, let S′=S and jump to step 2.

[0102] Step 5: If UAV n has cached service s, jump to step 4, otherwise, the enhanced UAV caches service s according to the cache capacity constraint of lightweight UAV n and the cache value w of service type s. n,s Priority, using the “least frequently used (LFU)” algorithm, update the service cache for lightweight drone n and jump to step 4;

[0103] Step 6: Output k-time slot service cache configuration decision Λ k .

[0104] 8. Sub-algorithm 2: Update the task offloading decision Ω of k time slots through the task offloading and resource allocation decision generator k and computing resource allocation decision Φ k

[0105] Step 1: Let M be the set of users who request task offloading in k time slots. k , initialize the task offloading decision of k time slots in, The binary decision variable for offloading tasks from user m to drone n in time slot k, and computing the resource allocation decision in, For the computing resources allocated to user m by drone n in time slot k, let the temporary set M = M k ;

[0106] Step 2: Calculate the task priority p based on the task request of each user m in the set M m,k , and according to p m,k The elements in the value pair set M are arranged in descending order, and the decision Λ is made based on the cache configuration of the drone service k And the task type requested by each user m in the set M, establish a candidate unloading drone set N for each user m m,k ;

[0107] Step 3: If Take out the first element m and jump to step 4, otherwise, jump to step 6;

[0108] Step 4: If Compute user m offloads tasks to N m,k The transmission delay of candidate drone n according to The value pair set N m,k Arrange the elements in ascending order, take out the first element n, and calculate the minimum computing resources that drone n needs to allocate for user m Jump to step 5, otherwise, jump to step 3;

[0109] Step 5: If the computing resources are available for UAV n Update the computational resource allocation decision Θ k , let user m make task offloading decision Jump to step 3, otherwise, jump to step 4;

[0110] Step 6: For the drone collection For each drone n in To M k Each The user, Update the computing resource allocation decision Φ k , jump to step 7, otherwise, jump to step 7;

[0111] Step 7: Output k-time slot task offloading decision Ω k and computing resource allocation decision Φ k .

[0112] A method for offloading tasks in a heterogeneous UAV network based on service cache, the specific implementation method of which includes the following steps:

[0113] Step 1: Use the double-delay deep deterministic policy gradient algorithm to build a deep reinforcement learning model, including the drone swarm action strategy generator, drone swarm state-action value generator, service cache update decision generator, task offloading and resource allocation decision generator, experience sample storage area, random sample set, divide the drone swarm's cruising time into K equal-length time slots, assume that the system state remains unchanged in any time slot k, and establish the system state vector [s k ,a k ,r k ,s k+1 ], where s k is the state vector of the k-time slot drone swarm, a k is the k-time slot UAV group action vector, r k is the k-time slot system reward function, s k+1 is the state vector of the drone swarm in time slot k+1, and k is initialized to 0;

[0114] Step 2: If k < K, let k = k + 1, and the service cache of the enhanced drone updates the decision generator according to the state vector s k And service popularity distribution, call sub-algorithm 1, update the service cache configuration decision Λ of the k-time slot lightweight drone swarm k , jump to step 3; otherwise, jump to step 6;

[0115] Step 3: Based on the task request of the ground user in time slot k, the state vector s of the drone swarm k and serving cache configuration decisions Λ k , through the task offloading and resource allocation decision generator, call sub-algorithm 2 to update the task offloading decision Ω of the k time slot k and computing resource allocation decision Φ k ;

[0116] Step 4: Obtain s through the drone swarm action strategy generator k The action vector of the drone group in state a k , according to {Λ k ,Ω k ,Φ k}Execute the task offloading in k time slots and obtain the drone group action reward r k , according to s k with a k Calculate and obtain the k+1 time slot drone swarm state vector s k+1 , the system state vector [s k ,a k ,r k ,s k+1 ] Added experience sample storage area;

[0117] Step 5: Randomly sample from the experience sample storage area to obtain a random sample set, and import the random sample set into the drone swarm action strategy generator and the drone swarm state-action value generator for training, and jump to step 2;

[0118] Step 6: The cruise work is completed.

[0119] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions.

[0120] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0121] The above embodiments should be understood to be only used to illustrate the present invention and not to limit the protection scope of the present invention. After reading the contents of the present invention, technicians can make various changes or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A heterogeneous UAV network task offloading method based on service cache, characterized in that: The following steps are involved:

101. A double-delay deep deterministic policy gradient algorithm is used to construct a deep reinforcement learning model, including a drone swarm action strategy generator, a drone swarm state-action value generator, a service cache update decision generator, a task offloading and resource allocation decision generator, an experience sample storage area, and a random sample set. The cruise working time of the drone swarm is divided into K equal-length time slots. Assuming that the system state remains unchanged in any time slot k, the system state vector [s k ,a k ,r k ,s k+1 ], where s k is the state vector of the k-time slot drone swarm, a k is the k-time slot drone group action vector, r k is the k-slot system reward function, s k+1 is the state vector of the drone swarm in time slot k+1, and k is initialized to 0; 102. If k<K, let k=k+1, the service cache update decision generator of the enhanced drone is based on the state vector s k and service popularity distribution, updating the service cache configuration decision Λ of the k-slot lightweight drone swarm k , jump to step 103; otherwise, jump to step 106; 103. According to the task request of the ground user in time slot k, the state vector s of the drone swarm k and serving cache configuration decisions Λ k , update the task offloading decision Ω of k time slot through the task offloading and resource allocation decision generator k and computing resource allocation decision Φ k ; 104. Obtain s through the drone swarm action strategy generator k The action vector of the drone group in state a k , perform task offloading for k time slots according to {Λk,Ωk,Φk}, and obtain the drone group action reward r k , according to s k with a k Calculate the k+1 time slot drone swarm state vector s k+1 , the system state vector [s k ,a k ,r k ,s k+1 ] Added experience sample storage area; 105. Randomly sample from the experience sample storage area to obtain a random sample set, and import the random sample set into the drone swarm action strategy generator and the drone swarm state-action value generator for training, and jump to step 102; 106. The patrol work is completed.

2. According to claim 1, a heterogeneous UAV network task offloading method based on service cache is characterized in that: In step 101, a system state vector [s k ,a k ,r k ,s k+1 ], where the state vector s of the drone swarm in time slot k is k , UAV swarm motion vector a k 、System reward function r k , drone swarms in s k Execute action a in state k After that, the state vector s of the drone swarm in time slot k+1 is k+1 , as shown in formulas (1), (2), (3), and (4) respectively: In formula (1), represents the three-dimensional coordinates of UAV n in time slot k, represents the set of all drones, including enhanced drones and all lightweight drones; in formula (2), is the horizontal flight angle of UAV n in time slot k, is the horizontal flight distance of UAV n in time slot k, is the vertical flight distance of drone n in time slot k; in formula (3), ε is the drone action reward weight factor, where 0≤ε≤1, ψ k represents the task completion rate of the k-time slot system, represents the normalized system unit task delay of k time slots, and the calculation method is shown in formulas (5) and (6): In formula (5), if the task of user m is offloaded within the task tolerance delay, then ψ m =1, otherwise, ψ m =0,M k represents the set of users who request task offloading in k time slots; in formula (6), represents the completion delay of unit task offloading of user m in time slot k, and the calculation method is shown in formula (7): In formula (7), T m,k is the task offloading completion delay of user m in time slot k, and the transmission delay from the task offloading of user m in time slot k to the execution of drone n Computational delay Composition, d m,k Request task data volume for user m in time slot k.

3. According to the method of claim 1, a heterogeneous UAV network task offloading method based on service cache is characterized in that: In step 102, the enhanced UAV service cache update decision generator updates the service cache configuration decision Λ of the k-time slot lightweight UAV group. k The method is as follows: 1) Let the set of lightweight drones be N, the set of service functions be S, and initialize the temporary set N′=N,S′=S; 2) If Take out the first element n in N′ and jump to step 3), otherwise, jump to step 6); 3) Based on the local popularity n of each service type s in the coverage area of ​​lightweight drone n s , calculate the cache value w of service type s on lightweight UAV n n,s , according to w n,s The value of arranging the elements in the set S′ in descending order; 4) If Take out the first element s in S′ and jump to step 5); otherwise, let S′=S and jump to step 2); 5) If UAV n has cached service s, jump to step 4), otherwise, the enhanced UAV caches the service type s according to the cache capacity constraint of the lightweight UAV n and the cache value w n,s Priority, using the "least frequently used LFU" algorithm, update the service cache of lightweight drone n and jump to step 4); 6) Output k-time slot service cache configuration decision Λ k .

4. According to claim 3, a heterogeneous UAV network task offloading method based on service cache is characterized in that: The cache value w of the service type s on the lightweight UAV n in step 3) n,s The calculation method is shown in formula (8): In formula (8), i s Indicates the amount of cached data of service s, n s represents the local popularity of service type s in the coverage area of ​​lightweight UAV n.

5. The method for offloading heterogeneous UAV network tasks based on service cache according to claim 1 is characterized in that: In step 103, the task offloading decision Ω of the k time slot is updated by the task offloading and resource allocation decision generator. k and computing resource allocation decision Φ k The method is as follows: 7) Let the set of users who request task offloading in time slot k be M k , initialize the task offloading decision of k time slots in, The binary decision variable for offloading tasks from user m to drone n in time slot k, and computing the resource allocation decision in, For the computing resources allocated to user m by drone n in time slot k, let the temporary set M = M k ; 8) Calculate the task priority p based on the task request of each user m in the set M m,k , and according to p m,k The elements in the value pair set M are arranged in descending order, and the decision Λ is made based on the cache configuration of the drone service k And the task type requested by each user m in the set M, establish a candidate unloading drone set N for each user m m,k ; 9) If Take out the first element m and jump to step 10), otherwise, jump to step 12); 10) If Compute user m offloads tasks to N m,k The transmission delay of candidate drone n according to The value pair set N m,k Arrange the elements in ascending order, take out the first element n, and calculate the minimum computing resources that drone n needs to allocate for user m Jump to step 11), otherwise, jump to step 9); 11) If the computing resources available on drone n Update the computational resource allocation decision Θ k , let user m make task offloading decision Jump to step 9), otherwise, jump to step 10); 12) For drone collection For each drone n in To M k Each The user, Update the computing resource allocation decision Φ k , jump to step 13), otherwise, jump to step 13); 13) Output k-time slot task offloading decision Ω k and computing resource allocation decision Φ k .

6. A heterogeneous UAV network task offloading method based on service cache according to claim 5, characterized in that: In step 8), the task priority p of each user m request m,k The calculation method of is shown in formula (9): In formula (9), is the maximum tolerable delay of a task requested by user m in time slot k.

7. The method for offloading heterogeneous UAV network tasks based on service cache according to claim 5 is characterized in that: In step 10), user m offloads the task to N m,k The transmission delay of candidate drone n and the minimum computing resources that drone n needs to allocate to user m The calculation method of is shown in formulas (10) and (11): In formula (10), is the association state between user m and drone n in time slot k. If user m is within the coverage of drone n, then otherwise, is the offloading relationship between user m and drone n in time slot k. If the task of user m is offloaded to drone n, then otherwise, That is, the associated drone acts as a relay to forward the task to the unloading drone for processing; represents the transmission rate from user m to associated drone n in time slot k, represents the transmission rate of the k-time slot associated UAV n and the unloaded UAV n′. In formula (11), c m,k represents the task complexity requested by user m in time slot k; and The calculation methods are shown in formulas (12) and (13) respectively: In formula (12), is the channel bandwidth of user m, is the communication channel gain from user m to drone n in time slot k, n0 is the noise power, is the transmission power of user m; in formula (13), represents the channel bandwidth of UAV n, is the communication channel gain from UAV n to UAV n′ in time slot k, is the transmission power of UAV n.

8. The method for offloading heterogeneous UAV network tasks based on service cache according to claim 1 is characterized in that: In step 104, according to s k with a k Calculate to obtain k+1 The methods include: Will Import the UAV swarm action strategy generator to obtain the k-time slot UAV swarm action vector Get it again in,

Images