A service cache-based heterogeneous unmanned aerial vehicle network task offloading method

Abstract

The application claims a service cache-based heterogeneous unmanned aerial vehicle network task offloading method, belonging to the field of communication technology. In view of the problem that light unmanned aerial vehicle networks are difficult to provide sufficient cache resources, which may lead to uneven service cache configuration and poor user service experience, a heterogeneous unmanned aerial vehicle network task offloading method is proposed by introducing an enhanced unmanned aerial vehicle into the light unmanned aerial vehicle network. The method adopts a deep reinforcement learning algorithm, dynamically plans the cruising trajectory of the heterogeneous unmanned aerial vehicle group according to the location distribution and task offloading request of the ground user, constructs a service cache decision according to the cache capacity constraint of the light unmanned aerial vehicle and the service popularity distribution of the associated user, and constructs a task offloading and resource allocation decision according to the on-board computing resource and service cache state, so as to improve the service cache efficiency of the unmanned aerial vehicle group, maximize the task offloading completion rate and reduce the system unit task offloading delay.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of communication, and particularly relates to a heterogeneous unmanned aerial vehicle network task offloading method based on service cache. BACKGROUND

[0002] Unmanned aerial vehicles have become an important part of future networks due to their high mobility and flexibility. By deploying low-altitude unmanned aerial edge networks in areas where ground communication facilities are weak, good communication and edge computing services can be provided for ground users.

[0003] Existing research on unmanned aerial vehicle network task offloading mostly focuses only on the scheduling of offloaded tasks and the allocation of computing resources, and less considers the serious contradiction between the diversity of service types and the limitations of on-board cache resources. Existing low-altitude edge networks usually use homogeneous unmanned aerial vehicles for networking, while lightweight unmanned aerial networks based on network cost considerations are usually difficult to equip with sufficient cache resources, which may lead to uneven distribution of service caches and affect the service experience of users.

[0004] To solve the above problems, the application proposes a heterogeneous unmanned aerial vehicle network task offloading method in which an enhanced unmanned aerial vehicle is introduced into a lightweight unmanned aerial vehicle network. This method uses a double-delay deep deterministic policy gradient algorithm [1] to establish a deep reinforcement learning model, plans the cruise trajectory of the unmanned aerial vehicle group adaptively, dynamically plans the allocation of on-board service cache and computing resources, maximizes the service cache efficiency and computing resource utilization of the unmanned aerial vehicle group, thereby effectively improving the task completion rate and reducing the system unit task offloading delay.

[0005] [1] F. Scott, V. H. Herke, M. David. Addressing Function Approximation Error in Actor-Critic Methods [C]. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden, 2018: 2587-2601. SUMMARY

[0006] The application aims to solve the problems of the above prior art. A heterogeneous unmanned aerial vehicle network task offloading method based on service cache is proposed. The technical solution of the application is as follows:

[0007] A heterogeneous unmanned aerial vehicle network task offloading method based on service cache includes the following steps:

[0008] 101. A double-delay deep deterministic policy gradient algorithm is used to construct a deep reinforcement learning model, including a drone swarm action strategy generator, a drone swarm state-action value generator, a service cache update decision generator, a task offloading and resource allocation decision generator, an experience sample storage area, and a random sample set. The cruising working time of the drone swarm is divided into K equal-length time slots. Assuming that the system state remains unchanged in any time slot k, the system state vector [s k ,a k ,r k ,s k+1 ], where s k is the state vector of the k-time slot drone swarm, a k is the k-time slot UAV swarm action vector, r k is the reward function of the k-time slot system, s k+1 is the state vector of the UAV swarm in time slot k+1, and k is initialized to 0;

[0009] 102. If k < K, let k = k + 1, the service cache of the enhanced drone updates the decision generator according to the state vector s k and service popularity distribution, update the service cache configuration decision Λ of the k-time slot lightweight drone swarm k , jump to step 103; otherwise, jump to step 106;

[0010] 103. According to the task request of the ground user in time slot k, the state vector s of the UAV swarm k and serving cache configuration decisions Λ k , update the task offloading decision Ω of k time slot through task offloading and resource allocation decision generator k and computing resource allocation decision Φ k ;

[0011] 104. Obtain s through the drone swarm action strategy generator k The motion vector of the drone group in the state a k , according to {Λ k ,Ω k ,Φ k}Execute the task offloading in k time slots and obtain the drone group action reward r k , according to s k with a k Calculate the k+1 time slot drone swarm state vector s k+1 , the system state vector [s k ,a k ,r k ,s k+1 ] Added experience sample storage area;

[0012] 105. Randomly sample from the experience sample storage area to obtain a random sample set, and import the random sample set into the drone swarm action strategy generator and the drone swarm state-action value generator for training, and jump to step 102;

[0013] 106. The patrol work is completed.

[0014] Furthermore, in step 101, the system state vector [s k ,a k ,r k ,s k+1 ], where the state vector s of the drone swarm in time slot k is k , UAV swarm motion vector a k , system reward function r k , drone swarms in s k Execute action a in state k After that, the state vector s of the drone group in time slot k+1 is k+1 , as shown in formulas (1), (2), (3), and (4) respectively:

[0015]

[0016]

[0017]

[0018]

[0019] In formula (1), represents the three-dimensional coordinates of UAV n in time slot k, represents the set of all UAVs, including enhanced UAVs and all lightweight UAVs; in formula (2), is the horizontal flight angle of UAV n in time slot k, is the horizontal flight distance of UAV n in time slot k, is the vertical flight distance of UAV n in time slot k; In formula (3), ε is the UAV action reward weight factor, where 0≤ε≤1, ψ k represents the task completion rate of the k-time slot system, represents the normalized system unit task delay of k time slots, and the calculation method is shown in formulas (5) and (6):

[0020]

[0021]

[0022] In formula (5), if the task of user m is completed within the task tolerance delay, then ψ m=1, otherwise, ψ m =0,M k represents the set of users who request task offloading in k time slots; in formula (6), represents the unit task offloading completion delay of user m in time slot k, and is calculated as shown in formula (7):

[0023]

[0024] In formula (7), T m,k is the task offloading completion delay of user m in time slot k, which is the transmission delay from the task offloaded by user m in time slot k to the execution on drone n. and computational latency Composition, d m,k Request task data volume for user m in time slot k.

[0025] Furthermore, in step 102, the service cache update decision generator of the enhanced UAV updates the service cache configuration decision Λ of the k-time slot lightweight UAV group. k The method is as follows:

[0026] 1) Let the set of lightweight drones be N, the set of service functions be S, and initialize the temporary set N′=N, S′=S;

[0027] 2) If Take out the first element n in N′ and jump to step 3), otherwise, jump to step 6);

[0028] 3) Based on the local popularity n of each service type s in the coverage area of ​​lightweight drone n s , calculate the cache value w of service type s on lightweight UAV n n,s , according to w n,s The value of is used to sort the elements in the set S′ in descending order;

[0029] 4) If Take out the first element s in S' and jump to step 5); otherwise, let S' = S and jump to step 2);

[0030] 5) If UAV n has cached service s, jump to step 4), otherwise, the enhanced UAV caches the service type s according to the cache capacity constraint of the lightweight UAV n and the cache value w n,s Priority, using the "least frequently used LFU" algorithm, update the service cache of lightweight drone n and jump to step 4);

[0031] 6) Output k-time slot service cache configuration decision Λ k .

[0032] Furthermore, the cache value w of the service type s on the lightweight drone n in step 3) is n,s The calculation method is shown in formula (8):

[0033]

[0034] In formula (8), i s Indicates the cache data volume of service s, n s represents the local popularity of service type s in the coverage area of ​​lightweight UAV n.

[0035] Furthermore, in step 103, the task offloading decision Ω of the k time slot is updated by the task offloading and resource allocation decision generator. k and computing resource allocation decision Φ k The method is as follows:

[0036] 7) Let the set of users who request task offloading in time slot k be M k , initialize the task offloading decision of k time slots in, The resource allocation decision is calculated as the binary decision variable for user m in time slot k to offload tasks to drone n. in, For the computing resources allocated to user m by drone n in time slot k, let the temporary set M = M k ;

[0037] 8) Calculate the task priority p based on the task request of each user m in the set M m,k , and according to p m,k The elements in the value pair set M are sorted in descending order, and the decision Λ is made based on the drone service cache configuration. k And the task type requested by each user m in the set M, establish a candidate unloading drone set N for each user m m,k ;

[0038] 9) If Take out the first element m and jump to step 10), otherwise, jump to step 12);

[0039] 10) If Compute user m offloads tasks to N m,k The transmission delay of candidate drone n according to The value pair set N m,k Arrange the elements in ascending order, take out the first element n, and calculate the minimum computing resources that drone n needs to allocate for user m Jump to step 11), otherwise, jump to step 9);

[0040] 11) If the computing resources available to drone n Update computing resource allocation decision Θ k , let user m make task offloading decision Jump to step 9), otherwise, jump to step 10);

[0041] 12) For drone collection For each drone n in To M k Each The user, Update computing resource allocation decision Φ k , jump to step 13), otherwise, jump to step 13);

[0042] 13) Output k-time slot task offloading decision Ω k and computing resource allocation decision Φ k .

[0043] Furthermore, the task priority p requested by each user m in step 8) m,k The calculation method is shown in formula (9):

[0044]

[0045] In formula (9), is the maximum tolerable delay of a task requested by user m in time slot k.

[0046] Furthermore, in step 10), user m offloads the task to N m,k The transmission delay of candidate drone n and the minimum computing resources that drone n needs to allocate to user m The calculation method is shown in formulas (10) and (11):

[0047]

[0048]

[0049] In formula (10), is the association state between user m and drone n in time slot k. If user m is within the coverage of drone n, then otherwise, is the offloading relationship between user m and drone n in time slot k. If the task of user m is offloaded to drone n, then otherwise, That is, the associated drone acts as a relay to forward the task to the unloading drone for processing; represents the transmission rate of user m to the associated drone n in time slot k, represents the transmission rate of the k-time slot associated UAV n and the unloaded UAV n′. In formula (11), c m,k represents the task complexity requested by user m in time slot k; and The calculation methods are shown in formulas (12) and (13):

[0050]

[0051]

[0052] In formula (12), is the channel bandwidth of user m, is the communication channel gain from user m to drone n in time slot k, n0 is the noise power, is the transmission power of user m; in formula (13), represents the channel bandwidth of UAV n, is the communication channel gain from UAV n to UAV n′ in time slot k, is the transmission power of UAV n.

[0053] Furthermore, in step 104, according to s k with a k Calculate to obtain s k+1 The methods include:

[0054] Will Import the drone swarm action strategy generator to obtain the k-time slot drone swarm action vector Get it again in,

[0055]

[0056] The advantages and beneficial effects of the present invention are as follows:

[0057] This invention discloses a task offloading method for heterogeneous UAV networks based on service caching. Existing research on task offloading in UAV networks has mostly focused solely on the scheduling of offloaded tasks and the allocation of computing resources, with little consideration of the significant conflict between the diversity of service types and the limitations of onboard cache resources. Existing low-altitude edge networks typically utilize homogeneous UAV networks. However, lightweight UAV networks, driven by network cost considerations, often struggle to equip sufficient cache resources, potentially leading to an uneven distribution of service caches and impacting user experience. To address these issues, this invention proposes a task offloading method for heterogeneous UAV networks that introduces an enhanced UAV into a lightweight UAV network. This method utilizes a dual-delay deep deterministic policy gradient algorithm to establish a deep reinforcement learning model. This method adaptively plans the UAV swarm's cruising trajectory based on the location distribution of ground users and task offloading requests. Service caching decisions are then made based on the lightweight UAV cache capacity constraints and the local popularity distribution of various service types in the coverage area. Task offloading and resource allocation decisions are then made based on onboard computing resources and service cache status. This maximizes the service cache efficiency of the UAV swarm, effectively improving task completion rates and reducing system-unit task offloading latency. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 The present invention provides a flow chart of a heterogeneous UAV network task offloading method based on service cache in a preferred embodiment. DETAILED DESCRIPTION

[0059] The following will describe the technical solutions in the embodiments of the present invention in detail with reference to the accompanying drawings. The described embodiments are only a part of the embodiments of the present invention.

[0060] The technical solution of the present invention to solve the above technical problems is:

[0061] The concepts and models involved in the present invention are as follows:

[0062] The present invention assumes that the low-altitude network is composed of multiple drones, each of which carries an edge server, and each server can cache a certain number of different types of services. Assuming that ground users do not participate in task calculations due to limited computing power, if the drone server caches the service type of the task requested by the associated user and has sufficient computing and communication resources, the ground user can offload the task to the associated drone for execution. Otherwise, the associated drone forwards the task to other drones that cache the corresponding service for execution. The cruising working time of the drone group is divided into K equal-length time slots, and the system state remains unchanged in any kth (∈K) time slot. Let M k represents the set of users who request task offloading in k time slots, All unmanned aerial vehicles, including enhanced unmanned aerial vehicles and all lightweight unmanned aerial vehicles, a service type set S = {s}, wherein the enhanced unmanned aerial vehicles can store all services, and each lightweight unmanned aerial vehicle can only store a part of services in the set S due to limited storage space.

[0063] The technical solutions of the present application are as follows:

[0064] 1. Network state vector, action vector, and reward function of the air-ground environment

[0065] Unmanned aerial vehicle group state vector s at k time slot k Unmanned aerial vehicle group action vector a k System reward function r k Unmanned aerial vehicle group state vector s at k+1 time slot after the unmanned aerial vehicle group performs action a in state s k Unmanned aerial vehicle group state vector s at k+1 time slot after the unmanned aerial vehicle group performs action a in state s k Unmanned aerial vehicle group state vector s at k+1 time slot after the unmanned aerial vehicle group performs action a in state s k+1 , as shown in formulas (1), (2), (3), and (4) respectively:

[0066]

[0067]

[0068]

[0069]

[0070] In formula (1), represents the three-dimensional coordinates of unmanned aerial vehicle n at k time slot, represents the set of all unmanned aerial vehicles, including enhanced unmanned aerial vehicles and all lightweight unmanned aerial vehicles; in formula (2), is the horizontal flight angle of unmanned aerial vehicle n at k time slot, is the horizontal flight distance of unmanned aerial vehicle n at k time slot, is the vertical flight distance of unmanned aerial vehicle n at k time slot; in formula (3), ε is the action reward weight factor of the unmanned aerial vehicle, wherein 0≤ε≤1, ψ k represents the system task completion rate at k time slot, represents the normalized system unit task delay at k time slot, and the calculation method is shown in formulas (5) and (6):

[0071]

[0072]

[0073] In formula (5), if the task of user m is completed within its task tolerance delay, then ψ m = 1, otherwise, ψ m= 0, M k denotes the set of users that request task offloading in k time slot; formula (6) is denotes the unit task offloading completion delay of user m in k time slot, and the calculation method is shown in formula (7):

[0074]

[0075] In formula (7), T m,k is the task offloading completion delay of user m in k time slot, which is composed of the transmission delay of user m in k time slot offloading task to the unmanned aerial vehicle n for execution and the calculation delay , d m,k is the task data volume requested by user m in k time slot;

[0076] 2. The method for calculating the caching value w n,s of service type s on the light unmanned aerial vehicle n

[0077] The caching value w n,s of service type s on the light unmanned aerial vehicle n is calculated as shown in formula (8):

[0078]

[0079] In formula (8), i s denotes the caching data volume of service s, n s denotes the local popularity of service type s in the coverage area of the light unmanned aerial vehicle n;

[0080] 3. The method for calculating the task priority p m,k requested by each user m

[0081] The calculation method of the task priority p m,k requested by each user m is shown in formula (9):

[0082]

[0083] In formula (9), T is the maximum tolerable delay of the task requested by user m in k time slot;

[0084] 4. The method for calculating the transmission delay of user m offloading task to the candidate unmanned aerial vehicle n in N m,k

[0085] The calculation method of the transmission delay of user m offloading task to the candidate unmanned aerial vehicle n in N m,k is shown in formula (10):

[0086] ​​​

[0087] In formula (10), is the association state between user m and drone n in time slot k. If user m is within the coverage of drone n, then otherwise, is the offloading relationship between user m and drone n in time slot k. If the task of user m is offloaded to drone n, then otherwise, That is, the associated drone acts as a relay to forward the task to the unloading drone for processing; represents the transmission rate of user m to the associated drone n in time slot k, represents the transmission rate of the k-time slot associated UAV n and the unloaded UAV n′, and The calculation methods are shown in formulas (11) and (12):

[0088]

[0089]

[0090] In formula (11), is the channel bandwidth of user m, is the communication channel gain from user m to drone n in time slot k, n0 is the noise power, is the transmission power of user m; in formula (12), represents the channel bandwidth of UAV n, is the communication channel gain from UAV n to UAV n′ in time slot k, is the transmission power of UAV n;

[0091] 5. Calculate the minimum computing resources that drone n needs to allocate to user m Method

[0092] The minimum computing resources that drone n needs to allocate to user m The calculation method is shown in formula (13):

[0093]

[0094] In formula (13), c m,k represents the task complexity requested by user m in time slot k;

[0095] 6. According to s k with a k Calculate to obtain s k+1 Method

[0096] Will Import the drone swarm action strategy generator to obtain the k-time slot drone swarm action vector Get it again in,

[0097] 7. Sub-algorithm 1: Enhanced UAV service cache update decision generator updates the service cache configuration decision Λ of the k-time slot lightweight UAV swarm k

[0098] Step 1: Let the set of lightweight drones be N, the set of service functions be S, and initialize the temporary set N′=N, S′=S;

[0099] Step 2: If Take out the first element n in N′ and jump to step 3. Otherwise, jump to step 6.

[0100] Step 3: Based on the local popularity n of each service type s in the coverage area of ​​lightweight drone n s , calculate the cache value w of service type s on lightweight UAV n n,s , according to w n,s The value of arranging the elements in the set S′ in descending order;

[0101] Step 4: If Take out the first element s in S' and jump to step 5. Otherwise, let S' = S and jump to step 2.

[0102] Step 5: If UAV n has cached service s, jump to step 4, otherwise, the enhanced UAV caches the service type s according to the cache capacity constraint of the lightweight UAV n and the cache value w n,s Priority, using the “least frequently used (LFU)” algorithm, update the service cache for lightweight UAV n and jump to step 4;

[0103] Step 6: Output k-time slot service cache configuration decision Λ k .

[0104] 8. Sub-algorithm 2: Update the task offloading decision Ω of k time slots through task offloading and resource allocation decision generator k and computing resource allocation decision Φ k

[0105] Step 1: Let the set of users who request task offloading in k time slots be M k , initialize the task offloading decision of k time slots in, The resource allocation decision is calculated as the binary decision variable for user m in time slot k to offload tasks to drone n. in, For the computing resources allocated to user m by drone n in time slot k, let the temporary set M = M k ;

[0106] Step 2: Calculate the task priority p based on the task request of each user m in the set M m,k , and according to p m,k The elements in the set M are sorted in descending order according to the drone service cache configuration decision Λ k And the task type requested by each user m in the set M, establish a candidate unloading drone set N for each user m m,k ;

[0107] Step 3: If Take out the first element m and jump to step 4, otherwise, jump to step 6;

[0108] Step 4: If Compute user m offloads tasks to N m,k The transmission delay of candidate drone n according to The value pair set N m,k Arrange the elements in ascending order, take out the first element n, and calculate the minimum computing resources that drone n needs to allocate for user m Jump to step 5, otherwise, jump to step 3;

[0109] Step 5: If the computing resources are available for drone n Update computing resource allocation decision Θ k , let user m make task offloading decision Jump to step 3, otherwise, jump to step 4;

[0110] Step 6: For the Drone Collection For each drone n in To M k Each The user, Update computing resource allocation decision Φ k , jump to step 7, otherwise, jump to step 7;

[0111] Step 7: Output k-time slot task offloading decision Ω k and computing resource allocation decision Φ k .

[0112] A method for offloading tasks from a heterogeneous UAV network based on service caching, the specific implementation method of which includes the following steps:

[0113] Step 1: Using the double-delay deep deterministic policy gradient algorithm, a deep reinforcement learning model is constructed, including a drone swarm action strategy generator, a drone swarm state-action value generator, a service cache update decision generator, a task offloading and resource allocation decision generator, an experience sample storage area, and a random sample set. The cruising working time of the drone swarm is divided into K equal-length time slots. Assuming that the system state remains unchanged in any time slot k, the system state vector [s k ,a k ,r k ,s k+1 ], where s k is the state vector of the k-time slot drone swarm, a k is the k-time slot UAV group motion vector, r k is the reward function of the k-time slot system, s k+1 is the state vector of the UAV swarm in time slot k+1, and k is initialized to 0;

[0114] Step 2: If k < K, let k = k + 1, and the service cache of the enhanced UAV updates the decision generator according to the state vector s k And the service popularity distribution, call sub-algorithm 1, update the service cache configuration decision Λ of the k-time slot lightweight drone swarm k , jump to step 3; otherwise, jump to step 6;

[0115] Step 3: Based on the task request of the ground user in time slot k, the state vector s of the UAV swarm k and serving cache configuration decisions Λ k , through the task offloading and resource allocation decision generator, call sub-algorithm 2 to update the task offloading decision Ω of the k time slot k and computing resource allocation decision Φ k ;

[0116] Step 4: Obtain s through the drone swarm action strategy generator k The motion vector of the drone group in the state a k , according to {Λ k ,Ω k ,Φ k}Execute the task offloading of k time slots and obtain the drone group action reward r k , according to s k with a k Calculate the k+1 time slot drone swarm state vector s k+1 , the system state vector [s k ,a k ,r k ,s k+1 ] Added experience sample storage area;

[0117] Step 5: Random sample set is obtained from experience sample storage area by random sampling, and the random sample set is respectively introduced into the UAV group action policy generator and the UAV group state-action value generator for training, and jumping to step 2;

[0118] Step 6: The cruising work is ended.

[0119] The system, the device, the module or the unit illustrated in the above embodiments can be specifically implemented by a computer chip or an entity, or by a product with certain functions.

[0120] It should be further noted that the terms "comprising", "containing" or any other variants thereof are intended to cover non-exclusive inclusions, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but also other elements not explicitly listed, or other elements inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of other identical elements in the process, method, article or apparatus comprising the element.

[0121] The above embodiments should be understood as only for illustrating the present application but not for limiting the protection scope of the present application. After reading the description of the present application, those skilled in the art can make various changes or modifications to the present application, and these equivalent changes and modifications also fall within the scope defined by the claims of the present application.

Claims

1. A heterogeneous UAV network task offloading method based on service cache, characterized in that: The following steps are involved:

101. A double-delay deep deterministic policy gradient algorithm is used to construct a deep reinforcement learning model, including a drone swarm action strategy generator, a drone swarm state-action value generator, a service cache update decision generator, a task offloading and resource allocation decision generator, an experience sample storage area, and a random sample set. The cruising working time of the drone swarm is divided into K equal-length time slots. Assuming that the system state remains unchanged in any time slot k, the system state vector [s k ,a k ,r k ,s k+1 ], where s k is the state vector of the k-time slot drone swarm, a k is the k-time slot UAV group motion vector, r k is the reward function of the k-time slot system, s k+1 is the state vector of the UAV swarm in time slot k+1, and k is initialized to 0; 102. If k < K, let k = k + 1, the service cache of the enhanced drone updates the decision generator according to the state vector s k and service popularity distribution, updating the service cache configuration decision Λ of the k-slot lightweight drone swarm k , jump to step 103; otherwise, jump to step 106; 103. According to the task request of the ground user in time slot k, the state vector s of the UAV swarm k and serving cache configuration decisions Λ k , update the task offloading decision Ω of k time slot through task offloading and resource allocation decision generator k and computing resource allocation decision Φ k ; 104. Obtain s through the drone swarm action strategy generator k The motion vector of the drone group in the state a k , according to {Λ k ,Ω k ,Φ k }Execute the task offloading of k time slots and obtain the drone group action reward r k , according to s k with a k Calculate the k+1 time slot drone swarm state vector s k+1 , the system state vector [s k ,a k ,r k ,s k+1 ] Added experience sample storage area; 105. Randomly sample from the experience sample storage area to obtain a random sample set, and import the random sample set into the drone swarm action strategy generator and the drone swarm state-action value generator for training, and jump to step 102; 106. The patrol work is completed; In step 102, the enhanced UAV service cache update decision generator updates the service cache configuration decision Λ of the k-time slot lightweight UAV swarm. k The method is as follows: 1) Let the set of lightweight drones be N, the set of service functions be S, and initialize the temporary set N′=N, S′=S; 2) If Take out the first element n in N′ and jump to step 3), otherwise, jump to step 6); 3) Based on the local popularity n of each service type s in the coverage area of ​​lightweight drone n s , calculate the cache value w of service type s on lightweight UAV n n,s , according to w n,s The value of arranging the elements in the set S′ in descending order; 4) If Take out the first element s in S' and jump to step 5); otherwise, let S' = S and jump to step 2); 5) If UAV n has cached service s, jump to step 4), otherwise, the enhanced UAV caches the service type s according to the cache capacity constraint of the lightweight UAV n and the cache value w n,s Priority, using the "least frequently used LFU" algorithm, update the service cache of lightweight drone n and jump to step 4); 6) Output k-time slot service cache configuration decision Λ k ; In step 103, the task offloading decision Ω of the k time slot is updated by the task offloading and resource allocation decision generator. k and computing resource allocation decision Φ k The method is as follows: 7) Let the set of users who request task offloading in time slot k be M k , initialize the task offloading decision of k time slots in, The resource allocation decision is calculated as the binary decision variable for user m in time slot k to offload tasks to drone n. in, For the computing resources allocated to user m by drone n in time slot k, let the temporary set M = M k ; 8) Calculate the task priority p based on the task request of each user m in the set M m,k , and according to p m,k The elements in the value pair set M are sorted in descending order, and the decision Λ is made based on the drone service cache configuration. k And the task type requested by each user m in the set M, establish a candidate unloading drone set N for each user m m,k ; 9) If Take out the first element m and jump to step 10), otherwise, jump to step 12); 10) If Compute user m offloads tasks to N m,k The transmission delay of candidate drone n according to The value pair set N m,k Arrange the elements in ascending order, take out the first element n, and calculate the minimum computing resources that drone n needs to allocate for user m Jump to step 11), otherwise, jump to step 9); 11) If the computing resources available to drone n Update the computing resource allocation decision Φ k , let user m make task offloading decision Jump to step 9), otherwise, jump to step 10); 12) For drone collection For each drone n in To M k Each The user, Update the computing resource allocation decision Φ k , jump to step 13), otherwise, jump to step 13); 13) Output k-time slot task offloading decision Ω k and computing resource allocation decision Φ k .

2. A heterogeneous UAV network task offloading method based on service cache according to claim 1, characterized in that: In step 101, the system state vector [s k ,a k ,r k ,s k+1 ], where the state vector s of the drone swarm in time slot k is k , UAV swarm motion vector a k , system reward function r k , drone swarms in s k Execute action a in state k After that, the state vector s of the drone group in time slot k+1 is k+1 , as shown in formulas (1), (2), (3), and (4) respectively: In formula (1), represents the three-dimensional coordinates of UAV n in time slot k, represents the set of all UAVs, including enhanced UAVs and all lightweight UAVs; in formula (2), is the horizontal flight angle of UAV n in time slot k, is the horizontal flight distance of UAV n in time slot k, is the vertical flight distance of UAV n in time slot k; In formula (3), ε is the UAV action reward weight factor, where 0≤ε≤1, ψ k represents the task completion rate of the k-time slot system, represents the normalized system unit task delay of k time slots, and the calculation method is shown in formulas (5) and (6): In formula (5), if the task of user m is completed within the task tolerance delay, then ψ m =1, otherwise, ψ m =0,M k represents the set of users who request task offloading in k time slots; in formula (6), represents the unit task offloading completion delay of user m in time slot k, and is calculated as shown in formula (7): In formula (7), T m,k is the task offloading completion delay of user m in time slot k, which is the transmission delay from the task offloaded by user m in time slot k to the execution on drone n. and computational latency Composition, d m,k Request task data volume for user m in time slot k.

3. The method for offloading tasks in heterogeneous UAV networks based on service cache according to claim 1 is characterized in that: The cache value w of the service type s on the lightweight UAV n in step 3) n,s The calculation method is shown in formula (8): In formula (8), i s Indicates the cache data volume of service s, n s represents the local popularity of service type s in the coverage area of ​​lightweight UAV n.

4. The method for offloading tasks in heterogeneous UAV networks based on service cache according to claim 1, characterized in that: The task priority p requested by each user m in step 8) m,k The calculation method is shown in formula (9): In formula (9), is the maximum tolerable delay of a task requested by user m in time slot k.

5. The method for offloading heterogeneous UAV network tasks based on service cache according to claim 1 is characterized in that: In step 10), user m offloads the task to N m,k The transmission delay of candidate drone n and the minimum computing resources that drone n needs to allocate to user m The calculation method is shown in formulas (10) and (11): In formula (10), is the association state between user m and drone n in time slot k. If user m is within the coverage of drone n, then otherwise, is the offloading relationship between user m and drone n in time slot k. If the task of user m is offloaded to drone n, then otherwise, That is, the associated drone acts as a relay to forward the task to the unloading drone for processing; represents the transmission rate of user m to the associated drone n in time slot k, represents the transmission rate of the k-time slot associated UAV n and the unloaded UAV n′. In formula (11), c m,k represents the task complexity requested by user m in time slot k; and The calculation methods are shown in formulas (12) and (13): In formula (12), is the channel bandwidth of user m, is the communication channel gain from user m to drone n in time slot k, n0 is the noise power, is the transmission power of user m; in formula (13), represents the channel bandwidth of UAV n, is the communication channel gain from UAV n to UAV n′ in time slot k, is the transmission power of UAV n.

6. The method for offloading tasks in heterogeneous UAV networks based on service cache according to claim 1, characterized in that: In step 104, according to s k with a k Calculate to obtain s k+1 The methods include: Will Import the drone swarm action strategy generator to obtain the k-time slot drone swarm action vector Get it again in,

Images