A dynamic user request-oriented unmanned aerial vehicle air-ground collaborative caching method

Abstract

The application discloses a kind of unmanned aerial vehicle air-ground collaborative caching methods for dynamic user request, belong to unmanned aerial vehicle communication and edge computing field.The application constructs the air-ground collaborative caching system consisting of macro base station, multiple unmanned aerial vehicles and dynamic mobile user, defines caching strategy, node association strategy, resource allocation strategy and unmanned aerial vehicle trajectory deployment strategy, and establishes joint optimization problem with global cache hit rate and content delivery average time delay as optimization target.It is decomposed into three sub-problems using hierarchical nested dynamic collaborative optimization framework: inner layer optimizes node association by clustering, and optimizes resource allocation by continuous convex approximation;outer layer adopts mean-field multi-agent deep deterministic policy gradient algorithm to jointly optimize cache state and unmanned aerial vehicle trajectory, and inner layer result is fed back to outer layer to form dynamic closed loop.The application effectively deals with user random movement and dynamic change of content request, reduces content delivery delay, improves system caching efficiency and service quality.

Description

Technical Field

[0001] This invention belongs to the field of UAV communication and edge computing technology, specifically relating to a UAV air-ground collaborative caching method for dynamic user requests. Background Technology

[0002] With the development of mobile internet and intelligent connected technologies, users' demands for low-latency and high-reliability content services are constantly increasing. The traditional centralized content distribution model based on macro base stations is difficult to meet the real-time service needs in highly dynamic scenarios due to problems such as long transmission distances and network congestion.

[0003] To reduce content retrieval latency, Mobile Edge Computing (MEC) technology deploys caching and computing resources at the network edge, enabling content to be processed and distributed closer to the user. In recent years, drones, with their flexible deployment and rapid response capabilities, have been introduced into edge computing systems, forming a drone-assisted air-ground collaborative network structure, thereby further improving system coverage and service capabilities.

[0004] In existing technologies, UAV-assisted caching systems primarily improve cache hit rate and system performance by optimizing cache state variables or resource allocation strategies. However, in practical applications, users exhibit random movement, leading to dynamic changes in network topology. This makes it difficult to maintain consistently high cache hit rates and low content delivery latency using fixed caching strategies or static deployment methods. Furthermore, while multi-UAV collaborative caching can improve the overall cache hit rate, the lack of reasonable collaboration range control and dynamic association mechanisms can easily introduce redundant transmissions, increasing system load and transmission latency. Simultaneously, there is a strong coupling relationship between cache placement, node association, communication resource allocation, and UAV trajectories; existing technologies typically optimize only a single factor, making it difficult to achieve optimal overall performance.

[0005] Therefore, how to achieve efficient content distribution in a dynamic user environment by comprehensively considering caching strategies, collaboration mechanisms, resource allocation, and drone trajectories has become a key technical problem that urgently needs to be solved. Summary of the Invention

[0006] The purpose of this invention is to provide a UAV air-to-ground collaborative caching method for dynamic user requests, which addresses the problem of coupling between cache placement, node association, communication resource allocation, and UAV trajectory deployment under conditions of random user movement and random changes in content request events in each time slot. This invention constructs a hierarchical nested dynamic collaborative optimization mechanism that combines cache placement, node association, resource allocation, trajectory deployment, and status feedback. Based on user location, content request, node cache status, and communication link status, the cache status variable, node association strategy, resource allocation strategy, and UAV trajectory deployment strategy are iteratively updated.

[0007] This invention is achieved through the following technical solution:

[0008] A drone air-to-ground collaborative caching method for dynamic user requests includes the following steps:

[0009] S1: Establish an air-ground collaborative caching system model consisting of macro base stations, multiple drones, and dynamic mobile users; construct a collaborative network caching, user random movement, and edge caching communication model.

[0010] S2: Define caching strategy, node association strategy, resource allocation strategy and UAV trajectory deployment strategy. With global cache hit rate and average content delivery latency as optimization objectives, construct a joint optimization problem that satisfies cache capacity constraints, user single association constraints, constraints to avoid repeated collaborative transmission, bandwidth allocation constraints, transmission power constraints, UAV movement distance constraints and UAV collision avoidance constraints.

[0011] S3: A hierarchical nested dynamic collaborative optimization framework is adopted to decompose the joint optimization problem into three sub-problems;

[0012] Sub-problem A: Under the conditions of fixed cache state variables, resource allocation strategy, and UAV trajectory deployment strategy, optimize the node association strategy using clustering method;

[0013] Subproblem B: Under the conditions of fixed cache state variables, node association strategy, and UAV trajectory deployment strategy, optimize the resource allocation strategy using a continuous convex approximation method;

[0014] Subproblem C: Under the conditions of fixed node association strategy and resource allocation strategy, use mean field multi-agent deep deterministic strategy gradient algorithm to jointly optimize cached state variables and UAV trajectory deployment strategy;

[0015] In this process, subproblems A and B are inner-layer optimizations, and subproblem C is an outer-layer optimization. The results of the inner layer are fed back to the outer layer, and the updated results of the outer layer are applied to the next round of inner-layer optimization, forming a dynamic closed loop.

[0016] Further, in step S1, the dynamic mobile user set is represented as ,in, Represents a user index; multiple drones form a drone ensemble. ,in, The drone index is used to provide content delivery services for dynamically mobile users; the macro base station is used to cache content files; the set of content files is represented as... ,in, Indicates file index, content file Data size is represented as The discrete time slot model is used to represent the set of discrete time slots as follows: ,in, This represents the slot index, and the duration of a single slot is expressed as... ;in, , , and These represent the number of users, the number of drones, the number of cached files, and the total number of discrete time slots, respectively.

[0017] The construction of the air-ground collaborative caching system model includes the following:

[0018] A three-tiered content collaborative transmission mechanism is constructed, including local cache direct transmission, collaborative cache relay, and base station source supplementation;

[0019] The local cache direct transmission means that when the associated drone caches the content file requested by the dynamic mobile user, the associated drone directly delivers it to the dynamic mobile user.

[0020] The collaborative caching relay is as follows: when the associated drone does not cache the content file requested by the dynamic mobile user, but the collaborative node caches the content file, the collaborative node transmits the content file to the associated drone, and then the associated drone forwards the content file to the dynamic mobile user; wherein, the collaborative node refers to other drones located in the set of dynamic collaborative nodes of the associated drone, which establish a communication link with the associated drone and provide it with content files;

[0021] The base station back-to-source supplementation is as follows: when neither the associated drone nor its cooperating nodes have cached the content file, the macro base station transmits the content file to the associated drone, and the associated drone then forwards it.

[0022] In the aforementioned air-to-ground collaborative caching system model, users are associated with corresponding drones based on their location; using This represents the service node associated variable, where, Indicates user With drones Establish service node associations. Indicates user With drones No service node association was established; the service node refers to the drone that has established a service node association with the dynamic mobile user and provides content delivery services to the dynamic mobile user; the drone redeploys its location based on the user's movement; and Representing users respectively and drones In the time slot The position coordinates within; where, and Representing users respectively In the time slot The x and y coordinates; and They represent drones In the time slot The x and y coordinates; Indicates drone The fixed safe flight altitude; the fixed location coordinates of the macro base station are... ,in, and These represent the horizontal and vertical coordinates of the macro base station, respectively.

[0023] Furthermore, in step S1, in the air-ground collaborative caching system model, a collaborative network caching model is constructed to determine the caching status of each UAV and the collaborative transmission relationship between UAVs, so as to support local cache direct transmission and collaborative cache relay in the three-level content collaborative transmission mechanism.

[0024] The construction of the cooperative network caching model includes the following:

[0025] A1. Construct a multi-drone collaborative caching network, where multiple drones share content and transmit it collaboratively through a collaborative caching mechanism; define the drone cache state variable as follows. The drone cache state variable is used to indicate the drone Should the requested content file be cached? ,in, Indicates drone Cached content files , Indicates drone Uncached content files And establish drone cache capacity constraints: ;in, Indicates the maximum cache capacity of the drone; Represents content file Data size;

[0026] A dynamic set of cooperative nodes is constructed based on the spatial relationships and communication link quality among the drones; the dynamic set of cooperative nodes is represented as follows:

[0027]

[0028] in, Indicates the index of candidate collaborative drones; Indicates drone Collaborating with candidate drones In the time slot The Euclidean distance; Indicates drone In the time slot The maximum cooperation radius is calculated based on the current node location and communication link status.

[0029] ;

[0030] in, Indicates drone Euclidean distance between the macro base station and the macro base station; Indicates drone Collaborating with candidate drones The Euclidean distance between them; Indicates macro base station to drone A fixed transmission power is used when transmitting content; Indicates drone To drones A fixed transmit power is used when transmitting collaborative buffer content; Macro base stations and drones Small-scale fading coefficient of the communication link between them; Indicates the channel power gain at the reference distance; The path loss index is represented; the collaborative link between UAVs adopts a normalized line-of-sight propagation model.

[0031] Define a collaborative cache association variable to indicate whether a collaborative cache association has been established between the drone and the collaborative node for the target file. The collaborative cache association variable is represented as follows:

[0032] ;

[0033] in, Indicates associated drones In collaboration with nodes For content files Establish collaborative cache associations. This indicates that no collaborative cache association has been established; the collaborative cache association must meet the following conditions:

[0034] ;

[0035] ;

[0036] The first constraint indicates that only if the cooperating node Cache content files At that time, establish a collaborative cache association; the second constraint indicates that only when the drone Uncached content files At that time, establish collaborative cache associations;

[0037] When drones Uncached content files At that time, from the dynamic collaborative node set The cached files contain the aforementioned content. The collaborative nodes are selected, and the closest collaborative node is chosen to establish a collaborative cache association; the collaborative cache association also satisfies: This constraint indicates that the drone For the same content file It can establish a collaborative cache association with at most one collaborating node;

[0038] A2. A content popularity model is established using the Zipf distribution, and users... For content files The request probability is represented as follows:

[0039] ;

[0040] in, Indicates user Request content file The probability of; This represents the Zipf parameter, which characterizes the concentration of content request probabilities. This represents the virtual index of the file in the summation of the denominator; according to the above model, each user uses the same Zipf content popularity distribution;

[0041] A3. Calculate the number of local cache hits for the drone and the number of cache hits for the dynamic collaborative node set based on the aforementioned request probabilities, wherein the drone... Local cache hit count:

[0042] ;

[0043] Dynamic collaborative node set The cache hit count is expressed as:

[0044] ;

[0045] A4. Based on the local cache hit count and the cache hit count of the dynamic collaborative node set, the global cache hit rate of the collaborative network is obtained as follows:

[0046] ;

[0047] The collaborative network caching model is defined by the UAV cache state variables, UAV cache capacity constraints, the dynamic collaborative node set, collaborative cache association variables and their constraints, and the content popularity model, forming a complete collaborative network caching model.

[0048] Further, in step S1, a user random movement model is constructed, including the following:

[0049] Set the user's maximum movement area to ,in, and These represent the maximum range of the user's movement area in the horizontal and vertical directions, respectively; user In the time slot The direction of movement is Movement speed is ,in, Indicates user Maximum movement speed; when When the current time slot is filled, it indicates that the user remains stationary; at the start of the next time slot, the user's movement direction and speed are regenerated independently.

[0050] Configure User In the time slot The position coordinates are The user during the duration of Moving at a constant speed within a time slot, the user's position coordinates in the next time slot are: ;

[0051] Setting up drones Maximum flight speed is The flight constraints for the drone are: ;

[0052] Establish minimum safe flight distance between drones ,ensure To avoid collisions between drones.

[0053] Furthermore, in step S1, the edge cache communication model is constructed, including the following:

[0054] B1. Small-scale fading is represented using the Ricean channel model as follows:

[0055] ;

[0056] in, Represents the Rice factor, used to characterize the ratio of the line-of-sight component power to the non-line-of-sight scattering component power; and Representing nodes respectively With nodes The LOS and NLOS components of the communication link between them; and These represent the node indices at both ends of the communication link;

[0057] B2. Constructing the channel coefficient model for the two ends of the communication link:

[0058] ;

[0059] in, Represents communication node and Complex channel coefficients of the communication link between them. and These represent the position coordinates of the nodes at both ends of the communication link. This indicates the distance path loss of the corresponding communication link;

[0060] B3. Establish the signal-to-noise ratio (SNR) model from the drone to the user, the SNR model from the base station to the drone, and the SNR model between drones, respectively:

[0061] ;

[0062] ;

[0063] ;

[0064] in, Indicates drone To users The signal-to-noise ratio of the link; Indicates macro base station to drone The signal-to-noise ratio of the link; Indicating collaborative drones To drones The signal-to-noise ratio of the link; Indicates drone For users Allocated transmission power; Indicates macro base station to drone A fixed transmission power is used when transmitting content; Indicating collaborative drones To drones A fixed transmission power is used when transmitting content; This represents the noise power at the receiving end; the cooperative link between UAVs adopts a normalized line-of-sight propagation model.

[0065] B4. Establish the transmission rate models for drone-to-user, base station-to-drone, and drone-to-drone transmission rates, respectively:

[0066] ;

[0067] ;

[0068] ;

[0069] in, This indicates the transmission bandwidth occupied by the link from the macro base station to the drone; This represents the total bandwidth between the drone and ground users. It is the subcarrier frequency division coefficient. It is the bandwidth between the drone and the collaborative network. A virtual index for summing collaborative nodes, used solely for drone statistics. The number of currently established collaborative caching links; Only Used when there is no collaborative cache association; when there is no collaborative cache association, the corresponding collaborative network transmission latency term is set to 0 and not calculated. ;

[0070] B5. Define the overall content delivery latency for users as follows:

[0071]

[0072] in, ;

[0073] ;

[0074] ;

[0075] This indicates the latency of content transmission between the drone and the user. Indicates transmission latency within a collaborative network. This indicates the transmission delay between the base station and the drone.

[0076] Furthermore, in step S2, the optimization objectives of global cache hit rate and average content delivery latency include the following:

[0077] C1. Define the caching strategy, node association strategy, resource allocation strategy, and drone trajectory deployment strategy as follows:

[0078] ;

[0079] ;

[0080] ;

[0081] ;

[0082] in, This represents the caching strategy comprised of the cached state variables of each drone for each file; This represents the node association strategy, which consists of user-drone service node association variables and drone inter-drone collaborative cache association variables. This refers to the resource allocation strategy, which consists of the proportion of bandwidth and transmission power allocated by the drone to the user. This represents the drone trajectory deployment strategy, which consists of the position coordinates of each drone in different time slots.

[0083] C2. Based on the above set of strategies, with global cache hit rate and average content delivery latency as optimization objectives, the following joint optimization problem is constructed:

[0084] ;

[0085] in, Indicates by caching strategy Node association strategy The system's global cache hit rate is determined jointly. Indicates user Overall content delivery latency under given caching strategy, node association strategy, resource allocation strategy, and drone trajectory deployment strategy; This represents the cache hit rate weight adjustment coefficient, used to coordinate the dimensionless global cache hit rate with the time-based average content delivery latency in the objective function.

[0086] C3. The joint optimization problem satisfies the following constraints:

[0087]

[0088]

[0089]

[0090] Among them, the cache state constraint: constraint C1 represents that the cache state of any drone for any file is a binary variable; when the drone When acting as a collaborative node, it accesses files The corresponding cache state is represented as ;

[0091] Cache capacity constraint: Constraint C2 indicates that the total amount of data cached by each drone must not exceed its maximum cache capacity. ;

[0092] Service node association constraints: Constraint C3 indicates that the service node association variable is a binary variable; Constraint C4 indicates that each user establishes a service node association with only one drone in the same time slot;

[0093] Collaborative cache association constraints: Constraint C5 indicates that the collaborative cache association variable is a binary variable; Constraint C6 indicates that the associated drone is associated. For the same target file A collaborative cache association can be established with at most one collaborating node, and only if the collaborating node... The target file has been cached. Related drones A collaborative cache association can only be established if the target file is not cached by the cache itself.

[0094] Bandwidth allocation constraints: Constraint C7 states that the bandwidth ratio allocated by a drone to a user ranges from 0 to 1; Constraint C8 states that the sum of the bandwidth ratios allocated by each drone to its served users must not exceed 1, and only if the user... With drones When establishing service node associations, drones Only then can users Allocate bandwidth resources; when At that time, by achievable ;

[0095] Transmit power constraint: Constraint C9 indicates that the total transmit power allocated to each UAV for its service users shall not exceed the uniform maximum transmit power. And only when the user With drones When establishing service node associations, drones Only then can users Allocate transmission power; when At that time, by achievable ;

[0096] Drone movement distance constraint: Constraint C10 means that the movement distance of a drone between adjacent time slots does not exceed the product of its maximum flight speed and the duration of a single time slot;

[0097] Drone collision avoidance constraint: Constraint C11 means that the distance between any two drones in any time slot is not less than the minimum safe flight distance. .

[0098] Furthermore, in step S3, the sub-problem A specifically includes:

[0099] D1. Variables associated with service nodes Associated variables with collaborative cache To optimize variables, cache state variables. Resource allocation strategy and drone trajectory deployment strategy Under the condition that all nodes are fixed, construct a node association strategy. Optimization subproblems :in, Indicates the iteration round index of hierarchical nested dynamic collaborative optimization:

[0100]

[0101]

[0102] in, and They represent the first In the process of dynamic collaborative optimization with nested layers, associated drones and candidate collaborative drones For content files The cache state;

[0103] D2. Solve the subproblem of node association strategy optimization, and perform service node association optimization and collaborative cache association optimization in sequence:

[0104] D21. Service Node Association Optimization: The K-Means clustering algorithm is used to cluster dynamically moving users based on their location, and the service node association between users and drones is determined based on the clustering results. Then, the collaborative cache association is determined based on the dynamic collaborative node set of the current time slot, the node cache state, and the minimum distance criterion, thus obtaining a node association strategy that satisfies constraints C3-C6. ;

[0105] Specifically, this includes: the initialization phase, the clustering iteration phase, and the service drone matching phase;

[0106] Initialization phase: based on time slots Location coordinates selection for internal dynamic mobile users There are 1 initial cluster center, with the number of clusters being the same as the number of drones, forming a set of cluster centers:

[0107]

[0108] in, Indicates time slot Inner The location of the cluster center for each user cluster. Represents a user clustering index;

[0109] Clustering Iteration Phase: Calculate the Euclidean distance between each user and each cluster center, and assign each user to the nearest cluster center, forming... Each user is clustered; the corresponding cluster center is updated based on the arithmetic mean of the location coordinates of all users in each user cluster. The user assignment and cluster center update are repeated until the position change of each cluster center in two adjacent clustering iterations does not exceed the preset clustering convergence threshold.

[0110] Service Drone Matching Phase: After user clustering converges, following the order of cluster center indices, for each cluster center, select the drone with the smallest Euclidean distance to that cluster center from among the drones that have not yet established service node relationships with other user clusters. This drone will then be designated as the service drone for that user cluster. Furthermore, establish service node associations between all users within that cluster and the selected drone, and configure the corresponding service node association variables. The associated variables of the remaining service nodes are set to ;

[0111] At the beginning of each time slot, the location coordinates of each user are updated according to the user random movement model, and user clustering, cluster center update and service drone matching are re-executed based on the updated user locations to obtain the service node association results for the current time slot.

[0112] D22. Collaborative Cache Association Optimization: Based on the completion of service node association, calculate the maximum collaborative radius according to the UAV's location in the current time slot and the communication link status, and follow the dynamic collaborative node set in step S1. The construction rules determine the drone Candidate collaborating nodes in the current time slot;

[0113] From macro base stations to drones The signal-to-noise ratio (SNR) of the link serves as the minimum SNR condition for inter-UAV cooperative links. The SNR of the inter-UAV cooperative link at the maximum cooperative distance is set to be equal to the SNR of the link from the macro base station to the UAV, i.e.:

[0114] ;

[0115] Based on the above signal-to-noise ratio relationship, the UAV is obtained In the time slot Maximum collaboration radius:

[0116] ;

[0117] When a drone establishes a service node association with a user The content file requested by the user was not cached. At that time, from the dynamic collaborative node set The cached files contain the aforementioned content. Collaborating nodes; if the filter results are not empty, select those that are compatible with drones. Nearest Collaborating Node Establish collaborative cache association; when ,or There is no such condition as satisfying When collaborating with nodes, no collaborative cache association is established.

[0118] Furthermore, in step S3, the sub-problem B specifically includes:

[0119] E1. Using bandwidth ratio allocation variables and transmit power allocation variables as optimization variables, and considering the buffer state variables... Node association strategy and drone trajectory deployment strategy Under the condition that all resources are fixed, a subproblem for optimizing resource allocation strategy is constructed. The subproblem for optimizing resource allocation strategy aims to minimize the average content delivery latency, and is expressed as:

[0120]

[0121] ;

[0122] E2. Using the maximum user content delivery latency as the upper bound of the average content delivery latency, based on the resource allocation strategy optimization sub-problem P3, construct an upper bound replacement problem P3.1 that minimizes the maximum user content delivery latency:

[0123] ;

[0124]

[0125] ;

[0126] Among them, auxiliary variables are introduced. This represents the common upper bound of the delivery latency for all user content;

[0127] And ensure that the content delivery latency for each user does not exceed Therefore, problem P3.1 can be equivalently rewritten as a minimization problem by introducing a common upper bound auxiliary variable:

[0128]

[0129]

[0130] Wherein, constraint C12 indicates that the content delivery latency for each user does not exceed the maximum user latency auxiliary variable. ;

[0131] E3. Define the original spectral efficiency function:

[0132] ;

[0133] The specification for the user content delivery delay expression is as follows:

[0134] ;

[0135] in, This indicates that under the current fixed drone trajectory deployment strategy, the drone... To users The actual spectral efficiency of the link, its value is determined by the drone. For users The allocated transmit power, the small-scale fading coefficient of the communication link, and the distance between nodes are all used to determine the transmission power. Indicates user Content delivery latency under the current fixed cache state variables, node association strategy, and drone trajectory deployment strategy;

[0136] E4. Introducing spectral efficiency slack variables ,make And set:

[0137] ;

[0138] The user is obtained using the spectral efficiency slack variable. Upper bound of content delivery latency:

[0139] ;

[0140] At this point, constraint C12 is transformed into a time delay constraint after spectral efficiency relaxation. ,in, It still exhibits non-convexity. To process the product term, it is expressed as an equivalent form of the difference between the following two convex quadratic functions:

[0141] ;

[0142] product terms Let be the difference between two convex quadratic functions, and perform a first-order Taylor expansion of the convex function at the current continuous convex approximation iteration point; let The index of the internal iteration round of the continuous convex approximation method in subproblem B is given, thus yielding the first iteration round. In each successive convex approximation iteration, a concave lower bound is constructed based on the product of the bandwidth ratio and the spectral efficiency relaxation variable at the current iteration point. :

[0143]

[0144] The continuous convex approximation method is used to obtain the first In the next successive convex approximation iteration, the user Convex upper bound of content delivery latency :

[0145] ;

[0146] E5. Based on the concave lower bound of the product term and the convex upper bound of the user delay mentioned above, the original non-convex problem is constructed into the following convex approximation problem:

[0147]

[0148]

[0149] Set the initial resource allocation results to satisfy bandwidth allocation constraints, transmit power constraints, and positive value conditions. and And calculate the spectral efficiency relaxation variable based on the initial transmit power. ; in the In the next iteration, construct based on the current iteration point. and The problem P3.3 is solved using a convex optimization solver to obtain the updated bandwidth ratio, transmit power, and spectral efficiency relaxation variables. The concave lower bound construction and convex approximation problem solving are repeated until the objective function value changes between two adjacent iterations do not exceed the preset convergence threshold, and the resource allocation strategy for the current round is output.

[0150] Furthermore, in step S3, the sub-problem C specifically includes:

[0151] For the In the process of nested optimization, the node association strategy obtained from subproblems A and B. Resource allocation strategy Cache state variables With drone trajectory deployment strategy To optimize the variables, the following optimization problem is constructed:

[0152]

[0153] ;

[0154] Since the above optimization problem involves both discrete buffer state variables and continuous trajectory control variables, and there is a strong coupling relationship between these variables, it belongs to a high-dimensional hybrid non-convex optimization problem. The mean-field multi-agent deep deterministic policy gradient algorithm is used to solve it.

[0155] F1. Cache and trajectory joint optimization modeling based on multi-agent Markov decision process:

[0156] The joint optimization process of collaborative caching and UAV trajectory is modeled as a multi-agent Markov decision process, denoted as […]. ,in, This represents a set of drone agents, where each drone corresponds to an agent with independent decision-making capabilities.

[0157] state space : Intelligent agents in The state at time t is determined by the first The process consists of the round-robin node association strategy, resource allocation strategy, user location, drone location, user request probability, and the drone's current cache state, and is represented as follows:

[0158] ;

[0159] in, Indicates time slot The global state; the state space includes the UAV's state in the time slot. Cache state before starting The drone intelligent agent, based on the state Generation time slot caching decision ;

[0160] Action space Unmanned Aerial Vehicle (UAV) Intelligent Agent In the time slot The action is represented as , among which, respectively using Indicates drone In the time slot The cache decision vector; Indicates drone In the time slot The direction of movement; Indicates drone In the time slot The distance to move within, and satisfying: Use respectively and This indicates caching policy actions and trajectory deployment actions;

[0161] Observation space Unmanned Aerial Vehicle (UAV) Intelligent Agent In the time slot Local observation global state The local state information obtained by the drone includes information related to the drone itself. Establish user location and request probability associated with service nodes, and drones The location and current cache status, the location and cache status of dynamic collaborative nodes, and the first In the round-robin node association strategy and resource allocation strategy, and with drones The relevant parts;

[0162] reward function : Indicates that the drone intelligent agent is in the time slot The instant rewards received; among them, This represents the penalty incurred by a drone agent when its candidate trajectory violates the drone movement distance constraint C10 or the collision avoidance constraint C11. The training objective is to maximize the cumulative discount reward. Representing the discount factor, the drone intelligent agent From time slot The initial cumulative discount reward is represented as follows:

[0163] ;

[0164] State transition : Put all drone agents in time slots The combined action is recorded as The state transition probability of a multi-UAV system is expressed as: The various drone agents execute joint actions. Then, the environment changes from state to state based on the state transition probability. Transition to the next state The state transition is jointly influenced by the user's random movement, user content request events randomly generated based on the request probability, and the drone's trajectory and actions;

[0165] F2. GST-based caching strategy action refactoring:

[0166] F21, for drone intelligent agents In the time slot For the document The caching strategy outputs a binary logistic regression vector from the caching strategy network:

[0167] ,

[0168] in, and These represent files that are not cached. and cache files Action tendency value;

[0169] The binary logistic regression vector is converted into action selection probabilities using the Softmax function.

[0170] ;

[0171] in, Represents unmanned aerial vehicle intelligent agents In the time slot For the document Select the The probability of class caching actions; Indicates the index of the current cached action category. This represents the virtual index of the cache action category in the summation of the denominator, and Category 0 indicates that files are not cached. Category 1 indicates cached files. ;

[0172] F22. Use the Gumbel-Max method to perform discrete action sampling on the binary logistic regression values ​​of cache actions:

[0173]

[0174] ;

[0175] in, It is independently sampled Gumbel noise. It is a One-Hot encoding; when When, it means ;when When, it indicates the caching strategy. ;

[0176] F23. Introduce two deterministic perturbation functions. and The deterministic perturbation function is used to adjust the binary action logic value vector so that the maximum action category corresponding to the continuous approximation result is consistent with the discrete sampling result, and a preset numerical difference is formed between the selected category and the unselected category, thereby obtaining a continuous and differentiable GST approximation function:

[0177] ;

[0178] F24. Perform capacity feasibility processing on the initial caching operation:

[0179] First, satisfy The document is used as a drone In the time slot Candidate cache files are selected based on the probability of the cache action. Sort the files from largest to smallest; then, evaluate the candidate files according to the sorting results, ensuring that the total file data size after inclusion does not exceed the cache capacity. The file set as the final cache file will cause the cumulative file data size to exceed the cache capacity after being added. Files that were not selected as candidate cache files are set to not be cached; all other files that were not selected as candidate cache files are also set to not be cached.

[0180] After capacity feasibility processing, the drone In the time slot The final caching decision satisfies:

[0181] ;

[0182] ;

[0183] The cached state variables, after capacity feasibility processing, are used to form the drone. In the time slot The final cache action vector:

[0184] ;

[0185] F25, for continuous trajectory actions Normalize each component according to its allowed value range:

[0186] ;

[0187] The normalized continuous trajectory motion satisfies:

[0188] ;

[0189] in, and These represent the minimum and maximum allowable values ​​for continuous trajectory actions, respectively; the subtraction and division operations are performed according to the corresponding components of the action vector; candidate positions of the UAV are calculated based on the normalized direction and distance of movement; when a candidate position violates constraint C10 or C11, the candidate position is made feasible and adjusted to a feasible position that satisfies the constraints; if a feasible adjustment position cannot be obtained, the UAV is kept at the position of the previous time slot.

[0190] F3, MF-MADDPG cache and trajectory joint optimization algorithm:

[0191] F31, each drone agent is equipped with an online Actor network. Online Critic Network and the corresponding target network and ;in, , , and These represent the network parameters of the online Actor network, online Critic network, target Actor network, and target Critic network, respectively. Represents unmanned aerial vehicle intelligent agents In the time slot The obtained local observations; Indicates time slot The global state; Represents unmanned aerial vehicle intelligent agents The combined action of executing the cache and the trajectory; Indicates the intelligent agent of drones The average action of the interactive collaborative agents;

[0192] F32, Centralized Training vs. Distributed Execution:

[0193] During training, each UAV agent sends its local observations and actions to the macro base station. The macro base station forms a global state and joint action information required for training based on the information from each UAV agent. The Critic network of each UAV agent is trained centrally with the global state, its own actions, and the average actions of the cooperating agents as inputs. The Actor network of each UAV agent is trained with its own local observations as inputs.

[0194] After training, each UAV agent no longer relies on the complete global state. Its online Actor network generates cached actions and continuous trajectory actions only based on its own local observations, thereby achieving distributed execution. The cached action logic values ​​output by the online Actor network are processed for discrete action reconstruction and cache capacity feasibility according to the GST method in F2, and the continuous trajectory actions are mapped according to the corresponding allowed value range.

[0195] F33, Mean Field Dimensionality Reduction: For Unmanned Aerial Vehicle Agents Combine it with the set of dynamic collaborative nodes The interactions with other agents are approximated by the interactions with the average agent, whose actions are represented as follows:

[0196] ;

[0197] in, Indicates time slot Internal dynamic collaborative node set The number of collaborative drones included; Represents collaborative drone intelligent agents In the time slot The execution of cache and trajectory joint actions; when At that time, Set to preset zero action; through the mean field approximation, the online Critic network does not need to directly input the actions of all UAV agents in the dynamic cooperative node set, but uses the mean agent action to characterize the comprehensive influence of other agents on UAV agent m;

[0198] F34. Exploration Noise: During the training phase, exploration noise sampled from a Gaussian distribution with a mean of 0 is added to the continuous trajectory actions output by the online Actor network, and the amplitude of the exploration noise is gradually reduced as the training rounds increase; for discrete cached actions, action exploration is performed through Gumbel-Max sampling and GST reparameterization mechanism in D2; the exploration noise is no longer added in the distributed execution phase after training is completed.

[0199] F35, Experience Replay: The global state, local observations, current actions, average agent actions, instantaneous rewards, global state and local observations of the next time slot generated by the interaction between each UAV agent and the environment are stored in a shared experience replay buffer; during training, a small batch of experience data is randomly extracted from the shared experience replay buffer for parameter updates of the online Actor network and the online Critic network.

[0200] F36. Network Update: The Critic network constructs a time-difference objective based on the current immediate reward and the target Critic network's value estimate for the next state, and updates its network parameters by minimizing the time-difference error; the Actor network updates its network parameters using a deterministic policy gradient; the target Actor network and the target Critic network update their network parameters using a soft update method; the process of environmental interaction, experience storage, mini-batch experience sampling, online network parameter updates, and target network soft updates is repeated until the preset training termination condition is met; based on the output of the Actor network after training, a caching strategy that satisfies the cache capacity constraint, the UAV movement distance constraint, and the collision avoidance constraint is obtained. and drone trajectory deployment strategy The strategy will then be used in the next round of hierarchical nested dynamic collaborative optimization process.

[0201] Compared with the prior art, the advantages of the present invention are as follows:

[0202] (1) The present invention constructs a joint optimization mechanism for cached state variables, node association strategies, resource allocation strategies and UAV trajectory deployment strategies, so that the four types of strategies can be iteratively updated under the same optimization framework;

[0203] (2) The present invention establishes a dynamic set of cooperative nodes based on the spatial distance and communication link conditions between UAVs, and establishes a cooperative cache association for the target file to reduce the situation where multiple cooperative nodes transmit the same target file at the same time;

[0204] (3) The present invention adopts a hierarchical nested dynamic collaborative optimization framework, which divides the joint optimization problem into three sub-problems: node association optimization, resource allocation optimization, and cache and trajectory joint optimization, and solves them respectively using K-Means clustering, continuous convex approximation and MF-MADDPG algorithm;

[0205] (4) The present invention introduces the mean field approximation in the MF-MADDPG algorithm, and approximates the interaction between the UAV agent and multiple cooperative agents as the interaction between the average agent actions, so as to reduce the input dimension of the joint actions in the Critic network.

[0206] (5) The present invention converts discrete cache actions into continuous differentiable approximate representations through the GST reparameterization method, enabling discrete cache actions to be jointly trained with continuous trajectory actions under the policy gradient framework. Attached Figure Description

[0207] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0208] Figure 1 This is a flowchart illustrating a UAV air-to-ground collaborative caching method for dynamic user requests according to the present invention.

[0209] Figure 2 A model diagram of an UAV air-to-ground collaborative caching system for dynamic user requests provided in an embodiment of the present invention;

[0210] Figure 3 A flowchart of the node association strategy optimization provided in this embodiment of the invention;

[0211] Figure 4 A flowchart illustrating the resource allocation strategy optimization provided in this embodiment of the invention;

[0212] Figure 5 A flowchart illustrating the joint optimization of caching strategy and UAV trajectory deployment strategy provided in this embodiment of the invention. Detailed Implementation

[0213] To clearly and completely describe the technical solution and its specific working process of the present invention, the specific embodiments of the present invention are as follows, in conjunction with the accompanying drawings:

[0214] Example 1

[0215] like Figure 1-5 As shown in the figure, the UAV air-to-ground collaborative caching method for dynamic user requests proposed in this embodiment includes the following steps:

[0216] S1: Establish an air-ground collaborative caching system model consisting of macro base stations, multiple drones, and dynamic mobile users; construct a collaborative network caching, user random movement, and edge caching communication model.

[0217] S11, the air-ground collaborative caching system model includes:

[0218] Further, in step S1, the set of dynamic mobile users is represented as ,in, Represents a user index; multiple drones form a drone ensemble. ,in, The drone index is used to provide content delivery services for dynamically mobile users; the macro base station is used to cache content files; the set of content files is represented as... ,in, Indicates file index, content file Data size is represented as The discrete time slot model is adopted, and the set of discrete time slots is represented as follows: ,in, This represents the slot index, and the duration of a single slot is expressed as... ;in, , , and These represent the number of users, the number of drones, the number of cached files, and the total number of discrete time slots, respectively.

[0219] The construction of the air-ground collaborative caching system model includes the following:

[0220] A three-tiered content collaborative transmission mechanism is constructed, including local cache direct transmission, collaborative cache relay, and base station source return supplementation. Local cache direct transmission means that when an associated drone caches a content file requested by a dynamic mobile user, the associated drone directly delivers the file to the dynamic mobile user. Collaborative cache relay means that when an associated drone does not cache a content file requested by a dynamic mobile user, but a collaborating node caches the content file, the collaborating node transmits the content file to the associated drone, which then forwards it to the dynamic mobile user. The collaborating node refers to other drones located within the set of dynamic collaborating nodes of the associated drone that have established a communication link with the associated drone and provide it with content files. Base station source return supplementation means that when neither the associated drone nor its collaborating nodes cache the content file, a macro base station transmits the content file to the associated drone, which then forwards it.

[0221] The system considers the random movement characteristics of users and associates users with corresponding drones based on their location; using This represents the service node associated variable, where, Indicates user With drones Establish service node associations. This indicates that no service node association has been established between the two; the drone will redeploy its location based on the user's movement to support the user's content requests; and Representing users respectively and drones In the time slot The position coordinates within; where, and Representing users respectively In the time slot The x and y coordinates; and They represent drones In the time slot The x and y coordinates; Indicates drone The fixed safe flight altitude; the fixed location coordinates of the macro base station are... ,in, and These represent the horizontal and vertical coordinates of the macro base station, respectively.

[0222] S12. Construct a collaborative network caching model, including the following:

[0223] A dynamic set of collaborative nodes and a collaborative cache association model are constructed to build a multi-UAV collaborative cache network. Multiple UAVs can share content and transmit it collaboratively through a collaborative cache mechanism.

[0224] Define the drone cache state variable as Used to represent drones For the file The cache state; where, Indicates drone Cache content files , Indicates drone Uncached content files A dynamic set of collaborative nodes is constructed based on the spatial relationships between drones and the quality of communication links.

[0225] drones In the time slot The set of dynamically cooperating nodes is represented as:

[0226]

[0227] in, Indicates the index of candidate collaborative drones; Indicates drone Collaborating with candidate drones In the time slot The Euclidean distance; Indicates drone In the time slot The maximum cooperation radius is calculated based on the current node location and communication link status.

[0228] ;

[0229] in, Indicates drone Euclidean distance between the macro base station and the macro base station; Indicates drone Collaborating with candidate drones The Euclidean distance between them; Indicates macro base station to drone A fixed transmission power is used when transmitting content; Indicates drone To drones A fixed transmit power is used when transmitting collaborative buffer content; Macro base stations and drones Small-scale fading coefficient of the communication link between them; Indicates the channel power gain at the reference distance; The path loss exponent is represented; the cooperative link between UAVs adopts a normalized line-of-sight propagation model, in which the reference channel gain and deterministic line-of-sight components have been normalized and absorbed. middle.

[0230] To avoid redundant transmission, the following variables are defined to represent the association between the drone and the collaborating nodes:

[0231] ;

[0232] in, Indicates associated drones In collaboration with nodes For the document Establish collaborative cache associations. This indicates that the collaborative cache association has not been established; the collaborative cache association satisfies:

[0233] ;

[0234] ;

[0235] The first constraint states that only if the cooperating node cache files Only when [the drone] is in use can a collaborative cache association be established; the second constraint indicates that a collaborative cache association can only be established when [the drone] is in use. Uncached files Only then is a collaborative cache association established.

[0236] Because drones have limited storage space, each drone will only cache a portion of the files; therefore, each file may be cached on different drones. The maximum storage space limit for a drone is specified. The cache constraints resulting from the capacity limit are expressed as follows:

[0237] ;

[0238] When drones Uncached files At that time, from the set of collaborating nodes The file is filtered in the cache. The collaborative nodes are selected, and the closest collaborative node is chosen to establish a collaborative cache association; the collaborative cache association also satisfies:

[0239] ;

[0240] The constraint represents the drone. For the same file It can establish a collaborative cache association with at most one collaborating node;

[0241] A content popularity model is established using the Zipf distribution, and users... For the target file The request probability is represented as follows:

[0242] ;

[0243] in, Indicates user Request file The probability of; file index It also indicates the popularity ranking of the documents after they are sorted in descending order of content popularity; This represents the Zipf parameter, which characterizes the concentration of content request probabilities. This represents the virtual index of the file in the summation of the denominator; according to the above model, each user adopts the same Zipf content popularity distribution. The request probability is used to describe the statistical distribution of content popularity; in different time slots, the content files actually requested by each user are randomly generated according to the content popularity distribution, therefore, the content files actually requested by users in each time slot can change with the random sampling results.

[0244] Based on the request probability, the number of local cache hits by the drone and the number of cache hits by the set of cooperating nodes are calculated respectively, wherein the drone Local cache hit count:

[0245] ;

[0246] Collaborative Node Set The cache hit count is expressed as:

[0247] ;

[0248] Based on the local cache hit count and the cache hit count of the cooperative node set, the global cache hit rate of the cooperative network is obtained as follows:

[0249] .

[0250] S13. Construct a user random movement model, including the following:

[0251] Set the user's maximum movement area to ,in, and These represent the maximum range of the user's movement area in the horizontal and vertical directions, respectively; user In the time slot The direction of movement is Movement speed is ,in, Indicates user Maximum movement speed; when This indicates that the user remains stationary within the current time slot; at the start of the next time slot, the user's movement direction and speed are regenerated independently. In the time slot The position coordinates are .

[0252] The user has a length of Moving at a constant speed within a time slot, the user's position coordinates in the next time slot are: ;

[0253] At the start of each time slot, the movement direction and speed for each user are generated independently. The user's movement direction and speed are not retained as state variables across time slots; the user's position in the next time slot is determined by the position in the current time slot and the movement direction and speed generated in the current time slot. When the user's position obtained according to the above position update formula exceeds the preset maximum movement area, the user's position is adjusted to the boundary of the maximum movement area, and the user's movement direction and speed are regenerated in the next time slot.

[0254] Setting up drones Maximum flight speed is The flight constraints for the drone are: ;

[0255] Establish minimum safe flight distance between drones ,ensure To avoid collisions between drones.

[0256] S14. The construction of the edge cache communication model includes the following:

[0257] Small-scale fading can be represented using the Ricean channel model as follows:

[0258] ;

[0259] in, Represents the Rice factor, used to characterize the ratio of the line-of-sight component power to the non-line-of-sight scattering component power; and Representing nodes respectively With nodes The LOS and NLOS components of the communication link between them; and These represent the node indices at both ends of the communication link.

[0260] Construct a channel coefficient model for the nodes at both ends of the communication link:

[0261] ;

[0262] in, Represents communication node and Complex channel coefficients of the communication link between them. and These represent the position coordinates of the nodes at both ends of the communication link. This indicates the distance path loss of the corresponding communication link;

[0263] The signal-to-noise ratio (SNR) models for drone-to-user, base station-to-drone, and drone-to-drone connections are established as follows:

[0264] ;

[0265] ;

[0266] ;

[0267] in, Indicates drone To users The signal-to-noise ratio of the link; Indicates macro base station to drone The signal-to-noise ratio of the link; Indicating collaborative drones To drones The signal-to-noise ratio of the link; Indicates drone For users Allocated transmission power; Indicates macro base station to drone A fixed transmission power is used when transmitting content; Indicating collaborative drones To drones A fixed transmission power is used when transmitting content; This represents the noise power at the receiving end; the cooperative link between UAVs uses a normalized line-of-sight propagation model, therefore... Only transmit power, inter-node distance, path loss index, and noise power are retained.

[0268] The transmission rate models for drone-to-user, base station-to-drone, and drone-to-drone are established as follows:

[0269] ;

[0270] ;

[0271] ;

[0272] in, This indicates the transmission bandwidth occupied by the link from the macro base station to the drone; This represents the total bandwidth between the drone and ground users. It is the subcarrier frequency division coefficient. It is the bandwidth between the drone and the collaborative network. A virtual index for summing collaborative nodes, used solely for drone statistics. The number of currently established collaborative caching links; Only Used when there is no collaborative cache association; when there is no collaborative cache association, the corresponding collaborative network transmission latency term is set to 0 and not calculated. .

[0273] Define the overall content delivery latency for users as:

[0274]

[0275] in, ;

[0276] ;

[0277] ;

[0278] This indicates the latency of content transmission between the drone and the user. Indicates transmission latency within a collaborative network. This indicates the transmission delay between the base station and the drone.

[0279] S2: Define cache state variables, node association strategies, resource allocation strategies, and UAV trajectory deployment strategies. Using global cache hit rate and average content delivery latency as optimization metrics, construct a joint optimization problem that satisfies constraints such as cache capacity, single user association, at most one collaborative node for the same target file, bandwidth, transmission power, UAV movement distance, and anti-collision constraints.

[0280] The caching strategy, node association strategy, resource allocation strategy, and drone trajectory deployment strategy are defined as follows:

[0281] ;

[0282] ;

[0283] ;

[0284] ;

[0285] in, This represents the caching strategy comprised of the cached state variables of each drone for each file; This represents the node association strategy, which consists of user-drone service node association variables and drone inter-drone collaborative cache association variables. This refers to the resource allocation strategy, which consists of the proportion of bandwidth and transmission power allocated by the drone to the user. This represents the drone trajectory deployment strategy, which consists of the position coordinates of each drone in different time slots.

[0286] Based on the above set of strategies, and using the weighted sum of global cache hit rate and average content delivery latency as a quantitative indicator of user experience quality, the following joint optimization problem is constructed:

[0287] ;

[0288] in, Indicates by caching strategy Node association strategy The system's global cache hit rate is determined jointly. Indicates user Overall content delivery latency under given caching strategy, node association strategy, resource allocation strategy, and drone trajectory deployment strategy. This represents the cache hit rate weighting adjustment coefficient, used to coordinate the dimensionless global cache hit rate with the time-based average content delivery latency in the objective function.

[0289] The joint optimization problem satisfies the following constraints:

[0290]

[0291]

[0292]

[0293] Among them, the cache state constraint: constraint C1 represents that the cache state of any drone for any file is a binary variable; when the drone When acting as a collaborative node, it accesses files The corresponding cache state is represented as .

[0294] Cache capacity constraint: Constraint C2 indicates that the total amount of data cached by each drone must not exceed its maximum cache capacity. .

[0295] Service node association constraints: Constraint C3 indicates that the service node association variable is a binary variable; Constraint C4 indicates that each user establishes a service node association with only one drone in the same time slot.

[0296] Collaborative cache association constraints: Constraint C5 indicates that the collaborative cache association variable is a binary variable; Constraint C6 indicates that the associated drone is associated. For the same target file A collaborative cache association can be established with at most one collaborating node, and only if the collaborating node... The target file has been cached. Related drones A collaborative cache association can only be established when the target file is not cached by the cache itself.

[0297] Bandwidth allocation constraints: Constraint C7 states that the bandwidth ratio allocated by a drone to a user ranges from 0 to 1; Constraint C8 states that the sum of the bandwidth ratios allocated by each drone to its served users must not exceed 1, and only if the user... With drones When establishing service node associations, drones Only then can users Allocate bandwidth resources. When At that time, by achievable .

[0298] Transmit power constraint: Constraint C9 indicates that the total transmit power allocated to each UAV for its service users shall not exceed the uniform maximum transmit power. And only when the user With drones When establishing service node associations, drones Only then can users Allocate transmission power; when At that time, by achievable .

[0299] Drone movement distance constraint: Constraint C10 means that the movement distance of a drone between adjacent time slots does not exceed the product of its maximum flight speed and the duration of a single time slot.

[0300] Drone collision avoidance constraint: Constraint C11 means that the distance between any two drones in any time slot is not less than the minimum safe flight distance. .

[0301] S3: A hierarchical nested dynamic collaborative optimization framework is adopted to decompose the joint optimization problem into three sub-problems;

[0302] Sub-problem A: Under the conditions of fixed cache state variables, resource allocation strategy, and UAV trajectory deployment strategy, optimize the node association strategy using clustering method;

[0303] Subproblem B: Under the conditions of fixed cache state variables, node association strategy, and UAV trajectory deployment strategy, optimize the resource allocation strategy using a continuous convex approximation method;

[0304] Subproblem C: Under the conditions of fixed node association strategy and resource allocation strategy, use mean field multi-agent deep deterministic strategy gradient algorithm to jointly optimize cached state variables and UAV trajectory deployment strategy.

[0305] In this process, subproblems A and B are inner-layer optimizations, and subproblem C is an outer-layer optimization. The results of the inner layer are fed back to the outer layer, and the updated results of the outer layer are applied to the next round of inner-layer optimization, forming a dynamic closed loop.

[0306] S31. Sub-problem A is node association strategy optimization: Under the conditions of fixed cache state variables, resource allocation strategy, and UAV trajectory deployment strategy, the K-Means clustering algorithm is used to determine the service node association between users and UAVs, and the collaborative cache association is determined based on the dynamic collaborative node set, node cache state, and distance between nodes, specifically including:

[0307] For the The round-robin nested dynamic collaborative optimization process is based on cached state variables. Resource allocation strategy and drone trajectory deployment strategy Constructing a subproblem of optimizing node association strategies ,in, Indicates the iteration round index of hierarchical nested dynamic collaborative optimization:

[0308]

[0309]

[0310] in, and They represent the first In the process of dynamic collaborative optimization of nested layers, associated drones and candidate collaborative drones For the file The cache state; due to cache state variables Resource allocation strategy and drone trajectory deployment strategy All variables are fixed in subproblem A; therefore, subproblem A only considers the service node association variables. Associated variables with collaborative cache Node association strategy To optimize variables.

[0311] For the node association strategy optimization subproblem, firstly, the K-Means clustering algorithm is used to cluster dynamically moving users based on their locations, and the service node association between users and drones is determined based on the clustering results. Then, the collaborative cache association is determined based on the dynamic cooperative node set in the current time slot, the node cache state, and the minimum distance criterion, thereby obtaining a node association strategy that satisfies constraints C3-C6. .

[0312] The service node association optimization includes an initialization phase, a clustering iteration phase, and a service drone matching phase.

[0313] Initialization phase: based on time slots Location coordinates selection for internal dynamic mobile users There are 1 initial cluster center, with the number of clusters being the same as the number of drones, forming a set of cluster centers:

[0314]

[0315] in, Indicates time slot Inner The location of the cluster center for each user cluster. This represents the user clustering index.

[0316] Clustering Iteration Phase: Calculate the Euclidean distance between each user and each cluster center, and assign each user to the nearest cluster center, forming... Each user is clustered; the corresponding cluster center is updated based on the arithmetic mean of the location coordinates of all users in each user cluster. The user assignment and cluster center update are repeated until the position change of each cluster center in two adjacent clustering iterations does not exceed the preset clustering convergence threshold.

[0317] Service Drone Matching Phase: After user clustering converges, following the order of cluster center indices, for each cluster center, select the drone with the smallest Euclidean distance from the drones that have not yet established service node relationships with other user clusters. This drone will then be designated as the service drone for that user cluster. Furthermore, establish service node associations between all users within that cluster and the selected drone, and configure the corresponding service node association variables. The associated variables of the remaining service nodes are set to .

[0318] At the beginning of each time slot, the location coordinates of each user are updated according to the user random movement model, and user clustering, cluster center update and service drone matching are re-executed based on the updated user locations to obtain the service node association results for the current time slot.

[0319] After completing the service node association, the maximum cooperation radius is calculated based on the UAV's location in the current time slot and the communication link status, and then the dynamic cooperation node set in step S1 is used. The construction rules determine the drone Candidate collaborating nodes in the current time slot.

[0320] From macro base stations to drones The signal-to-noise ratio (SNR) of the link serves as the minimum SNR condition for inter-UAV cooperative links. The SNR of the inter-UAV cooperative link at the maximum cooperative distance is set to be equal to the SNR of the link from the macro base station to the UAV, i.e.:

[0321] ;

[0322] Based on the above signal-to-noise ratio relationship, the UAV is obtained In the time slot Maximum collaboration radius:

[0323] ;

[0324] When a drone establishes a service node association with a user The target file requested by the user was not cached. At that time, from the dynamic collaborative node set The target file is filtered and cached. Collaborating nodes; if the filter results are not empty, select those that are compatible with drones. Nearest Collaborating Node Establish collaborative cache association; when ,or There is no such condition as satisfying When collaborating with nodes, no collaborative cache association is established.

[0325] S32. Sub-problem B is resource allocation strategy optimization: Under the conditions of fixed cache state variables, node association strategy, and UAV trajectory deployment strategy, a continuous convex approximation method is used to optimize the resource allocation strategy, specifically including:

[0326] For the The cache state variables given during the round-robin nested dynamic collaborative optimization process Node association strategy and trajectory deployment strategy Global cache hit rate does not depend on resource allocation strategy Therefore, the resource allocation strategy optimization in subproblem B can be expressed as a problem of minimizing the average content delivery latency:

[0327] ;

[0328] ;

[0329] The resource allocation strategy uses the bandwidth ratio and transmit power allocated by the UAV to different users as optimization variables. Since the resource allocation to different users is coupled due to constraints from the total bandwidth and total transmit power of the same UAV, to control the worst-case user content delivery latency while also considering latency balance among different users, the maximum user content delivery latency is used as the upper bound of the average content delivery latency. This is because for any feasible resource allocation strategy, the following applies: Therefore, based on problem P3, we construct the maximum user content delivery latency minimization problem P3.1 as an upper bound replacement problem for problem P3:

[0330] ;

[0331] ;

[0332] Introducing auxiliary variables This represents the common upper bound of the delivery latency for all user content:

[0333] ;

[0334] And ensure that the content delivery latency for each user does not exceed Therefore, problem P3.1 can be equivalently rewritten as a minimization problem by introducing a common upper bound auxiliary variable:

[0335]

[0336]

[0337] Wherein, constraint C12 indicates that the content delivery latency for each user does not exceed the maximum user latency auxiliary variable. Define the original spectral efficiency function:

[0338] ;

[0339] The specification for the user content delivery delay expression is as follows:

[0340] ;

[0341] in, This indicates that under the current fixed drone trajectory deployment strategy, the drone... To users The actual spectral efficiency of the link, its value is determined by the drone. For users The allocated transmit power, the small-scale fading coefficient of the communication link, and the distance between nodes are all used to determine the transmission power. Indicates user Content delivery latency under the current fixed cache state variables, node association strategy, and drone trajectory deployment strategy. (Top right corner) This indicates that the corresponding variable or function is related to the resource allocation strategy, but does not indicate exponential operations.

[0342] Then, a spectral efficiency slack variable is introduced. ,make And set:

[0343] ;

[0344] The user is obtained using the spectral efficiency slack variable. Upper bound of content delivery latency:

[0345] ;

[0346] Therefore, constraint C12 is transformed into a time delay constraint after spectral efficiency relaxation. ,in, It still exhibits non-convexity. To process the product term, it is rewritten as an equivalent form of the difference between two convex quadratic functions:

[0347] ;

[0348] product terms This is expressed as the difference between two convex quadratic functions, and a first-order Taylor expansion is performed on the convex function at the current continuous convex approximation iteration point. Let... The index of the internal iteration round of the continuous convex approximation method in subproblem B is given, thus yielding the first iteration round. In each successive convex approximation iteration, a concave lower bound is constructed based on the product of the bandwidth ratio and the spectral efficiency relaxation variable at the current iteration point. :

[0349]

[0350] The continuous convex approximation method is used to obtain the first In the next successive convex approximation iteration, the user Convex upper bound of content delivery latency :

[0351] ;

[0352] Based on the concave lower bound of the product term and the convex upper bound of the user delay mentioned above, the original non-convex problem is constructed into the following convex approximation problem:

[0353]

[0354]

[0355] Set the initial resource allocation results to satisfy bandwidth allocation constraints, transmit power constraints, and positive value conditions. and And calculate the spectral efficiency relaxation variable based on the initial transmit power. ; in the In the next iteration, construct based on the current iteration point. and The problem P3.3 is solved using a convex optimization solver to obtain updated bandwidth ratio, transmit power, and spectral efficiency slack variables. The concave lower bound construction and convex approximation problem solving are repeated until the change in the objective function value between two adjacent iterations does not exceed a preset convergence threshold. The result is then output. Round resource allocation strategy .

[0356] S33. Sub-problem C is the joint optimization of the caching strategy and the UAV trajectory deployment strategy: Under the conditions of fixed node association strategy and resource allocation strategy, the mean-field multi-agent deep deterministic strategy gradient algorithm is used to jointly optimize the caching strategy and the UAV trajectory deployment strategy, specifically including:

[0357] For the In the process of nested optimization, the node association strategy obtained from subproblems A and B. Resource allocation strategy Cache state variables With drone trajectory deployment strategy To optimize the variables, the following optimization problem is constructed:

[0358]

[0359] ;

[0360] Since the above optimization problem involves both discrete buffer state variables and continuous trajectory control variables, and there is a strong coupling relationship between these variables, it belongs to a high-dimensional hybrid non-convex optimization problem. The mean-field multi-agent deep deterministic policy gradient algorithm is used to solve it, specifically including:

[0361] Joint optimization modeling of cache and trajectory based on multi-agent Markov decision process:

[0362] The joint optimization process of collaborative caching and UAV trajectory is modeled as a multi-agent Markov decision process, denoted as […]. ,in, This represents a set of drone agents, where each drone corresponds to an agent with independent decision-making capabilities.

[0363] state space : Intelligent agents in The state at time t is determined by the first The process consists of the round-robin node association strategy, resource allocation strategy, user location, drone location, user request probability, and the drone's current cache state, and is represented as follows:

[0364] ;

[0365] in, Indicates time slot The global state is defined as follows: Since each user's movement direction and speed are generated independently at the beginning of each time slot and are not retained as state variables across time slots, user movement direction and speed are not separately included in the global state. The state space includes the drone's position within each time slot. Cache state before starting The drone intelligent agent, based on the state Generation time slot caching decision ;

[0366] Action space Unmanned Aerial Vehicle (UAV) Intelligent Agent In the time slot The action is represented as , among which, respectively using Indicates drone In the time slot The cache decision vector; Indicates drone In the time slot The direction of movement; Indicates drone In the time slot The distance to move within, and satisfying: Use respectively and This indicates caching policy actions and trajectory deployment actions.

[0367] Observation space Unmanned Aerial Vehicle (UAV) Intelligent Agent In the time slot Local observation global state The local state information that the drone can obtain includes information related to the drone itself. Establish user location and request probability associated with service nodes, and drones The location and current cache status, the location and cache status of dynamic collaborative nodes, and the first In the round-robin node association strategy and resource allocation strategy, and with drones The relevant parts.

[0368] reward function : Indicates that the drone intelligent agent is in the time slot The instant rewards received; among them, This represents the penalty incurred by a drone agent when its candidate trajectory violates the drone movement distance constraint C10 or the collision avoidance constraint C11. The training objective is to maximize the cumulative discount reward. Representing the discount factor, the drone intelligent agent From time slot The initial cumulative discount reward is represented as follows:

[0369] ;

[0370] State transition : Put all drone agents in time slots The combined action is recorded as The state transition probability of a multi-UAV system is expressed as: Each drone intelligent agent executes joint actions. Then, the environment changes from state to state based on the state transition probability. Transition to the next state The state transition is influenced by the user's random movement, user content request events randomly generated based on the request probability, and the drone's trajectory and actions.

[0371] GST-based caching strategy action refactoring:

[0372] For drone intelligent agents In the time slot For the document The caching strategy outputs a binary logistic regression vector from the caching strategy network:

[0373] ,

[0374] in, and These represent files that are not cached. and cache files Action tendency value.

[0375] The binary logistic regression vector is converted into action selection probabilities using the Softmax function.

[0376] ;

[0377] in, Represents unmanned aerial vehicle intelligent agents In the time slot For the document Select the The probability of class caching actions; Indicates the index of the current cached action category. This represents the virtual index of the cache action category in the summation of the denominator, and Category 0 indicates that files are not cached. Category 1 indicates cached files. .

[0378] The Gumbel-Max method is used to perform discrete action sampling on the binary logistic regression values ​​of cache actions:

[0379]

[0380] ;

[0381] in, It is independently sampled Gumbel noise. It is a One-Hot encoding; when When, it means ;when When, it indicates the caching strategy. ;

[0382] To enable discrete cache actions to be trained within the policy gradient framework, two deterministic perturbation functions are introduced. and The deterministic perturbation function is used to adjust the binary action logic value vector so that the maximum action category corresponding to the continuous approximation result is consistent with the discrete sampling result, and a preset numerical difference is formed between the selected category and the unselected category, thereby obtaining a continuous and differentiable GST approximation function:

[0383] ;

[0384] Since the caching policy network generates caching actions for each file separately, the initial caching actions for each file may not meet the drone's cache capacity constraints. Therefore, capacity feasibility processing is performed on the initial caching actions:

[0385] First, satisfy The document is used as a drone In the time slot Candidate cache files are selected based on the probability of the cache action. Sort the files from largest to smallest; then, evaluate the candidate files according to the sorting results, ensuring that the total file data size after inclusion does not exceed the cache capacity. The file set as the final cache file will cause the cumulative file data size to exceed the cache capacity after being added. Files that are not selected as candidate cache files are set to not be cached; all other files that are not selected as candidate cache files are also set to not be cached.

[0386] After capacity feasibility processing, the drone In the time slot The final caching decision satisfies:

[0387] ;

[0388] ;

[0389] The cached state variables, after capacity feasibility processing, are used to form the drone. In the time slot The final cache action vector:

[0390] ;

[0391] For continuous trajectory actions Normalize each component according to its allowed value range:

[0392] ;

[0393] The normalized continuous trajectory motion satisfies:

[0394] ;

[0395] in, and These represent the minimum and maximum allowable values ​​for continuous trajectory actions, respectively; the subtraction and division operations are performed according to the corresponding components of the action vector; candidate positions of the UAV are calculated based on the normalized direction and distance of movement; when a candidate position violates constraint C10 or C11, the candidate position is made feasible and adjusted to a feasible position that satisfies the constraints; if a feasible adjustment position cannot be obtained, the UAV is kept at the position of the previous time slot.

[0396] MF-MADDPG cache and trajectory joint optimization algorithm:

[0397] Each unmanned aerial vehicle (UAV) agent is equipped with an online Actor network. Online Critic Network and the corresponding target network and ;in, , , and These represent the network parameters of the online Actor network, online Critic network, target Actor network, and target Critic network, respectively. Represents unmanned aerial vehicle intelligent agents In the time slot The obtained local observations; Indicates time slot The global state; Represents unmanned aerial vehicle intelligent agents The combined action of executing the cache and the trajectory; Indicates the intelligent agent of drones The average action of the interactive collaborative agents.

[0398] Centralized Training and Distributed Execution: During training, each UAV agent sends its local observations and actions to the macro base station. The macro base station then uses the information from each UAV agent to form a global state and the joint action information required for training. The Critic network of each UAV agent is trained centrally using the global state, its own actions, and the average actions of the collaborating agents as inputs. The Actor network of each UAV agent is trained using its own local observations as inputs. After training, each UAV agent no longer relies on the complete global state; its online Actor network generates cached actions and continuous trajectory actions only based on its own local observations, thus achieving distributed execution. The cached action logic values ​​output by the online Actor network are processed for discrete action reconstruction and cache capacity feasibility using the GST method in F2, and continuous trajectory actions are mapped according to their corresponding allowed value ranges.

[0399] Mean field dimensionality reduction: for drone intelligent agents Combine it with the set of dynamic collaborative nodes The interactions with other agents are approximated by the interactions with the average agent, whose actions are represented as follows:

[0400]

[0401] in, Indicates time slot Internal dynamic collaborative node set The number of collaborative drones included; Represents collaborative drone intelligent agents In the time slot The execution of the cache and trajectory joint action; when At that time, The default action is set to zero; through the mean field approximation, the online Critic network does not need to directly input the actions of all UAV agents within the dynamic cooperative node set, but instead uses the average agent action to characterize the actions of other agents towards the UAV agents. The combined impact.

[0402] Exploration noise: During the training phase, exploration noise, sampled from a Gaussian distribution with a mean of 0, is added to the continuous trajectory actions output by the online Actor network. The amplitude of the exploration noise is gradually reduced as the training epochs increase. For discrete cached actions, action exploration is performed using Gumbel-Max sampling and GST reparameterization mechanisms in D2. The exploration noise is no longer added during the distributed execution phase after training is complete.

[0403] Experience replay: The global state, local observations, current actions, average agent actions, immediate rewards, next time-slot global state, and next time-slot local observations generated by the interaction between each UAV agent and the environment are stored in a shared experience replay buffer. During training, small batches of experience data are randomly extracted from the shared experience replay buffer for parameter updates of the online Actor network and the online Critic network.

[0404] Network Update: The Critic network constructs a temporal difference objective based on the current immediate reward and the target Critic network's value estimate for the next state, and updates its network parameters by minimizing the temporal difference error. The Actor network updates its network parameters using a deterministic policy gradient update. The target Actor network and the target Critic network update their network parameters using a soft update method. The process of environmental interaction, experience storage, mini-batch experience sampling, online network parameter updates, and target network soft updates is repeated until the preset training termination condition is met. Based on the output of the trained Actor network, a caching strategy that satisfies the cache capacity constraint, the UAV movement distance constraint, and the collision avoidance constraint is obtained. and drone trajectory deployment strategy The strategy will then be used in the next round of hierarchical nested dynamic collaborative optimization process.

[0405] The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the specific details of the above embodiments. Within the scope of the technical concept of the present invention, various simple modifications can be made to the technical solution of the present invention, and these simple modifications all fall within the protection scope of the present invention.

[0406] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present invention will not describe the various possible combinations separately.

[0407] Furthermore, various different embodiments of the present invention can be combined in any way, as long as they do not violate the spirit of the present invention, they should also be regarded as the content disclosed by the present invention.

Claims

1. A UAV air-to-ground collaborative caching method for dynamic user requests, characterized in that, Includes the following steps: S1: Establish an air-ground collaborative caching system model consisting of macro base stations, multiple drones, and dynamic mobile users; construct a collaborative network caching, user random movement, and edge caching communication model. S2: Define caching strategy, node association strategy, resource allocation strategy and UAV trajectory deployment strategy. With global cache hit rate and average content delivery latency as optimization objectives, construct a joint optimization problem that satisfies cache capacity constraints, user single association constraints, constraints to avoid repeated collaborative transmission, bandwidth allocation constraints, transmission power constraints, UAV movement distance constraints and UAV collision avoidance constraints. S3: A hierarchical nested dynamic collaborative optimization framework is adopted to decompose the joint optimization problem into three sub-problems; Sub-problem A: Under the conditions of fixed cache state variables, resource allocation strategy, and UAV trajectory deployment strategy, optimize the node association strategy using clustering method; Subproblem B: Under the conditions of fixed cache state variables, node association strategy, and UAV trajectory deployment strategy, optimize the resource allocation strategy using a continuous convex approximation method; Subproblem C: Under the conditions of fixed node association strategy and resource allocation strategy, use mean field multi-agent deep deterministic strategy gradient algorithm to jointly optimize cached state variables and UAV trajectory deployment strategy; In this process, subproblems A and B are inner-layer optimizations, and subproblem C is an outer-layer optimization. The results of the inner-layer optimizations are fed back to the outer-layer optimizations, and the updated results of the outer-layer optimizations are applied to the next round of inner-layer optimizations, forming a dynamic closed loop. In step S2, the optimization objectives of global cache hit rate and average content delivery latency include the following: C1. Define the caching strategy, node association strategy, resource allocation strategy, and drone trajectory deployment strategy as follows: ； ； ； ； in, This represents the caching strategy comprised of the cached state variables of each drone for each file; This represents the node association strategy, which consists of user-drone service node association variables and drone inter-drone collaborative cache association variables. This refers to the resource allocation strategy, which consists of the proportion of bandwidth and transmission power allocated by the drone to the user. This represents the drone trajectory deployment strategy, which consists of the position coordinates of each drone in different time slots. C2. Based on the above set of strategies, with global cache hit rate and average content delivery latency as optimization objectives, the following joint optimization problem is constructed: ； in, Indicates by caching strategy Node association strategy The system's global cache hit rate is determined jointly. Indicates user Overall content delivery latency under given caching strategy, node association strategy, resource allocation strategy, and drone trajectory deployment strategy; This represents the cache hit rate weight adjustment coefficient, used to coordinate the dimensionless global cache hit rate with the time-based average content delivery latency in the objective function. C3. The joint optimization problem satisfies the following constraints: ； ； ； Among them, the cache state constraint: constraint C1 represents that the cache state of any drone for any file is a binary variable; when the drone When acting as a collaborative node, it accesses files The corresponding cache state is represented as ; Cache capacity constraint: Constraint C2 indicates that the total amount of data cached by each drone must not exceed its maximum cache capacity. ; Service node association constraints: Constraint C3 indicates that the service node association variable is a binary variable; Constraint C4 indicates that each user establishes a service node association with only one drone in the same time slot; Collaborative cache association constraints: Constraint C5 indicates that the collaborative cache association variable is a binary variable; Constraint C6 indicates that the associated drone is associated. For the same target file A collaborative cache association can be established with at most one collaborating node, and only if the collaborating node... The target file has been cached. Related drones A collaborative cache association can only be established if the target file is not cached by the cache itself. Bandwidth allocation constraints: Constraint C7 states that the bandwidth ratio allocated by a drone to a user ranges from 0 to 1; Constraint C8 states that the sum of the bandwidth ratios allocated by each drone to its served users must not exceed 1, and only if the user... With drones When establishing service node associations, drones Only then can users Allocate bandwidth resources; when At that time, by achievable ; Transmit power constraint: Constraint C9 indicates that the total transmit power allocated to each UAV for its service users shall not exceed the uniform maximum transmit power. And only when the user With drones When establishing service node associations, drones Only then can users Allocate transmission power; when At that time, by achievable ; Drone movement distance constraint: Constraint C10 means that the movement distance of a drone between adjacent time slots does not exceed the product of its maximum flight speed and the duration of a single time slot; Drone collision avoidance constraint: Constraint C11 means that the distance between any two drones in any time slot is not less than the minimum safe flight distance. ; In step S1, the set of dynamic mobile users is represented as ,in, Represents a user index; multiple drones form a drone ensemble. ,in, The drone index is used to provide content delivery services for dynamically mobile users; the macro base station is used to cache content files; the set of content files is represented as... ,in, Indicates file index, content file Data size is represented as The discrete time slot model is adopted, and the set of discrete time slots is represented as follows: ,in, This represents the slot index, and the duration of a single slot is expressed as... ;in, , , and These represent the number of users, the number of drones, the number of cached files, and the total number of discrete time slots, respectively. The construction of the air-ground collaborative caching system model includes the following: A three-tiered content collaborative transmission mechanism is constructed, including local cache direct transmission, collaborative cache relay, and base station source supplementation; The local cache direct transmission means that when the associated drone caches the content file requested by the dynamic mobile user, the associated drone directly delivers it to the dynamic mobile user. The collaborative caching relay is as follows: when the associated drone does not cache the content file requested by the dynamic mobile user, but the collaborative node caches the content file, the collaborative node transmits the content file to the associated drone, and then the associated drone forwards the content file to the dynamic mobile user; wherein, the collaborative node refers to other drones located in the set of dynamic collaborative nodes of the associated drone, which establish a communication link with the associated drone and provide it with content files; The base station back-to-source supplementation is as follows: when neither the associated drone nor its cooperating nodes have cached the content file, the macro base station transmits the content file to the associated drone, and the associated drone then forwards it. In the aforementioned air-to-ground collaborative caching system model, users are associated with corresponding drones based on their location; using This represents the service node associated variable, where, Indicates user With drones Establish service node associations. Indicates user With drones No service node association was established; the service node refers to the drone that has established a service node association with the dynamic mobile user and provides content delivery services to the dynamic mobile user; the drone redeploys its location based on the user's movement; and Representing users respectively and drones In the time slot The position coordinates within; where, and Representing users respectively In the time slot The x and y coordinates; and They represent drones In the time slot The x and y coordinates; Indicates drone The fixed safe flight altitude; the fixed location coordinates of the macro base station are... ,in, and These represent the x and y coordinates of the macro base station, respectively. In step S1, a collaborative network cache model is constructed in the air-ground collaborative caching system model to determine the cache status of each UAV and the collaborative transmission relationship between UAVs, so as to support the local cache direct transmission and collaborative cache relay in the three-level content collaborative transmission mechanism. The construction of the cooperative network caching model includes the following: A1. Construct a multi-drone collaborative caching network, where multiple drones share content and transmit it collaboratively through a collaborative caching mechanism; define the drone cache state variable as follows. The drone cache state variable is used to indicate the drone Should the requested content file be cached? ,in, Indicates drone Cached content files , Indicates drone Uncached content files And establish drone cache capacity constraints: ;in, Indicates the maximum cache capacity of the drone; Represents content file Data size; A dynamic set of cooperative nodes is constructed based on the spatial relationships and communication link quality among the drones; the dynamic set of cooperative nodes is represented as follows: ； in, Indicates the index of candidate collaborative drones; Indicates drone Collaborating with candidate drones In the time slot The Euclidean distance; Indicates drone In the time slot The maximum cooperation radius is calculated based on the current node location and communication link status. ； in, Indicates drone Euclidean distance between the macro base station and the macro base station; Indicates drone Collaborating with candidate drones The Euclidean distance between them; Indicates macro base station to drone A fixed transmission power is used when transmitting content; Indicates drone To drones A fixed transmit power is used when transmitting collaborative buffer content; Macro base stations and drones Small-scale fading coefficient of the communication link between them; Indicates the channel power gain at the reference distance; The path loss index is represented; the collaborative link between UAVs adopts a normalized line-of-sight propagation model. Define a collaborative cache association variable to indicate whether a collaborative cache association has been established between the drone and the collaborative node for the target file. The collaborative cache association variable is represented as follows: ； in, Indicates associated drones In collaboration with nodes For content files Establish collaborative cache associations. This indicates that no collaborative cache association has been established; the collaborative cache association must meet the following conditions: ； ； The first constraint indicates that only if the cooperating node Cache content files At that time, establish a collaborative cache association; the second constraint indicates that only when the drone Uncached content files At that time, establish collaborative cache associations; When drones Uncached content files At that time, from the dynamic collaborative node set The cached files contain the aforementioned content. The collaborative nodes are selected, and the closest collaborative node is chosen to establish a collaborative cache association; the collaborative cache association also satisfies: This constraint indicates that the drone For the same content file It can establish a collaborative cache association with at most one collaborating node; A2. A content popularity model is established using the Zipf distribution, and users... For content files The request probability is represented as follows: ； in, Indicates user Request content file The probability of; This represents the Zipf parameter, which characterizes the concentration of content request probabilities. This represents the virtual index of the file in the summation of the denominator; according to the above model, each user uses the same Zipf content popularity distribution; A3. Calculate the number of local cache hits for the drone and the number of cache hits for the dynamic collaborative node set based on the aforementioned request probabilities, wherein the drone... Local cache hit count: ； Dynamic collaborative node set The cache hit count is expressed as: ； A4. Based on the local cache hit count and the cache hit count of the dynamic collaborative node set, the global cache hit rate of the collaborative network is obtained as follows: ； The collaborative network caching model is defined by the UAV cache state variables, UAV cache capacity constraints, the dynamic collaborative node set, collaborative cache association variables and their constraints, and the content popularity model, forming a complete collaborative network caching model. In step S1, the edge cache communication model is constructed, including the following: B1. Small-scale fading is represented using the Ricean channel model as follows: ； in, Represents the Rice factor, used to characterize the ratio of the line-of-sight component power to the non-line-of-sight scattering component power; and Representing nodes respectively With nodes The LOS and NLOS components of the communication link between them; and These represent the node indices at both ends of the communication link; B2. Constructing the channel coefficient model for the two ends of the communication link: ； in, Represents communication node and Complex channel coefficients of the communication link between them. and These represent the position coordinates of the nodes at both ends of the communication link. This indicates the distance path loss of the corresponding communication link; B3. Establish the signal-to-noise ratio (SNR) model from the drone to the user, the SNR model from the base station to the drone, and the SNR model between drones, respectively: ； ； ； in, Indicates drone To users The signal-to-noise ratio of the link; Indicates macro base station to drone The signal-to-noise ratio of the link; Indicating collaborative drones To drones The signal-to-noise ratio of the link; Indicates drone For users Allocated transmission power; Indicates macro base station to drone A fixed transmission power is used when transmitting content; Indicating collaborative drones To drones A fixed transmission power is used when transmitting content; This represents the noise power at the receiving end; the cooperative link between UAVs adopts a normalized line-of-sight propagation model. B4. Establish the transmission rate models for drone-to-user, base station-to-drone, and drone-to-drone transmission rates, respectively: ； ； ； in, This indicates the transmission bandwidth occupied by the link from the macro base station to the drone; This represents the total bandwidth between the drone and ground users. It is the subcarrier frequency division coefficient. It is the bandwidth between the drone and the collaborative network. A virtual index for summing collaborative nodes, used solely for drone statistics. The number of currently established collaborative caching links; Only Used when there is no collaborative cache association; when there is no collaborative cache association, the corresponding collaborative network transmission latency term is set to 0 and not calculated. ; B5. Define the overall content delivery latency for users as follows: ； in, ; ； ； This indicates the latency of content transmission between the drone and the user. Indicates transmission latency within a collaborative network. This indicates the transmission delay between the base station and the drone.

2. The UAV air-to-ground collaborative caching method for dynamic user requests as described in claim 1, characterized in that, In step S1, a user random movement model is constructed, including the following: Set the user's maximum movement area to ,in, and These represent the maximum range of the user's movement area in the horizontal and vertical directions, respectively; user In the time slot The direction of movement is Movement speed is ,in, Indicates user Maximum movement speed; when When the current time slot is filled, it indicates that the user remains stationary; at the start of the next time slot, the user's movement direction and speed are regenerated independently. Configure User In the time slot The position coordinates are The user during the duration of Moving at a constant speed within a time slot, the user's position coordinates in the next time slot are: ; Setting up drones Maximum flight speed is The flight constraints for the drone are: ; Establish minimum safe flight distance between drones ,ensure To avoid collisions between drones.

3. The UAV air-to-ground collaborative caching method for dynamic user requests as described in claim 1, characterized in that, In step S3, sub-problem A specifically includes: D1. Variables associated with service nodes Associated variables with collaborative cache To optimize variables, cache state variables. Resource allocation strategy and drone trajectory deployment strategy Under the condition that all nodes are fixed, construct a node association strategy. Optimization subproblems :in, Indicates the iteration round index of hierarchical nested dynamic collaborative optimization: ； ； in, and They represent the first In the process of dynamic collaborative optimization of nested layers, associated drones and candidate collaborative drones For content files The cache state; D2. Solve the subproblem of node association strategy optimization, and perform service node association optimization and collaborative cache association optimization in sequence: D21. Service Node Association Optimization: The K-Means clustering algorithm is used to cluster dynamically moving users based on their location, and the service node association between users and drones is determined based on the clustering results. Then, the collaborative cache association is determined based on the dynamic collaborative node set of the current time slot, the node cache state, and the minimum distance criterion, thus obtaining a node association strategy that satisfies constraints C3-C6. ; Specifically, this includes: the initialization phase, the clustering iteration phase, and the service drone matching phase; Initialization phase: based on time slots Location coordinates selection for internal dynamic mobile users There are 1 initial cluster center, with the number of clusters being the same as the number of drones, forming a set of cluster centers: ； in, Indicates time slot Inner The location of the cluster center for each user cluster. Represents a user clustering index; Clustering Iteration Phase: Calculate the Euclidean distance between each user and each cluster center, and assign each user to the nearest cluster center, forming... Each user is clustered; the corresponding cluster center is updated based on the arithmetic mean of the location coordinates of all users in each user cluster. The user assignment and cluster center update are repeated until the position change of each cluster center in two adjacent clustering iterations does not exceed the preset clustering convergence threshold. Service Drone Matching Phase: After user clustering converges, following the order of cluster center indices, for each cluster center, select the drone with the smallest Euclidean distance to that cluster center from among the drones that have not yet established service node relationships with other user clusters. This drone will then be designated as the service drone for that user cluster. Furthermore, establish service node associations between all users within that cluster and the selected drone, and configure the corresponding service node association variables. The associated variables of the remaining service nodes are set to ; At the beginning of each time slot, the location coordinates of each user are updated according to the user random movement model, and user clustering, cluster center update and service drone matching are re-executed based on the updated user locations to obtain the service node association results for the current time slot. D22. Collaborative Cache Association Optimization: Based on the completion of service node association, calculate the maximum collaborative radius according to the UAV's location in the current time slot and the communication link status, and follow the dynamic collaborative node set in step S1. The construction rules determine the drone Candidate collaborating nodes in the current time slot; From macro base stations to drones The signal-to-noise ratio (SNR) of the link serves as the minimum SNR condition for inter-UAV cooperative links. The SNR of the inter-UAV cooperative link at the maximum cooperative distance is set to be equal to the SNR of the link from the macro base station to the UAV, i.e.: ； Based on the above signal-to-noise ratio relationship, the UAV is obtained In the time slot Maximum collaboration radius: ； When a drone establishes a service node association with a user The content file requested by the user was not cached. At that time, from the dynamic collaborative node set The cached files contain the aforementioned content. Collaborating nodes; if the filter results are not empty, select those that are compatible with drones. Nearest Collaborating Node Establish collaborative cache association; when ,or There is no such condition as satisfying When collaborating with nodes, no collaborative cache association is established.

4. The UAV air-to-ground collaborative caching method for dynamic user requests as described in claim 3, characterized in that, In step S3, sub-problem B specifically includes: E1. Using bandwidth ratio allocation variables and transmit power allocation variables as optimization variables, and considering the buffer state variables... Node association strategy and drone trajectory deployment strategy Under the condition that all resources are fixed, a subproblem of optimizing resource allocation strategy is constructed. The subproblem of optimizing resource allocation strategy aims to minimize the average content delivery latency, and is expressed as: ； ； E2. Using the maximum user content delivery latency as the upper bound of the average content delivery latency, based on the resource allocation strategy optimization sub-problem P3, construct an upper bound replacement problem P3.1 that minimizes the maximum user content delivery latency: ； ； ； Among them, auxiliary variables are introduced. This represents the common upper bound of the delivery latency for all user content; And ensure that the content delivery latency for each user does not exceed Therefore, problem P3.1 can be equivalently rewritten as a minimization problem by introducing a common upper bound auxiliary variable: ； ； Wherein, constraint C12 indicates that the content delivery latency for each user does not exceed the maximum user latency auxiliary variable. ; E3. Define the original spectral efficiency function: ； The specification for the user content delivery delay expression is as follows: ； in, This indicates that under the current fixed drone trajectory deployment strategy, the drone... To users The actual spectral efficiency of the link, its value is determined by the drone. For users The allocated transmit power, the small-scale fading coefficient of the communication link, and the distance between nodes are all used to determine the transmission power. Indicates user Content delivery latency under the current fixed cache state variables, node association strategy, and drone trajectory deployment strategy; E4. Introducing spectral efficiency slack variables ,make And set: ； The user is obtained using the spectral efficiency slack variable. Upper bound of content delivery latency: ； At this point, constraint C12 is transformed into a time delay constraint after spectral efficiency relaxation. ,in, It still exhibits non-convexity. To process the product term, it is expressed as an equivalent form of the difference between the following two convex quadratic functions: ； product terms Let be the difference between two convex quadratic functions, and perform a first-order Taylor expansion of the convex function at the current continuous convex approximation iteration point; let The index of the internal iteration round of the continuous convex approximation method in subproblem B is given, thus yielding the first iteration round. In each successive convex approximation iteration, a concave lower bound is constructed based on the product of the bandwidth ratio and the spectral efficiency relaxation variable at the current iteration point. : ； The continuous convex approximation method is used to obtain the first In the next successive convex approximation iteration, the user Convex upper bound of content delivery latency : ； E5. Based on the concave lower bound of the product term and the convex upper bound of the user delay mentioned above, the original non-convex problem is constructed into the following convex approximation problem: ； ； Set the initial resource allocation results to satisfy bandwidth allocation constraints, transmit power constraints, and positive value conditions. and And calculate the spectral efficiency relaxation variable based on the initial transmit power. ; in the In the next iteration, construct based on the current iteration point. and The problem P3.3 is solved using a convex optimization solver to obtain the updated bandwidth ratio, transmit power, and spectral efficiency relaxation variables. The concave lower bound construction and convex approximation problem solving are repeated until the objective function value changes between two adjacent iterations do not exceed the preset convergence threshold, and the resource allocation strategy for the current round is output.

5. A UAV air-to-ground collaborative caching method for dynamic user requests as described in claim 4, characterized in that, In step S3, the sub-problem C specifically includes: For the In the process of nested optimization, the node association strategy obtained from subproblems A and B. Resource allocation strategy Cache state variables With drone trajectory deployment strategy To optimize the variables, the following optimization problem is constructed: ； ； Since the above optimization problem includes both discrete buffer state variables and continuous trajectory control variables, and there is a strong coupling relationship between the variables, it belongs to a high-dimensional hybrid non-convex optimization problem. The mean-field multi-agent deep deterministic policy gradient algorithm is used to solve it. F1. Cache and trajectory joint optimization modeling based on multi-agent Markov decision process: The joint optimization process of collaborative caching and UAV trajectory is modeled as a multi-agent Markov decision process, denoted as […]. ,in, This represents a set of drone agents, where each drone corresponds to an agent with independent decision-making capabilities. state space : Intelligent agents in The state at time t is determined by the first The process consists of the round-robin node association strategy, resource allocation strategy, user location, drone location, user request probability, and the drone's current cache state, and is represented as follows: ； in, Indicates time slot The global state; the state space includes the UAV's state in the time slot. Cache state before starting The drone intelligent agent, based on the state Generation time slot caching decision ; Action space Unmanned Aerial Vehicle (UAV) Intelligent Agent In the time slot The action is represented as , among which, respectively using Indicates drone In the time slot The cache decision vector; Indicates drone In the time slot The direction of movement; Indicates drone In the time slot The distance to move within, and satisfying: Use respectively and This indicates caching policy actions and trajectory deployment actions; Observation space Unmanned Aerial Vehicle (UAV) Intelligent Agent In the time slot Local observation global state The local state information obtained by the drone includes information related to the drone itself. Establish user location and request probability associated with service nodes, and drones The location and current cache status, the location and cache status of dynamic collaborative nodes, and the first In the round-robin node association strategy and resource allocation strategy, and with drones The relevant parts; reward function : Indicates that the drone intelligent agent is in the time slot The instant rewards received; among them, This represents the penalty incurred by a drone agent when its candidate trajectory violates the drone movement distance constraint C10 or the collision avoidance constraint C11. The training objective is to maximize the cumulative discount reward. Representing the discount factor, the drone intelligent agent From time slot The initial cumulative discount reward is represented as follows: ； State transition : Put all drone agents in time slots The combined action is recorded as The state transition probability of a multi-UAV system is expressed as: The various drone agents execute joint actions. Then, the environment changes from state to state based on the state transition probability. Transition to the next state The state transition is jointly influenced by the user's random movement, user content request events randomly generated based on the request probability, and the drone's trajectory and actions; F2. GST-based caching strategy action refactoring: F21, for drone intelligent agents In the time slot For the document The caching strategy outputs a binary logistic regression vector from the caching strategy network: ； in, and These represent files that are not cached. and cache files Action tendency value; The binary logistic regression vector is converted into action selection probabilities using the Softmax function. ； in, Represents unmanned aerial vehicle intelligent agents In the time slot For the document Select the The probability of class caching actions; Indicates the index of the current cached action category. This represents the virtual index of the cache action category in the summation of the denominator, and Category 0 indicates that files are not cached. Category 1 indicates cached files. ; F22. Use the Gumbel-Max method to perform discrete action sampling on the binary logistic regression values ​​of cache actions: ； ； in, It is independently sampled Gumbel noise. It is a One-Hot encoding; when When, it means ;when When, it indicates the caching strategy. ; F23. Introduce two deterministic perturbation functions. and The deterministic perturbation function is used to adjust the binary action logic value vector so that the maximum action category corresponding to the continuous approximation result is consistent with the discrete sampling result, and a preset numerical difference is formed between the selected category and the unselected category, thereby obtaining a continuous and differentiable GST approximation function: ； F24. Perform capacity feasibility processing on the initial caching operation: First, satisfy The document is used as a drone In the time slot Candidate cache files are selected based on the probability of the cache action. Sort the files from largest to smallest; then, evaluate the candidate files according to the sorting results, ensuring that the total file data size after inclusion does not exceed the cache capacity. The file set as the final cache file will cause the cumulative file data size to exceed the cache capacity after being added. Files that were not selected as candidate cache files are set to not be cached; all other files that were not selected as candidate cache files are also set to not be cached. After capacity feasibility processing, the drone In the time slot The final caching decision satisfies: ； ； The cached state variables, after capacity feasibility processing, are used to form the drone. In the time slot The final cache action vector: ； F25, for continuous trajectory actions Normalize each component according to its allowed value range: ； The normalized continuous trajectory motion satisfies: ； in, and These represent the minimum and maximum allowable values ​​for continuous trajectory actions, respectively; the subtraction and division operations are performed according to the corresponding components of the action vector; candidate positions of the UAV are calculated based on the normalized direction and distance of movement; when a candidate position violates constraint C10 or C11, the candidate position is made feasible and adjusted to a feasible position that satisfies the constraints; if a feasible adjustment position cannot be obtained, the UAV is kept at the position of the previous time slot. F3, MF-MADDPG cache and trajectory joint optimization algorithm: F31, each drone agent is equipped with an online Actor network. Online Critic Network and the corresponding target network and ;in, , , and These represent the network parameters of the online Actor network, online Critic network, target Actor network, and target Critic network, respectively. Represents unmanned aerial vehicle intelligent agents In the time slot The obtained local observations; Indicates time slot The global state; Represents unmanned aerial vehicle intelligent agents The combined action of executing the cache and the trajectory; Indicates the intelligent agent of drones The average action of the interactive collaborative agents; F32, Centralized Training vs. Distributed Execution: During training, each UAV agent sends its local observations and actions to the macro base station. The macro base station forms a global state and joint action information required for training based on the information from each UAV agent. The Critic network of each UAV agent is trained centrally with the global state, its own actions, and the average actions of the cooperating agents as inputs. The Actor network of each UAV agent is trained with its own local observations as inputs. After training, each UAV agent no longer relies on the complete global state. Its online Actor network generates cached actions and continuous trajectory actions only based on its own local observations, thereby achieving distributed execution. The cached action logic values ​​output by the online Actor network are processed for discrete action reconstruction and cache capacity feasibility according to the GST method in F2, and the continuous trajectory actions are mapped according to the corresponding allowed value range. F33, Mean Field Dimensionality Reduction: For Unmanned Aerial Vehicle Agents Combine it with the set of dynamic collaborative nodes The interactions with other agents are approximated by the interactions with the average agent, whose actions are represented as follows: ； in, Indicates time slot Internal dynamic collaborative node set The number of collaborative drones included; Represents collaborative drone intelligent agents In the time slot The execution of the cache and trajectory joint action; when = At that time, The default action is set to zero; through the mean field approximation, the online Critic network does not need to directly input the actions of all UAV agents within the dynamic cooperative node set, but instead uses the average agent action to characterize the actions of other agents towards the UAV agents. The combined impact; F34. Exploration Noise: During the training phase, exploration noise sampled from a Gaussian distribution with a mean of 0 is added to the continuous trajectory actions output by the online Actor network, and the amplitude of the exploration noise is gradually reduced as the training rounds increase; for discrete cached actions, action exploration is performed through Gumbel-Max sampling and GST reparameterization mechanism in D2; the exploration noise is no longer added in the distributed execution phase after training is completed. F35, Experience Replay: The global state, local observations, current actions, average agent actions, instantaneous rewards, global state and local observations of the next time slot generated by the interaction between each UAV agent and the environment are stored in a shared experience replay buffer; during training, a small batch of experience data is randomly extracted from the shared experience replay buffer for parameter updates of the online Actor network and the online Critic network. F36. Network Update: The Critic network constructs a time-difference objective based on the current immediate reward and the target Critic network's value estimate for the next state, and updates its network parameters by minimizing the time-difference error; the Actor network updates its network parameters using a deterministic policy gradient; the target Actor network and the target Critic network update their network parameters using a soft update method; the process of environmental interaction, experience storage, mini-batch experience sampling, online network parameter updates, and target network soft updates is repeated until the preset training termination condition is met; based on the output of the Actor network after training, a caching strategy that satisfies the cache capacity constraint, the UAV movement distance constraint, and the collision avoidance constraint is obtained. and drone trajectory deployment strategy The strategy will then be used in the next round of hierarchical nested dynamic collaborative optimization process.

Images