Multi-unmanned aerial vehicle cooperative caching and trajectory planning method based on time delay and energy perception

Abstract

The invention discloses a multi-unmanned aerial vehicle cooperative caching and trajectory planning method based on time delay and energy perception, and the method comprises the steps: constructing a multi-unmanned aerial vehicle cooperative caching network architecture, and defining an unmanned aerial vehicle and user movement model, a content caching model, a communication model, and an energy consumption model; constructing an optimization problem which takes the limited battery energy of the unmanned aerial vehicle as a constraint and minimizes the sum of content retrieval delays of all users as a target; secondly, classifying the users by adopting a clustering algorithm based on K-means according to the diversity of geographic positions and content requirements of the users; decomposing the proposed time delay optimization problem into a plurality of sub-problems according to a divide-and-conquer method principle, and solving the sub-problems; and finally, an efficient iterative algorithm is introduced, and an alternate optimization method is adopted for the sub-problems until convergence. The method is easy to operate and high in practicability, and low-delay mobile user content delivery is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of delay optimization, and particularly relates to a delay and energy-aware multi-UAV cooperative caching and trajectory planning method. Background Art

[0002] The UAV caching network can solve the problem that the mobile edge caching network cannot meet the user's demand for hot content in a timely manner in specific scenarios such as natural disasters and temporary hot events. However, the caching capacity of UAVs is limited, and only some hot content can be stored. The content that is not cached still needs to be frequently downloaded from the macro base station, which will cause a large communication overhead. To alleviate this problem, a multi-UAV cooperative caching system has emerged. However, there are still multiple challenges in achieving low-latency content delivery in a multi-UAV cooperative caching system. The primary challenge lies in how to design an efficient cooperative caching placement strategy to achieve a high content hit rate on the basis of considering multiple factors such as the diverse content demands of different users, the caching capacity of UAVs, and the cooperation among multiple UAVs. Secondly, the heterogeneity of caching resources and battery capacity among UAVs, as well as the diversity of user geographical locations and content demands, further increases the complexity of the UAV-user association mechanism. Moreover, user mobility poses higher requirements for UAV trajectory planning. UAVs need to dynamically adjust their flight trajectories according to the user's movement trajectory, and UAV trajectory planning is a non-convex optimization problem with continuous variables, which is difficult to solve by conventional optimization methods. Finally, the limited on-board battery energy of UAVs results in a limited service time, which makes it a challenge to balance the service time of UAVs and the quality of experience of users. Therefore, there is an urgent need to design an optimization method that takes into account UAV-user association, cooperative caching placement, and trajectory planning to achieve low-latency content delivery under the constraint of limited battery energy of UAVs.

[0003] Some scholars at home and abroad have begun to pay attention to the UAV wireless caching network, aiming to improve the user experience by optimizing the caching placement strategy and UAV trajectory planning. The literature (T. Zhang, Y. Wang, W. Yi, Y. Liu, and A. Nallanathan, "Joint Optimization of Caching Placement and Trajectory for UAV-D2D Networks," in IEEE Transactions on Communications, vol. 70, no. 8, pp. 5514-5527, 2022.) proposed a collaborative caching architecture between UAVs and user terminals, and used a multi-to-multi exchange matching algorithm, approximate convex optimization, and dynamic programming methods to optimize the user terminal caching placement, UAV trajectory, and UAV caching placement respectively, in order to achieve the purpose of providing high transmission rate content delivery and personalized video viewing quality for hot spots. To minimize the download content delay, the literature (J. Chen, F. Tan, H. Chen, and S. Li, "Caching Strategy and Resource Allocation in Cache-Enabled UAV Networks: A Deep Reinforcement Learning Approach," in 2024 IEEE / CIC International Conference on Communications in China, pp. 283-288, 2024.) studied the joint optimization problem of caching decision and resource allocation in the UAV wireless caching network, and designed a deep reinforcement learning framework based on double-clip proximal policy optimization (PPO) to dynamically adjust the caching strategy, UAV trajectory, and transmission power. Considering the energy limitation of UAVs, the literature (Y. Xiao, Z. Lin, X. Cao, Y. Chen, and X. Lu, "AoI-Energy-Efficient Edge Caching in UAV-Assisted Vehicular Networks," in IEEE Internet of Things Journal, 2024, doi: 10.1109 / JIOT.2024.3492535.) studied the joint optimization problem in the cacheable UAV-assisted vehicle integration network for remote suburban scenarios with non-physical infrastructure, and solved the cache refresh period and cache content placement through an alternating optimization algorithm, so as to minimize the weighted sum of the content information age and UAV energy consumption.However, the above work ignores the cooperative caching among UAVs and fails to fully exploit the distributed advantages of UAV clusters.

[0004] To provide users with more efficient content delivery services, the literature (H. Wu, X. Tao, N. Zhang, and X. Shen, "Cooperative UAV Cluster-Assisted Terrestrial Cellular Networks for Ubiquitous Coverage," in IEEE Journal on Selected Areas in Communications, vol. 36, no. 9, pp. 2045-2058, 2018.) proposed a user-centric cooperative UAV cluster scheme to improve the coverage performance of ground mobile terminals through traffic offloading and diversity gain. Similarly, the literature (W. Tang, and H. Zhang, "A Tunable Caching Distribution Model for Unmanned Aerial Vehicle Networks," in IEEE Internet of Things Journal, vol. 9, no. 11, pp. 8646-8656, 2022.) studied a cooperative UAV clustering scheme, which caches the most popular content on each UAV in the cooperative group and delivers it to users together, while storing other content in the remaining cache space probabilistically to achieve content diversity and a high cache coverage probability. The literature (Y. Sun, X. Zhong, F. Wu, X. Chen, S. Zhang, and N. Dong, "Multi-UAV Content Caching Strategy and Cooperative, Complementary Content Transmission Based on Coalition Formation Game," in Sensors, vol. 22, no. 9, p. 3123, 2022.) explored the issues of content caching strategies and cooperative transmission among UAVs in a multi-UAV-assisted wireless communication network. In response to the challenges of limited battery capacity and cache capacity of UAVs, a content caching strategy based on user clustering was proposed, and a coalition formation game model was used to optimize the cooperative and complementary content transmission among multiple UAVs, thus achieving the maximization of network utility. The above work all assumes that the user location is fixed and fails to take into account the mobility of users. The change of user location will further increase the complexity of cache placement strategies and UAV trajectory planning.

[0005] Some scholars have considered the dynamic user scenario and studied the cache placement and trajectory planning problems in a multi-UAV cooperative caching network with user mobility awareness. The literature (Y.-J. Chen, K.-M. Liao, M.-L. Ku, F. P. Tso, and G.-Y. Chen, "Multi-Agent Reinforcement Learning Based 3D Trajectory Design in Aerial-Terrestrial Wireless Caching Networks," in IEEE Transactions on Vehicular Technology, vol. 70, no. 8, pp. 8201-8215, 2021.) studied the dynamic three-dimensional flight trajectory design problem of multiple caching UAVs in a user mobility-aware wireless device-to-device caching network. And a new multi-agent reinforcement learning-based framework was designed to determine the optimal three-dimensional flight trajectory of each UAV in a distributed manner without a central coordinator, so as to maximize the long-term network throughput. The literature (S. Anokye, D. Ayepah-Mensah, A. M. Seid, G. O. Boateng, and G. Sun, "Deep Reinforcement Learning-Based Mobility-Aware UAV Content Caching and Placement in Mobile Edge Networks," in IEEE Systems Journal, vol. 16, no. 1, pp. 275-286, 2022.) proposed a general algorithm to predict the positions of multiple UAVs and the content cached on the UAVs by combining human-centered features, a random waypoint user mobility model, and a dual deep reinforcement learning algorithm, so as to maximize the quality of user experience satisfaction and reduce the transmission power of the UAVs. Although the above optimization methods can enable UAVs to provide efficient services for mobile users in a highly dynamic environment, they all ignore the limited battery energy of UAVs.

[0006] In addition, it is crucial to consider the UAV-user association mechanism in the multi-UAV multi-user scenario. By allocating the most suitable UAV for each user, content delivery can be effectively optimized, thereby providing more efficient and personalized services for users. The literature (A. Bera, S. Misra, and C. Chatterjee, "QoE Analysis in Cache-Enabled Multi-UAV Networks," in IEEE Transactions on Vehicular Technology, vol. 69, no. 6, pp. 6680-6687, 2020.) proposed a content delivery framework based on quality of experience (QoE) in cache-enabled multi-UAV networks. This framework first uses the K-means algorithm to cluster users and assigns a UAV to each user cluster, then optimizes the deployment positions of UAVs in three-dimensional space, and finally uses linear regression and Zipf distribution to find the optimized caching strategies for all UAVs. Similarly, the literature (J. Luo, J. Song, F.-C. Zheng, L. Gao, and T. Wang, "User-Centric UAV Deployment and Content Placement in Cache-Enabled Multi-UAV Networks," in IEEE Transactions on Vehicular Technology, vol. 71, no. 5, pp. 5656-5660, 2022.) also proposed a K-means algorithm based on user location to cluster users, associate each user with a unique cluster, and randomly assign a UAV to each cluster to achieve the association between UAVs and users. The UAV-user association mechanism in the above work is only constructed based on user geographical location information and fails to comprehensively consider the impacts of multiple factors such as the diversity of content requirements and UAV load balancing on this association mechanism. This one-dimensional association strategy will have a negative effect on UAV service efficiency and quality of user experience to a certain extent.

[0007] Therefore, there is an urgent need to design a joint optimization framework that takes into account UAV-user association, collaborative cache placement, and trajectory planning in a multi-UAV cooperative caching system considering user mobility, user geographical location, diversity of content requirements, and heterogeneity of UAV battery capacity and cache resources, so as to achieve low-latency content delivery under the constraint of limited battery energy of UAVs. Summary of the Invention

[0008] The object of the present invention is to provide a delay optimization method considering the limited battery energy of unmanned aerial vehicles (UAVs) in a multi-UAV cooperative caching network for the problems existing in the above-mentioned prior art. Under the constraint of the limited battery energy of UAVs, by optimizing the association between UAVs and users, the UAV caching decision, and the three-dimensional coordinates of UAVs, the sum of the content retrieval delays of all users is minimized.

[0009] The technical solution for realizing the object of the present invention is as follows: On the one hand, a delay and energy-aware multi-UAV cooperative caching and trajectory planning method is provided, and the method includes:

[0010] Step 1, construct a multi-UAV cooperative caching network architecture including a ground control station, multiple UAVs, and multiple mobile users, and define the UAV and user movement models, content caching models, communication models, and UAV energy consumption models;

[0011] Step 2, construct an optimization problem with the limited battery energy of UAVs as the constraint and the minimization of the sum of the content retrieval delays of all users as the objective;

[0012] Step 3, for the diversity of the geographical locations of mobile users and the requirements for hot content, classify the mobile users into large clusters and small clusters by using a K-means-based clustering algorithm;

[0013] Step 4, decompose the optimization problem into multiple sub-problems according to the principle of the divide-and-conquer method and solve them;

[0014] Step 5, introduce an iterative algorithm, and use the alternating optimization method for multiple sub-problems until convergence to achieve low-delay content delivery considering the limited battery energy of UAVs.

[0015] On the other hand, a delay and energy-aware multi-UAV cooperative caching and trajectory planning system is provided, and the system includes:

[0016] The first module is used to construct a multi-UAV cooperative caching network architecture including a ground control station, multiple UAVs, and multiple mobile users, and define the UAV and user movement models, content caching models, communication models, and UAV energy consumption models;

[0017] The second module is used to construct an optimization problem with the limited battery energy of UAVs as the constraint and the minimization of the sum of the content retrieval delays of all users as the objective;

[0018] The third module is used to classify the mobile users into large clusters and small clusters by using a K-means-based clustering algorithm for the diversity of the geographical locations of mobile users and the requirements for hot content;

[0019] The fourth module is used to decompose the optimization problem into multiple sub-problems according to the principle of the divide-and-conquer method and solve them;

[0020] The fifth module is used to introduce an iterative algorithm and adopt an alternating optimization method for multiple sub-problems until convergence, so as to achieve low-latency content delivery considering the limited battery energy of the UAV.

[0021] Compared with the prior art, the remarkable advantages of the present invention are as follows:

[0022] (1) The present invention utilizes the multi-UAV collaborative caching technology to forward the content requested by the user from other UAVs to the associated UAV, and then the associated UAV forwards the popular content to the user, achieving the purpose of effectively improving the performance of the associated UAV, realizing a high content hit rate, and reducing the content delivery delay; by optimizing the sum of the content retrieval delays of all users, the quality of experience of the users can be improved; when performing delay optimization, the limited on-board battery energy of the UAV is considered, ensuring the high availability of the UAV.

[0023] (2) Aiming at the diversity of user geographical locations and content requirements, a clustering algorithm based on K-means is designed to classify users into large clusters and small clusters. This method can not only make the UAV cache popular content more targeted when providing services, but also ensure the load balance of the UAV.

[0024] (3) For the NP-hard mixed integer non-linear programming problem, the proposed problem is decomposed into three simple sub-problems according to the principle of the branch method for solution, so that it can achieve low-latency content delivery under the constraint of the limited on-board battery energy of the UAV, thereby improving the quality of experience of the users.

[0025] (4) The clustering algorithm based on K-means proposed by the present invention is simple to operate, has a small amount of calculation, and is highly practical, and can quickly cluster users; the collaborative cache placement strategy optimization method based on multi-agent reinforcement learning proposed by the present invention can quickly obtain a stable optimal cache decision, has a low algorithm complexity, requires a short decision-making time, and is highly practical, and can achieve a high content hit rate; the UAV flight trajectory optimization method based on Lyapunov optimization and adaptive differential evolution proposed by the present invention has a low algorithm complexity and is highly practical, and can obtain a stable optimal flight trajectory; the UAV-user association optimization method based on bipartite graph and LP relaxation proposed by the present invention is simple to operate, has a low algorithm complexity, requires a short decision-making time, and is highly practical, and can achieve low-latency content delivery. Generally speaking, the present invention is simple to operate, highly practical, can achieve low-latency content delivery under the constraint of the limited on-board battery energy of the UAV, and effectively improves the quality of experience of the users.

[0026] The present invention will be further described in detail below with reference to the accompanying drawings. Description of the Drawings

[0027] Figure 1 It is a schematic flowchart of a delay optimization method considering the limited battery energy of UAVs in the multi-UAV cooperative caching network of the present invention.

[0028] Figure 2 It is a schematic diagram of the multi-UAV cooperative caching network architecture. Detailed implementation manners

[0029] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0030] It should be noted that if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between various embodiments can be combined with each other, but it must be based on the fact that those skilled in the art can implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope required by the present invention.

[0031] The delay optimization method considering the limited battery energy of UAVs in the multi-UAV cooperative caching system proposed by the present invention takes into account the limited on-board battery energy of UAVs during hot content transmission to optimize the content retrieval delay of all users. To solve this problem, the present invention first classifies users into large clusters and small clusters by using a K-means-based clustering algorithm according to the diversity of user geographical locations and content requirements; then decomposes the proposed delay optimization problem into three sub-problems according to the principle of divide and conquer, and proposes solutions for each sub-problem, namely a cooperative caching placement strategy optimization method based on multi-agent reinforcement learning, a UAV flight trajectory optimization method based on Lyapunov optimization and adaptive differential evolution, and a UAV-user association optimization method based on bipartite graph and LP relaxation; finally, an efficient iterative algorithm is introduced, and the three sub-problems are alternately optimized until convergence to achieve low-latency content delivery considering the limited battery energy of UAVs.

[0032] In one embodiment, in combination with Figure 1 and Figure 2 , a delay and energy-aware multi-UAV cooperative caching and trajectory planning method is provided, and the method includes:

[0033] Step 1, construct a multi-UAV collaborative caching network architecture including a ground control station, multiple UAVs, and multiple mobile users, and define the UAV and user mobility models, content caching models, communication models, and UAV energy consumption models;

[0034] Step 2, construct an optimization problem with the limited battery energy of the UAVs as a constraint and the goal of minimizing the sum of the content retrieval delays of all users;

[0035] Step 3, for the diversity of the geographical locations of mobile users and the demand for popular content, use the K-means-based clustering algorithm to classify mobile users into large clusters and small clusters;

[0036] Step 4, decompose the optimization problem into multiple sub-problems according to the principle of the divide-and-conquer method and solve them;

[0037] Step 5, introduce an iterative algorithm, and use the alternating optimization method for multiple sub-problems until convergence to achieve low-latency content delivery considering the limited battery energy of the UAVs.

[0038] Further, in one embodiment, Step 1 specifically includes:

[0039] Step 1-1, construct a multi-UAV collaborative caching network architecture, which includes a ground control station, U UAVs, and M mobile users; a ground macro base station MBS is equipped in the ground control station, which has all the popular content; the UAVs are aircraft controlled by the ground control station, and are equipped with computing and caching modules to provide content delivery services for surrounding mobile users; use the set to represent the UAVs, and the set to represent the mobile users;

[0040] Since the temporal variation of the popularity of popular content is slower than the content delivery time variation caused by the mobility of UAVs and users, a two-time-scale model is constructed; among them, the large time scale represents the content caching period, and the index is represented by c∈{1,…,C}, t c represents the c-th content caching period, and the period length is T c , C is the total number of content caching periods; each content caching period is divided into multiple time slots for content delivery, and each time slot is represented as n∈{1,…,N}, the duration of each time slot is δ, n represents the n-th time slot, and N represents the total number of time slots;

[0041] Before the start of the content caching period t c , first cluster the mobile users according to their spatial positions, and divide the mobile users into K c large clusters, K c≤U; Then, perform secondary clustering according to the number of mobile users and the demand of users for hot content. For the k-th large cluster, divide it into small clusters, where 1 ≤ k ≤ K c , and there are mobile users in the i-th small cluster of the k-th large cluster. Each mobile user is assigned to a small cluster, and the number of all small clusters corresponds to the number of drones, that is During the content caching period t c , the set of all small clusters is denoted as where represents the i-th small cluster in the k-th large cluster during the caching period t c ;

[0042] is represented by which represents the association between drone u and the i-th small cluster in the k-th large cluster during the content caching period t c . If represents that drone u serves the mobile users in the i-th small cluster of the k-th large cluster during the caching period t c , otherwise After dividing the user clusters and determining the association between the drones and the mobile user clusters, each drone will fly to the corresponding small cluster to provide content delivery services for the mobile users in this small cluster;

[0043] Step 1-2, establish the movement model of drones and mobile users, specifically:

[0044] Consider a three-dimensional Cartesian coordinate system. The three-dimensional coordinates of drone u in the time slot are represented as The three-dimensional coordinates of mobile user m in the time slot are represented as Then the distance between drone u and mobile user m in the time slot is obtained by ; In addition, assume that mobile users move over time, and consider two user movement models, namely the random roaming model and the directional walking model;

[0045] Step 1-3, establish the content caching model, specifically:

[0046] Use the set to represent Q hot contents, where the data volume of hot content q ∈ Q is λ q ; For the content caching period t cthe small clusters in The set of request probabilities for all hot content is where represents the request probability of mobile user m for all hot content and needs to satisfy the constraint is the request probability of mobile user m for hot content q;

[0047] UAVs can cache hot content in advance, but the cache capacity of each UAV is limited. The cache capacity of UAV u is defined as σ u , and not all hot content can be cached on UAVs at the same time; therefore, a binary variable is introduced to represent the caching decision of UAV u for hot content q in content caching period t c , means that UAV u caches hot content q, otherwise

[0048] Steps 1 - 4: Based on the retrieval method for obtaining the hot content requested by mobile users, establish the air - to - ground (A2G) communication model between UAVs and mobile users, the air - to - air (A2A) communication model between UAVs, and the ground - to - air (G2A) communication model between the ground macro - base station MBS and UAVs respectively;

[0049] Steps 1 - 5: Establish the UAV energy consumption model, specifically:

[0050] The power consumption of UAVs mainly consists of two parts: communication - related power consumption and propulsion power consumption; the former can be ignored compared with the propulsion power consumption, and the latter is necessary for UAV flight and hovering, usually depending on the flight speed of the UAV, that is, the propulsion power consumption is a function of the flight speed;

[0051]

[0052] where

[0053]

[0054] In the formula, is the total propulsion energy consumption of UAV u in each time slot , and δ is the duration of each time slot; is the propulsion power consumption of UAV u in each time slot ; P0 and P1 respectively represent the blade - type power and induced power in the hovering state of the UAV, ω b is the rotor tip speed, v0 is the average rotor induced speed during hovering, d0 and s are the fuselage drag ratio and rotor solidity respectively, ρ and A are the air density and rotor disk area respectively; is the propulsion power consumption of UAV u in each time slot The speed, are respectively small time slots Small time slots The three-dimensional coordinates of.

[0055] Here, in some embodiments, in steps 1-3, the mobile user The requested hot content is retrieved in the following way:

[0056] Edge content retrieval: When the UAV associated with the small cluster caches the requested hot content, the mobile user m can directly obtain the required hot content q only through a single-hop air-to-ground A2G link;

[0057] And / or, collaborative content retrieval: To improve the cache hit rate, a UAV collaborative cache model with data forwarding function is constructed; When the associated UAV does not cache the hot content q requested by the mobile user m, but other UAVs that can be accessed within the maximum forwarding hop count of the associated UAV cache the hot content, the mobile user m obtains the content q through a single-hop or multi-hop air-to-air A2A link and a single-hop A2G link;

[0058] And / or, remote content retrieval: If the associated UAV and other UAVs that can be accessed within its maximum forwarding hop count do not cache the hot content q requested by the mobile user m, the mobile user m can only obtain the content q through a single-hop ground-to-air G2A link and an A2G link.

[0059] Here, in some embodiments, step 1-4 specifically includes:

[0060] (1) For edge content retrieval:

[0061] Edge content retrieval only requires a single-hop A2G link to directly transmit the requested content from the UAV u associated with the small cluster to the user above; For the A2G communication link, there are many scatterings or obstacles in the real environment, and the resulting shadow or scattering will cause the radio signal to not be able to propagate in free space, thus generating additional path loss; Therefore, a probability path loss model considering Los and NLos communications is introduced to model A2G communication, specifically:

[0062] In the small time slot The probability path loss between the UAV u and the mobile user m is expressed as:

[0063]

[0064] Wherein,

[0065]

[0066]

[0067] wherein, are the LoS and NLoS path losses from the UAV u to the mobile user m in the small time slot respectively; are the occurrence probabilities of LoS communication and NLoS communication between the UAV u and the mobile user m in the small time slot respectively; f c and V l are the carrier frequency and the speed of light respectively, and η LoS and η NLoS are the average additional losses of the LoS and NLoS links respectively; and are constants determined by the carrier frequency and the system environment respectively;

[0068] In the content caching period t c the edge content retrieval delay of the UAV u for the mobile user m of the requested hot content q is

[0069]

[0070] wherein,

[0071]

[0072] wherein, is the average transmission rate from the UAV u to the mobile user m in the content caching period t c respectively; are the channel gain and transmission rate from the UAV u to the mobile user m in the small time slot respectively; is the A2G communication link bandwidth of the UAV u, is the transmission power of the UAV u, σ is the white Gaussian noise power, is the interference signal power of the UAV u when sending content to other mobile users belonging to the same small cluster as the mobile user m to the mobile user m; is the transmission rate from the UAV u to the mobile user m' in the small time slot respectively;

[0073] (2) For collaborative content retrieval:

[0074] Collaborative content retrieval requires one-hop or multi-hop A2A links and one-hop A2G link to obtain the content required by the user; the user The requested content q is first transmitted from other drones u' that have cached the content q to the drone u associated with the small cluster in sequence through the shortest path calculated by the link state (LS) routing algorithm, via one-hop or multi-hop A2A links, and then from the drone u to the user m via a one-hop A2G link; For the A2A communication link, considering that the flight altitude of the drones is relatively high and is hardly affected by obstacles, the communication environment between the drones can be approximated as a free space. The free space path loss model is adopted to describe the A2A communication. Among them, in the small time slot

[0075] the path loss between drone i and drone j is:

[0076]

[0077] where

[0078]

[0079] In the formula, is the distance between drone i and drone j in the small time slot ; are the three-dimensional coordinates of drone i and drone j respectively in the small time slot ;

[0080] During the content caching period t c the collaborative content retrieval delay for the hot content q to be transmitted from the drone u' to the drone u in sequence according to the shortest path first and then from the drone u to the mobile user m is

[0081]

[0082] where

[0083]

[0084] In the formula, is the shortest forwarding hop count between drones u and u', indicates that the drone u' that can perform collaborative caching must be a drone that can be accessed within the maximum forwarding hop count of drone u; indicates the shortest path from drone u' to u during the caching period t c ; is the average transmission rate from drone i to drone j during the memory caching period t c ; are respectively the small time slot ​Channel gain and transmission rate from UAV i to UAV j; is the A2A communication link bandwidth of UAV i, is the transmission power of UAV i;

[0085] (3) For remote content retrieval:

[0086] The hotspot content q requested by mobile user m is first transmitted from the ground macro base station MBS to the UAV u associated with the small cluster through the G2A link, and then from the UAV u to the mobile user m through the A2G link. The remote content retrieval delay for this process is

[0087]

[0088] where R G2A is the data transmission rate of the G2A communication link.

[0089] Furthermore, in one embodiment, the optimization problem in step 2 is:

[0090]

[0091] C1:

[0092] C2:

[0093] C3:

[0094] C4:

[0095] C5:

[0096] C6:

[0097] C7:

[0098] C8:

[0099] C9:

[0100] C10:

[0101] C11:

[0102] C12:

[0103] C13: ​

[0104] C14:

[0105] Among them,

[0106]

[0107] In the formula, the decision variables are the association between the UAV and the cluster UAV caching decision and the three-dimensional coordinates of the UAV is the association between UAV u and the i-th small cluster in the k-th large cluster during the content caching period t c , is the caching decision of UAV u for the hot content q during the content caching period t c , λ q is the data volume of the hot content q, σ u is the caching capacity of UAV u, respectively represent whether to select the edge content retrieval, collaborative content retrieval, and remote content retrieval methods for content caching; is the two-dimensional coordinate of UAV u in the small time slot , is the height of UAV u in the small time slot , H min , H max respectively represent the maximum horizontal distance, vertical distance, minimum height, and maximum height of the UAV, D min is the distance between UAVs that should not be less than the minimum distance, T c is the length of the c-th content caching period, is the maximum allowable energy consumption of UAV u in each content caching period, is the total propulsion energy consumption of UAV u in the small time slot , Q u is the long-term energy consumption upper limit of UAV u; respectively represent the three-dimensional coordinates of UAV u and UAV u' in the small time slot .

[0108] C1 - C3 are the association constraints between the UAV and the clusters, requiring that each small cluster is served by only one UAV, and each UAV serves one and only one small cluster; C4 - C5 are the cache decision constraints, which limit the cache capacity of the UAVs; C6 - C7 indicate that each user can obtain the requested content in only one way, and it is related to the cache decision; C8 - C10 are the UAV three - dimensional coordinate constraints, which limit the horizontal, vertical movement distance and height of the UAV in each time slot; C11 requires that no collision occurs between any two UAVs, and it is related to the UAV three - dimensional coordinates; C12 is the content retrieval delay constraint, requiring that the content must be sent to the user within the user's request period; C13 indicates that the energy consumption of each UAV in each cache cycle cannot exceed the maximum allowable energy consumption; C14 is the long - term energy constraint of the UAV, requiring that the long - term average total energy consumption of the UAV does not exceed the upper limit.

[0109] Further, in one of the embodiments, step 3 specifically includes:

[0110] Step 3 - 1: According to the spatial location information of the mobile users, use the silhouette coefficient method to determine the optimal K value, and based on the determined K value, classify the mobile users into large clusters according to the K - means algorithm;

[0111] Step 3 - 2: According to the number of mobile users and the demand of mobile users for hot content, classify the mobile users in each large cluster into small clusters.

[0112] Preferably, in some embodiments, step 3 - 1, which is to determine the optimal K value according to the spatial location information of the mobile users, use the silhouette coefficient method, and classify the mobile users into large clusters according to the K - means algorithm based on the determined K value, specifically includes:

[0113] Determine the K value and classify the users into large clusters according to the spatial location of the mobile users using the silhouette coefficient method; the silhouette coefficient is an index used to evaluate the quality of clustering results and can be understood as an index describing the clarity of the contours of each category after aggregation; the formula of the silhouette coefficient is as follows:

[0114]

[0115] Where,

[0116]

[0117] In the formula, |C i1Let \(n(i1)\) denote the number of samples in the cluster to which sample \(i1\) belongs. \(a(i1)\) represents the cohesion of the sample point, which is the average distance between sample \(i1\) and other samples within its cluster (referred to as the within-cluster distance). \(j1\) represents other sample points within the same cluster as sample \(i1\), and \(distance(i1,j1)\) is used to calculate the distance between sample \(i1\) and sample \(j1\). The smaller \(a(i1)\) is, the tighter the cluster. \(b(i1)\) represents the separation of the sample point, which is the minimum average distance between sample \(i1\) and samples in any other cluster (referred to as the nearest-cluster distance), where \(j2\) represents other sample points in different clusters from sample \(i1\).

[0118] Step 3 - 1 - 1: Select the range of \(K\) values, i.e., from 2 to \(U - 1\).

[0119] Step 3 - 1 - 2: Perform clustering for each \(K\) value and calculate the silhouette coefficient of the clustering result.

[0120] Step 3 - 1 - 3: Select the \(K\) value with the largest silhouette coefficient as the final \(K\) value and perform clustering again.

[0121] Preferably, in some embodiments, in step 3 - 2, classifying the mobile users in each large cluster into small clusters according to the number of mobile users and the demand of mobile users for hot content specifically includes:

[0122] Step 3 - 2 - 1: Determine the number of small clusters in each large cluster according to the proportion of the number of mobile users in each large cluster.

[0123] Step 3 - 2 - 2: Determine the number of mobile users in each small cluster within each large cluster according to the determined number of small clusters, ensuring an average to guarantee load balancing.

[0124] Step 3 - 2 - 3: Use the K - means algorithm for partitioning according to the request probability of mobile users for hot content: First, calculate the similarity between the request probability \(Re\) of mobile user \(j\) for hot content in large cluster \(C\) i and the centroid \(k\) j of mobile user \(j\) for hot content in large cluster \(C\). Let \(N\) i represent the number of small clusters in the \(i\) - th large cluster; then assign mobile user \(j\) to the small cluster \(C\) jk with the smallest \(S\), i,k and finally update the centroid.

[0125] Step 3 - 2 - 4: If the number of mobile users in the partitioned small cluster exceeds the specified number of mobile users, delete the mobile user with the smallest similarity in this small cluster and assign it to another cluster \(C\) i,t ≠ \(C\) i,kIn the middle; repeat this step until the number of mobile users in all clusters reaches the preset legal quantity.

[0126] Furthermore, in one embodiment, step 4 of decomposing the optimization problem into multiple sub-problems according to the principle of divide and conquer and solving them specifically includes:

[0127] Step 4-1, decompose the optimization problem into: UAV cooperative caching placement strategy optimization, UAV flight trajectory optimization, and UAV-mobile user association optimization;

[0128] Step 4-2, for the UAV cooperative caching placement strategy optimization, adopt a multi-agent reinforcement learning algorithm, let each UAV be an agent, and learn the optimal caching strategy by interacting with other agents in a dynamic environment;

[0129] Step 4-3, for the UAV flight trajectory optimization, first handle the long-term energy constraint based on the Lyapunov optimization framework, and then solve it through an adaptive differential evolution algorithm;

[0130] Step 4-4, for the UAV-mobile user association optimization, use the method based on bipartite graph and LP relaxation to reduce the problem complexity and solve it.

[0131] Preferably, in some embodiments, in step 4-2, for the UAV cooperative caching placement strategy optimization, adopt a multi-agent reinforcement learning algorithm, let each UAV be an agent, and learn the optimal caching strategy by interacting with other agents in a dynamic environment, specifically including:

[0132] For the UAV cooperative caching placement strategy optimization problem, a UAV cooperative caching method based on multi-agent double-delayed deep deterministic policy gradient (MATD3) is proposed to solve it, where the UAV corresponds to the agent making the caching decision, and the multi-UAV cooperative caching system corresponds to the environment; there are three key elements in reinforcement learning, namely the state space, action space, and reward function, which are specifically described in this problem as:

[0133] 1) State Space: Define as the state of UAV u in content caching period t c including the UAV caching capacity, the remaining caching capacity of the UAV, the three-dimensional coordinates of the UAV, the content data size, the association between the UAV and the small cluster, and the user's request probability for the content; therefore, the state space can be represented as a vector with elements:

[0134]

[0135] Among them, is the remaining cache capacity of the UAV u, is the three-dimensional coordinates of the UAV u in the time slot , which is set as the centroid of the small cluster providing services; the global state space is the set of the states of all UAVs, that is

[0136] 2) Action Space: Define as the action of the UAV u in the content caching period t c , which contains Q cache decision elements:

[0137]

[0138] Similarly, the global action space is the set of the actions of all UAVs, that is

[0139] 3) Reward Function: Define as the reward function of the UAV u in the content caching period t c ; the reward function of the UAV u is designed as:

[0140]

[0141] where M1 is a negative constant, and when the action violates the constraint of the agent u (i.e., the cache capacity constraint C2 in step 2), a certain penalty will be given. Γ c is the objective function when no constraint is violated, and the formula is:

[0142]

[0143] where and are the weights of edge content retrieval and collaborative content retrieval respectively, satisfying It should be noted that in order to maximize the weighted content hit rate of the system, all agents share the same reward, so the global reward is

[0144] The specific process of the UAV collaborative caching algorithm based on MATD3 includes:

[0145] Step 4-2-1, initialize the global experience replay memory buffer χ, the Actor policy network parameter θ u and the two Critic value network parameters ω i,u (i = 1, 2), the Actor target network parameter θ' u = θ uand the parameters ω' of the two Critic target networks i,u = ω i,u (i = 1, 2) and the states s of all agents c ;

[0146] Step 4-2-2, for each drone u, first select the current continuous action according to the current state Then use a threshold function to achieve the mapping from the continuous action to the discrete action . The expression of the threshold function is:

[0147]

[0148] Step 4-2-3, each drone u executes the current discrete action and interacts with the environment to obtain the reward and the next state The Actor network sends the experience to the control center, and the control center collects the experiences of all agents and synthesizes e c = (s c , a c , r c , s c,new ) and stores it in the global experience replay memory buffer χ;

[0149] Step 4-2-4, each drone u randomly extracts min_batch groups of experience data e c = {s c , a c , r c , s c,new} from the buffer χ for network parameter update. The parameter update process specifically includes: First, calculate the TD target of agent u according to the time difference method:

[0150]

[0151] where γ is the discount factor, is the action generated by the Actor target network and the noise φ; Then, update the parameters ω i,u (i = 1, 2) of the two Critic value networks by minimizing the loss function. The loss function can be expressed as:

[0152]

[0153] where Q i (s c , a c ; ωi,u ) The evaluation of the global state s c and action a c by the Critic value network of the agent u at cycle t c ; then, after a certain number of iterations, the parameters of the Actor network and the Critic target network are updated; the parameters θ u of the Actor policy network can be updated by the deterministic policy gradient algorithm, and the gradient of its loss function can be expressed as:

[0154]

[0155] The parameters θ’ u of the Actor target network and the parameters ω’ i,u (i = 1, 2) are updated by the soft update policy, that is:

[0156] θ’ u = ψθ u +(1 - ψ)θ’ u

[0157] ω’ i,u = ψω i,u +(1 - ψ)ω’ i,u , i = 1, 2

[0158] where ψ is the soft update rate of the target network parameters.

[0159] Step 4 - 2 - 5, repeat Step 4 - 2 - 2 to Step 4 - 2 - 4 max_epoch times to obtain the best caching decisions of all agents.

[0160] Preferably, in some embodiments, in Step 4 - 3, for the UAV flight trajectory optimization, first, the long - term energy constraint is processed based on the Lyapunov optimization framework, and then it is solved by the adaptive differential evolution algorithm, specifically including:

[0161] For the UAV flight trajectory optimization problem, first, the original problem is converted into a real - time optimization problem for each large time slot based on the Lyapunov optimization framework. The specific process includes:

[0162] First, establish a virtual energy queue to represent the energy constraint C14 in Step 2:

[0163]

[0164] where, is the queuing backlog of UAV u at time slot t c , representing the deviation between the current energy consumption and the energy constraint; then define the Lyapunov function:

[0165]

[0166] where is the vector of the current queue backlog; based on these two values, the conditional Lyapunov drift at time slot t c is defined as:

[0167] ΔL(Q c ) = E{L(Q c+1 ) - L(Q c )|Q c}

[0168] where Finally, the drift-plus-penalty at time slot t c can be obtained as:

[0169] ΔL(Q c ) + VE{D c |Q c}

[0170] where V is a parameter that trades off the content retrieval delay and energy consumption of the control system; using Lyapunov optimization techniques, the long-term optimization problem can be decomposed into an optimization problem for a single large time slot. The theoretical bound of the drift-plus-penalty function at time slot t c is:

[0171]

[0172] where According to the Lyapunov optimization framework, minimizing the right-hand side of the above equation can further obtain sub-problem 2, whose decision variables are all three-dimensional coordinates of all UAVs at time slot t c :

[0173]

[0174] Then, an adaptive differential evolution with external archive (JADE) algorithm is used to solve sub-problem 2. The specific process includes:

[0175] Step 4-3-1, initialize the mutation factor F c , the crossover factor CR c and the population Q c (τ). The initialized population Q c (τ) should satisfy the constraints C8 - C12 in Step 2;

[0176] Step 4-3-2, enter the main loop of differential evolution. For each individual in the population, at the beginning of each iteration of evolution, the mutation factor and crossover probability need to be updated. The parameter update method is as follows:

[0177]

[0178] Among them, generated according to the Cauchy distribution with a mean of μ F , and a standard deviation of 0.1, generated according to the normal distribution with a mean of μ CR , and a standard deviation of 0.1;

[0179] Step 4-3-3, for each individual in the population generate a mutant individual through a new mutation strategy DE / current-to-best That is:

[0180]

[0181] Among them, is an individual randomly selected from the top p*NP individuals in the population sorted by fitness, where p is a given ratio; is an individual randomly selected from the population, is an individual randomly selected from the current population and the external archive set. The external archive set stores the individuals that failed in mutation and crossover during each iteration of evolution, and its size is fixed at NP. If the size exceeds NP, some of the individuals are randomly deleted; is the individual 's mutation factor, adaptively adjusted by the successful mutation factors recorded in history;

[0182] Note that for each drone in the mutant individual the three-dimensional coordinates at each time slot need to satisfy the boundary constraints. If they exceed the boundary values, boundary processing needs to be performed on the three-dimensional coordinates, that is:

[0183]

[0184] Step 4-3-4, based on the mutant individual and the current individual perform a crossover operation to generate a trial individual That is:

[0185]

[0186] Among them, is the crossover probability of the current individual , adaptively adjusted by the successful crossover probabilities recorded in history; rnbr(i) = (u', n', j') represents a randomly selected sequence used to ensure that the trial individual receives at least from the mutant individual Obtain a parameter;

[0187] Since all three-dimensional coordinates of the current individual satisfy constraints C8 - C9, while the three-dimensional coordinates of the mutant individual may not satisfy the constraints, it is necessary to reprocess all three-dimensional coordinates in the trial individual to determine whether they satisfy constraints C8 - C9. If they do not satisfy the constraints, the coordinates that violate the constraints need to be changed to the corresponding coordinates of the current individual ;

[0188] Step 4 - 3 - 5, the trial individual obtained after the above mutation and crossover operations must satisfy constraints C8 - C10, but may not satisfy constraints C11 - C12, and the process of solving these two constraints is complex. Therefore, the method of adding penalty terms to the fitness function is adopted to obtain a solution that satisfies all constraints through iterative evolution; the penalty terms for the UAV collision constraint C11 and the content retrieval delay constraint C12 of the current individual are:

[0189]

[0190]

[0191] where represents the content retrieval delay for UAV u to provide content q to user m in cache cycle t c ;

[0192] Furthermore, based on the fitness function of the current individual can be obtained:

[0193]

[0194] where represents the energy consumption of the u-th UAV in the i-th individual at the τ-th iteration in the time slot ;

[0195] Step 4 - 3 - 6, then compare the trial individual obtained after mutation and crossover with the current individual and select the one with better penalty term value and fitness value as the new individual. The selection formula is expressed as:

[0196]

[0197] where and respectively represent the trial individual The penalty terms for the collision constraint C11 and content retrieval delay constraint C12 between drones represent the trial individual fitness value;

[0198] Step 4-3-7, at the end of each iterative evolution, it is necessary to update μ F and μ CR , and their update methods are as follows:

[0199] μ F =(1 - c)·μ F + c·mean L (S F )

[0200] μ CR =(1 - c)·μ CR + c·mean A (S CR )

[0201] where S F and S CR are the sets corresponding to the parameters of the individuals with successful mutation and crossover in this iterative evolution The parameter c is the weight, set to 0.5; mean A (S CR ) represents the arithmetic mean of all CRs in S, and mean L (S F ) is the Lehmermean, whose role is to transmit a larger F to improve the convergence speed;

[0202] Step 4-3-8, repeat steps 4-3-2 to 4-3-7 τ max times to obtain the optimal flight trajectories of all drones.

[0203] Preferably, in some embodiments, for the drone-user association optimization described in step 4-4, a method based on bipartite graph and LP relaxation is used to reduce the problem complexity and solve it. Specifically:

[0204] For the drone-user association optimization, a method based on bipartite graph and LP relaxation is used to reduce the problem complexity and solve it. The specific process includes:

[0205] Given the drone caching decision and all three-dimensional coordinates of the drones within time slot t c , sub-problem 3 can be expressed as follows:

[0206]

[0207] ​s.t.C1:

[0208] C2:

[0209] C3: where the cumulative content retrieval delay is fixed given the known UAV caching decisions and the 3D coordinates of the UAVs at time slot t c and can be expressed as

[0210] Define a bipartite graph G=(V, E), where the vertex set V is divided into two disjoint and independent sets, i.e., V = V1 ∪ V2; specifically corresponds to the set of UAVs, while corresponds to the set of small clusters; for any vertex in V1 and any vertex in V2, there exists an edge with weight (v 1,u , v 2,k,i ); thus, sub - problem 3 is equivalent to finding a subset of edges with the minimum total weight such that each vertex in V1 and V2 corresponds to exactly one edge;

[0211] Let be whether to add the edge (v 1,u , v 2,k,i ) ∈ E to ; then, sub - problem 3 is equivalent to the following integer linear problem SP3(1):

[0212]

[0213]

[0213] s.t.C1:

[0214] C2:

[0215] C3: By relaxing the binary variable to a linear constraint, i.e., allowing to take any value in the interval [0, 1], the LP relaxation problem SP3(2) can be obtained:

[0216]

[0217] s.t.C1:

[0218] C2:

[0219] C3: Note that problem SP3(2) is a standard linear programming problem, which can be effectively solved by the interior-point method of the simplex algorithm to obtain the optimal solution of SP3(1), all of whose are equal to 0 or 1.

[0220] Furthermore, in one of the embodiments, the iterative algorithm introduced in step 5 is adopted, and the alternating optimization method is used for multiple sub-problems until convergence to achieve low-latency content delivery considering the limited battery energy of the UAVs. Specifically, it includes:

[0221] Step 5-1: Before performing iterative alternating optimization, first cluster the mobile users through steps 3-1 and 3-2 to obtain U small clusters; then initialize the association variables according to the distances between the UAVs and the mobile users within the clusters If several UAVs have the shortest distances to the same small cluster, select the UAV with the least remaining battery power to serve this cluster;

[0222] Step 5-2: Enter the iterative loop process. First, at the beginning of each content caching period t c obtain the optimal UAV caching decision using the MATD3-based UAV collaborative caching algorithm in step 4-2, then obtain the optimal flight trajectories of all UAVs in each caching period t c using the adaptive differential evolution with external archive (JADE) algorithm in step 4-3, and finally obtain the optimal association variables using the method based on bipartite graph and LP relaxation in step 4-4 and flight trajectories under the condition of the given UAV caching decision

[0223] Step 5-3: Repeat step 5-2 until the convergence condition is reached, that is, the change in the objective function value between two adjacent iterations is less than a certain threshold ∈.

[0224] In one embodiment, a multi-UAV collaborative caching and trajectory planning system with delay and energy awareness is provided. The system includes:

[0225] The first module is used to construct a multi-UAV collaborative caching network architecture including a ground control station, multiple UAVs, and multiple mobile users, and define the UAV-user movement model, content caching model, communication model, and UAV energy consumption model;

[0226] The second module is used to construct an optimization problem with the limited battery energy of the UAVs as the constraint and the goal of minimizing the sum of the content retrieval delays of all users;

[0227] The third module is used to classify mobile users into large clusters and small clusters by using a K-means-based clustering algorithm for the diversity of mobile user geographical locations and hot content requirements;

[0228] The fourth module is used to decompose the optimization problem into multiple sub-problems according to the principle of the divide-and-conquer method and solve them;

[0229] The fifth module is used to introduce an iterative algorithm and adopt an alternating optimization method for multiple sub-problems until convergence to achieve low-latency content delivery considering the limited battery energy of the drones.

[0230] For the specific limitations of the multi-UAV cooperative caching and trajectory planning system regarding delay and energy awareness, reference can be made to the limitations of the multi-UAV cooperative caching and trajectory planning method for delay and energy awareness in the above text, which will not be elaborated here. Each module in the above multi-UAV cooperative caching and trajectory planning system regarding delay and energy awareness can be implemented in whole or in part through software, hardware, and combinations. The above modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form to facilitate the processor to call and execute the operations corresponding to the above respective modules.

[0231] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the following is implemented:

[0232] Step 1: Construct a multi-UAV cooperative caching network architecture including a ground control station, multiple UAVs, and multiple mobile users, and define the UAV-user movement model, content caching model, communication model, and UAV energy consumption model;

[0233] Step 2: Construct an optimization problem with the limited battery energy of the UAVs as a constraint and the goal of minimizing the sum of content retrieval delays of all users;

[0234] Step 3: For the diversity of mobile user geographical locations and hot content requirements, classify mobile users into large clusters and small clusters by using a K-means-based clustering algorithm;

[0235] Step 4: Decompose the optimization problem into multiple sub-problems according to the principle of the divide-and-conquer method and solve them;

[0236] Step 5: Introduce an iterative algorithm and adopt an alternating optimization method for multiple sub-problems until convergence to achieve low-latency content delivery considering the limited battery energy of the UAVs.

[0237] For the specific limitations of each step, reference can be made to the limitations of the multi-UAV cooperative caching and trajectory planning method for delay and energy awareness in the above text, which will not be elaborated here.

[0238] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, it realizes:

[0239] Step 1, construct a multi-UAV collaborative caching network architecture including a ground control station, multiple UAVs, and multiple mobile users, and define the UAV and user movement models, content caching models, communication models, and UAV energy consumption models;

[0240] Step 2, construct an optimization problem with the limited battery energy of the UAV as a constraint and the goal of minimizing the sum of the content retrieval delays of all users;

[0241] Step 3, aiming at the diversity of the geographical locations and hot content requirements of mobile users, use the K-means-based clustering algorithm to classify mobile users into large clusters and small clusters;

[0242] Step 4, decompose the optimization problem into multiple sub-problems according to the principle of the divide-and-conquer method and solve them;

[0243] Step 5, introduce an iterative algorithm, and use the alternating optimization method for multiple sub-problems until convergence to achieve low-latency content delivery considering the limited battery energy of the UAV.

[0244] For the specific limitations of each step, reference can be made to the limitations of the multi-UAV collaborative caching and trajectory planning method for time delay and energy awareness in the above text, which will not be elaborated here.

[0245] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art of this industry should understand that the present invention is not limited by the above embodiments. What is described in the above embodiments and the specification only illustrates the principle of the present invention. Without departing from the spirit and scope of the present invention, any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A time-delay and energy-aware multi-UAV cooperative caching and trajectory planning method, characterized in that The method includes: Step 1, construct a multi-UAV collaborative caching network architecture including a ground control station, multiple UAVs, and multiple mobile users, and define the UAV and user mobility models, content caching models, communication models, and UAV energy consumption models; Step 2, construct an optimization problem with the limited battery energy of the UAV as a constraint and the goal of minimizing the sum of content retrieval delays of all users; Step 3, for the diversity of the geographical locations of mobile users and the requirements for hot content, use the K-means based clustering algorithm to classify mobile users into large clusters and small clusters; Step 4, decompose the optimization problem into multiple sub-problems according to the principle of the divide-and-conquer method and solve them; Step 5, introduce an iterative algorithm, and use the alternating optimization method for multiple sub-problems until convergence to achieve low-latency content delivery considering the limited battery energy of the UAV.

2. The method for delay and energy-aware multi-UAV cooperative caching and trajectory planning according to claim 1, characterized in that Step 1 specifically includes: Step 1-1, construct a multi-UAV cooperative caching network architecture, which includes a ground control station, U UAVs and M mobile users; a ground macro base station MBS is equipped in the ground control station, which has all the hot content; the UAVs are aircrafts controlled by the ground control station, and are equipped with computing and caching modules to provide content delivery services for the surrounding mobile users; use the set to represent the UAVs, and the set to represent the mobile users; Construct a two - time - scale model; where the large time - scale represents the content caching period, and the index is denoted by \(c\in\{1,\ldots,C\}\), \(t\) c represents the \(c\) - th content caching period, and the period length is \(T\) c , \(C\) is the total number of content caching periods; each content caching period is divided into multiple hourly slots for content delivery, and each hourly slot is denoted as \(n\in\{1,\ldots,N\}\), the duration of each hourly slot is \(\delta\), \(n\) represents the \(n\) - th hourly slot, and \(N\) represents the total number of hourly slots; Before the content caching period t c First, the mobile users are clustered according to their spatial positions, and the mobile users are divided into K c large clusters, where K c ≤U; then, a secondary clustering is performed according to the number of mobile users and the demand of users for hot content. For the k-th large cluster, it is divided into small clusters according to the number of mobile users and the demand of mobile users for content, where 1 ≤ k ≤ K c . There are mobile users in the i-th small cluster of the k-th large cluster, where 1 ≤ i ≤ Each mobile user is assigned to a small cluster, and the number of all small clusters corresponds to the number of UAVs, that is During the content caching period t c , the set of all small clusters is denoted as where represents the i-th small cluster in the k-th large cluster during the caching period t c ; From denote the association between the drone u and the i-th small cluster in the k-th large cluster during the content caching period t c If denote the drone u serving the mobile users in the i-th small cluster in the k-th large cluster during the caching period t c Otherwise After partitioning the user clusters and determining the associations between the drones and the mobile user clusters, each drone will fly to the corresponding small cluster to provide content delivery services for the mobile users in this small cluster; Step 1-2, establish the UAV and mobile user mobility models, specifically: Consider a three-dimensional Cartesian coordinate system, where the UAV u has three-dimensional coordinates in the time slot represented as The mobile user m has three-dimensional coordinates in the time slot represented as Then, the distance between the UAV u and the mobile user m in the time slot is obtained by In addition, assume that the mobile user moves over time. Consider two user mobility models, namely the random walk model and the directional walk model. Step 1-3, establish the content caching model, specifically: Use a set to represent Q hot contents, where the data volume of hot content q ∈ Q is λ q ; for the small clusters in the content caching period t c the set of request probabilities for all hot contents is where represents the request probability of mobile user m for all hot contents and needs to satisfy the constraint is the request probability of mobile user m for hot content q; Introduce a binary variable indicating the caching decision of the UAV u for the hot content q in the content caching period t c where 1 means the UAV u caches the hot content q, otherwise it means the UAV u does not cache the hot content q Step 1-4, based on the retrieval methods for obtaining the hot content requested by mobile users, establish the air-to-ground communication model between the UAV and mobile users, the air-to-air communication model between UAVs, and the ground-to-air communication model between the ground macro base station MBS and the UAV respectively; Step 1-5, establish the UAV energy consumption model, specifically: Among them, In the formula, is the total propulsion energy consumption of the UAV u in each time slot , where δ is the duration of each time slot; is the propulsion power consumption of the UAV u in each time slot . P0 and P1 respectively represent the profile power and induced power of the UAV in the hovering state. ω b is the rotor tip speed, v0 is the average rotor induced speed during hovering, d0 and s are the fuselage drag ratio and rotor solidity respectively, and ρ and A are the air density and rotor disk area respectively; is the speed of the UAV u in each time slot ; are the three-dimensional coordinates of the time slot and the time slot respectively.

3. The method for delay and energy-aware multi-UAV cooperative caching and trajectory planning according to claim 2, characterized in that In steps 1-3, the mobile user The requested hotspot content is retrieved in the following manner: Edge content retrieval: When the UAV associated with the small cluster caches the requested hot content, the mobile user m can directly obtain the required hot content q through only one-hop air-to-ground A2G link; And / or, collaborative content retrieval: construct a UAV collaborative caching model with data forwarding function; when the associated UAV does not cache the hot content q requested by mobile user m, but other UAVs within the maximum forwarding hops of the associated UAV cache this hot content, mobile user m obtains content q through one-hop or multi-hop air-to-air A2A links and one-hop A2G link; And / or, remote content retrieval: if the associated UAV and other UAVs within its maximum forwarding hops do not cache the hot content q requested by mobile user m, mobile user m can only obtain content q through one-hop ground-to-air G2A link and one A2G link.

4. The method for delay and energy-aware multi-UAV collaborative caching and trajectory planning according to claim 3, wherein Step 1-4 specifically includes: (1) For edge content retrieval: Introduce a probabilistic path loss model considering Los and NLos communications to model A2G communication, specifically: In the small time slot The probabilistic path loss between the UAV u and the mobile user m is expressed as: Among them, wherein, are the LoS and NLoS path losses from the UAV u to the mobile user m in the small time slot respectively; are the occurrence probabilities of LoS communication and NLoS communication between the UAV u and the mobile user m in the small time slot respectively; f c and V l are the carrier frequency and the speed of light respectively, and η LoS and η NLoS are the average additional losses of the LoS and NLoS links respectively; and are constants determined by the carrier frequency and the system environment respectively; During the content caching period t c the edge content retrieval latency of the mobile user m of the requested hot content q by the UAV u is Among them, wherein, is the average transmission rate from the UAV u to the mobile user m during the content caching period t c ; are the channel gain and transmission rate from the UAV u to the mobile user m during the small time slot respectively; is the A2G communication link bandwidth of the UAV u, is the transmit power of the UAV u, σ is the white Gaussian noise power, is the interference signal power to the mobile user m when the UAV u sends content to other mobile users belonging to the same small cluster as the mobile user m; is the transmission rate from the UAV u to the mobile user m' during the small time slot ; (2) For collaborative content retrieval: In a short time slot Path loss between drone i and drone j is as follows: Among them, Wherein, is the distance between the UAV i and the UAV j in the small time slot ; are respectively the three-dimensional coordinates of the UAV i and the UAV j in the small time slot ; During the content caching period t c The collaborative content retrieval delay for the hot content q to be first transmitted from the drone u' to the drone u in sequence according to the shortest path and then from the drone u to the mobile user m is Among them, Wherein, is the shortest forwarding hop count between drones u and u'; indicates that the drone u' capable of collaborative caching must be the drone that can be accessed within the maximum forwarding hop count of drone u; within; indicates the shortest path from drone u' to u during the caching period t c ; is the average transmission rate from drone i to drone j during the memory caching period t c ; respectively represent the channel gain and transmission rate of the small time slot from drone i to drone j; is the A2A communication link bandwidth of drone i, and is the transmission power of drone i; (3) For remote content retrieval: The hot content q requested by the mobile user m is first transmitted from the ground macro base station MBS to the drone u associated with the small cluster through the G2A link, and then transmitted from the drone u to the mobile user m through the A2G link. The remote content retrieval delay of this process is and then, and the remote content retrieval delay of this process is Wherein, R G2A is the data transmission rate of the G2A communication link.

5. The method for delay and energy-aware multi-UAV collaborative caching and trajectory planning according to claim 4, characterized in that The optimization problem in Step 2 is: C1: C2: C3: C4: C5: C6: C7: C8: C9: C10: C11: C12: C13: C14: Among them, In the formula, the decision variables are the associations between the UAVs and the clusters UAV caching decision and the 3D coordinates of the UAV is the association between UAV u and the i-th small cluster in the k-th large cluster during the content caching period t c is the caching decision of UAV u for the hot content q during the content caching period t c , λ q is the data volume of the hot content q, σ u is the caching capacity of UAV u respectively represent whether to select edge content retrieval, collaborative content retrieval, and remote content retrieval methods for content caching are the 2D coordinates of UAV u in the small time slot are the 2D coordinates of UAV u in the small time slot H min , H max respectively represent the maximum horizontal distance, vertical distance, minimum height, and maximum height of the UAV, D min is that the distance between UAVs should not be less than the minimum distance, T c is the length of the c-th content caching period is the maximum allowable energy consumption of UAV u in each content caching period is the total propulsion energy consumption of UAV u in the small time slot Q u is the long-term energy consumption upper limit of UAV u are the 3D coordinates of UAV u and UAV u' in the small time slot ​​​​ 6. The method for time delay and energy-aware multi-UAV cooperative caching and trajectory planning according to claim 2, wherein Step 3 specifically includes: Step 3-1, according to the spatial location information of mobile users, use the silhouette coefficient method to determine the optimal K value, and based on the determined K value, classify mobile users into large clusters according to the K-means algorithm; Step 3-2, according to the number of mobile users and the requirements of mobile users for hot content, classify the mobile users in each large cluster into small clusters.

7. The method for delay and energy-aware multi-UAV collaborative caching and trajectory planning according to claim 6, wherein The specific content of Step 3-1, which is to use the silhouette coefficient method to determine the optimal K value according to the spatial location information of mobile users and classify mobile users into large clusters based on the determined K value, specifically includes: Determine the K value and classify users into large clusters according to the spatial location of mobile users; the formula for the silhouette coefficient is as follows: Among them, where |C i1 | represents the number of samples in the cluster to which sample i1 belongs, a(i1) represents the cohesion of the sample point, which is the average distance between sample i1 and other samples in its cluster, j1 represents other sample points in the same cluster as sample i1, distance(i1, j1) is used to calculate the distance between sample i1 and sample j1, and the smaller a(i1) is, the tighter the cluster; b(i1) represents the separation of the sample point, which is the minimum average distance between sample i1 and other samples in any other cluster, where j2 represents other sample points in a different cluster from sample i1; Step 3-1-1: Select the range of the value of K, that is, from 2 to U-1; Step 3-1-2: Perform clustering for each value of K and calculate the silhouette coefficient of the clustering result; Step 3-1-3: Select the value of K with the largest silhouette coefficient as the final value of K and perform clustering again.

8. The method for delay and energy-aware multi-UAV cooperative caching and trajectory planning according to claim 6, wherein Step 3-2: According to the number of mobile users and the demand of mobile users for hot content, classify the mobile users in each large cluster into small clusters, which specifically includes: Step 3-2-1: Determine the number of small clusters in each large cluster according to the proportion of the number of mobile users in each large cluster; Step 3-2-2: Determine the number of mobile users in each small cluster within each large cluster according to the determined number of small clusters, and it is necessary to ensure an average to ensure load balancing; Step 3-2-3, perform partitioning using the K-means algorithm according to the request probability of mobile users for hot content: First, calculate the similarity S between the request probability Re of mobile user j for hot content in large cluster C i and the centroid k j as S = ||Re jk - μ j || i,k , where k ≤ N , and N i represents the number of small clusters in the i-th large cluster; then assign mobile user j to the small cluster C jk with the smallest S i,k , and finally update the centroid . i for the request probability Re of mobile user j for hot content in j the similarity S with the centroid k jk =||Re j -μ i,k ||, k≤N i ,N i represents the number of small clusters in the i-th large cluster; then assign mobile user j to the small cluster C jk with the smallest S i,k and finally update the centroid Step 3-2-4, if the number of mobile users in the small cluster obtained by partitioning exceeds the specified number of mobile users, then delete the mobile user with the smallest similarity in this small cluster and assign it to another cluster C i,t ≠C i,k that is most similar to this mobile user; repeat this step until the number of mobile users in all clusters reaches the preset legal quantity.

9. The method for delay and energy-aware multi-UAV collaborative caching and trajectory planning according to claim 2, wherein Step 4: Decompose the optimization problem into multiple sub-problems and solve them according to the principle of the divide-and-conquer method, which specifically includes: Step 4-1: Decompose the optimization problem into: optimization of the UAV cooperative caching placement strategy, optimization of the UAV flight trajectory, and optimization of the UAV-mobile user association; Step 4-2: For the optimization of the UAV cooperative caching placement strategy, adopt a multi-agent reinforcement learning algorithm, let each UAV be an agent, and learn the optimal caching strategy by interacting with other agents in a dynamic environment; For the optimization of the UAV flight trajectory, first handle the long-term energy constraint based on the Lyapunov optimization framework, and then solve it through the adaptive differential evolution algorithm; For the optimization of the UAV-mobile user association, use the method based on bipartite graph and LP relaxation to reduce the problem complexity and solve it.

10. The method for delay and energy-aware multi-UAV collaborative caching and trajectory planning according to claim 9, characterized in that, Step 5: Introduce an iterative algorithm and adopt an alternating optimization method for multiple sub-problems until convergence to achieve low-latency content delivery considering the limited battery energy of the UAV, which specifically includes: Step 5-1: Before performing iterative alternating optimization, first cluster the mobile users in the manner of Step 3 to obtain U small clusters; then initialize the association variable according to the distance between the UAVs and the mobile users within the clusters If the distances from several UAVs to the same small cluster are the shortest, select the UAV with the least remaining power to serve this cluster; Step 5-2, enter the iterative loop process: First, at each content caching period t c At the beginning, obtain the optimal UAV caching decision through Step 4-2, and the optimal flight trajectories of all UAVs at each content caching period t c Finally, given the UAV caching decision and the flight trajectory obtain the optimal association variable between the UAV and the mobile user Step 5-3: Repeat Step 5-2 until the convergence condition is reached, that is, the change in the objective function value between two adjacent iterations is less than a certain threshold ∈.

Images