An edge collaborative caching method and system for space-air-ground integrated network

Abstract

The application belongs to the technical field of edge computing, and relates to an edge cooperative caching method and system for space-air-ground integrated networks, the method comprising: calculating the average cache hit rate of content requested by a user equipment and the average content access delay of the user equipment; modeling the cache replacement of an edge server on a UAV or a satellite as a Markov decision process; reconstructing a reward function by maximizing the reward of a negative scalar to achieve the purpose of minimizing the system cost; and learning the optimal cache replacement strategy by maximizing the expected long-term negative reward based on the reconstructed reward function as an optimization target. The application avoids the privacy leakage problem of user data in the training process through the framework of federated learning, and uses a discrete flexible actor critic model to cope with the heterogeneous requests of a large number of users and the dynamics of the network, so as to improve the training efficiency and robustness of the network.

Description

Technical Field

[0001] This invention belongs to the field of edge computing technology and relates to an edge collaborative caching method and system for integrated air-space-ground networks. Background Technology

[0002] With the rapid development of the Internet of Things (IoT), the number of terrestrial IoT terminal devices and applications is constantly increasing. However, due to the high cost of building basic network communication infrastructure and geographical limitations, as well as the limited computing power of terminal devices in remote areas, a series of time-sensitive service requests cannot be processed and responded to in a timely manner. Although traditional cloud computing can provide higher computing power for user requests in these remote areas, the long transmission distance will cause significant transmission latency and consume huge bandwidth resources.

[0003] To address this issue, an integrated air-space-ground network is combined with edge computing. Leveraging the wide coverage of satellite communication networks, aerial drone networks can further assist satellite networks, alleviating congestion on the satellite backbone and reducing uplink load. By deploying a series of edge servers at the network edge of satellites or drones, the storage capacity of MEC servers can be utilized to cache frequently requested content. Mobile edge caching has attracted significant attention from industry and academia as a promising solution, helping to reduce content retrieval latency for users. Since popular content may be frequently requested by multiple users, caching and retrieving this content from MEC servers is more efficient than caching and retrieving it from a remote cloud. However, the cache size of MEC servers is generally limited compared to the ever-increasing data traffic. Furthermore, the constantly updating popularity of requested content and dynamic changes in network topology can easily lead to cache hit failures. Therefore, a carefully designed edge collaborative caching and cache replacement scheme is needed to improve cache hit rate and reduce latency for user-requested content access, thereby improving the QoE of user requests.

[0004] Some traditional rule-based algorithms (such as LRU and LFU) can only roughly predict content popularity to make ordinary cache replacement decisions. These traditional algorithms cannot cope with dynamic, large-scale user requests and complex mobile edge network environments. Existing large-scale artificial intelligence algorithms, such as deep learning and deep reinforcement learning, are considered promising technologies for handling dynamic network environments and large-scale user requests, and are widely used in edge caching. However, with the continuous uploading of user data during training, there is a high risk of user privacy leaks. Therefore, combining federated learning and deep reinforcement learning to propose a federated deep reinforcement learning-based edge collaborative caching strategy is of great significance in improving the cache hit rate of user requests and minimizing user content access latency while protecting user privacy. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes an edge collaborative caching method and system for integrated air-space-ground networks.

[0006] On the one hand, this invention proposes an edge collaborative caching method for integrated air-space-ground networks, comprising:

[0007] S1. Calculate the average cache hit rate of the content requested by the user equipment based on the popularity of the content requested by the ground user equipment and the cache resources on the edge servers of the associated drones and satellites.

[0008] S2. Calculate the average content access latency of the user equipment based on the content access latency of the ground user equipment on the corresponding drone or satellite;

[0009] S3. Based on the optimization objectives of maximizing the cache hit rate of user device requests and minimizing the average access latency of content, the cache replacement of edge servers on drones or satellites is modeled as a Markov decision process (MDP), which consists of a state space, an action space and a reward function.

[0010] S4. Reconstruct the reward function to minimize system cost by maximizing the reward of the negative scalar;

[0011] S5. Using a Federated Discrete Flexible Actor Critic (FDSAC) algorithm, based on the reconstructed reward function as the optimization objective, the optimal cache replacement strategy π is learned by maximizing the expected long-term negative reward. * .

[0012] On the other hand, this invention proposes an edge collaborative caching system for integrated air-space-ground networks, comprising:

[0013] Ground user equipment terminals are used to send content requests to edge servers on drones or satellites in the integrated air-space-ground network.

[0014] The global controller is used for centralized control and information management of inter-domain controllers in the drone swarm; the inter-domain controller is responsible for the allocation and management of various resource information within the swarm; the edge server is responsible for providing services to user devices for content requests.

[0015] The collaborative caching module is used to analyze the collaborative caching of edge servers in drones or satellite clusters and further increase the cache hit rate of user device requests.

[0016] The cache replacement module uses the FDSAC algorithm to obtain the optimal cache replacement strategy on drones and satellite edge servers, ultimately aiming to maximize the average cache hit rate of user devices and minimize the average access latency of content.

[0017] The beneficial effects of this invention are:

[0018] 1. This invention provides a three-layer edge computing network architecture of satellite-drone-ground. The drone network in the air can further assist the satellite network, reduce the congestion problem of the satellite backbone network and the burden on the uplink channel. By deploying MEC servers on satellites and drones and utilizing the caching capabilities of MEC servers to cache some popular content of users, the service requests of user devices in remote areas can be resolved.

[0019] 2. This invention provides a collaborative caching method. Through cluster collaboration, requests initiated by ground user equipment can be forwarded by drones or satellites within the corresponding cluster to edge servers on neighboring drones or satellites for retrieval, increasing the cache hit rate of user requests. Furthermore, each drone cluster has an inter-domain controller, and the satellite network has a global controller, ensuring the collaborative and stable operation of the entire network.

[0020] 3. This invention addresses the challenges of constantly updating popularity of user-requested content, limited caching capacity of MEC servers, and the dynamic nature of network topology by proposing a Federated Discrete Flexible Actor-Commentator (FDSAC) algorithm to solve the cache replacement problem and minimize user content access latency. It avoids privacy leaks of user data during training by employing a federated learning framework and utilizes the Discrete Flexible Actor-Commentator (DSAC) model to handle heterogeneous requests from a large number of users and the dynamic nature of the network, thereby improving network training efficiency and robustness. Attached Figure Description

[0021] Figure 1This is an example diagram of a three-layer edge computing network architecture consisting of satellite, drone, and ground, provided in an embodiment of the present invention.

[0022] Figure 2 A flowchart of edge collaborative caching in an integrated air-space-ground network provided in an embodiment of the present invention;

[0023] Figure 3 The Discrete Flexible Actor Critic (DSAC) model provided for embodiments of the present invention employs a framework diagram of federated learning;

[0024] Figure 4 This diagram illustrates the training process of the edge cache replacement algorithm based on federated deep reinforcement learning, as provided in an embodiment of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] like Figure 1 As shown in the diagram, this architecture primarily addresses the challenges of poor network communication infrastructure and geographical limitations in remote areas. Leveraging the wide coverage of satellite networks, it provides comprehensive network services to user devices in these regions. Furthermore, it utilizes aerial drone networks to alleviate congestion on the satellite backbone and reduce uplink load, serving as a supplementary means to the satellite network. Simultaneously, the satellite connects to a remote cloud server with ample computing and storage resources, ensuring the processing of user requests from the satellite.

[0027] This embodiment 1 provides the construction of a three-layer edge computing network consisting of satellite, drone, and ground, such as... Figure 1 As shown, it includes the following steps:

[0028] S11. Deploy MEC servers on both satellites and drones. Each MEC server has a certain caching capability to cache some popular content requested by users.

[0029] S12. Deploy an inter-domain controller in the aerial drone swarm to facilitate collaborative caching among drones. The inter-domain controller can uniformly manage the caching information and collaboration of drones throughout the entire swarm.

[0030] S13. Deploy a global controller in the satellite network to centrally control the aerial drone swarm and manage the collaborative operation between satellites. At the same time, deploy a pre-trained cache replacement algorithm based on federated deep reinforcement learning in the global controller to ensure the high-speed and stable operation of the entire integrated air-space-ground network.

[0031] like Figure 2 As shown in the figure, this embodiment provides a flowchart of edge collaborative caching in an integrated air-space-ground network. The specific implementation steps are as follows:

[0032] S21 Consider a real-world scenario where user requests are random and follow a Zipf distribution. Let p... f Let f be the local popularity of content in the edge server. Then the global popularity distribution is P = {p1, p2, ..., p...} F}, where p F Let F represent the popularity of the F-th content. For any edge server, the probability formula for a ground user device requesting content f in time slot t is: And there are in, This indicates the number of times the ground equipment requests content f in time slot t. This represents the total number of requests made by ground equipment.

[0033] All content requests from the S22 ground user equipment are retrieved by being transmitted in parallel to the edge servers on the corresponding satellites or drones.

[0034] When the content requested by a ground user is sent to the corresponding drone, the edge server checks whether the content is cached. If the drone does not have the requested content cached, the request will be sent to its neighboring drone within the cluster. m A hopping drone, if adjacent n m If the drone within the hop also caches the content, it's considered a cache hit; otherwise, the cache hit fails. If a cache hit fails, the request is forwarded to the satellite to retrieve the content. Therefore, the cache hit formula on the drone is:

[0035]

[0036] in, This indicates a cache hit on the corresponding drone m; otherwise... This indicates that in the case of drone m, n are adjacent m If the drone collaboration cache within the jump is hit, otherwise...

[0037] and The specific calculation formula is as follows:

[0038]

[0039] in, This indicates the relationship between the requested content f and the cache on the drone's edge server. If content f has already been cached on drone m, then... otherwise, Similarly; This represents the correlation between ground user equipment u and drone m. If user equipment u is within the coverage area of ​​drone m, that is, if l is satisfied... u,m =||loc u -loc m If ||≤R1, then we have otherwise l u,m This represents the distance information between user equipment u and drone m, loc u The coordinate position information of user equipment u, loc m R1 represents the coordinates of the drone m, and R1 represents the radius of the communication coverage area of ​​the drone m to the ground user equipment.

[0040] S24 requests content from the user equipment on the drone or adjacent n m The formula for drone cache hits within a hop, and the formula for calculating the access latency of content requests on the drone edge server are as follows:

[0041]

[0042] Here, α∈{0,1} is a binary variable. When α=1, it means that the content request was cached on the corresponding drone; otherwise, if it was not cached on the local drone, the request will be sent to its neighboring n. m The drone inside the jump; R m,m For the transmission rate between drones, R u,m This indicates the transmission rate between the user device and the drone. c is the speed of light, d m h is the distance between adjacent drones. m,υ R is the distance from the drone to the satellite. m,υ This refers to the transmission rate from the drone to the satellite.

[0043] When the content requested by a ground user is sent to the corresponding satellite, the edge server checks whether the content is cached. If the corresponding satellite does not have the requested content cached, the request will be sent to its neighboring n satellites within the cluster. s A hopping satellite, if adjacent n sIf the satellite within the hop does not cache the content, it is considered a cache failure, and the request will eventually be forwarded to the cloud server for processing. Therefore, the cache hit formula on the satellite is:

[0044]

[0045] in, This indicates a cache hit on the corresponding satellite v; otherwise... This indicates that satellite v is adjacent to n. s If the satellite cooperative cache within the jump is hit, otherwise...

[0046] and The specific calculation formula is as follows:

[0047]

[0048] in, This indicates the relationship between the requested content f and the cache on the satellite's upper edge server. If content f was previously cached on satellite v, then... otherwise, Similarly; This represents the correlation between ground user equipment u and satellite v. If user equipment u is within the coverage area of ​​satellite v, i.e., satisfying l u,υ =||loc u -loc υ If ||≤R2, then we have otherwise l u,υ This represents the distance information between user equipment u and satellite v, loc u The coordinate position information of user equipment u, loc υ R1 represents the coordinates of satellite v, and R2 represents the radius of the communication coverage area of ​​satellite v to ground user equipment.

[0049] S26 requests content from user equipment on satellite or neighboring n s The formula for satellite cache hits within a hop, and the formula for calculating the access latency of content requests on the satellite edge server are as follows:

[0050]

[0051] Here, β∈{0,1} is also a binary variable. When β=1, it means that the content request was cached on the corresponding satellite; otherwise, if it was not cached on the local satellite, the request will be sent to its neighboring n. s Satellite within the jump; R υ,υ R is the transmission rate between satellites.u,υ Indicates the transmission rate between the user equipment and the satellite. c is the speed of light, d υ d represents the distance between adjacent satellites. c R is the distance from the satellite to the cloud server. υ,c This refers to the transmission rate from the satellite to the cloud server.

[0052] S27 uses a cache hit formula on edge servers for drones and satellites. The average cache hit rate of the content requested by ground user equipment can be obtained as follows:

[0053]

[0054] Where U represents the total number of user devices on the ground. It represents the popularity of the content requested by users.

[0055] S28 can calculate the average content access latency for ground user equipment by using the cache hit rate of content requests on edge servers on satellites and drones, as well as the content access latency of requests on satellites and drones.

[0056]

[0057] S29 When a user device's requested content fails to hit the cache of an edge server on a drone or satellite, the edge server should replace the cached content to improve the cache hit rate of the user device's requested content and reduce the average content access latency. The cache replacement of the edge server on the drone or satellite is modeled as a Markov Decision Process (MDP), which consists of a state space, an action space, and a reward function.

[0058] State space: It consists of the edge server's cache state and the user device's request state. The user device's request state is represented as... The content caching status of the edge server on the drone or satellite is as follows: This indicates that the content f is cached on the corresponding drone or satellite; otherwise,

[0059] Action space: set For cache replacement actions on edge servers of drones or satellites, where when This indicates that the cached content f should be replaced; otherwise, the cached content will not be replaced, and the user request will be processed directly.

[0060] Reward function: Since cache hit cost is negatively correlated with cache hit rate, we express cache hit cost as... To maximize the cache hit rate of user device requests and minimize the average access latency of content, the reward function is defined as a weighted sum of cache hit cost and latency cost, where λ1 and λ2 represent the weights of cache hit cost and content access latency cost, respectively.

[0061] Furthermore, to simplify the calculation, the reward is represented as... Our goal is to minimize system cost by maximizing the reward of a negative scalar. Therefore, our optimization objective is to learn the optimal policy π by maximizing the expected long-term cumulative negative reward. * ,Right now

[0062]

[0063] Where γ∈(0,1] represents the attenuation factor.

[0064] like Figure 3 As shown, based on the solution to the cache replacement problem, a Federated Discrete Flexible Actor-Critic (FDSAC) algorithm is proposed to make the optimal cache replacement strategy. This embodiment provides a framework diagram of the Discrete Flexible Actor-Critic (DSAC) model using federated learning. It is a multi-intelligence discrete variant of the Flexible Actor-Critic (SAC), consisting of an actor network and a critic network. The actor network is responsible for learning the strategy to make replacement actions, and the critic network is responsible for evaluating the strategy to improve training efficiency. Then, the actor network is improved by integrating Long Short-Term Memory (LSTM) to capture the temporal dependencies of replacement actions; finally, the critic network is improved by combining federated learning with an attention mechanism to accelerate network training efficiency and protect user privacy. The design of the neural network in the Discrete Flexible Actor-Critic (DSAC) is as follows:

[0065] The actor network and critic network in S31 DSAC are both constructed from fully connected networks;

[0066] S32 will and Let θ and θ be the parameterized flexible Q-function and parameterized policy function of the deep neural network (DNN), respectively. Set as parameters for the DNN;

[0067] S33 adds an LSTM layer between the hidden layer and the output layer, which contains a set of hidden neurons for extracting hidden information and predicting future changes in the time series;

[0068] S34 combines the output of the fully connected layer and the hidden matrix H(t) as input, where the hidden matrix is ​​responsible for dynamically recording all historical information of observation and action;

[0069] S35 finally outputs the replacement action for the current time slot by using the feature representation of the replacement actions of past time slots through the LSTM layer.

[0070] Specifically, DSAC's policy π is used to maximize reward and entropy, that is:

[0071]

[0072] Where, π * This represents the optimal strategy. Let δ represent the entropy used to measure the uncertainty of a random variable, and let δ represent the temperature parameter indicating the relative importance of equilibrium entropy and reward. This represents the state information of the edge server when i = t, γ i-t This represents the attenuation factor when i = t. This represents the reconstructed reward value.

[0073] To evaluate all finite discrete actions, the flexible Q-function of DSAC corresponds to the Q-value of the output for each possible action. Therefore, the flexible Q-function is defined as:

[0074]

[0075] in, This represents a flexible value function with entropy-enhancing cumulative returns, where γ represents the decay factor. This indicates the status information of the edge server at time t+1.

[0076] To ensure that the optimal policy is discrete in the state-action space and can be found within a finite number of iterations, a functional approximation (i.e., a deep neural network (DNN)) is used to find the optimal policy for DSAC. and Let θ and θ be the parameterized flexible Q-function and parameterized policy function of the DNN, respectively. These are the parameters of the DNN. The mean square error of the flexible Bellman residual is minimized as... The objective function, i.e.

[0077]

[0078] Where D is the experience replay buffer, used to store historical information; The target flexible Q function, This indicates that minimizing the mean square error of the flexible Bellman residual is used as... The objective function.

[0079] J Q The gradient formula for (θ) is:

[0080]

[0081] Similarly, minimizing the KL divergence is used as... Objective function:

[0082]

[0083] Because this strategy outputs a precise action distribution, unlike the SAC strategy which requires parameter re-initialization for continuous actions, it provides a more accurate action distribution. The objective function can be further written as:

[0084]

[0085] The gradient formula is:

[0086]

[0087] The federated learning process in the FDSAC algorithm includes:

[0088] First, the DSAC model is deployed on edge servers on drones and satellites, and the critic network of the DSAC model is deployed on cloud servers as the central critic network.

[0089] Then, the actor network and critic network in the DSAC model on the edge server are separated;

[0090] Next, the trained parameters of the critic network in the DSAC model on all edge servers are aggregated into the central critic network on the cloud server;

[0091] Finally, the global parameters of the central critic network are updated and passed down to the DSAC model in the edge server for the next round of training. Furthermore, the attention mechanism is used to enhance better agents and reduce worse agents based on all the rewards of the agents.

[0092] like Figure 4 This example provides a training process diagram for an edge cache replacement algorithm based on federated deep reinforcement learning. It employs a federated discrete flexible actor critic (FDSAC) algorithm, using the reconstructed reward function as the optimization objective, and learns the optimal cache replacement policy π by maximizing the expected long-term negative reward. *

[0093] The S40 Discrete Flexible Actor Critic (DSAC) model is pre-trained on the margins using historical data;

[0094] S41 Initializes network parameters θ. Total rewards for edge servers Aggregate the quantity Δ and the experience replay buffer, and transmit the initialization parameters to the cloud server;

[0095] S42 observes the local state on the edge server's agent. And make the corresponding replacement actions.

[0096] S43 further derives its current reward based on the status and replacement actions of the edge server. and the next state

[0097] The total rewards for the S44 edge server have been updated to... Initially, the total reward for edge servers was... Initialized to 0, it will receive a reward value after the corresponding replacement action. Therefore, the total reward update for the edge server is equal to... Equivalent to

[0098] S45 will use a quaternion set Store in the playback buffer D;

[0099] S46 then extracts a batch of tuple data from the playback buffer to participate in learning and passes it through J. Q (θ) and The gradient formula is used to update the parameters θ of the actor network and the critic network.

[0100] After multiple iterations, when the number of training episodes equals a multiple of the number of aggregations, all agents on the edge servers upload their evaluator network parameters and total rewards to the central evaluator network.

[0101] S48 then uses a cloud server to calculate the parameters of the central critic network. And update it.

[0102] S49 will again update the central commentator network parameters Download to the network of critics on all edge servers;

[0103] After a certain number of iterations, S50 achieved the training accuracy for its model parameters and output the optimal replacement strategy π. * .

[0104] An edge collaborative caching system for an integrated air-space-ground network includes: a ground user equipment terminal for sending content requests to an edge server on a drone or satellite in the integrated air-space-ground network;

[0105] The global controller is used for centralized control and information management of inter-domain controllers in the drone swarm; the inter-domain controller is responsible for the allocation and management of various resource information within the swarm; the edge server is responsible for providing services to user devices for content requests.

[0106] The collaborative caching module is used to analyze the collaborative caching of edge servers in drones or satellite clusters and further increase the cache hit rate of user device requests.

[0107] The cache replacement module uses the FDSAC algorithm to obtain the optimal cache replacement strategy on drones and satellite edge servers, ultimately aiming to maximize the average cache hit rate of user devices and minimize the average access latency of content.

[0108] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An edge collaborative caching method for integrated air-space-ground networks, characterized in that, include: S1. Calculate the average cache hit rate of the content requested by the user equipment based on the popularity of the content requested by the ground user equipment and the cache resources on the edge servers of the associated drones and satellites. S2. Calculate the average content access latency of the user equipment based on the content access latency of the ground user equipment on the corresponding drone or satellite; S3. Based on the optimization objectives of maximizing the cache hit rate of user device requests and minimizing the average access latency of content, the cache replacement of the edge server on the drone or satellite is modeled as a Markov decision process (MDP), which consists of a state space, an action space and a reward function. S4. Reconstruct the reward function to minimize system cost by maximizing the reward of the negative scalar; S5. Using a Federated Discrete Flexible Actor Critic (FDSAC) algorithm, based on the reconstructed reward function as the optimization objective, the optimal cache replacement strategy is learned by maximizing the expected long-term negative reward. ; The aforementioned Federation Discrete Flexible Actor Critic (FDSAC) algorithm includes: An actor network and a critic network are constructed through a fully connected network. The actor network and the critic network together form a multi-intelligent discrete variant of the flexible actor-critic (SAC). Improve the actor network by integrating long short-term memory (LSTM) to capture the temporal sequence dependence of alternative actions, and improve the critic network by combining federated learning with attention mechanisms; The improvement of the actor network by integrating Long Short-Term Memory (LSTM) to capture the temporal dependencies of alternative actions includes: Will and Let these be the parameterized flexible Q-function and parameterized policy function of a deep neural network (DNN), respectively. and Set as parameters for the DNN. This indicates the local state of the edge server; Add an LSTM layer between the hidden layer and the output layer, which contains a set of hidden neurons for extracting hidden information and predicting future changes in the time series; The output of the fully connected layer and the hidden matrix H(t) are combined as input, where the hidden matrix is ​​responsible for dynamically recording all historical information of observation and action; The replacement action for the current time slot is output by using the feature representation of the replacement actions of past time slots through the LSTM layer; The improvement of the commentator network by combining federated learning with attention mechanisms includes: Discrete Flexible Actor-Critic (DSAC) models are deployed on edge servers on drones and satellites, and a network of critics for the discrete Flexible Actor-Critic (DSAC) models is deployed on cloud servers as the central critic network. Separate the actor network and critic network in the discrete flexible actor-critic (DSAC) model on the edge server; The trained parameters from the critic networks of all discrete Flexible Actor Critic (DSAC) models on all edge servers are aggregated into a central critic network on a cloud server; The global parameters of the central critic network are updated and then passed down to the discrete flexible actor critic (DSAC) model in the edge server for the next round of training. Attention mechanisms are also used to enhance better agents and reduce worse agents based on all the rewards of the agents.

2. The edge collaborative caching method for an integrated air-space-ground network according to claim 1, characterized in that, Calculate the average cache hit rate for content requested by ground user equipment, including: The popularity of the requested content is obtained based on the probability formula of the ground user equipment requesting content f in time slot t. ; The cache hit rate of the user's requested content on the drone and satellite is obtained based on the cache relationship between the user equipment's request content and the edge server on the drone or satellite, as well as the association between the ground user equipment and the drone or satellite. and ; The average cache hit rate of ground user equipment requests is calculated based on the popularity of the requested content and the cache hit rate of the requested content on drones and satellites. .

3. The edge collaborative caching method for an integrated air-space-ground network according to claim 2, characterized in that, The average cache hit rate of ground user equipment requests is calculated based on the popularity of the requested content and the cache hit rate of the requested content on drones and satellites. ,include: ； Where U represents the total number of ground user equipment, , These represent cache hits on the drone and satellite, respectively. This indicates the popularity of the content requested by the user.

4. The edge collaborative caching method for an integrated air-space-ground network according to claim 1, characterized in that, Calculate the average content access latency for user devices, including: The uplink transmission rates from ground user equipment to drones and satellites were calculated using Shannon's formula. and ; When a user device requests content on an associated drone or in a nearby location When the drone cache is hit within the jump, the content access latency at the drone location is obtained. ; When a user device requests content on an associated satellite or neighboring satellite... When the satellite cache is hit within the jump, the content access latency at the satellite is obtained. ; Based on content access latency at the drone location Delay in accessing content from satellite Calculate the average content access latency of terrestrial user equipment .

5. The edge collaborative caching method for an integrated air-space-ground network according to claim 4, characterized in that, Based on content access latency at the drone location Delay in accessing content from satellite Calculate the average content access latency of terrestrial user equipment ,include: ； in, This indicates the transmission delay from the drone to the satellite. This represents the transmission latency from the satellite to the cloud server, where U represents the total number of ground user devices. Indicates the popularity of the content requested by the user. , These represent cache hits on the drone and satellite, respectively. This indicates a delay in content access at the drone location. This indicates a delay in accessing content at the satellite location.

6. The edge collaborative caching method for an integrated air-space-ground network according to claim 1, characterized in that, Modeling cache replacement on edge servers of drones or satellites as a Markov decision process (MDP) includes: State space: ,in, This indicates the local state of the edge server. Indicates the request status of the user device. Indicates the amount of content requested by the user device; This indicates the content caching status of the edge server on the drone or satellite. This indicates that the content f is cached on the corresponding drone or satellite; otherwise, ; Action space: set For cache replacement actions on edge servers of drones or satellites, where ,when This indicates that the cached content f should be replaced; otherwise, the cached content will not be replaced, and the user request will be processed directly. Reward function: ,in, Indicates the reward value. This represents the average content access latency for terrestrial user equipment; This represents the cache hit cost. Since the cache hit cost is negatively correlated with the cache hit rate, the cache hit cost is expressed as... ;in, , These represent the weights of cache hit cost and content access latency cost, respectively. This represents the average cache hit rate.

7. The edge collaborative caching method for an integrated air-space-ground network according to claim 1, characterized in that, The reward function is reconstructed to minimize system cost by maximizing the reward of the negative scalar, including: ； in, Indicates the caching strategy. This indicates the operation of finding the expected value. This represents the attenuation factor when i=t. This represents the reconstructed reward value when i=t. , This indicates a cache replacement action on the edge server of a drone or satellite. This indicates the local state of the edge server. This represents the reward value.

8. The edge collaborative caching method for an integrated air-space-ground network according to claim 1, characterized in that, The Federated Discrete Flexible Actor Critic (FDSAC) algorithm, based on the reconstructed reward function, learns the optimal cache replacement strategy by maximizing the expected long-term negative reward. ,include: Initialize network parameters Actor network parameters , Commentator network parameters , Central commentator network parameters Total rewards for edge servers =0, number of aggregates And the experience replay buffer D; The agent on the edge server observes the local state. And make the corresponding replacement actions. ; Based on the local state of the edge server and replacement action Further obtain the current rewards and the next state ,in, Indicates the reward value; Update the total reward for the edge server to the current reward plus the initial reward; quaternion Store in the experience replay buffer D; A batch of tuple data is drawn from the experience replay buffer D to participate in learning and update the parameters of the actor network and critic network. and ; After multiple iterations, when the number of training episodes is equal to a multiple of the number of aggregations, all agents on the edge servers upload their commentator network parameters and total rewards to the central commentator network. The parameters of the central critic network are calculated by the cloud server. And update it; Update the central critic network parameters Download to the network of critics on all edge servers; After multiple iterations, until the model parameters reach the required training accuracy, the optimal replacement strategy is output. .

9. An edge collaborative caching system for an integrated air-space-ground network, used to implement the edge collaborative caching method for an integrated air-space-ground network as described in any one of claims 1-8, characterized in that, include: Ground user equipment terminals are used to send content requests to edge servers on drones or satellites in the integrated air-space-ground network. The global controller is used for centralized control and information management of inter-domain controllers in the drone swarm; the inter-domain controller is responsible for the allocation and management of various resource information within the swarm; the edge server is responsible for providing services to user devices for content requests. The collaborative caching module is used to analyze the collaborative caching of edge servers in drones or satellite clusters and further increase the cache hit rate of user device requests. The cache replacement module uses the FDSAC algorithm to obtain the optimal cache replacement strategy on drones and satellite edge servers, ultimately aiming to maximize the average cache hit rate of user devices and minimize the average access latency of content.

Images