Space-air-ground integrated network content caching method and system

Abstract

The application discloses a space-air-ground integrated network content caching method and system, which comprises the following steps: establishing a network model, constructing a ground and sky wireless communication model on the network model; a user in the network model initiates a content request to a terminal based on the communication model, and the terminal requests the content from a nearest available edge caching node; the edge caching node checks whether the local caching database contains the content, if the local database contains the content, the content is directly delivered to the user terminal; if the local database does not contain the content, a deep reinforcement learning method based on a DQN algorithm framework is used to update the local caching content, and the caching hit rate is calculated. The application uses the space-air-ground integrated network to cache the content, and has the advantages that the space-air-ground integrated network has a larger coverage range than a traditional ground cellular network, wherein an aerial network composed of unmanned aerial vehicles and geosynchronous orbit satellites and the like provides a flexible network access method for ground equipment, and can meet the individualized needs of users.

Description

Technical Field

[0001] This invention belongs to the field of data caching technology and relates to a method and system for caching network content in an integrated space-air-ground system. Background Technology

[0002] Space-Air-Ground Integrated Networks (SAGIN) are widely recognized as an important 6G layered architecture. The heterogeneous communication and computing environment enables SAGIN to provide diverse, seamless, and resilient network access for various services and applications, among which content caching is a very important use case.

[0003] Storing content at different layers of an integrated air-space-ground network, rather than solely on a remote cloud server, can significantly reduce user latency, alleviate pressure on cloud servers, reduce the load on the core network, achieve differentiated traffic transmission quality, and meet diverse user requirements for service quality. However, due to the different network service and transmission quality requirements within an integrated air-space-ground network, and the heterogeneous hardware with varying storage capacities, it is essential to determine the optimal content caching location and content.

[0004] However, existing research focuses on optimizing the latency and resource consumption of content transmission in different scenarios. Due to the highly dynamic nature of user demand, content popularity has temporal and regional characteristics. The optimal content caching location and cached content become more difficult to determine with the dynamic changes in spatiotemporal content popularity and user demand. Therefore, there are still many gaps in content caching research that considers dynamic spatiotemporal content popularity. Summary of the Invention

[0005] The purpose of this invention is to provide an integrated air-space-ground network content caching method and system to solve the above-mentioned problems.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides an integrated air-space-ground network content caching method, comprising:

[0008] Establish a network model that includes users, terminals, and edge buffer nodes, and build terrestrial and sky wireless communication models on the network model;

[0009] In the network model, users initiate content requests to the terminal based on the communication model, and the terminal requests the content from the nearest available edge cache node.

[0010] The edge cache node checks whether the content is contained in the local cache database. If the content is contained in the local database, the content is delivered directly to the user terminal.

[0011] If the local database does not contain the content, the edge cache node requests the content from the data center and then delivers it to the user. The deep reinforcement learning method of the DQN algorithm framework is used to update the local cached content and calculate the cache hit rate.

[0012] Optionally, the network model may specifically include:

[0013] The network consists of three layers: a space-based network, a sky-based network, and a ground-based network. Public information content is stored on ground-based cloud servers, with geostationary satellites providing access to these cloud servers, and multiple edge servers deployed on drones. A network model is constructed using N = {EU, AN, SN}, where N contains three subsets representing all nodes in the network model. The specific interpretation of N is as follows: EU = {E1, E2, ..., E...} r ,...,E R} is the complete set of R end-user nodes, E r Represents user nodes; AN = {v1, v2, ..., v j ,...,v J} is the complete set of J drone nodes, v j SN represents a drone node; SN = {g} is the complete set of satellite nodes, where g represents a geostationary satellite node.

[0014] Optionally, the communication model is defined as follows:

[0015] Using a logarithmic distance path loss model and based on Shannon's theorem, the wireless communication data transmission rates for ground and sky are derived mathematically as follows:

[0016]

[0017] Among them, B AG P represents the channel bandwidth for communication between the UAV and the ground; AG This indicates the signal power used for communication between the drone and the ground. The Gaussian noise power within the communication channel between the UAV and the ground; d H This indicates the hovering altitude of the drone; n represents the path loss exponent.

[0018] Using the free-space propagation model commonly used in satellite systems, the free-space path loss is defined as follows:

[0019]

[0020] Among them, G t Indicates the antenna gain of the drone; G r Indicates the antenna gain of the satellite; λ S Indicates the signal wavelength; d SIndicates the distance between the drone and the satellite;

[0021] Introducing a loss factor α, the data transmission rate between the UAV and the geostationary satellite is defined as follows:

[0022]

[0023] Among them, B SA P represents the channel bandwidth for communication between the drone and the satellite; SA This indicates the transmission power of the drone's communication with the satellite; This represents the Gaussian noise power within the channel for communication between the drone and the satellite.

[0024] Optionally, the local cached content can be updated using a deep reinforcement learning method within the DQN algorithm framework, and the cache hit rate can be calculated.

[0025] Initialize the current network parameters θ and the target network parameters θ - ;

[0026] The current state s(t) is obtained from previous requests in the environment, the current cache state, and the content popularity; the local cache of the node is queried according to the index of the current request content sequence, and the decision to update the cache is determined through mapping, i.e., action a(t);

[0027] Execute action a(t), observe the reward r(t) and the next state s(t+1);

[0028] Through Q(s′,a′;θ) - Calculate the target Q value by using the value network Q value calculated through Q(s,a;θ);

[0029] Calculate the loss of the neural network and backpropagate to update θ;

[0030] Every N steps, the parameters of the current value network are copied to the target value function, i.e., θ. - =θ;

[0031] Let s(t) = s(t+1) and a(t) = a(t+1), and repeat the above steps until the set number of training steps is reached.

[0032] Optionally, the Cache Hit Ratio (CHR) represents the proportion of content successfully retrieved from the cache, and is a key indicator for evaluating the effectiveness of a caching system; suppose that within a certain time period t, the number of drone nodes v j The total number of user requests received was E f Then calculate the long-term cache hit rate H. T The mathematical expression is as follows:

[0033]

[0034] Where: the cache flag is

[0035] Suppose there are L cached content files, denoted by f∈{f1,f2,...,f...} l ,...,f L Let} represent each file, and let CMB be the storage capacity of each drone. Let user E be defined at time t. r The requested file f l In drone v j The cache flag in is Its mathematical expression is:

[0036]

[0037] Considering that the number of user requests varies at different times, let the number of user requests follow a Poisson distribution with parameter p as follows:

[0038]

[0039] User requests arrive one by one, and the popularity distribution of files among all users is approximately described by a Zipf distribution with parameter α. Assuming the files are arranged in descending order of content popularity, and the file sequence changes with time t, the probability that a user requests the content ranked nth in popularity at time t is expressed as:

[0040]

[0041] in, User E at time t r The requested file f l Right now An index of request frequency.

[0042] Optional, objective function:

[0043] The cached content model is modeled as a long-term cache hit rate H that maximizes given parameters. T The problem, specifically the objective function, is expressed as:

[0044] maxH T

[0045] stC1:N i ∈N={EU,AN,SN}

[0046]

[0047] C3:H T ∈[0,1]

[0048]

[0049] Among them, C1 indicates that the communication node should be a node in SAGIN; C2 indicates that the cumulative size of all cached content on the drone v j does not exceed CMB; C3 indicates that the cache hit rate H T has a value range of [0, 1]; C4 indicates the cache flag takes 0 or 1 according to whether the cache is hit.

[0050] Optionally, the environment is characterized by the system state, and the tasks related to content caching and decision update are encapsulated within the framework of MDP;

[0051] The state space of the system is defined by previous requests, the current cache state, and content popularity. The state space at time t is defined as s(t) = [L(t - 1), C(t), P(t)], where L(t - 1) represents the request state at the previous moment, including the request frequencies of all contents during this period, C(t) represents the current cache state, indicating the file contents stored on the current node, and P(t) represents the current content popularity;

[0052] Assume that when updating the cache each time, the sequence of contents already cached in the node is (f 1 , f 2 , …, f Z ), and the sequence of requested contents is a partial content of all content sequences (f 1 , f 2 , …, f L ), Z < L; map the contents already cached to the sequence of requested contents to indicate the cache decision; when the agent outputs the key of the cache decision, retrieve the corresponding decision index to cache and update the content; the action space at time t is defined as a t = {∈ 1 , ε 2 , …, ∈ Z}, where ∈ z ∈ (f 1 , f 2 , …, f L ); [

[0053] The reward function is defined as the difference between the number of cache hits and the total number of requests, and this value is a non - positive number; in this scenario, the higher the cache hit rate, the closer the reward value is to 0, and the lower the cache hit rate, the closer the absolute value of the reward value is to the total number of requests, that is, the smaller the value; assume that the total number of requests within the time period t is E f , and the total number of hits of the neural network cache decision is The reward for this time period is defined as [

[0054] The popularity distribution of a file among all users follows a Zipf distribution, and the probability of a user requesting certain content at time t is defined as follows: in User E at time t r The requested file f l Right now The index of the request frequency, where α is the parameter of the Zipf distribution. Each user makes a serial request, and the environment provides a pre-defined distribution function for the popularity of cached content. The number of cache hits and requests is recorded during the interaction with the agent.

[0055] When a user sends a request to the terminal, the nearest available edge cache node searches its local cache database. If the cached content is found, it is returned directly to the user. If the content is not found in the local database, the edge cache node requests the content from the data center via a geostationary satellite.

[0056] Secondly, the present invention provides an integrated air-space-ground network content caching system, comprising:

[0057] The model building module is used to build a network model that includes users, terminals, and edge cache nodes, and to build terrestrial and sky wireless communication models on the network model.

[0058] The content request module is used by users in the network model to initiate content requests to the terminal based on the communication model, and the terminal requests the content from the nearest available edge cache node.

[0059] The judgment module is used by the edge cache node to check whether the local cache database contains the content. If the local database contains the content, the content is delivered directly to the user terminal.

[0060] If the local database does not contain the content, the edge cache node requests the content from the data center and then delivers it to the user. The deep reinforcement learning method of the DQN algorithm framework is used to update the local cached content and calculate the cache hit rate.

[0061] Thirdly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of an integrated air-space-ground network content caching method.

[0062] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of an integrated air-space-ground network content caching method.

[0063] Compared with the prior art, the present invention has the following technical effects:

[0064] This invention provides a content caching method based on a deep reinforcement learning-integrated air-space-ground network. The advantage of using an integrated air-space-ground network for content caching is that it offers a wider coverage area compared to traditional terrestrial cellular networks. The aerial network, composed of drones and geostationary satellites, provides flexible network access methods for ground devices, meeting users' personalized needs.

[0065] Modeling the content update problem using Markov Decision Processes (MDPs) offers several advantages. MDPs define the state and behavior of content caching as a sequence of states and actions, providing a formal framework to describe the interaction between the intelligent agent and its environment. This allows decision-makers to better understand the caching mechanism, helping them formulate optimal strategies. Furthermore, it facilitates simulation and evaluation of different content caching strategies, leading to optimized performance. Simultaneously, the model considers uncertainties in user distribution and network topology within an integrated air-space-ground network environment, such as user mobility and dynamic changes in network topology, resulting in more accurate and reliable decision-making.

[0066] By employing a deep reinforcement learning approach based on the DQN algorithm to optimize long-term cache hit rate, the advantages are that deep reinforcement learning utilizes neural networks to approximate values ​​or policy functions, effectively addressing the challenges of high-dimensional state and action spaces. The DQN algorithm can learn online, adjusting and optimizing according to different business scenarios and user behaviors, better adapting to the heterogeneous environment of dynamically changing cache capacity and content popularity in an integrated space-air-ground network. Simultaneously, our cache update algorithm introduces the concept of Double Q-learning on top of DQN, utilizing the current network to select actions and the target network to estimate Q-values, thus solving the overestimation problem in deep reinforcement learning and achieving better performance in high-dimensional state and action spaces.

[0067] The proposed method was evaluated using a randomly generated dataset. The results show that, compared with traditional First In First Out (FIFO), Least Frequently Used (LFU), and Random Caching (RC) caching strategies, this method can significantly improve the long-term cache hit rate under various conditions, effectively enhancing the user experience. Attached Figure Description

[0068] Figure 1 This is a diagram of the content caching layered architecture based on SAGIN as described in this invention;

[0069] Figure 2 This is a diagram of the DQN algorithm framework to which this invention pertains.

[0070] Figure 3 This is a comparison chart of the rewards of this scheme with FIFO, LFU, and RC at different training steps.

[0071] Figure 4 This is a comparison chart of the average latency of this solution with FIFO, LFU, and RC under different request numbers.

[0072] Figure 5 This is a comparison chart of the cache hit rates of this solution with FIFO, LFU, and RC under different request volumes. Detailed Implementation

[0073] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0074] The network model involved in the deep reinforcement learning-based integrated air-space-ground network content caching method described in this invention is defined as follows:

[0075] Reference Appendix Figure 1 As shown, the network model of this invention is divided into three layers: a space layer network, a sky layer network, and a ground layer network. Satellites and drones are responsible for collecting public information such as weather and terrain information, which is considered content to be stored. All content is stored in cloud servers deployed on the ground, with geostationary satellites providing access services to the cloud servers. Some content is stored on multiple edge servers deployed in drones based on its spatiotemporal popularity. Users can request content from drones or from the cloud server via geostationary satellites. If the requested content is already cached in the drone, it can be delivered directly to the user. Otherwise, the drone will request content from the cloud server via satellite. We construct the network model using N = {EU, AN, SN}, where N contains three subsets representing all nodes in the network model. The specific interpretation of N is as follows: EU = {E1, E2, ..., E...} r ,...,E R} is the complete set of R end-user nodes, E r Represents user nodes; AN = {v1, v2, ..., v j ,...,v J} is the complete set of J drone nodes, v j SN represents a drone node; SN = {g} is the complete set of satellite nodes, where g represents a geostationary satellite node.

[0076] The communication model involved in the deep reinforcement learning-based integrated air-space-ground network content caching method described in this invention is defined as follows:

[0077] In practical terrestrial and aerial wireless communication environments, the average received signal power decreases logarithmically with the distance between the transmitter and receiver. This phenomenon can be modeled using the path loss exponent n, which depends on the specific propagation environment, including density, type, building height, and vegetation. Therefore, using the logarithmic distance path loss model and based on Shannon's theorem, the mathematical expression for the terrestrial and aerial wireless communication data transmission rate is derived as follows:

[0078]

[0079] Among them, B AG P represents the channel bandwidth for communication between the UAV and the ground; AG This indicates the signal power used for communication between the drone and the ground. The Gaussian noise power within the communication channel between the UAV and the ground; d H This represents the hovering altitude of the drone; n represents the path loss index.

[0080] Using the free-space propagation model commonly used in satellite systems, the free-space path loss is defined as follows:

[0081]

[0082] Among them, G t Indicates the antenna gain of the drone; G r Indicates the antenna gain of the satellite; λ S Indicates the signal wavelength; d S This indicates the distance between the drone and the satellite.

[0083] Considering the impact of factors such as cloud cover and atmospheric conditions on satellite communication, we introduce a loss factor α, and the data transmission rate between the UAV and the geostationary satellite is defined as follows:

[0084]

[0085] Among them, B SA P represents the channel bandwidth for communication between the drone and the satellite; SA This indicates the transmission power of the drone's communication with the satellite; This represents the Gaussian noise power within the channel for communication between the drone and the satellite.

[0086] The following defines the caching and request models involved in the deep reinforcement learning-based integrated air-space-ground network content caching method described in this invention:

[0087] Suppose there are L cached content files, denoted by f∈{f1,f2,...,f...} l ,...,f LLet} represent each file, and let CMB be the storage capacity of each drone. Let user E be defined at time t. r The requested file f l In drone v j The cache flag in is Its mathematical expression is:

[0088]

[0089] Considering that the number of user requests varies at different times, let the number of user requests follow a Poisson distribution with parameter p as follows:

[0090]

[0091] User requests arrive one by one, and the popularity distribution of files among all users is approximately described by a Zipf distribution with parameter α. Assuming the files are arranged in descending order of content popularity, and the file sequence changes with time t, the probability that a user requests the content ranked nth in popularity at time t is expressed as:

[0092]

[0093] in, User E at time t r The requested file f l Right now An index of request frequency.

[0094] Cache hit rate represents the proportion of content successfully retrieved from cached content and is a key indicator for evaluating the effectiveness of a caching system; suppose that within a certain time period t, the number of drone nodes v j The total number of user requests received was E f Then calculate the long-term cache hit rate H. T The mathematical expression is as follows:

[0095]

[0096] In SAGIN, each drone node has a limited cache capacity, requiring the selection of an appropriate strategy to determine which caches to update. Therefore, the cache content model is modeled as maximizing the long-term cache hit rate H given parameters. T The problem, specifically the objective function, is expressed as:

[0097] maxH T

[0098] stC1:N i ∈N={EU,AN,SN}

[0099]

[0100] C3: H T ∈ [0, 1]

[0101]

[0102] Among them, C1 indicates that the communication node should be a node in the SAGIN; C2 indicates that the cumulative size of all cached contents on the drone v j does not exceed CMB; C3 indicates that the cache hit rate H T has a value range of [0, 1]; C4 indicates the cache flag takes 0 or 1 according to whether the cache is hit.

[0103] Model the content update problem according to the Markov decision process and describe the environment of content caching. Define the conditions in the MDP as follows:

[0104] In the considered scenario, each edge cache server is defined as an agent. The environment is characterized by the system state, and the tasks related to content caching and decision update are encapsulated within the framework of the MDP.

[0105] The state space of the system is defined by the previous requests, the current cache state, and the content popularity. The state space at time t is defined as s(t) = [L(t - 1), C(t), P(t)], where L(t - 1) represents the request state at the previous moment, including the request frequencies of all contents during this period, C(t) represents the current cache state, indicating the file contents stored on the current node, and P(t) represents the current content popularity.

[0106] Suppose that when updating the cache each time, the sequence of contents already cached in the node is (f 1 , f 2 , …, f Z ), and the sequence of requested contents is a partial content of all content sequences (f 1 , f 2 , …, f L ), Z < L. Map the contents already cached to the sequence of requested contents to indicate the cache decision. When the agent outputs the key of the cache decision, we retrieve the corresponding decision index to cache and update the content. The action space at time t is defined as a t = {∈ 1 , ∈ 2 , …, ε Z}, where ∈ z ∈ (f 1 , f 2 , …, f L ).

[0107] The reward function is defined as the difference between the number of cache hits and the total number of requests; this value is a non-positive number. In this scenario, the higher the cache hit rate, the closer the reward value is to 0; the lower the cache hit rate, the closer the absolute value of the reward value is to the total number of requests, i.e., the smaller the value. Let the total number of requests in time period t be E. f The total number of hits in the neural network cache decision is The reward for this period of time is defined as

[0108] The popularity distribution of a file among all users follows a Zipf distribution, and the probability of a user requesting certain content at time t is defined as follows: in User E at time t r The requested file f l Right now The index of the request frequency, where α is the parameter of the Zipf distribution. Each user makes a serial request, and the environment provides a pre-defined distribution function for the popularity of cached content. The number of cache hits and requests is recorded during the interaction with the agent.

[0109] When a user sends a request to the terminal, the nearest available edge caching node searches its local cache database. If the cached content is found, it is returned directly to the user. If the local database does not contain the content, the edge caching node requests the content from the data center of the geostationary orbit satellite and updates the cached content using the content caching strategy described in this invention. Specifically, this includes the following steps:

[0110] S1. The user initiates a content request to the terminal, and the terminal requests the content from the nearest available edge cache node.

[0111] S2. The edge cache node checks if the content is contained in the local cache database. If the content is contained in the local database, it is delivered directly to the user terminal.

[0112] S3. If the local database does not contain the content, the edge cache node requests the content from the data center and then delivers it to the user. The local cache content is updated using a deep reinforcement learning method within the DQN algorithm framework, and the cache hit rate is calculated.

[0113] Reference Appendix Figure Two As shown, the DQN algorithm framework used in this invention includes two networks: the current value network and the target value network. Specifically, updating the local cache using the DQN algorithm involves the following steps:

[0114] S301. Initialize the current network parameters θ and the target network parameters θ -

[0115] S302. Obtain the current state s(t) from previous requests in the environment, the current cache state, and the content popularity. Query the local cache of the node based on the index of the current request content sequence, and determine the decision to update the cache, i.e., action a(t), through mapping.

[0116] S303, Execute action a(t), observe reward r(t) and the next state s(t+1).

[0117] S304, via Q(s′,a′;θ) - Calculate the target Q value by using Q(s,a;θ) to calculate the Q value of the value network.

[0118] S305. Calculate the loss of the neural network and backpropagate to update θ.

[0119] S306. Every N steps, copy the parameters of the current value network to the target value function, i.e., θ. - =θ

[0120] S307. Let s(t) = s(t+1) and a(t) = a(t+1). Repeat the above steps until the set number of training steps is reached.

[0121] Reference Appendix Figure 3 As shown, attached Figure 3 A comparison chart of the rewards of this method and FIFO, LFU, and RC under different training steps is provided. The chart shows that the proposed method achieves the highest reward, the highest cache hit rate, and more stable performance with more training steps and more request events.

[0122] Reference Appendix Figure 4 As shown, attached Figure 4 A comparison chart of the average latency of this solution with FIFO, LFU, and RC under different request volumes is provided. The chart shows that as the number of requests increases, the system takes longer to process them, thus increasing the average latency for all methods. The method proposed in this invention exhibits a significantly lower average latency than the other three solutions under the same request volume, reducing it by approximately 40%. This demonstrates that the proposed solution has a faster cache hit speed, reduces the time overhead for users requesting cached content, and improves the user experience.

[0123] Reference Appendix Figure 5 As shown, attached Figure 5 This chart compares the cache hit rate of our proposed solution with FIFO, LFU, and RC under different request volumes. As can be seen from the chart, the cache hit rate of our proposed method is significantly higher than the other three solutions under different request volumes, and it remains stable around 0.8. This demonstrates that our invention can significantly improve the long-term cache hit rate under various conditions, effectively enhancing the user experience.

[0124] Example

[0125] Without loss of generality, we assign values ​​to the parameters of the content caching system proposed in this solution and provide a specific implementation. We set the number of drones used to provide content caching services to different regions to 3, the number of ground-based geostationary satellites used to store all cached content to 1, the Zipf parameter α to be 0.6, and the Poisson distribution parameter p to be 2.

[0126] To simulate user behavior, the environment provides a pre-cached content popularity distribution function and randomly samples from this distribution according to a defined probability density function. We also specify three file sizes: 150MB, 200MB, and 250MB, and set the total amount of content L that can be cached on the drone node to 15.

[0127] The drone's operating altitude is set to 15m, and the channel bandwidth B between it and the ground communication link is... AG The frequency is set to 6MHz, the path loss index n for communication with the user terminal is set to 2, and the signal transmission power P for communication with the user terminal is set to 2. AG The noise power of the channel between the ground and the ground is set to 2W. Set it to 0.01W.

[0128] The atmospheric conditions in space have a loss factor α of 5 for the system. The UAV's antenna gain G... t Antenna gain G of geostationary orbit satellites r They were set to 17dbi and 25dbi respectively.

[0129] The distance d between the ground node and the geostationary orbit satellite S Set to 10 6 m, wavelength λ of the communication signal S The channel bandwidth B is set to 0.2m when communicating with geostationary orbit satellites. SA Set to 10MHz, noise power The signal transmission power P is set to 3W when communicating with geostationary orbit satellites. SA Set it to 150W.

[0130] By simulating the performance of the proposed method under the above conditions compared with FIFO, LFU, and RC, the results show that the proposed method reduces the average latency by about 40% compared with the other methods. Under different request numbers, the CHR can reach about 0.75, which is significantly higher than other caching strategies, and it has high stability. This successfully demonstrates that the proposed method can adapt to the environmental conditions in the integrated air-space-ground network, effectively improve the user experience, and meet the diverse needs of users.

[0131] In another embodiment of the present invention, an integrated air-space-ground network content caching system is provided, which can be used to implement the above-described integrated air-space-ground network content caching method. Specifically, the system includes:

[0132] The model building module is used to build a network model that includes users, terminals, and edge cache nodes, and to build terrestrial and sky wireless communication models on the network model.

[0133] The content request module is used by users in the network model to initiate content requests to the terminal based on the communication model, and the terminal requests the content from the nearest available edge cache node.

[0134] The judgment module is used by the edge cache node to check whether the local cache database contains the content. If the local database contains the content, the content is delivered directly to the user terminal.

[0135] If the local database does not contain the content, the edge cache node requests the content from the data center and then delivers it to the user. The deep reinforcement learning method of the DQN algorithm framework is used to update the local cached content and calculate the cache hit rate.

[0136] The module division in this embodiment of the invention is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the invention can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0137] In another embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions from the computer storage medium to achieve a corresponding method flow or function. The processor described in this embodiment of the present invention can be used in the operation of an integrated air-space-ground network content caching method.

[0138] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory). This computer-readable storage medium is a memory device within a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium of the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the integrated air-space-ground network content caching method described in the above embodiments.

[0139] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0140] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure One One or more processes and / or boxes Figure One A device that provides the functions specified in one or more boxes.

[0141] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure One One or more processes and / or boxes Figure One The function specified in one or more boxes.

[0142] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure One One or more processes and / or boxes Figure One The steps of the function specified in one or more boxes.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for caching content in a space-air-ground integrated network, characterized in that, include: Establish a network model that includes users, terminals, and edge buffer nodes, and build terrestrial and sky wireless communication models on the network model; In the network model, users initiate content requests to the terminal based on the communication model, and the terminal requests the content from the nearest available edge cache node. The edge cache node checks whether the content is contained in the local cache database. If the content is contained in the local database, the content is delivered directly to the user terminal. If the local database does not contain the content, the edge cache node requests the content from the data center and then delivers it to the user. The deep reinforcement learning method of the DQN algorithm framework is used to update the local cache content and calculate the cache hit rate. The network model specifically includes: The system comprises three layers: a space-based network, a sky-based network, and a ground-based network. Public information content is stored on cloud servers deployed on the ground and multiple edge servers deployed on drones. Geostationary satellites establish connections with these ground-based cloud servers to provide access services. Build a network model, It contains three subsets, which include all nodes in the network model. The specific explanation is as follows: yes The complete set of all end-user nodes Represents user nodes; yes The complete set of all drone nodes. Represents a drone node; It is the complete set of satellite nodes. Represents a geostationary satellite node; The local cache content is updated using a deep reinforcement learning method based on the DQN algorithm framework, and the cache hit rate is calculated. Initialize current network parameters and target network parameters ; The current state is obtained from previous requests in the environment, the current cache state, and the content popularity. The local cache of the node is queried based on the index of the current request content sequence. The decision to update the cache, i.e., the action, is determined through mapping. ; Execute action Observation and reward With the next state ; pass Calculation target Value, through Computational value network value; Calculate the loss of the neural network and update it via backpropagation. ; Every N steps, the parameters of the current value network are copied to the target value function, i.e. ; make , Repeat the above steps until the set number of training steps is reached; Cache hit rate represents the proportion of content successfully retrieved from cached content and is a key indicator for evaluating the effectiveness of a caching system; let's assume a certain time period... Inside, drone nodes The total number of user requests received is Then calculate the long-term cache hit rate. The mathematical expression is as follows: Where: the cache flag is Suppose there exists A cached content file, used To represent each file, the storage capacity of each drone is... MB, and define the first Time users Requested file In drones The cache flag in is Its mathematical expression is: Considering that the number of user requests varies at different times, let the number of user requests follow the parameters as follows: Poisson distribution: User requests arrive one by one, and are accompanied by parameters. of The distribution approximates the popularity distribution of files across all users. Assuming files are arranged in descending order of content popularity, and the file sequence changes over time t, then the ranking of user request popularity at time t is... The probability of the content of a bit is expressed as: in, Indicates the first Time users Requested file Right now An index of request frequency.

2. The integrated air-space-ground network content caching method according to claim 1, characterized in that, The communication model is defined as follows: Using a logarithmic distance path loss model and based on Shannon's theorem, the wireless communication data transmission rates for ground and sky are derived mathematically as follows: in, This indicates the channel bandwidth for communication between the drone and the ground. This indicates the signal power used for communication between the drone and the ground. This represents the Gaussian noise power within the communication channel between the UAV and the ground. Indicates the hovering altitude of the drone; Indicates the path loss index; Using the free-space propagation model commonly used in satellite systems, the free-space path loss is defined as follows: in, Indicates the antenna gain of the drone; Indicates the antenna gain of the satellite; Indicates the signal wavelength; Indicates the distance between the drone and the satellite; A loss factor was introduced. The data transmission rate between the UAV and the geostationary satellite is defined as follows: in, This indicates the channel bandwidth for communication between the drone and the satellite; This indicates the transmission power of the drone's communication with the satellite; This represents the Gaussian noise power within the channel for communication between the drone and the satellite.

3. The integrated air-space-ground network content caching method according to claim 1, characterized in that, Objective value function: Model the cached content model as a long-term cache hit rate that maximizes given parameters. The problem, specifically the objective function, is expressed as: in, This indicates that the communication node should be a node in SAGIN; Indicates drone The total size of all cached content shall not exceed MB; Indicates cache hit rate The range of values ​​is ; Indicates cache flag The value is either 0 or 1 depending on whether a cache hit occurs.

4. The integrated air-space-ground network content caching method according to claim 1, characterized in that, The environment is characterized by system state, and tasks involving content caching and decision updates are encapsulated within the MDP framework; The system's state space is defined by previous requests, the current cache state, and content popularity. The state space at time t is defined as follows: ,in, This indicates the request status at a previous moment, including the request frequency for all content within that time period. This indicates the current cache state and shows the file content stored on the current node. Indicates the current popularity of the content; Let the sequence of cached content in a node be at each cache update. The requested content sequence is the entire content sequence. Part of the content, Mapping cached content to the requested content sequence to indicate caching decisions; when the proxy outputs the key of the caching decision, retrieving the corresponding decision index to cache and update the content; the action space at time t is defined as follows: ,in ; The reward function is defined as the difference between the number of cache hits and the total number of requests; this value is a non-positive number. The higher the cache hit rate, the closer the reward value is to 0; the lower the cache hit rate, the closer the absolute value of the reward value is to the total number of requests, i.e., the smaller the value. Let's assume a time period... The total number of requests within is The total number of hits in the neural network cache decision is The reward for this time period is defined as ; The popularity distribution of a file among all users follows a pattern The probability distribution of a user requesting certain content at time t is defined as follows: ,in Indicates the first Time users Requested file Right now Index of request frequency for The parameters of the distribution are as follows: each user makes a serial request, and the environment provides a pre-defined distribution function for the popularity of cached content, and the number of cache hits and requests are recorded during the interaction with the agent; When a user sends a request to the terminal, the nearest available edge cache node searches its local cache database. If the cached content is found, it is returned directly to the user. If the content is not found in the local database, the edge cache node requests the content from the data center via a geostationary satellite.

5. A space-air-ground integrated network content caching system, characterized in that, include: The model building module is used to build a network model that includes users, terminals, and edge cache nodes, and to build terrestrial and sky wireless communication models on the network model. The content request module is used by users in the network model to initiate content requests to the terminal based on the communication model, and the terminal requests the content from the nearest available edge cache node. The judgment module is used by the edge cache node to check whether the local cache database contains the content. If the local database contains the content, the content is delivered directly to the user terminal. If the local database does not contain the content, the edge cache node requests the content from the data center and then delivers it to the user. The deep reinforcement learning method of the DQN algorithm framework is used to update the local cache content and calculate the cache hit rate. The network model specifically includes: The system comprises three layers: a space-based network, a sky-based network, and a ground-based network. Public information content is stored on cloud servers deployed on the ground and multiple edge servers deployed on drones. Geostationary satellites establish connections with these ground-based cloud servers to provide access services. Build a network model, It contains three subsets, which include all nodes in the network model. The specific explanation is as follows: yes The complete set of all end-user nodes Represents user nodes; yes The complete set of all drone nodes. Represents a drone node; It is the complete set of satellite nodes. Represents a geostationary satellite node; The local cache content is updated using a deep reinforcement learning method based on the DQN algorithm framework, and the cache hit rate is calculated. Initialize current network parameters and target network parameters ; The current state is obtained from previous requests in the environment, the current cache state, and the content popularity. The local cache of the node is queried based on the index of the current request content sequence. The decision to update the cache, i.e., the action, is determined through mapping. ; Execute action Observation and reward With the next state ; pass Calculation target Value, through Computational value network value; Calculate the loss of the neural network and update it via backpropagation. ; Every N steps, the parameters of the current value network are copied to the target value function, i.e. ; make , Repeat the above steps until the set number of training steps is reached; Cache hit rate represents the proportion of content successfully retrieved from cached content and is a key indicator for evaluating the effectiveness of a caching system; let's assume a certain time period... Inside, drone nodes The total number of user requests received is Then calculate the long-term cache hit rate. The mathematical expression is as follows: Where: the cache flag is Suppose there exists A cached content file, used To represent each file, the storage capacity of each drone is... MB, and define the first Time users Requested file In drones The cache flag in is Its mathematical expression is: Considering that the number of user requests varies at different times, let the number of user requests follow the parameters as follows: Poisson distribution: User requests arrive one by one, and are accompanied by parameters. of The distribution approximates the popularity distribution of files across all users. Assuming files are arranged in descending order of content popularity, and the file sequence changes over time t, then the ranking of user request popularity at time t is... The probability of the content of a bit is expressed as: in, Indicates the first Time users Requested file Right now An index of request frequency.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the integrated air-space-ground network content caching method as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the integrated air-space-ground network content caching method as described in any one of claims 1 to 4.

Images