Cross-domain test network topology adaptive method for satellite-ground fusion

Abstract

The invention provides a cross-domain test network topology adaptive method for satellite-ground fusion, and belongs to the technical field of satellite-ground fusion networks. The method is applied to the military combined test, and on the basis of the satellite-ground fusion technology, the high speed and stability of combined test transmission are guaranteed by constructing the self-adaptive network topology.

Description

Technical Field

[0001] The present invention belongs to the technical field of satellite-ground fusion network, and in particular relates to a cross-domain test network topology self-adaptation method oriented to satellite-ground fusion. Background Art

[0002] At present, the standards and technical requirements for the integrated 5G network architecture and key technologies at home and abroad are still in the research stage, and there is no complete system of standard release and application cases. At present, some satellite-ground integrated network topology architectures have been proposed from different levels of logic and network organization.

[0003] From the perspective of fixed, mobile and satellite converged access, space-based networks initially adopted non-3GPP protocols such as DVB, while ground-based networks mainly adopted 3GPP protocols and also supported non-3GPP access methods. Space-based networks and ground-based networks developed independently in the early stages, supporting interconnection from the terminal to the access network level. Network topology based on 3GPP protocol is as follows: Figure 1 shown.

[0004] More specifically, the satellite access 5G core network interworking architecture introduces non-3GPP interworking functions (N3IWF), such as Figure 2 As shown, it is the same as the terrestrial 5G core network architecture. It supports the establishment of an IP security encrypted link with the UE, forwards the NAS signals of the uplink and downlink control planes between the UE and AMF, forwards the uplink and downlink user plane data packets between the UE and UPF, and performs N3 packet marking corresponding to QoS and local mobile anchoring functions for non-3GPP access.

[0005] From the perspective of the long-term integration and development of the two network topologies, the satellite-ground integrated network will be composed of space-based networks and ground-based networks, as shown in Appendix 3. The space-based network mainly includes the space-based access network and the space-based core network, and the ground-based network mainly includes the ground access network and the ground core network. The ground core network can be further divided into the public core network and the satellite core network. In the initial stage, it is recommended that the public core network and the satellite core network be independently networked, and in the later stage, the space-based and ground-based core networks be integrated.

[0006] The specific integration methods of the two networks are divided into the following three levels according to different stages of evolution:

[0007] (1) Interconnection and intercommunication stage: service integration. In this stage, two networks are interconnected through an intercommunication gateway, and terminals are connected from their respective networks.

[0008] (2) Hybrid access stage: system integration. This stage allows different types of single-mode / multi-mode terminals to access the space-based / ground-based access network separately or simultaneously, realizing the integration of the two network systems from the user's perspective.

[0009] (3) Unified networking stage: system integration. In this stage, the space-based and ground-based network protocols are unified to achieve true system integration.

[0010] The integration of existing ground network cache strategies into the satellite-ground fusion network is difficult to adapt to the characteristics of its diverse and heterogeneous network composition and dynamic changes in network topology. Especially in the military joint test network, the accessed test resources are characterized by large quantity, variety, and large geographical span, which makes the network environment complex and requires high stability. Summary of the invention

[0011] The present invention proposes a cross-domain test network topology adaptive solution for satellite-ground integration, which aims to solve the problem that the current cache strategy of the satellite-ground integration network cannot adapt to the complex heterogeneous network environment and dynamically changing network topology of military joint experiments.

[0012] The first aspect of the present invention provides a cross-domain test network topology adaptation method for satellite-ground integration, the method comprising:

[0013] Step S1: Modeling the satellite-ground fusion network, including a network connection model, a cache model, and a communication link model; wherein:

[0014] In the satellite-ground fusion network model, there are S satellites with a cache capacity of Cap s The set of low-Earth orbit satellites is represented by It also includes B caches, each with a capacity of Cap b The ground stations are represented by There are X cells evenly distributed within the ground station coverage area, with an average of Z users in each cell; and the remote core network contains all user request content;

[0015] Step S2: Based on the constructed satellite-ground fusion network model, the concurrent strategy of the satellite-ground fusion network is calculated using Markov decision making; wherein:

[0016] Each ground station and each satellite in the satellite-ground fusion network is identified as an intelligent agent;

[0017] The ground station state space is Among them, Req t represents the content request status received by the agent deployed on the ground station at time t;

[0018] The satellite state space is Req t Represents the content request status received by the agent at time t;

[0019] The action space is Each request content i corresponds to a set of actions Represented by 0 and 1 respectively, the agent selects the action to determine whether to cache the current content. When , it means that the node where the agent is located does not cache content i. When , it represents the cache content i of the node where the intelligent agent is located;

[0020] For the action chosen by the agent Determine whether content i exists in the request content received by the node where the current agent is located. If not, regardless of a i Regardless of the value, it is considered an invalid action and the agent does not update the cache;

[0021] The reward function is:

[0022]

[0023] Total_Utility represents the utility value of the cache resources of the agent unit at time t. t is the total utility obtained by the current agent, Cace resoutce is the cache capacity of the current agent; the utility value of the agent's cache resources is used as the reward function.

[0024] According to the method of the first aspect of the present invention, in step S1, regarding the network connection model:

[0025] Use S×(S+B) 0-1 matrix G t Indicates the connection status between nodes in the satellite-ground fusion network:

[0026]

[0027] in, represents the connection status of satellite s and node i in the satellite-ground fusion network including other satellites and all ground stations at time t, A value of 0 indicates that the satellite S is not connected to the network node i, and a value of 1 indicates that the satellite S is connected to the network node i.

[0028] According to the method of the first aspect of the present invention, in step S1, regarding the cache model:

[0029] The size of different types of business request content is m a , a∈{1,2,3}, m1 represents the average size of the first type of service request content, and so on; the cache state of satellite s is represented by vector express, When the value is 1, it means that the satellite s has cached content i at time t; when the value is 0, it means that the content is not cached; then the cache content set of the low-Earth orbit satellite is:

[0030]

[0031] Ground station cache status To express.

[0032] According to the method of the first aspect of the present invention, in step S1, for the communication link model:

[0033] The communication link includes the user link between the user and the ground station, the feeder link between the ground station and the low-Earth orbit satellite, the inter-satellite link between each low-Earth orbit satellite, and the return link between the low-Earth orbit satellite and the core network; for the downlink feeder link, the received signal-to-noise ratio is expressed as:

[0034] SNR FL (dB) = EIRP-LOSS+GKkB f

[0035] Where EIRP is the equivalent isotropic radiated power, P is the amplifier output power, G is the antenna gain, K is the system equivalent noise temperature, k is the Boltzmann constant, B f is the channel bandwidth, LOSS is the feeder loss, and the feeder link data transmission rate is the ideal Shannon capacity, then:

[0036] R FL =B f log(1+SNR FL )

[0037] Among them, R FL is the feeder link transmission rate, B f is the downlink feeder link channel bandwidth of the LEO beam.

[0038] According to the method of the first aspect of the present invention, in step S2:

[0039] To avoid mutual interference between strategies of agents, each agent uses an independent network to select actions, and each agent learns its own optimal strategy. Experience samples are used to train the strategy network, and random sampling is used to reduce the correlation between samples.

[0040] The input of the Actor network is: μ1,μ2,…,μ N , parameters θ1,θ2,…,θ N ;

[0041] The input of the Critic network is: Q1, Q2, ..., Q N ,parameter

[0042] The input of the Actor target network is: μ'1,μ'2,…,μ' N ;

[0043] The input of the Critic target network is: Q'1, Q'2, ..., Q' N ;

[0044] set up is the number of training sessions;

[0045] Initialize the satellite-ground fusion network environment s1, s2, ..., s N ; is the number of training steps; is the number of agents in the environment; add exploration noise: a i =μ i (S,θ i )+exploration_noise;

[0046] For each agent in the network, execute cache update actions α1, a2, ..., a N , get the state of the network at the next moment s'1,s'2,…,s' N , get the reward r1,r2,…,r for the action N , store the experience tuple (s,a,r,s') into the experience replay pool;

[0047] Extract S experience tuples from the experience replay pool; use the Actor target network μ' i Calculate the target action a' i ; Calculate the target Q value (Q'); Update the Critic network by minimizing the mean squared error between Q and Q'; Calculate the policy gradient of the Q value relative to the action:

[0048]

[0049] Update the Actor network; update the Actor target network and Critic target network parameters, τ is the soft update coefficient; then:

[0050] θ' i ←τθ i +(1-τ)θ' i

[0051]

[0052] Set s1=s'1,s2=s'2,…,s N =s' N , and output the trained concurrent strategy.

[0053] The second aspect of the present invention provides an electronic device. The electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps in the cross-domain test network topology adaptation method for satellite-ground integration of the first aspect of the present invention are implemented.

[0054] The third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the cross-domain test network topology adaptation method for satellite-ground integration of the first aspect of the present invention.

[0055] The application process of the present invention is divided into two parts: one is the satellite-ground fusion network modeling. Aiming at the network environment requirements of military joint simulation experiments, a satellite-ground fusion network topology model suitable for the joint test environment is constructed; the second is the satellite-ground fusion test network concurrency strategy. Due to the large number of nodes in the joint test network and the dynamic changes in the topological relationship, the actual application of single-agent DRL is difficult. It is necessary to propose a new network concurrency strategy to adapt to the application environment of the joint test and realize the self-adaptation of the test network.

[0056] The test network topology adaptation technology for satellite-ground integration proposed in the present invention is more suitable for the joint test environment than the existing satellite-ground integration network, has stronger anti-interference ability, has the ability to access heterogeneous resources and cross-regional resources, and contains intelligent management capabilities for ground station memory. It can flexibly respond to complex situations in joint tests and ensure the stability of transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0058] Figure 1 Provides a converged architecture diagram for fixed, mobile and satellite access;

[0059] Figure 2 This is the network architecture diagram of the non-3GPP interworking function (N3IWF);

[0060] Figure 3 This is a diagram of the integrated architecture of heaven and earth;

[0061] Figure 4 It is the flow chart of the overall technical solution;

[0062] Figure 5 It is a satellite-ground fusion network scene diagram;

[0063] Figure 6 This is a flow chart of Example 1. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0065] like Figure 4-5 As shown, the application process of the present invention is divided into two parts: one is the satellite-ground fusion network modeling. Aiming at the network environment requirements of the military joint simulation experiment, a satellite-ground fusion network model suitable for the joint test environment is constructed to provide a theoretical basis for the adaptive scheduling algorithm to analyze and evaluate the network environment; the second is the satellite-ground fusion test network concurrency strategy. Due to the large number of nodes in the joint test network and the dynamic changes in the topological relationship, the actual application of the single-agent DRL is difficult. It is necessary to propose a new network concurrency strategy to adapt to the application environment of the joint test and realize the adaptive adjustment of the network topology.

[0066] In the satellite-ground fusion network model proposed in the present invention, the cache capacity of S chips is Cap s LEO, whose set is represented by The capacity of each of the B caches is Cap b The ground stations are represented by There are X cells evenly distributed within the coverage area of ​​the ground station, and each cell has an average of Z users. It is assumed that the remote core network contains all user request content.

[0067] (1) Network connection model: Use an S×(S+B) 0-1 matrix G t Indicates the connection status between nodes in the satellite-ground fusion network.

[0068]

[0069] in, represents the connection status of satellite s and node i in the satellite-ground fusion network including other satellites and all ground stations at time t, A value of 0 indicates that the satellite S is not connected to the network node i, and a value of 1 indicates that the satellite S is connected to the network node i.

[0070] (2) Cache model: Set the content size of different types of business requests to m a, a∈{1,2,3}, m1 represents the average size of the Streaming service request content, and so on. The cache status available vector of satellite s express. When the value is 1, it means that at time t, satellite s has cached content i; when the value is 0, it means that the content is not cached. Thus, the cache content set of LEO is

[0071]

[0072] The same representation method can be used for the ground station cache situation, then

[0073] (3) Communication link model: The communication link includes the user link (UL) between the user and the ground station, the feeder link (FL) between the ground station and LEO, the inter-satellite link (ISL) between LEOs, and the backhaul link between LEO and the Core Network. For the downlink feeder link, the received signal-to-noise ratio can be expressed as:

[0074] SNR FL (dB) = EIRP-LOSS+GKkB f

[0075] Where EIRP is the equivalent isotropic radiated power, P is the amplifier output power, G is the antenna gain, K is the system equivalent noise temperature, k is the Boltzmann constant, B f is the channel bandwidth, LOSS is the feeder loss, and for the convenience of modeling, the data transmission rate of the feeder link is assumed to be the ideal Shannon capacity, thus:

[0076] R FL =B f log(1+SNR FL )

[0077] Among them, R FL is the feeder link transmission rate, B f is the downlink feeder link channel bandwidth of the LEO beam.

[0078] In view of the large number of nodes in the satellite-ground fusion network and the dynamic changes in topological relationships, the practical application of single-agent DRL is difficult. In terms of the concurrent strategy of the experimental network, the present invention expands on the MADDPG algorithm and proposes a set of multi-agent DRL algorithms suitable for military joint experimental ground-satellite networks. First, in order to avoid mutual interference between the strategies of agents, each agent uses an independent actor network to select actions. This decentralized execution mechanism enables each agent to learn its own optimal strategy. Secondly, experience replay is used to solve the problem of sample correlation, that is, the critic network uses previous experience samples to centrally train the policy network, and reduces the correlation between samples through random sampling.

[0079] The basic idea of ​​the algorithm is as follows:

[0080] (1) Markov decision modeling

[0081] Agent: Each ground station and each satellite in the satellite-ground two-level cache network is considered an agent.

[0082] Terrestrial-station state space: Among them, Req t Represents the status of content requests received by the agent deployed on the ground station at time t.

[0083] Satellite state space: Req t That is, the content request status received by the agent at time t.

[0084] Action space (action): Each request content i in the system corresponds to a set of actions Represented by 0 and 1 respectively, the agent selects the action to determine whether to cache the current content. When , it means that the node where the agent is located does not cache content i. When , it represents the cache content i of the node where the agent is located. For the action selected by the agent It is necessary to determine whether content i exists in the request content received by the node where the current agent is located. If not, regardless of a i Regardless of the value, it is considered an invalid action and the agent does not update the cache.

[0085] Reward function:

[0086]

[0087] That is, the utility value of the agent unit cache resource at time t, Total_Utility t is the total utility obtained by the current agent, Cache resource is the cache capacity of the current agent. If the total utility value is used directly as the reward function, the reward value will fluctuate greatly, making it difficult for the deep reinforcement learning algorithm to converge. Therefore, the utility value of the agent's unit cache resource is used as the reward function.

[0088] (2) Algorithm Process

[0089] The algorithm proposed in this application is designed based on two key elements: separation strategy and experience replay. First, to avoid mutual interference between strategies of agents, each agent uses an independent actor network to select actions. This strategy separation mechanism enables each agent to learn its own optimal strategy. Secondly, experience replay is used to solve the problem of sample correlation, that is, to use previous experience samples to train the policy network and reduce the correlation between samples through random sampling. To solve the problem of concurrent caching of multiple types of services, the algorithm flow is as follows:

[0090]

[0091]

[0092] Example 1

[0093] The test network topology adaptive technology for satellite-ground integration involved in the present invention is used to build a cross-regional joint test network, and satellite technology is used to build a stable and fast satellite-ground integration network between test bases located in different regions. In the implementation of military joint tests, the distance between military test bases is relatively far, and each military base needs to be regarded as an independent cell. The test resources such as equipment and software participating in the test in the base are connected to the cell, and the ground station communicates with the satellite. Figure 6 As shown, by establishing a ground station at the base where the experimental resources are located, a ground station network is formed in multiple regions, information is transmitted with the satellite network, and a ground station cache and cache management algorithm are equipped.

[0094] Example 2

[0095] The interaction between models and algorithms can work through a layered framework structure.

[0096] Data layer: responsible for collecting and storing network status information, including link status, node cache status, business requirements, etc. Provide data support required by other layers.

[0097] Model layer: Network connection model: defines the topology of the network, including detailed information of nodes and connections; cache model: manages cache strategies and cache status in nodes; communication link model: describes the communication characteristics of the link, such as bandwidth, latency, stability, etc. The model layer updates the model status based on the information provided by the data layer and provides decision-making basis for the strategy layer.

[0098] Strategy layer: Multi-type business concurrent caching strategy algorithm: According to the information provided by the model layer, dynamically adjust the cache strategy to optimize business processing. The algorithm of the strategy layer will formulate or adjust the strategy according to the current network status, business needs and model prediction results to optimize network performance.

[0099] Execution layer: responsible for executing the policies formulated by the policy layer, including adjusting cache content, optimizing link usage, scheduling business processing, etc. The operation results of the execution layer will be fed back to the data layer to form a closed-loop control.

[0100] Interaction between layers:

[0101] Data layer to model layer: The network status and business request information collected in real time by the data layer is passed to the model layer. The model layer updates the status of the network connection model, cache model, and communication link model based on this information.

[0102] Model layer to policy layer: The model layer provides detailed status and prediction information of the current network to the policy layer. The policy layer uses a multi-type business concurrent caching strategy algorithm to formulate or adjust strategies based on this information and preset goals (such as minimizing latency, maximizing throughput, etc.).

[0103] From the policy layer to the execution layer: The policies formulated by the policy layer are implemented through the execution layer, such as adjusting cache content, optimizing link allocation, etc. The operation results of the execution layer will affect the actual operation status of the network.

[0104] Execution layer to data layer: The operation results of the execution layer and the actual operating status of the network are collected by the data layer to form a closed-loop feedback, which is then provided to the model layer and strategy layer for the next round of decision-making and execution.

[0105] Example 3

[0106] The concurrent caching strategy process for multiple types of services is as follows:

[0107] Initialization: (1) According to the network connection model, determine the nodes (i.e., agents) and connections in the network, including satellites, ground stations, and user terminals. (2) Initialize the cache model and set the cache capacity C for each node. n , and initialize the cache content set of each satellite. (3) According to the communication link model, determine the link parameters such as transmission rate, bandwidth, delay, signal-to-noise ratio, etc.

[0108] Business demand analysis: Analyze the received business requests and identify the request volume and characteristics of each business type, including data size, real-time requirements, etc.

[0109] Cache demand prediction and link status evaluation: (1) Use historical data to predict the cache demand of each service type (taking the average value, etc.). (2) Evaluate the status of the current communication link, including the link's transmission rate, available bandwidth, latency, etc., to provide a basis for subsequent cache strategy adjustments.

[0110] Cache strategy adjustment and link optimization: Cache strategy adjustment and link optimization are performed based on business needs, cache demand prediction, and link status evaluation results. The algorithm is formalized as an optimization problem, with the goal of maximizing overall network performance, reducing total latency, and improving cache hit rate. The optimization goal can be expressed as:

[0111]

[0112] Among them, the weight ω i It needs to be set according to the actual importance and priority of the business, involving business negotiation or analysis based on historical data, c i,n represents the cache of service type i assigned to node n, d i is the demand of business i, l m,n is the link from node m to n, f(c i,n ,d i ) is the performance function, g(l m,n ) represents the link cost function, and λ is the balancing coefficient. Where:

[0113]

[0114] The specific form of the performance function is designed according to actual business needs and network characteristics, and may include linear, nonlinear, or machine learning-based prediction models. A simpler performance function is:

[0115]

[0116] Among them, α is a positive proportional coefficient used to adjust the sensitivity of the function.

[0117] Index term Indicates that as node n allocates cache c to service i i,n Close to its needs i When c i,n Much smaller than d i When c i,n Close to or equal to d iWhen the cache is further increased, the effect of improving the service satisfaction rate decreases.

[0118] function Ensures that the value of the performance function is in c i,n =0 is close to 0, that is, it hardly meets any business needs), and when c i,n Increase, especially when cache allocation is close to or meets business needs i , the function value quickly approaches 1 (that is, the business needs are well met).

[0119] The link cost function is designed according to the communication link model and the specific design of the business. A simpler cost function is:

[0120]

[0121] Among them, R and B are link l m,n The weights β and γ are used to adjust the influence of different factors on the cost function. ∈ is a small constant used to prevent the denominator from being zero.

[0122] The higher the transmission rate, the smaller the value, indicating that the link has high transmission efficiency and therefore low cost. This reflects the advantage of high-speed links when transmitting large amounts of data. The wider the bandwidth, the smaller the value, indicating that the link can transmit more data at the same time and therefore low cost. This reflects the advantage of wide-bandwidth links when transmitting multiple services concurrently.

[0123] Perform cache update and link adjustment: According to the optimization results, determine the cache size allocated to each node for different service types, update the cache content and size of each node, and adjust the routing strategy of the service flow.

[0124] Feedback and adjustment: Monitor actual service performance and network status, and collect feedback information for adjusting future cache allocation and link optimization strategies.

[0125] The test network topology adaptation technology for satellite-ground integration proposed in the present invention is more suitable for the joint test environment than the existing satellite-ground integration network, has stronger anti-interference ability, has the ability to access heterogeneous resources and cross-regional resources, and contains intelligent management capabilities for ground station memory. It can flexibly respond to complex situations in joint tests and ensure the stability of transmission.

[0126] Please note that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. The above embodiments only express several implementation methods of the present application, and their descriptions are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, without departing from the concept of the present application, several variations and improvements can be made, which all belong to the scope of protection of the present application. Therefore, the scope of protection of the patent in this application shall be based on the attached claims.

Claims

1. A cross-domain test network topology adaptive method for satellite-ground integration, characterized in that: The method comprises: Step S1: Modeling the satellite-ground fusion network, including a network connection model, a cache model, and a communication link model; wherein: In the satellite-ground fusion network model, there are S satellites with a cache capacity of Cap s The set of low-Earth orbit satellites is represented by It also includes B caches, each with a capacity of Cap b The ground stations are represented by There are X cells evenly distributed within the ground station coverage area, with an average of Z users in each cell; and the remote core network contains all user request content; Step S2: Based on the constructed satellite-ground fusion network model, the concurrent strategy of the satellite-ground fusion network is calculated using Markov decision making; wherein: Each ground station and each satellite in the satellite-ground fusion network is identified as an intelligent agent; The ground station state space is Among them, Req t represents the content request status received by the agent deployed on the ground station at time t; The satellite state space is Req t Represents the content request status received by the agent at time t; The action space is Each request content i corresponds to a set of actions Represented by 0 and 1 respectively, the agent selects the action to determine whether to cache the current content. When , it means that the node where the agent is located does not cache content i. When , it represents the cache content i of the node where the intelligent agent is located; For the action chosen by the agent Determine whether content i exists in the request content received by the node where the current agent is located. If not, regardless of a i Regardless of the value, it is considered an invalid action and the agent does not update the cache; The reward function is: Total_Utility represents the utility value of the cache resources of the agent unit at time t. t is the total utility obtained by the current agent, Cache resource is the cache capacity of the current agent; the utility value of the agent's cache resources is used as the reward function.

2. According to claim 1, a cross-domain test network topology adaptive method for satellite-ground integration is characterized in that: In the step S1, regarding the network connection model: Use S×(S+B) 0-1 matrix G t Indicates the connection status between nodes in the satellite-ground fusion network: in, represents the connection status of satellite s and node i in the satellite-ground fusion network including other satellites and all ground stations at time t, A value of 0 indicates that the satellite S is not connected to the network node i, and a value of 1 indicates that the satellite S is connected to the network node i.

3. The cross-domain test network topology adaptive method for satellite-ground integration according to claim 2 is characterized in that: In the step S1, regarding the cache model: The size of different types of business request content is m a , a∈{1,2,3}, m1 represents the average size of the first type of service request content, and so on; the cache state of satellite s is represented by vector express, When the value is 1, it means that the satellite s has cached content i at time t; when the value is 0, it means that the content is not cached; then the cache content set of the low-Earth orbit satellite is: Ground station cache status To express.

4. The cross-domain test network topology adaptive method for satellite-ground integration according to claim 3 is characterized in that: In the step S1, for the communication link model: The communication link includes the user link between the user and the ground station, the feeder link between the ground station and the low-Earth orbit satellite, the inter-satellite link between each low-Earth orbit satellite, and the return link between the low-Earth orbit satellite and the core network; for the downlink feeder link, the received signal-to-noise ratio is expressed as: SNR FL (dB)=EIRP-LOSS+GKkB f Where EIRP is the equivalent isotropic radiated power, P is the amplifier output power, G is the antenna gain, K is the system equivalent noise temperature, k is the Boltzmann constant, B f is the channel bandwidth, LOSS is the feeder loss, and the feeder link data transmission rate is the ideal Shannon capacity, then: R FL =B f log(1+SNR FL ) Among them, R FL is the feeder link transmission rate, B f is the downlink feeder link channel bandwidth of the LEO beam.

5. The cross-domain test network topology adaptive method for satellite-ground integration according to claim 4 is characterized in that: In step S2: To avoid mutual interference between strategies of agents, each agent uses an independent network to select actions, and each agent learns its own optimal strategy. Experience samples are used to train the strategy network, and random sampling is used to reduce the correlation between samples. The input of the Actor network is: μ1,μ2,…,μ N , parameters θ1,θ2,…,θ N ; The input of the Critic network is: Q1, Q2, ..., Q N ,parameter The input of the Actor target network is: μ'1,μ'2,…,μ' N ; The input of the Critic target network is: Q'1, Q'2, ..., Q' N ; Set K to the number of training times; Initialize the satellite-ground fusion network environment s1, s2, ..., s N ; M is the number of training steps; is the number of agents in the environment; add exploration noise: a i =μ i (S,θ i )+exploration_noise; For each agent in the network, execute cache update actions a1, a2, ..., a N , get the state of the network at the next moment s'1,s'2,…,s' N , get the reward r1,r2,…,r for the action N , store the experience tuple (s,a,r,s') into the experience replay pool; Extract S experience tuples from the experience replay pool; use the Actor target network μ' i Calculate the target action a' i ; Calculate the target Q value (Q'); Update the Critic network by minimizing the mean squared error between Q and Q'; Calculate the policy gradient of the Q value relative to the action: Update the Actor network; update the Actor target network and Critic target network parameters, τ is the soft update coefficient; then: I will i ←tth i +(1-τ)θ' i Set s1=s'1,s2=s'2,…,s N =s' N , and output the trained concurrent strategy.

6. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps in the cross-domain experimental network topology adaptation method for satellite-ground integration described in any one of claims 1-5 are implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps in the cross-domain experimental network topology adaptation method for satellite-ground integration described in any one of claims 1 to 5 are implemented.

Images