Satellite network adaptive routing method and system for time delay optimization

Abstract

The invention relates to a time delay optimized satellite network adaptive routing method and system, and belongs to the technical field of satellite communication. The method comprises the steps of multi-hop information age construction, multi-path routing, link state sensing, flow segmentation adaptive routing, dynamic routing decision and the like. A multi-hop information age model is adopted to consider the time-varying property of a link state, the real-time property of path information is accurately estimated, and the timeliness of data updating is ensured; a multi-path search algorithm predicts link occupation frequency according to ephemeris information to construct a plurality of disjoint transmission paths, so that the traffic congestion problem is relieved, and the link resource utilization rate is optimized; a flow segmentation adaptive routing algorithm adjusts the path flow distribution proportion, reduces the data packet loss rate and the information age, and guarantees the stability of time delay sensitive data transmission; according to the method, resource utilization is balanced in a high-dynamic satellite network, link congestion is effectively prevented, the overall transmission efficiency and reliability are improved, and a high-adaptability and efficient transmission scheme is provided for time delay sensitive services.

Description

Technical Field

[0001] The present invention relates to the field of satellite communication technology, and in particular to a delay-optimized satellite network adaptive routing method and system. Background Art

[0002] As the application of satellite networks in global communications continues to expand, their potential in battlefield situation awareness, disaster relief, remote area communications and other fields is becoming increasingly apparent. Due to the characteristics of wide coverage and low latency, satellite networks have shown unique advantages in responding to emergencies and ensuring critical communications. At the same time, the dynamic and multi-hop characteristics of satellite networks also put forward higher requirements for the routing mechanism of the network. Especially for delay-sensitive services, how to ensure the real-time performance of data has become an urgent problem to be solved. In satellite networks, satellite nodes move at high speeds, network topology changes frequently, and the availability and transmission delay of communication links fluctuate all the time. Traditional routing methods are mainly divided into static and dynamic categories: static routing builds paths based on preset orbital information, but it is difficult to respond to changes in network status in real time, often resulting in link congestion and data loss; dynamic routing can update the routing table in time and has better adaptability, but it needs to continuously obtain real-time link information between satellites, which increases the communication overhead of the system. Although the existing dynamic and static combined routing strategy can balance the network status update and resource consumption to a certain extent, its application in complex dynamic environments still has great limitations and is difficult to meet the high requirements of delay-sensitive services for data real-time performance. In satellite networks, data needs to be transmitted through multiple hops of multiple satellite nodes. Frequent interruptions in link status lead to dynamic changes in the path, and the selection of different paths has a direct impact on the age of information. Existing methods mainly focus on optimizing the shortest delay or maximum throughput of a single path, and fail to effectively consider the age of information in multi-hop transmission paths, and cannot guarantee the freshness of data, especially when link interruptions occur frequently. Existing solutions are difficult to achieve real-time data updates. The link state of satellite networks is highly time-varying. Link availability and delays in different time periods vary with the position of the satellite, affecting the selection of paths and the control of information age. Existing methods usually do not consider the time-varying nature of link status in path selection, resulting in congestion of business flows on high-load links, increasing transmission delays and packet loss rates, and affecting the timeliness of data.

[0003] Therefore, in view of the shortcomings of existing satellite networks in information age control and resource utilization, an effective multi-hop information age optimization method is urgently needed to meet the needs of delay-sensitive services. Some existing studies have alleviated the congestion problem of single-path transmission through multi-path routing and optimized link utilization within a limited range. However, these methods can often only optimize the path selection of a single transmission, while ignoring the dynamic control of multi-hop information age, making it difficult to achieve comprehensive information age optimization. In addition, the existing multi-path methods fail to consider the load balance of different paths in path selection, which can easily lead to frequent congestion of high-load links and fail to achieve effective data updates.

[0004] In summary, the multi-hop routing mechanism of existing satellite networks has obvious shortcomings in dealing with delay-sensitive business needs. Summary of the invention

[0005] In view of this, the problem to be solved by the present invention is: how to ensure the real-time transmission of data of delay-sensitive services and meet multiple network constraints in a satellite network by optimizing a multi-hop routing mechanism and information age management.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A satellite network adaptive routing method for delay optimization comprises the following steps:

[0008] S1: Construction of multi-hop information age, specifically:

[0009] S11: In the low-orbit satellite constellation system model, it is assumed that A fi (t) represents the instantaneous information age of the i-th hop satellite node of data stream f at time slot t, and the instantaneous information age is calculated;

[0010] S12: Introducing binary variable x fik (t) indicates whether the i-th hop satellite node of data flow f receives the k-th data packet in time slot t; if x fik (t) = 1 means that at time slot t, the i-th hop satellite node on data stream f successfully receives the k-th update packet, otherwise x fik (t) = 0;

[0011] S13: Describe the relationship between the information age of the current satellite node and the previous hop satellite node; if the data packet is received at time t-1, the information age is represented by A f(i-1) (t-1)+1; if no data packet is received, the information age is represented by A fi (t-1)+1; construct the basic model of information age;

[0012] S14: Each satellite node is considered as a service unit with limited service capacity and queue to store data packets waiting to be processed; the first-come-first-served queuing theory is used to model the waiting time of data packets in the queue;

[0013] S15: The congestion level of the satellite network is measured by comparing the service rate and packet generation rate of the satellite nodes, and the network congestion level is modeled:

[0014] S16: Modeling satellite network connectivity constraints:

[0015] S17: Construct a model to minimize information age;

[0016] S2: Multi-path routing, which achieves traffic diversion by dynamically calculating multiple non-intersecting candidate paths based on a path search algorithm optimized based on multi-hop information age;

[0017] S3: Link status perception. Each satellite periodically sends link status information to the control satellite, including the capacity of the satellite node to receive new data packets, the queue length of the current satellite node, the current network topology information, and the overall index of the network.

[0018] S4: Traffic segmentation adaptive routing, based on path selection, optimizes traffic allocation ratio and data generation rate according to dynamic link parameters through deep reinforcement learning method;

[0019] S5: Dynamic routing decision, based on the continuous iterative training of S4, the intelligent agent dynamically adjusts the traffic distribution ratio and data generation rate; uses the reward function to calculate the immediate reward, continuously optimizes the policy network and Q value network to minimize the information age; after completing the training, the agent regularly collects the network status and makes traffic segmentation decisions based on the trained policy network.

[0020] Furthermore, the instantaneous information age is calculated in S11, specifically:

[0021] The instantaneous information age is expressed as follows:

[0022]

[0023] Among them, g fk represents the generation time of the kth data packet of data stream f;

[0024] The S13 reconstructs the basic model of information age, specifically:

[0025] The relationship between the information age of the current satellite node and the previous hop satellite node is described as follows: The basic model for constructing information age is as follows:

[0026]

[0027] Where K represents the total number of data flows: K f represents the total number of packets in data stream f; x fik (t-1) is a binary variable indicating whether data packet k is successfully received in time slot t-1;

[0028] The S14 is specifically:

[0029] When a packet arrives at node j, the effect of congestion on the information age is modeled as the waiting time W of the queue i The increase is expressed as the following formula:

[0030] A i (t) = tg i +W i (t)

[0031] Among them, A i (t) represents the information age of data packet i at time t; g i W is the time when data packet i is generated at the source node; i (t) is the waiting time of packet i in the queue;

[0032] Assume that the service rate of each satellite node i is μ i , represents the number of packets that a node can process per second; let the arrival rate λ i , represents the average rate at which data packets arrive at satellite node i; let the queue length be L i (t), represents the number of data packets waiting to be processed in the queue of satellite node i at time t, expressed as the following formula:

[0033] L i (t+Δt)=L i (t)+λ i ·Δt-μ i Δt

[0034] Among them, Δt represents the time interval; λ represents the arrival rate;

[0035] The average service time is The waiting time W of the packet in the queue i (t) can be expressed as the following formula:

[0036]

[0037] The S15 is specifically:

[0038] Define the congestion index C congestion (t) represents the congestion degree of satellite node i at time t, which is expressed as the following formula:

[0039]

[0040] Among them, λ i (t) represents the arrival rate of satellite node i at time t; when C congestion When (t)>0, it means that satellite node i is in a congested state; when C congestion When (t) < 0, it means that satellite node i has surplus processing capacity; when C congestion When (t) = 0, it means that satellite node i has just reached the equilibrium state;

[0041] The data packet generation rate λ is dynamically adjusted according to the current network congestion status and the remaining capacity of the satellite node; the adjustment formula is expressed as follows:

[0042] λ(t)=f(C congestion (t),C i (t))

[0043] Where λ(t) is the adjusted packet generation rate, f(C congestion (t),C i (t)) is a function that adjusts the generation rate according to the congestion index and the remaining capacity of the node;

[0044] The S16 is specifically:

[0045] For each source node s and destination node d, there exists a path P s,d Set makes:

[0046] Among them, e i Represents the path P s,d The edge on the i ) represents edge e i The capacity, λ s Indicates the traffic demand of the source node;

[0047] The S17 is specifically:

[0048] The satellite network is modeled as a space-time graph, expressed as the following formula:

[0049] G={(S,E,T,k)}

[0050] Among them, the set of S satellites is represented by S = {s1, s2, ..., s N}, E is the link set represented by E = {e1, e2, ..., e L}, T is a vector consisting of k time slots, whose time interval is τ = T / k; in the time period τ i The number of links in The K paths between any nodes are defined as R to achieve effective traffic diversion, which is expressed as the following formula:

[0051]

[0052] in, Represents the time slice τ i All path sets in Represents the time slice τ i The first transmission path of the first data stream within; Represents the time slice τ i The Kth transmission path of the first data stream in; Represents the time slice τ i The Kth transmission path of the Mth data stream in the

[0053] Assume that the path space P = {P1,P2,...,P K The maximum flow C = {c1, c2, ..., c K}; where P i The maximum flow is the maximum capacity of the link; according to the bandwidth resources of the relevant paths, each source node can split the flow into M sub-flows that pass through K paths at the same time, let represents the sub-flow ratio between the m-th node pair through the x-th path, and the split solution W of all sub-flow flows between the source node and the destination node pair is expressed as the following formula:

[0054] W = {w 1,1 ,...,w 1,K ,...,w m,K}.

[0055] Further, the S17 is specifically:

[0056] By optimizing the path selection R and the traffic distribution strategy W, the average information age A of all paths in the network is minimized. avg It is expressed as the following formula:

[0057]

[0058] Where K represents the total number of paths, τ i Represents a state in the network state set T; A x (τ i ) represents the time slice τ i Inside, the information age of the xth path;

[0059] Setting constraints includes:

[0060] 1) Packet generation rate constraints to ensure the generation rate of each satellite node Not exceeding its maximum load capacity

[0061] 2) Satellite node processing capacity constraints to ensure the processing rate of each satellite node Not exceeding its maximum processing capacity

[0062] 3) Network connectivity constraints, ensuring that the total link capacity of the selected path can meet the data traffic demand λ s , that is, the sum of the capacity of each link on the path ∑capacity(e i ) must be greater than or equal to the flow rate λ s , and there must be at least one feasible path p i ∈P;

[0063] 4) Path constraints, ensuring that the path set P μ Only feasible paths that meet the link status and node capability conditions are included;

[0064] 5) Queue length constraint, requiring the queue length L of all nodes i Maintaining the maximum queue length The following avoids transmission delays caused by long queues;

[0065] 6) Information age constraint, which ensures that the information age on the transmission path increases, i.e., the candidate paths between node pairs satisfy Where 1≤x≤y≤K, m∈[1,M];

[0066] 7) Path split flow conservation constraint requires that the sum of the sub-flows of the total flow between any node pair distributed to all paths is 1, that is, m∈[1,M];

[0067] 8) Congestion index constraint, limiting the congestion index of each satellite node Does not exceed the predefined threshold C threshold , to avoid the loss of information timeliness due to congestion;

[0068] 9) Bandwidth constraints to ensure the flow of each link on the path Not exceeding the link bandwidth BW, where And e j ∈E S .

[0069] Further, the S2 specifically comprises the following steps:

[0070] S21: Input the source satellite node and the destination satellite node pair, ephemeris information and a predefined random weight range, and initialize them, specifically:

[0071] Input the satellite pair set (S, D) of the source node and the target node, the ephemeris information, and the predefined random weight range (r1<r2<r3), initialize the link weight matrix M and the candidate path set R to an empty set, and the link occupancy frequency Set to 0, where e represents the link, τ i Indicates a time segment;

[0072] S22: For each time slot τ i , using the Dijkstra shortest path algorithm based on information age to calculate the shortest path in the current time slot If the link is in the shortest path, the link occupation frequency is updated. To prevent the algorithm from falling into the local optimum, a threshold setting method is adopted. If the link occupation frequency Less than a given threshold The ground control center randomly assigns an integer between r1 and r2 as the new link metric; otherwise, the link metric is set to a random integer between r2 and r3;

[0073] S23: Delete the links in the transmission path with the shortest information age, and continue to select the second shortest path using the shortest path algorithm on the remaining links to make full use of the idle links.

[0074] Further, the S3 is specifically:

[0075] The state S of each satellite node contains the following elements: i : The capacity of the satellite node to receive new data packets; L i : The queue length of the current satellite node; G i : Current network topology information; C congestion,i : The overall index of the network.

[0076] Further, the S4 is specifically:

[0077] S41: Assume that state S includes C i and L i ; C i represents the capacity of the satellite node to receive new data packets; L i Indicates the queue length of the current satellite node; G i Indicates the current network topology information; C congestion,i represents the overall index of the network; action A is the traffic proportion and data generation rate of the candidate paths between all node pairs, expressed as: A = {a1, a2, ..., a M}, each action a j Contains: path traffic distribution weight, packet generation rate The reward R is the weighted sum of the data information age and the node congestion level, expressed as the following formula:

[0078] R(s,a)=-α·A avg (s,a)-β·C congestion (s)

[0079] Among them, R(s,a) represents the reward for taking action a in state s; A avg (s,a) represents the information age of taking action a in state s; C congestion (s) represents the link congestion degree in state s; α and β are the weight coefficients of information age and congestion degree respectively. The larger the reward value, the more effective the path selection and traffic allocation strategy is for optimizing information age.

[0080] S42: Input the path obtained by the path search algorithm based on multi-hop information age optimization in step S2 and the link state information obtained in step S3 into reinforcement learning, and train the intelligent agent to decide the flow ratio and data transmission rate allocated to each transmission path; wherein the update of the optimal strategy satisfies the following formula:

[0081]

[0082] Where E represents the expectation; α is the temperature coefficient (regularization coefficient), which is used to adjust the importance of entropy in the objective function; γ is the discount factor; r(s t ,a t ) indicates state s t Action a occurs in the case t Reward; H(π(·|s t )) is the entropy function, representing the state s t The entropy of the strategy is expressed as follows:

[0083] H(π(·|s t ))=E[-logπ(·|s t )]

[0084] Value function Q(s t ,a t ) is defined as the following formula:

[0085]

[0086] Among them, s t+1 Sampled from the experience replay pool D; V π (s t+1 ) represents the value function, which means that in state s t+1 The expected return when following strategy π.

[0087] State value function V(s t ) represents the expected reward in a certain state, which is defined as the following formula:

[0088]

[0089] Further, the S4 is specifically:

[0090] S41: Initialize reinforcement learning parameters, initialize network parameters in the Actor-Critic framework, including the initial weights of the policy network π and the value network Q, set the capacity of the experience pool M, and the weight parameters of the target network

[0091] S42: State observation: the system periodically collects the environmental state S of the satellite network, including: link utilization and delay information of the current path set R; queue length L of the satellite node i ; The link congestion index C of the network congestion ;

[0092] S43: Execute the action and allocate the traffic ratio for each path according to the policy network π and packet generation rate Apply these allocation parameters as action A to the current network environment;

[0093] S44: Calculate the reward function. At each time slice, calculate the reward value R(s,a):

[0094] R(s,a)=-α·A avg (s,a)-β·C congestion (s)

[0095] Among them, A avg (s,a) represents the average information age of the system after executing the action, C congestion (s) represents the congestion level of the link, α and β are weight parameters;

[0096] S45: Update the policy network and value network, store the result of the current action in the experience pool M; randomly extract small batches of data, and use the gradient descent method to optimize the policy network and value network;

[0097] S46: Online learning and routing adjustment. The system updates the policy network in real time and adjusts the traffic distribution of each path according to the latest environmental status to ensure that high-priority data packets can pass through low-latency paths first, thereby reducing the overall information age; outputs the trained optimal traffic distribution strategy and applies it to the data transmission process.

[0098] A satellite network adaptive routing system with time delay optimization, comprising:

[0099] Path selection module: Based on the known ephemeris information, a network topology snapshot is constructed by the real-time position of the satellite nodes; each time slice constitutes a topology map, and multiple candidate paths that meet the feasible path constraints are calculated through a multi-path routing algorithm;

[0100] Data packet generation rate adjustment module: adjusts the generation rate according to the processing capacity of the satellite node and the network congestion, and feeds the adjusted data packet rate back to the traffic allocation module; the adjustment of the data packet generation rate affects the subsequent path selection and traffic allocation;

[0101] Information age calculation module: calculates and updates the information age of each satellite node, provides data for the path selection module, and helps select the most suitable path; also feeds back the information age to the traffic allocation module to affect the optimal allocation of data flow; after updating the information age, feeds back the calculation results to the reinforcement learning module to provide model learning and adjustment strategies;

[0102] Path selection module: selects the optimal multi-hop path based on satellite ephemeris information and link status, and feeds the selection result back to the traffic allocation module;

[0103] Traffic distribution module: optimizes traffic distribution according to information age, network congestion and satellite node queue length, and feeds back traffic scheduling results to the data packet generation rate adjustment module and path selection module; by optimizing traffic distribution, it affects the data forwarding rate and path selection strategy;

[0104] Reinforcement learning module: Based on the network status, adjust the path selection and traffic distribution strategy, and optimize the long-term performance; this module interacts with the above modules to provide dynamic adjustment and adaptive optimization capabilities.

[0105] The beneficial effects of the present invention are:

[0106] The present invention significantly improves the data transmission timeliness and network resource utilization of delay-sensitive services by constructing a multi-hop information age optimization model for satellite networks, proposing a multi-path search algorithm and an information age-optimized traffic segmentation adaptive routing algorithm. The multi-hop information age model takes into account the time-varying nature of the link state, accurately estimates the real-time nature of the path information, and ensures the timeliness of data updates; the multi-path search algorithm predicts the link occupancy frequency based on the ephemeris information, constructs multiple non-intersecting transmission paths, effectively alleviates the traffic congestion problem, and optimizes the link resource utilization; and the traffic segmentation adaptive routing algorithm adaptively adjusts the path traffic distribution ratio through a deep reinforcement learning method, significantly reducing the data packet loss rate and information age, and ensuring the stability of delay-sensitive data transmission. Through the above optimization, the present invention achieves a comprehensive improvement in transmission real-time, link resource utilization efficiency, and transmission reliability in satellite networks.

[0107] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0108] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0109] Figure 1 This is a diagram of a satellite network data transmission application architecture according to an embodiment of the present invention;

[0110] Figure 2 This is an interaction diagram of various modules in a satellite network adaptive routing system for delay optimization according to an embodiment of the present invention.

[0111] Figure 3 A diagram of a dynamic routing decision training process based on a soft actor-critic algorithm (SAC) according to an embodiment of the present invention; DETAILED DESCRIPTION

[0112] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0113] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0114] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0115] The purpose of the present invention is to provide a satellite network adaptive routing method with optimized delay; by dynamically optimizing the information age in a multi-hop path, optimizing the path selection and traffic distribution, the freshness of the data and the timeliness of transmission are ensured. Combined with the packet generation rate, satellite node processing capability, bandwidth, queue length and energy constraints, the present invention can balance resource utilization in a highly dynamic satellite network, effectively prevent link congestion, improve overall transmission efficiency and reliability, and provide a highly adaptable and efficient transmission solution for delay-sensitive services. The specific steps of the method include the following steps:

[0116] S1: Construction of multi-hop information age

[0117] S11: In the low-orbit satellite constellation system model, it is assumed that A fi (t) represents the instantaneous information age of the i-th hop satellite node of data stream f at time slot t, and the instantaneous information age can be expressed as follows:

[0118]

[0119] Among them, g fk represents the generation time of the kth data packet of data stream f;

[0120] S12: Introducing binary variable x fik (t) indicates whether the i-th hop satellite node of data stream f receives the k-th data packet in time slot t. fik (t) = 1, which means that at time slot t, the i-th hop satellite node on data stream f successfully receives the k-th update packet, otherwise x fik (t)=0.

[0121] S13: The relationship between the information age of the current satellite node and the previous hop satellite node is described as follows: If a data packet is received at time t-1, the information age is represented as A f(i-1) (t-1)+1; if no data packet is received, the information age is represented by A fi (t-1)+1; The basic model for constructing information age is as follows:

[0122]

[0123] Where K represents the total number of data flows: K f represents the total number of packets in data stream f; x fik (t-1) is a binary variable indicating whether data packet k is successfully received in time slot t-1;

[0124] S14: In satellite networks, information age is dynamically updated as data packets are transmitted and network status changes. When a data packet arrives at node j, it may be delayed due to link interruption and queue waiting. The data packet will not be sent out, which will increase the information age of the transmitted data. Therefore, the impact of congestion on information age is modeled as the queue waiting time W i The increase is expressed as the following formula:

[0125] A i (t) = tg i +W i (t)

[0126] Among them, A i (t) represents the information age of data packet i at time t; g i W is the time when data packet i is generated at the source node; i (t) is the waiting time of packet i in the queue, which depends on the congestion level of the network and the processing capacity of the satellite node.

[0127] The waiting time in the queue will be analyzed below.

[0128] In satellite networks, each satellite node can be regarded as a service unit with limited service capacity and queues to store data packets waiting to be processed. The first-come-first-served queuing theory is used to model the waiting time of data packets in the queue.

[0129] Service rate: Each satellite node i has a service rate μ i , represents the number of data packets that a node can process per second. The service rate depends on the hardware performance and current workload of the satellite.

[0130] Packet generation rate: arrival rate λ i It refers to the average rate at which data packets arrive at satellite node i.

[0131] Queue length: Queue length L i (t) represents the number of data packets waiting to be processed in the queue of satellite node i at time t. It can be expressed as the following formula:

[0132] L i (t+Δt)=Li (t)+λ i ·Δt-μ i Δt

[0133] Among them, Δt represents the time interval; λ represents the arrival rate;

[0134] Waiting time: The waiting time W of the data packet in the queue i (t) can be expressed as the following formula:

[0135]

[0136] The service time follows an exponential distribution, that is, the processing time of a packet is random, but the average service time is constant.

[0137] S15: The degree of congestion of the satellite network is measured by comparing the service rate of the satellite node with the packet generation rate, and the network congestion degree is modeled. Specifically, the network congestion degree can be measured by comparing the service rate of the satellite node with the packet generation rate. When the arrival rate is greater than the service rate, the network will become congested. Therefore, the congestion index C is defined as congestion (t) represents the congestion degree of satellite node i at time t, which is expressed as the following formula:

[0138]

[0139] Among them, λ i (t) represents the arrival rate of satellite node i at time t; when C congestion When (t)>0, it means that satellite node i is in a congested state; when C congestion When (t) < 0, it means that satellite node i has surplus processing capacity; when C congestion When (t) = 0, it means that satellite node i has just reached a balanced state. The data packet generation rate λ can be dynamically adjusted according to the current network congestion status and the remaining capacity of the satellite node. The adjustment formula is expressed as follows:

[0140] λ(t)=f(C congestion (t),C i (t))

[0141] Where λ(t) is the adjusted packet generation rate, f(C congestion (t),C i (t)) is a function that adjusts the generation rate according to the congestion index and the remaining capacity of the node.

[0142] S16: Modeling satellite network connectivity constraints. In a satellite network, a data packet must be transmitted from a source node to a destination node via at least one feasible path. For each source node s and destination node d, there exists a path P s,d Set makes:

[0143]

[0144] Among them, e i Represents the path P s,d The edge on the i ) represents edge e i The capacity, λ s Indicates the traffic demand of the source node.

[0145] S17: Construct a model to minimize information age. The satellite network is modeled as a space-time graph, expressed as the following formula:

[0146] G={(S,E,T,k)}

[0147] Among them, the set of S satellites is represented by S = {s1, s2, ..., s N}, E is the link set represented by E = {e1, e2, ..., e L}, T is a vector consisting of k time slots, and its time interval is τ = T / k. i The number of links in Since there are multiple transmission paths between any source satellite and destination satellite pair, during each topology snapshot, K paths between any nodes can be obtained, defined as R, to achieve effective traffic diversion, which is expressed as the following formula:

[0148]

[0149] in, Represents the time slice τ i All path sets in Represents the time slice τ i The first transmission path of the first data stream within; Represents the time slice τ i The Kth transmission path of the first data stream in; Represents the time slice τ i The Kth transmission path of the Mth data stream in .

[0150] Assume that the path space P = {P1,P2,...,P K The maximum flow C = {c1, c2, ..., c K}. Among them, P iThe maximum flow is the maximum capacity of the link. According to the bandwidth resources of the relevant paths, each source node can split the flow into M sub-flows that pass through K paths at the same time. represents the sub-flow ratio between the m-th node pair through the x-th path. The split solution of all sub-flow flows between the source node and the destination node pair can be expressed as the following formula:

[0151] W = {w 1,1 ,...,w 1,K ,...,w m,K}

[0152] The present invention aims at the problem of adaptive routing in satellite networks and proposes an optimization model with the goal of minimizing the age of information. The details are as follows:

[0153] Objective function: Minimize the average information age A of all paths in the network by optimizing the path selection R and traffic distribution strategy W avg ,Right now:

[0154]

[0155] Where K represents the total number of paths, τ i Represents a state in the network state set T; A x (τ i ) represents the time slice τ i The information age of the x-th path.

[0156] The constraints mainly include:

[0157] (1) Packet generation rate constraint to ensure the generation rate of each satellite node Not exceeding its maximum load capacity

[0158] (2) Satellite node processing capacity constraints to ensure the processing rate of each satellite node Not exceeding its maximum processing capacity

[0159] (3) Network connectivity constraints, ensuring that the total link capacity of the selected path can meet the data traffic demand λ s , that is, the sum of the capacity of each link on the path ∑capacity(e i ) must be greater than or equal to the flow rate λ s , and there must be at least one feasible path p i ∈P.

[0160] (4) Path constraints, ensuring the path set P μ Only feasible paths that meet the link status and node capability conditions are included.

[0161] (5) Queue length constraint, requiring the queue length L of all nodes i Maintaining the maximum queue length The following avoids transmission delays caused by long queues.

[0162] (6) Information age constraint, which ensures that the information age on the transmission path increases, that is, the candidate paths between node pairs satisfy Where 1≤x≤y≤K, m∈[1,M].

[0163] (7) The path splitting flow conservation constraint requires that the sum of the sub-flows of the total flow between any node pair distributed to all paths is 1, that is, m∈[1,M].

[0164] (8) Congestion index constraint, limiting the congestion index of each satellite node Does not exceed the predefined threshold C threshold , to avoid the loss of information timeliness due to congestion.

[0165] (9) Bandwidth constraints to ensure the flow of each link on the path Not exceeding the link bandwidth BW, where And e j ∈E S .

[0166] In order to solve the above-mentioned target optimization model, the present invention relates to a path search algorithm based on multi-hop information age optimization, which is used to optimize the transmission efficiency of delay-sensitive data in the network. The algorithm calculates multiple transmission paths with lower delays, thereby optimizing the information age of the entire network. Specifically, the present invention achieves effective traffic diversion by dynamically calculating multiple non-intersecting candidate paths, thereby improving the utilization of the path while avoiding network congestion. The following are the specific implementation steps of the multi-path search algorithm of the present invention:

[0167] S2: Multipath routing. The specific steps are as follows:

[0168] S21: Input the satellite pair set (S, D) of the source node and the target node, the ephemeris information (i.e., the satellite position and topology information in each time segment), and the predefined random weight range (r1<r2<r3) to initialize the link weight matrix M and the candidate path set R to an empty set, and the link occupancy frequency Set to 0, where e represents the link, τ i Represents a time slice.

[0169] S22: For each time segment τ iBased on the satellite ephemeris information, the Dijkstra's Shortest Path algorithm (DSP) is used to calculate the initial path set. The calculated path is included in the candidate path set. At the same time, other candidate paths are calculated to increase the diversity of paths. For each satellite pair of source node S and target node D, the occupied frequency of the link is iteratively updated on all paths in turn. Specifically, in each shortest path On the path, increase the link occupancy frequency of each edge (a, b) in the path, that is, execute In order to prevent the algorithm from falling into a local optimal state, the present invention sets a threshold based on the link frequency calculation To dynamically adjust the link weight. Below threshold Then randomly reassign the weight to the link Its value is in the random range (r1, r2). Otherwise, it is randomly assigned in the higher weight range (r2, r3).

[0170] S23: Remove the links included in the calculated shortest path, and re-run the shortest path algorithm on the remaining link set to calculate the suboptimal path. In this way, unoccupied link resources can be utilized to the maximum extent, ensuring that the system still has backup paths when the network is congested. Repeat the above steps until K candidate paths are found between each pair of source nodes and target nodes, thereby meeting the multi-path diversion requirements. Through the above path screening process, the present invention can reserve redundant channels on the transmission path to improve the robustness and stability of data transmission.

[0171] S3: Link state awareness, the specific steps are as follows:

[0172] S31: Each satellite periodically sends link status information to the control satellite, including the capacity of the satellite node to receive new data packets, the queue length of the current satellite node, the current network topology information, and the overall index of the network. The state S of each satellite node includes the following elements: C i : The capacity of the satellite node to receive new data packets. L i : The queue length of the current satellite node. G i : Current network topology information. C congestion,i : The overall index of the network.

[0173] Based on the path selection, an adaptive traffic segmentation algorithm is proposed to transform the information age minimization problem into a Markov decision process. Through the deep reinforcement learning method, the traffic allocation ratio and data generation rate are optimized according to the dynamic link parameters. The specific method is as follows:

[0174] S4: Traffic segmentation adaptive routing, the specific steps are as follows:

[0175] S41: State S includes C i : The capacity of the satellite node to receive new data packets and L i : The queue length of the current satellite node. G i : Current network topology information. C congestion,i : The overall index of the network. Action A is the traffic proportion and data generation rate of the candidate paths between all node pairs, expressed as: A = {a1, a2, ..., a M}, each action a j Includes: path traffic distribution weight, packet generation rate The reward R is the weighted sum of the data information age and the node congestion level, expressed as the following formula:

[0176] R(s,a)=-α·A avg (s,a)-β·C congestion (s)

[0177] Among them, R(s,a) represents the reward for taking action a in state s; A avg (s,a) represents the information age of taking action a in state s; C congestion (s) represents the link congestion degree in state s; α and β are the weight coefficients of information age and congestion degree respectively. The larger the reward value, the more effective the path selection and traffic distribution strategy are for optimizing information age.

[0178] S42: Input the path obtained by the path search algorithm based on multi-hop information age optimization in step S2 and the link state information obtained in step S3 into reinforcement learning, and train the intelligent agent to decide the flow ratio and data transmission rate allocated to each transmission path. The update of the optimal strategy follows the following formula:

[0179]

[0180] Where E represents the expectation; α is the temperature coefficient (regularization coefficient), which is used to adjust the importance of entropy in the objective function; γ is the discount factor; r(s t ,a t ) indicates state s t Action a occurs in the case t Reward; H(π(·|s t )) is the entropy function, representing the state s t The entropy of the strategy is expressed as follows:

[0181] H(π(·|s t ))=E[-logπ(·|s t )]

[0182] Value function Q(s t ,a t ) is defined as the following formula:

[0183]

[0184] Among them, s t+1 Sampled from the experience replay pool D; V π (s t+1 ) represents the value function, which means that in state s t+1 The expected return when following strategy π.

[0185] State value function V(s t ) represents the expected reward in a certain state, which is defined as the following formula:

[0186]

[0187] In addition, the policy network π in the Soft Actor-Critic (SAC) algorithm φ (a t ,s t ), soft state value network, target state value network And two soft Q networks Each of them consists of different parameter sets φ, ψ, θ parameterization, where φ represents the parameters of the policy network, responsible for generating policies; ψ represents the parameters of the state value network, estimating the value of the state; represents the parameters of the target state value network, which is updated with a delay to provide a more stable target value; θ represents the parameter set of the two Q networks, which estimate the Q value of each state-action pair respectively; the objective function of the network is optimized by applying the stochastic gradient descent method.

[0188] S5: Dynamic routing decision, the specific steps are as follows

[0189] S51: Based on the continuous iterative training in S4, the intelligent agent dynamically adjusts the traffic distribution ratio and data generation rate. The reward function is used to calculate the immediate reward, and the policy network and Q-value network are continuously optimized to ensure that the information age is minimized. After completing the training, the agent regularly collects the network status and makes traffic splitting decisions based on the trained policy network.

[0190] The following is combined with Figures 1 to 3 The specific embodiments of the present invention are described in detail.

[0191] See also Figure 1, is a satellite network data transmission application architecture diagram of an embodiment of the present invention; after the source satellite node s obtains the freshness information, it transmits to the destination node d through paths p1 and p2, taking into account the connectivity between satellites, and then receives it through the ground receiving equipment (Ground Receiving Equipment, GRE) and sends it to the ground control center (Ground Control Center, GCC), and the data center (Data Center, DC) updates the real-time monitoring data. It is assumed that each satellite node has the ability to transmit, process and store information, and can forward data through multi-hop paths. The system should be able to dynamically collect information from each node, including information age (information age), queue length, network load, etc. Each satellite node updates the information age according to the time when the data packet is received. If a data packet is successfully received and removed from the queue, the node's information age will be updated.

[0192] See also Figure 2 , is an interaction diagram of various modules in a satellite network adaptive routing system for delay optimization according to an embodiment of the present invention. The system includes:

[0193] The path selection module, based on the known ephemeris information, constructs a network topology snapshot through the real-time position of the satellite nodes. Each time slice constitutes a topology graph, and multiple candidate paths that meet the feasible path constraints are calculated through the multi-path routing algorithm.

[0194] Data packet generation rate adjustment module: adjusts the generation rate according to the processing capacity of the satellite node and the network congestion, and feeds the adjusted data packet rate back to the traffic allocation module. The adjustment of the data packet generation rate affects the subsequent path selection and traffic allocation.

[0195] Information age calculation module: Calculates and updates the information age (information age) of each satellite node, provides data for the path selection module, and helps select the most suitable path. It also feeds back the information age to the traffic allocation module to affect the optimal allocation of data flow. After updating the information age, the calculation results are fed back to the reinforcement learning module for model learning and strategy adjustment.

[0196] Path selection module: selects the optimal multi-hop path based on satellite ephemeris information and link status, and feeds the selection result back to the traffic allocation module. The optimization of path selection is affected by information age and queue status, so it interacts closely with the information age calculation module.

[0197] Traffic distribution module: optimizes traffic distribution based on information age, network congestion and satellite node queue length, and feeds back traffic scheduling results to the packet generation rate adjustment module and path selection module. By optimizing traffic distribution, the data forwarding rate and path selection strategy are affected.

[0198] Reinforcement learning module: Based on the network status (including information age, path selection, traffic distribution and other feedback information), adjust the path selection and traffic distribution strategy and optimize the long-term performance. This module interacts with all modules and provides dynamic adjustment and adaptive optimization capabilities.

[0199] The specific workflow of this system is as follows:

[0200] (1) Initialize path weight and link occupancy frequency

[0201] The system first initializes the path set R and the link occupancy frequency matrix Set the initial value to zero. At the same time, based on the satellite ephemeris information, a network topology graph G is constructed, which contains all nodes V and links E.

[0202] (2) Calculate the shortest path

[0203] Based on each time slice τ i The topology information is used to calculate the shortest path between the source node and the target node using the Dijkstra algorithm. The shortest path is stored in the path set R, and the link occupancy frequency on the path is updated.

[0204] (3) Dynamically assign link weights

[0205] In order to avoid the path selection falling into the local optimum, the link weight is dynamically adjusted: if the link occupies the frequency Less than threshold The weights are randomly assigned in the interval [r1, r2]. If the link occupancy frequency is greater than the threshold, the weights are assigned in the interval [r2, r3].

[0206] (4) Iterative search for candidate paths

[0207] After each shortest path is selected, the links used in the current path are deleted from the graph, and the shortest paths on the remaining links are recalculated. The above process is repeated until K candidate paths that meet the conditions are found.

[0208] (5) Output candidate paths

[0209] The final multiple non-intersecting paths are output, which can be used for subsequent data traffic segmentation and transmission to ensure load balancing and improve link utilization and data transmission efficiency.

[0210] See also Figure 3 , is a diagram of a dynamic routing decision training process based on a soft actor-critic algorithm (SAC) according to an embodiment of the present invention; the dynamic routing decision training process is composed of Figure 2The reinforcement learning module in the algorithm is executed, based on multiple candidate paths selected by the path selection module, and the model is trained according to the satellite network status to minimize the information age in multi-path transmission. In order to cope with the dynamic changes of transmission links and the differences in resource distribution, the algorithm models the routing decision problem as a Markov decision process (MDP) and uses the soft actor-critic algorithm (SAC) in deep reinforcement learning to dynamically optimize the routing strategy. The specific steps are as follows:

[0211] (1) Initialize reinforcement learning parameters

[0212] Initialize the network parameters in the Actor-Critic framework, including the initial weights of the policy network π and the value network Q, set the capacity of the experience pool M, and the weight parameters of the target network

[0213] (2) State Observation

[0214] The system periodically collects the environmental status S of the satellite network, including: the link utilization and delay information of the current path set R; the queue length L of the satellite node i ; The link congestion index C of the network congestion .

[0215] (3) Execute action

[0216] According to the policy network π, the traffic ratio is allocated to each path and packet generation rate Apply these allocation parameters as action A to the current network environment.

[0217] (4) Calculate the reward function

[0218] At each time slice, calculate the reward value R(s,a):

[0219] R(s,a)=-α·A avg (s,a)-β·C congestion (s)

[0220] Among them, A avg (s,a) represents the average information age of the system after executing the action, C congestion (s) represents the congestion level of the link, and α and β are weight parameters.

[0221] (5) Update the strategy network and value network

[0222] The result of the current action (state, reward, next state) is stored in the experience pool M. Small batches of data are randomly extracted, and the policy network and value network are optimized using the gradient descent method.

[0223] (6) Online learning and routing adjustment

[0224] The system adjusts the traffic distribution of each path according to the latest environmental status by updating the policy network in real time, ensuring that high-priority data packets can be given priority through low-latency paths, thereby reducing the overall information age. The system outputs the trained optimal traffic distribution strategy and applies it to the data transmission process.

[0225] The present invention proposes a delay-optimized satellite network adaptive routing method and system, which combines the dynamic topology characteristics of the satellite network and optimizes the information age in the multi-hop path to ensure the timeliness of the data. In addition, by integrating the link time-varying characteristics into the routing strategy, the present invention provides a highly adaptable solution to improve the transmission efficiency of the satellite network and meet the data real-time requirements of delay-sensitive services.

[0226] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A satellite network adaptive routing method with time delay optimization, characterized in that: The following steps are involved: S1: Construction of multi-hop information age, specifically: S11: In the low-orbit satellite constellation system model, it is assumed that A fi (t) represents the instantaneous information age of the i-th hop satellite node of data stream f at time slot t, and the instantaneous information age is calculated; S12: Introducing binary variable x fik (t) indicates whether the i-th hop satellite node of data flow f receives the k-th data packet in time slot t; if x fik (t) = 1 means that at time slot t, the i-th hop satellite node on data stream f successfully receives the k-th update packet, otherwise x fik (t) = 0; S13: Describe the relationship between the information age of the current satellite node and the previous hop satellite node; if the data packet is received at time t-1, the information age is represented by A f(i-1) (t-1)+1; if no data packet is received, the information age is represented by A fi (t-1)+1; construct the basic model of information age; S14: Each satellite node is considered as a service unit with limited service capacity and queue to store data packets waiting to be processed; the first-come-first-served queuing theory is used to model the waiting time of data packets in the queue; S15: The congestion level of the satellite network is measured by comparing the service rate and packet generation rate of the satellite nodes, and the network congestion level is modeled: S16: Modeling satellite network connectivity constraints: S17: Construct a model to minimize information age; S2: Multi-path routing, which achieves traffic diversion by dynamically calculating multiple non-intersecting candidate paths based on a path search algorithm optimized based on multi-hop information age; S3: Link status perception. Each satellite periodically sends link status information to the control satellite, including the capacity of the satellite node to receive new data packets, the queue length of the current satellite node, the current network topology information, and the overall index of the network. S4: Traffic segmentation adaptive routing, based on path selection, optimizes traffic allocation ratio and data generation rate according to dynamic link parameters through deep reinforcement learning method; S5: Dynamic routing decision, based on the continuous iterative training of S4, the intelligent agent dynamically adjusts the traffic distribution ratio and data generation rate; uses the reward function to calculate the immediate reward, continuously optimizes the policy network and Q value network to minimize the information age; after completing the training, the agent regularly collects the network status and makes traffic segmentation decisions based on the trained policy network.

2. The delay-optimized satellite network adaptive routing method according to claim 1, characterized in that: The instantaneous information age is calculated in S11, specifically: The instantaneous information age is expressed as follows: Among them, g fk represents the generation time of the kth data packet of data stream f; The S13 reconstructs the basic model of information age, specifically: The relationship between the information age of the current satellite node and the previous hop satellite node is described as follows: The basic model for constructing information age is as follows: Where K represents the total number of data flows: K f represents the total number of packets in data stream f; x fik (t-1) is a binary variable indicating whether data packet k is successfully received in time slot t-1; The S14 is specifically: When a packet arrives at node j, the effect of congestion on the information age is modeled as the waiting time W of the queue i The increase is expressed as the following formula: A i (t)=t-g i +W i (t) Among them, A i (t) represents the information age of data packet i at time t; g i W is the time when data packet i is generated at the source node; i (t) is the waiting time of packet i in the queue; Assume that the service rate of each satellite node i is μ i , represents the number of packets that a node can process per second; let the arrival rate λ i , represents the average rate at which data packets arrive at satellite node i; let the queue length be L i (t), represents the number of data packets waiting to be processed in the queue of satellite node i at time t, expressed as the following formula: L i (t+Δt)=L i (t)+λ i ·Δt-μ i Δt Among them, Δt represents the time interval; λ represents the arrival rate; The average service time is The waiting time W of the packet in the queue i (t) can be expressed as the following formula: The S15 is specifically: Define the congestion index C congestion (t) represents the congestion degree of satellite node i at time t, which is expressed as the following formula: Among them, λ i (t) represents the arrival rate of satellite node i at time t; when C congestion When (t)>0, it means that satellite node i is in a congested state; when C congestion When (t) < 0, it means that satellite node i has surplus processing capacity; when C congestion When (t) = 0, it means that satellite node i has just reached the equilibrium state; The data packet generation rate λ is dynamically adjusted according to the current network congestion status and the remaining capacity of the satellite node; the adjustment formula is expressed as follows: λ(t)=f(C congestion (t),C i (t)) Where λ(t) is the adjusted packet generation rate, f(C congestion (t),C i (t)) is a function that adjusts the generation rate according to the congestion index and the remaining capacity of the node; The S16 is specifically: For each source node s and destination node d, there exists a path P s,d The set makes: Among them, e i Represents the path P s,d The edge on the i ) represents edge e i The capacity, λ s Indicates the traffic demand of the source node; The S17 is specifically: The satellite network is modeled as a space-time graph, expressed as the following formula: G={(S,E,T,k)} Among them, the set of S satellites is represented by S = {s1, s2, ..., s N }, E is the link set represented by E = {e1, e2, ..., e L }, T is a vector consisting of k time slots, whose time interval is τ = T / k; in the time period τ i The number of links in The K paths between any nodes are defined as R to achieve effective traffic diversion, which is expressed as the following formula: in, Represents the time slice τ i All path sets in Represents the time slice τ i The first transmission path of the first data stream within; Represents the time slice τ i The Kth transmission path of the first data stream in; Represents the time slice τ i The Kth transmission path of the Mth data stream in the Assume that the path space P = {P1,P2,...,P K The maximum flow C = {c1, c2, ..., c K }; where P i The maximum flow is the maximum capacity of the link; according to the bandwidth resources of the relevant paths, each source node can split the flow into M sub-flows that pass through K paths at the same time, let represents the sub-flow ratio between the m-th node pair through the x-th path, and the split solution W of all sub-flow flows between the source node and the destination node pair is expressed as the following formula: In={in 1,1 ,...,In 1,K ,...,In m,K }。 3. The delay-optimized satellite network adaptive routing method according to claim 2, characterized in that: The S17 is specifically: By optimizing the path selection R and the traffic distribution strategy W, the average information age A of all paths in the network is minimized. avg It is expressed as the following formula: Where K represents the total number of paths, τ i Represents a state in the network state set T; A x (τ i ) represents the time slice τ i Inside, the information age of the xth path; Setting constraints includes: 1) Packet generation rate constraints to ensure the generation rate of each satellite node Not exceeding its maximum load capacity 2) Satellite node processing capacity constraints to ensure the processing rate of each satellite node Not exceeding its maximum processing capacity 3) Network connectivity constraints, ensuring that the total link capacity of the selected path can meet the data traffic demand λ s , that is, the sum of the capacity of each link on the path ∑capacity(e i ) must be greater than or equal to the flow rate λ s , and there must be at least one feasible path p i ∈P; 4) Path constraints, ensuring that the path set P μ Only feasible paths that meet the link status and node capability conditions are included; 5) Queue length constraint, requiring the queue length L of all nodes i Keep at maximum queue length The following avoids transmission delays caused by long queues; 6) Information age constraint, which ensures that the information age on the transmission path increases, i.e., the candidate paths between node pairs satisfy Where 1≤x≤y≤K, m∈[1,M]; 7) Path split flow conservation constraint requires that the sum of the sub-flows of the total flow between any node pair distributed to all paths is 1, that is, m∈[1,M]; 8) Congestion index constraint, limiting the congestion index of each satellite node Does not exceed the predefined threshold C threshold , to avoid the loss of information timeliness due to congestion; 9) Bandwidth constraints to ensure the flow of each link on the path Not exceeding the link bandwidth BW, where And e j ∈E S .

4. The delay-optimized satellite network adaptive routing method according to claim 1, characterized in that: The S2 specifically comprises the following steps: S21: Input the source satellite node and the destination satellite node pair, ephemeris information and a predefined random weight range, and initialize them, specifically: Input the satellite pair set (S, D) of the source node and the target node, the ephemeris information, and the predefined random weight range (r1<r2<r3), initialize the link weight matrix M and the candidate path set R to an empty set, and the link occupancy frequency Set to 0, where e represents the link, τ i Indicates a time segment; S22: For each time slot τ i , using the Dijkstra shortest path algorithm based on information age to calculate the shortest path in the current time slot If the link is in the shortest path, the link occupation frequency is updated. To prevent the algorithm from falling into the local optimum, a threshold setting method is adopted. If the link occupation frequency Less than a given threshold The ground control center randomly assigns an integer between r1 and r2 as the new link metric; otherwise, the link metric is set to a random integer between r2 and r3; S23: Delete the links in the transmission path with the shortest information age, and continue to select the second shortest path using the shortest path algorithm on the remaining links to make full use of the idle links.

5. The delay-optimized satellite network adaptive routing method according to claim 1, characterized in that: The S3 is specifically: The state S of each satellite node contains the following elements: i : The capacity of the satellite node to receive new data packets; L i : The queue length of the current satellite node; G i : Current network topology information; C congestion,i : The overall index of the network.

6. The delay-optimized satellite network adaptive routing method according to claim 1, characterized in that: The S4 is specifically: S41: Assume that state S includes C i and L i ; C i represents the capacity of the satellite node to receive new data packets; L i Indicates the queue length of the current satellite node; G i Indicates the current network topology information; C congestion,i represents the overall index of the network; action A is the traffic proportion and data generation rate of the candidate paths between all node pairs, expressed as: A = {a1, a2, ..., a M }, each action a j Contains: path traffic distribution weight, packet generation rate The reward R is the weighted sum of the data information age and the node congestion level, expressed as the following formula: R(s,a)=-α·A avg (s,a)-β·C congestion (s) Among them, R(s,a) represents the reward for taking action a in state s; A avg (s,a) represents the information age of taking action a in state s; C congestion (s) represents the link congestion degree in state s; α and β are the weight coefficients of information age and congestion degree respectively. The larger the reward value, the more effective the path selection and traffic allocation strategy is for optimizing information age. S42: Input the path obtained by the path search algorithm based on multi-hop information age optimization in step S2 and the link state information obtained in step S3 into reinforcement learning, and train the intelligent agent to decide the flow ratio and data transmission rate allocated to each transmission path; wherein the update of the optimal strategy satisfies the following formula: Where E represents the expectation; α is the temperature coefficient (regularization coefficient), which is used to adjust the importance of entropy in the objective function; γ is the discount factor; r(s t ,a t ) indicates state s t Action a occurs in the case t Reward; H(π(·|s t )) is the entropy function, representing the state s t The entropy of the strategy is expressed as follows: H(π(·|s t ))=E[-logπ(·|s t )] Value function Q(s t ,a t ) is defined as the following formula: Among them, s t+1 Sampled from the experience replay pool D; V π (s t+1 ) represents the value function, which means that in state s t+1 The expected return when following strategy π. State value function V(s t ) represents the expected reward in a certain state, which is defined as the following formula:

7. The delay-optimized satellite network adaptive routing method according to claim 1, characterized in that: The S4 is specifically: S41: Initialize reinforcement learning parameters, initialize network parameters in the Actor-Critic framework, including the initial weights of the policy network π and the value network Q, set the capacity of the experience pool M, and the weight parameters of the target network S42: State observation: the system periodically collects the environmental state S of the satellite network, including: link utilization and delay information of the current path set R; queue length L of the satellite node i ; The link congestion index C of the network congestion ; S43: Execute the action and allocate the traffic ratio for each path according to the policy network π and packet generation rate Apply these allocation parameters as action A to the current network environment; S44: Calculate the reward function. At each time slice, calculate the reward value R(s,a): R(s,a)=-α·A avg (s,a)-β·C congestion (s) Among them, A avg (s,a) represents the average information age of the system after executing the action, C congestion (s) represents the congestion level of the link, α and β are weight parameters; S45: Update the policy network and value network, store the result of the current action in the experience pool M; randomly extract small batches of data, and use the gradient descent method to optimize the policy network and value network; S46: Online learning and routing adjustment. The system updates the policy network in real time and adjusts the traffic distribution of each path according to the latest environmental status to ensure that high-priority data packets can pass through low-latency paths first, thereby reducing the overall information age; outputs the trained optimal traffic distribution strategy and applies it to the data transmission process.

8. A satellite network adaptive routing system with time delay optimization, characterized in that: include: Path selection module: Based on the known ephemeris information, a network topology snapshot is constructed by the real-time position of the satellite nodes; each time slice constitutes a topology map, and multiple candidate paths that meet the feasible path constraints are calculated through a multi-path routing algorithm; Data packet generation rate adjustment module: adjusts the generation rate according to the processing capacity of the satellite node and the network congestion, and feeds the adjusted data packet rate back to the traffic allocation module; the adjustment of the data packet generation rate affects the subsequent path selection and traffic allocation; Information age calculation module: calculates and updates the information age of each satellite node, provides data for the path selection module, and helps select the most suitable path; It also feeds back the information age to the traffic allocation module, affecting the optimal allocation of data flows; After updating the information age, the calculation results are fed back to the reinforcement learning module to provide model learning and adjustment strategies; Path selection module: selects the optimal multi-hop path based on satellite ephemeris information and link status, and feeds the selection result back to the traffic allocation module; Traffic distribution module: optimizes traffic distribution according to information age, network congestion and satellite node queue length, and feeds back traffic scheduling results to the data packet generation rate adjustment module and path selection module; by optimizing traffic distribution, it affects the data forwarding rate and path selection strategy; Reinforcement learning module: Based on the network status, adjust the path selection and traffic distribution strategy, and optimize the long-term performance; this module interacts with other modules in the system to dynamically adjust and optimize adaptation.

Images