Satellite constellation lightweight anti-congestion intelligent routing method and device based on stackelberg game, equipment, medium and product

Abstract

This application discloses a lightweight, anti-blocking intelligent routing method, apparatus, device, medium, and product for satellite constellations based on Stackelberg game theory, relating to the field of satellite communication networks. The method includes: constructing a satellite constellation network simulation environment and establishing a Stackelberg game model, defining the blocker as the leader and satellites as followers; designing a hierarchical routing framework, including a routing selection module based on an Actor-Critic architecture and an anti-blocking module based on Q-Learning; updating routing network parameters using a near-end policy optimization algorithm and utilizing Q-Learning to quickly reselect paths after detecting blockage; and achieving algorithm lightweighting through artificial dimensionality reduction of the state space and knowledge distillation techniques. This application effectively improves the robustness and reliability of satellite constellation networks in intelligent blocking environments, realizes a lightweight design of the intelligent algorithm, proves the existence of Stackelberg equilibrium, and significantly improves routing success rate and reduces average latency.

Description

Technical Field

[0001] This application relates to the field of satellite communication networks, and in particular to a lightweight, anti-blocking intelligent routing method, apparatus, device, medium, and product for satellite constellations based on Stackelberg game theory. Background Technology

[0002] In recent years, the rapid development of terrestrial network technology has driven the expansion of network construction into space. Satellites, with their advantages of high viewpoint and wide field of view, can effectively monitor ground information and play a vital role in disaster early warning, scientific exploration, navigation, and positioning. A constellation network composed of multiple satellites can achieve global coverage, and is particularly capable of transmitting information back to remote areas in real time, which has significant strategic value for the nation.

[0003] However, on the one hand, the complex structure, high dynamism, and resource constraints of constellation networks significantly increase the difficulty and urgency of constellation routing research. On the other hand, satellite constellations, due to their periodic visibility and fixed orbits, are vulnerable to intelligent blocking attacks. Traditional routing algorithms, including virtual node routing, virtual topology routing, and AODV, mainly focus on route optimization for networks with known topologies, while relatively insufficient consideration is given to anti-blocking design. Although deep reinforcement learning-based intelligent methods have shown some potential in the field of anti-blocking, their computational complexity is high, their convergence speed is slow, and they are prone to getting trapped in local optima, making it difficult to quickly adapt to changes in blocking.

[0004] Therefore, there is an urgent need for an anti-blocking routing algorithm that combines intelligent decision-making capabilities with lightweight characteristics to improve the robustness of satellite networks in highly dynamic adversarial environments. Existing technical solutions often struggle to simultaneously address the intelligence of game-theoretic adversarial scenarios, the real-time nature of routing decisions, and the constraints of satellite node resources, leading to decreased communication reliability in highly adversarial environments. Summary of the Invention

[0005] The purpose of this application is to provide a lightweight anti-blocking intelligent routing method, device, equipment, medium and product for satellite constellations based on Stackelberg game theory. It relies on technologies such as Stackelberg game theory, reinforcement learning and deep learning to optimize the routing selection strategy, thereby improving routing performance and algorithm robustness, enabling it to effectively cope with intelligent blocking attacks in dynamic and complex satellite network environments.

[0006] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a lightweight, anti-blocking intelligent routing method for satellite constellations based on Stackelberg game theory, including: Establish a satellite constellation network simulation environment to simulate the dynamic topology of the satellite constellation network; Based on the established satellite constellation network simulation environment, a Stackelberg game model is performed. The blocker is defined as the leader who first decides the interference strategy, and the satellite is defined as the follower who observes the blocking behavior and then chooses the transmission path. The existence of Stackelberg equilibrium in this game is proven. A satellite constellation intelligent routing model resistant to blocking attacks is constructed, including a routing selection module and an anti-blocking module. The routing selection module is used to generate an available set of routes, and the anti-blocking module makes decisions using a subset of routes generated by the routing selection module as the decision space. The routing module and the anti-blocking module are trained and optimized using a reinforcement learning strategy. The routing module adopts an Actor-Critic architecture, and the anti-blocking module adopts the Q-Learning algorithm. The trained intelligent routing model undergoes network lightweighting, including manual dimensionality reduction of the state space and knowledge distillation of the policy network, and the lightweighted algorithm is deployed on satellite nodes.

[0007] Optionally, the establishment of the satellite constellation network simulation environment specifically includes: A satellite constellation network is constructed based on Kepler orbital parameters. The constellation consists of N satellites distributed across O orbital planes. Each satellite establishes two inter-satellite links within the same orbit and two inter-satellite links outside the same orbit. A distance threshold is defined for establishing links between satellites. When the angle between satellites exceeds a preset angle threshold, the inter-satellite link is disconnected. An end-to-end delay model is defined, which includes transmission delay, propagation delay, and queuing delay.

[0008] Optionally, the Stackelberg game modeling specifically includes: The Q-Learning algorithm is used to build a blocker as the leader. Based on the estimation of the network state, key nodes and power in the routing path are selected for blocking attacks to maximize the blocker's utility function. As a follower, the satellite, after observing or inferring congestion behavior, dynamically selects the optimal transmission path from a pre-selected subset of available routes to minimize the satellite's utility function.

[0009] Optionally, training and optimizing the routing module using a reinforcement learning strategy specifically includes: The Actor network uses a 2-layer LSTM to encode the input state space, resulting in a processed state sequence. The processed state sequence is input into a 4-layer MLP network to complete the analysis and decision-making of the state sequence; The input state of the Critic network is first encoded through a 2-layer LSTM and a 1-layer MLP network to obtain the encoded sequence. The encoded sequence and the actions output by the Actor network are passed together through a 4-layer MLP network to finally output the value evaluation of the policy, providing feedback information for the training of the Actor network; Design reward function Includes queue time penalty Distance Reduction Reward and additional rewards and penalties The network parameters are updated using a near-end strategy optimization algorithm, and the parameter update formula is as follows: ; ; in, and These represent the gradient parameters of the updated Actor network and Critic network, respectively. and These are the gradient parameters of the Actor network and the Critic network, respectively. It is the learning rate of the Actor network. It is the learning rate of the Critic network. It is the immediate reward for the k-th sample. It is a discount factor. It is the value estimate of the k-th state. It is policy entropy. It is the gradient operator. Is the current strategy in the state? Select action The probability of KL and Let KL divergence and entropy function represent the two functions, respectively. and It is KL and coefficient, Indicates the old strategy in state Select action The probability, Indicates the current policy in state The probability distribution of actions for all feasible region nodes. Indicates the old strategy in state The probability distribution of actions for all feasible region nodes. , , They are , and The coefficient.

[0010] Optionally, training and optimizing the anti-blocking module using a reinforcement learning strategy specifically includes: Blocker pair function Blocking node strategy Blocking power strategy as well as the learning rate and Boltzmann parameters Perform initialization; At each time step, the blocker operates according to the current policy. and Select blocking node and blocking power ; The blocker directs to the selected node. Transmit blocking power is The blocking signal; Blocker Calculation Blocker Utility Function Come to function Update and based on the updated... Function-to-blocking strategy and The formula has been adjusted and updated as follows: ; ; ; in, This represents the updated action value function of the blocker J in the state after the (s+1)th iteration. This represents the updated action value function of the blocker J in the state after the (s+1)th iteration. Indicates the learning rate. This represents the updated policy probability of the blocker selecting a blocking target in round s+1. It is an action index variable, representing the summation of the possible action space of the blocker. This represents the updated policy probability of the blocker selecting the blocking power in round s+1. This represents the index for summing the traversal of the blocking power action space. The first blocker One candidate target, The first blocker Candidate blocking power.

[0011] Optionally, after observing or inferring congestion behavior, the optimal transmission path is dynamically selected from a pre-selected subset of available routes to minimize the satellite utility function, specifically including: First, initialize the Q function. Routing strategy This ensures that the probability of selecting all available routing paths is equal in the initial phase; where, This indicates the nth routing action selected by the user in the initial phase. Indicates the size of the routing action space; At each time step, the blocker is invoked according to the blocking policy. and Select blocking node and blocking power ; After sensing the congestion and link status in the network, the current routing selection strategy is applied. Re-establish the route; Calculate the reward based on the designed reward function. ; According to the reward Update Q function ; And based on the updated Q function Adjust routing strategy The updated formula is as follows: ; in, This represents the updated Q-value after the user's iteration (t+1). Indicates the learning rate. Indicates the discount factor. The index represents the summation index of the action space traversal. This represents the updated policy probability that a user chooses the nth routing action in round t+1. This represents the temperature coefficient of the Softmax strategy.

[0012] Secondly, this application provides a lightweight, anti-blocking intelligent routing device for satellite constellations based on Stackelberg game theory, comprising: The simulation environment building unit is used to establish a satellite constellation network simulation environment and simulate the dynamic topology of the satellite constellation network. The game modeling unit is used to perform Stackelberg game modeling based on the established satellite constellation network simulation environment. It defines the blocker as the leader who first decides the interference strategy, and the satellite as the follower who observes the blocking behavior and then chooses the transmission path. It also proves that the game has a Stackelberg equilibrium. A satellite constellation intelligent routing model construction unit is used to construct a satellite constellation intelligent routing model resistant to blocking attacks. It includes a routing selection module and an anti-blocking module. The routing selection module is used to generate an available set of routes, and the anti-blocking module makes decisions using a subset of routes generated by the routing selection module as the decision space. The training and optimization unit is used to train and optimize the routing module and the anti-blocking module using a reinforcement learning strategy, wherein the routing module adopts an Actor-Critic architecture and the anti-blocking module adopts the Q-Learning algorithm. The lightweight processing unit is used to perform network lightweighting processing on the trained intelligent routing model, including manual dimensionality reduction of the state space and knowledge distillation of the policy network, and then deploys the lightweight algorithm on satellite nodes.

[0013] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the lightweight, anti-blocking intelligent routing method for satellite constellations based on Stackelberg game theory as described above.

[0014] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the lightweight, anti-blocking intelligent routing method for satellite constellations based on Stackelberg game theory described above.

[0015] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the lightweight, anti-blocking intelligent routing method for satellite constellations based on Stackelberg game theory described above.

[0016] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a lightweight anti-blocking intelligent routing method, device, equipment, medium, and product for satellite constellations based on Stackelberg game theory. Firstly, this application effectively improves the robustness and reliability of satellite constellation networks in intelligent blocking environments. The method integrates Stackelberg game theory, deep reinforcement learning, and knowledge distillation techniques to construct a hierarchical routing optimization framework. It models the anti-blocking routing problem as a Stackelberg game, where the intelligent blocker acts as the leader and the satellites as followers, pursuing maximum utility through repeated policy interactions, thereby effectively responding to intelligent blocking attacks. Specifically, the anti-blocking module employs the Q-Learning algorithm, which can quickly select the optimal path from the selected route set to avoid blocking nodes, achieving a rapid response to blocking attacks.

[0017] Secondly, this application achieves a lightweight design for the intelligent algorithm, improving its applicability in resource-constrained environments. Addressing the limited computational resources of satellite nodes, this application employs techniques such as manual pruning and dimensionality reduction of the algorithm's state space and knowledge distillation of the policy network. Manual dimensionality reduction removes redundant features such as absolute coordinates. Simultaneously, knowledge distillation of the Actor network is performed using a teacher-student model architecture, significantly reducing the number of network parameters while maintaining high performance, thereby ensuring the stable operation and deployment of the intelligent routing algorithm in resource-constrained satellite environments.

[0018] This application proves the existence of the proposed Stackelberg equilibrium. Simulation results verify the significant improvement in key indicators such as routing success rate and average latency. It provides an anti-blocking routing solution with both intelligent decision-making capabilities and lightweight characteristics for high-reliability communication routing optimization tasks of low-Earth orbit satellite constellation networks in intelligent congestion environments. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating a lightweight, anti-blocking intelligent routing method for satellite constellations based on Stackelberg game theory, provided as an embodiment of this application; Figure 2 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] In one exemplary embodiment, such as Figure 1As shown, a lightweight, anti-blocking intelligent routing method for satellite constellations based on Stackelberg game theory is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method includes the following steps: Step 101: Establish a satellite constellation network simulation environment to simulate the dynamic topology of the satellite constellation network.

[0024] The constellation network consists of N satellites distributed across O orbital planes, each orbital plane It can be approximated as a circular orbit, deployed above the Earth's surface. At an altitude of one kilometer, with a given angle of inclination. The orbital period is ,in, It is a semi-major axis. It is the Earth's gravitational constant. Due to the high dynamics of constellation networks, inter-satellite connectivity varies over time. Satellite connections adopt an 'X'-shaped inter-satellite link configuration, with each satellite establishing two intra-orbit inter-satellite links and two inter-orbit inter-satellite links. A routing delay model is defined, including transmission delay, propagation delay, and queuing delay. Transmission delay is affected by various factors, including channel capacity, transmission power, noise level, and the distance between satellites. Channel capacity... It is a key metric for measuring link performance, and its calculation is based on Shannon's formula: ; in, It is the decline of free space. , , They represent the first Link The channel bandwidth, transmission power, and channel noise. Assume the amount of data transmitted each time is... bits, then link The transmission delay is: ; Inter-satellite propagation delay refers to the physical propagation time required for a data packet to travel from the sender to the receiver. In constellation network routing, It is the spherical distance between the current satellite and the next satellite. If the speed is light, then the interstellar propagation delay can be expressed as: ; The communication latency of satellite nodes is not only related to the incoming traffic. This also depends on the existing local traffic within the current node. It assumes that there is already traffic existing within the current node. Obtain the parameter as Poisson distribution: ; ; in, This represents an existing local traffic random variable.

[0025] The communication delay can then be expressed as: ; in, This represents the total number of hops in the inter-satellite data transmission path.

[0026] Step 102: Based on the established satellite constellation network simulation environment, perform Stackelberg game modeling, define the blocker as the leader to make the first decision on the interference strategy, define the satellite as the follower to observe the blocking behavior and choose the transmission path, and prove that the game has a Stackelberg equilibrium.

[0027] Stackelberg game modeling defines the anti-congestion routing problem in satellite constellation networks as a Stackelberg game, defining the blocker as the leader, who first decides its interference strategy, with the goal of causing the maximum transmission delay with the minimum power cost. Its utility function is defined as: ; in, The transmission time sensed and estimated by the blocker. This is the estimated minimum transmission time without congestion; the difference between the two is the additional delay caused by the blocker's blocking attack on the route transmission time. Since the blocker cannot accurately obtain feedback after blocking, a normal distribution coefficient is introduced here. This reflects the uncertainty in assessing the effectiveness of blocking. Represents the selectable blocking power. The blocking cost per unit power. As a follower, the satellite, upon observing changes in the network environment, makes its own routing response, aiming to minimize the multi-target routing cost. Its utility function is defined as: , It is the routing cost. Furthermore, it can be proven that the strategy combination... A Stackelberg equilibrium is established if and only if: ; ; That is, neither side can improve its own utility by unilaterally deviating from the equilibrium strategy.

[0028] Step 103: Construct a satellite constellation intelligent routing model resistant to blocking attacks, including a routing selection module and an anti-blocking module. The routing selection module is used to generate an available set of routes, and the anti-blocking module makes decisions using a subset of routes generated by the routing selection module as the decision space.

[0029] Step 104: The routing module and the anti-blocking module are trained and optimized using a reinforcement learning strategy. The routing module adopts an Actor-Critic architecture, and the anti-blocking module adopts the Q-Learning algorithm.

[0030] The intelligent blocker is trained using the Q-Learning algorithm. First, the blocker trains the intelligent blocker using the Q-function. Blocking node strategy Blocking power strategy as well as the learning rate and Boltzmann parameters Initialize; then at each time step, the blocker will adjust according to the current strategy. and Go to select the blocking node and blocking power The blocker then moves to the selected node. Transmit blocking power is When a blocking signal is received, the link performance in the constellation network will decrease due to the blocking, specifically manifested as reduced channel capacity and increased end-to-end latency; then the blocker calculates its utility function. To its Q function Update the function and adjust the blocking strategy accordingly. and The formula has been adjusted and updated as follows: ; ; ; A routing model is constructed using an Actor-Critic architecture. The Actor network uses a two-layer LSTM to encode the input state space, and the processed state sequence is further input into a four-layer MLP network to complete the analysis and decision-making of the state sequence. The Critic network's input state is first encoded through a two-layer LSTM and a one-layer MLP network. Then, the encoded sequence and the actions output by the Actor network are passed through a four-layer MLP network, finally outputting the value evaluation of the policy, providing feedback information for the training of the Actor network. A reward function is designed. It includes queue time penalties, distance reduction rewards, and additional reward / penalty items. Network parameters are updated using the Proximity Policy Optimization (PPO) algorithm, with the parameter update formula as follows: ; ; in, and These are the gradient parameters of the Actor network and the Critic network, respectively. It is the learning rate of the Actor network. It is the learning rate of the Critic network. It is the first Instant reward for each sample It is a discount factor. It is the first Value estimation of each state It is policy entropy. It is the gradient operator. Is the current strategy in the state? Select action The probability of KL and Let KL divergence and entropy function represent the two functions, respectively. and These are the corresponding coefficients. Through maximum entropy regularization, the agent is encouraged to explore under different utility conditions to maintain generalization ability.

[0031] When affected by congestion, the Q-Learning algorithm is used to quickly reselect the optimal path from the selected route set to avoid the blocking node. The specific steps are as follows: First, initialize the Q function. Routing strategy This ensures that the probability of selecting all available routing paths is equal in the initial phase. Subsequently, at each time step, the blocker is invoked according to its blocking policy. and Select blocking node and blocking power After sensing network congestion and link status, based on the current strategy... Choose a new route and re-establish the route. Calculate the reward based on the designed reward function. According to the reward Update its Q function and adjust the routing strategy based on the updated Q function. The updated formula is as follows: ; .

[0032] Repeat the above process until the predetermined stopping condition is met.

[0033] Step 105: Perform network lightweighting on the trained intelligent routing model, including manual dimensionality reduction of the state space and knowledge distillation of the policy network, and deploy the lightweight algorithm on satellite nodes.

[0034] Artificial dimensionality reduction of the state space specifically includes: manually pruning and reducing the dimensionality of the algorithm's state space, discarding redundant information such as absolute coordinates; retaining key decision features, reducing the input dimension, thereby reducing the algorithm's demand for satellite node computing resources.

[0035] Policy network knowledge distillation specifically includes: using a teacher-student model architecture to distill and train only the Actor network; designing a distillation loss function suitable for reinforcement learning scenarios to significantly compress the number of network parameters while maintaining model performance; and deploying the lightweight and trained intelligent routing algorithm on resource-constrained satellite nodes to ensure its stable operation in the satellite environment.

[0036] Based on the same inventive concept, this application also provides a Stackelberg game-based lightweight anti-blocking intelligent routing device for implementing the Stackelberg game-based lightweight anti-blocking intelligent routing method for satellite constellations described above. The solution provided by this device is similar to the implementation described in the above method. Therefore, the specific limitations of one or more Stackelberg game-based lightweight anti-blocking intelligent routing device embodiments provided below can be found in the limitations of the Stackelberg game-based lightweight anti-blocking intelligent routing method for satellite constellations described above, and will not be repeated here.

[0037] In one exemplary embodiment, a lightweight, anti-blocking intelligent routing device for satellite constellations based on Stackelberg game theory is provided, comprising: The simulation environment building unit is used to establish a satellite constellation network simulation environment and simulate the dynamic topology of the satellite constellation network. The game modeling unit is used to perform Stackelberg game modeling based on the established satellite constellation network simulation environment. It defines the blocker as the leader who first decides the interference strategy, and the satellite as the follower who observes the blocking behavior and then chooses the transmission path. It also proves that the game has a Stackelberg equilibrium. A satellite constellation intelligent routing model construction unit is used to construct a satellite constellation intelligent routing model resistant to blocking attacks. It includes a routing selection module and an anti-blocking module. The routing selection module is used to generate an available set of routes, and the anti-blocking module makes decisions using a subset of routes generated by the routing selection module as the decision space. The training and optimization unit is used to train and optimize the routing module and the anti-blocking module using a reinforcement learning strategy, wherein the routing module adopts an Actor-Critic architecture and the anti-blocking module adopts the Q-Learning algorithm. The lightweight processing unit is used to perform network lightweighting processing on the trained intelligent routing model, including manual dimensionality reduction of the state space and knowledge distillation of the policy network, and then deploys the lightweight algorithm on satellite nodes.

[0038] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 2 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores lightweight, anti-blocking intelligent routing data for satellite constellations based on Stackelberg game theory. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a lightweight, anti-blocking intelligent routing method for satellite constellations based on Stackelberg game theory.

[0039] Those skilled in the art will understand that Figure 2 The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0040] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0041] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0042] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0043] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0044] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0045] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0046] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A lightweight, anti-blocking intelligent routing method for satellite constellations based on Stackelberg game theory, characterized in that, The lightweight, anti-blocking intelligent routing method for satellite constellations based on Stackelberg game theory includes: Establish a satellite constellation network simulation environment to simulate the dynamic topology of the satellite constellation network; Based on the established satellite constellation network simulation environment, a Stackelberg game model is performed. The blocker is defined as the leader who first decides the interference strategy, and the satellite is defined as the follower who observes the blocking behavior and then chooses the transmission path. The existence of Stackelberg equilibrium in this game is proven. A satellite constellation intelligent routing model resistant to blocking attacks is constructed, including a routing selection module and an anti-blocking module. The routing selection module is used to generate an available set of routes, and the anti-blocking module makes decisions using a subset of routes generated by the routing selection module as the decision space. The routing module and the anti-blocking module are trained and optimized using a reinforcement learning strategy. The routing module adopts an Actor-Critic architecture, and the anti-blocking module adopts the Q-Learning algorithm. The trained intelligent routing model undergoes network lightweighting, including manual dimensionality reduction of the state space and knowledge distillation of the policy network, and the lightweighted algorithm is deployed on satellite nodes.

2. The lightweight, anti-blocking intelligent routing method for satellite constellations based on Stackelberg game theory as described in claim 1, characterized in that, The establishment of the satellite constellation network simulation environment specifically includes: A satellite constellation network is constructed based on Kepler orbital parameters. The constellation consists of N satellites distributed across O orbital planes. Each satellite establishes two inter-satellite links within the same orbit and two inter-satellite links outside the same orbit. A distance threshold is defined for establishing links between satellites. When the angle between satellites exceeds a preset angle threshold, the inter-satellite link is disconnected. An end-to-end delay model is defined, which includes transmission delay, propagation delay, and queuing delay.

3. The lightweight, anti-blocking intelligent routing method for satellite constellations based on Stackelberg game theory as described in claim 1, characterized in that, The Stackelberg game modeling specifically includes: The Q-Learning algorithm is used to build a blocker as the leader. Based on the estimation of the network state, key nodes and power in the routing path are selected for blocking attacks to maximize the blocker's utility function. As a follower, the satellite, after observing or inferring congestion behavior, dynamically selects the optimal transmission path from a pre-selected subset of available routes to minimize the satellite's utility function.

4. The lightweight, anti-blocking intelligent routing method for satellite constellations based on Stackelberg game theory as described in claim 1, characterized in that, The training and optimization of the routing module using reinforcement learning strategies specifically includes: The Actor network uses a 2-layer LSTM to encode the input state space, resulting in a processed state sequence. The processed state sequence is input into a 4-layer MLP network to complete the analysis and decision-making of the state sequence; The input state of the Critic network is first encoded through a 2-layer LSTM and a 1-layer MLP network to obtain the encoded sequence. The encoded sequence and the actions output by the Actor network are passed together through a 4-layer MLP network to finally output the value evaluation of the policy, providing feedback information for the training of the Actor network; Design reward function Includes queue time penalty Distance Reduction Reward and additional rewards and penalties The network parameters are updated using a near-end strategy optimization algorithm. The parameter update formula is as follows: ； ； in, and These represent the gradient parameters of the updated Actor network and Critic network, respectively. and These are the gradient parameters of the Actor network and the Critic network, respectively. It is the learning rate of the Actor network. It is the learning rate of the Critic network. It is the immediate reward for the k-th sample. It is a discount factor. It is the value estimate of the k-th state. It is policy entropy. It is the gradient operator. Is the current strategy in the state? Select action The probability of KL and Let KL divergence and entropy function represent the two functions, respectively. and It is KL and coefficient, Indicates the old strategy in state Select action The probability, Indicates the current policy in state The probability distribution of actions for all feasible region nodes. Indicates the old strategy in state The probability distribution of actions for all feasible region nodes. , , They are , and The coefficient.

5. The lightweight, anti-blocking intelligent routing method for satellite constellations based on Stackelberg game theory as described in claim 1, characterized in that, The training and optimization of the anti-blocking module using reinforcement learning strategies specifically includes: Blocker pair function Blocking node strategy Blocking power strategy as well as the learning rate and Boltzmann parameters Perform initialization; At each time step, the blocker operates according to the current policy. and Select blocking node and blocking power ; The blocker directs to the selected node. Transmit blocking power is The blocking signal; Blocker Calculation Blocker Utility Function Come to function Update and based on the updated... Function-to-blocking strategy and The formula has been adjusted and updated as follows: ； ； ； in, This represents the updated action value function of the blocker J in the state after the (s+1)th iteration. This represents the updated action value function of the blocker J in the state after the (s+1)th iteration. Indicates the learning rate. This represents the updated policy probability of the blocker selecting a blocking target in round s+1. It is an action index variable, representing the summation of the possible action space of the blocker. This represents the updated policy probability of the blocker selecting the blocking power in round s+1. This represents the index for summing the traversal of the blocking power action space. The first blocker One candidate target, The first blocker Candidate blocking power.

6. The lightweight, anti-blocking intelligent routing method for satellite constellations based on Stackelberg game theory as described in claim 3, characterized in that, After observing or inferring congestion behavior, the optimal transmission path is dynamically selected from a pre-selected subset of available routes to minimize the satellite utility function, specifically including: First, initialize the Q function. Routing strategy This ensures that the probability of selecting all available routing paths is equal in the initial phase; where, This represents the nth routing action selected by the user in the initial phase. Indicates the size of the routing action space; At each time step, the blocker is invoked according to the blocking policy. and Select blocking node and blocking power ; After sensing the congestion and link status in the network, the current routing selection strategy is applied. Re-establish the route; Calculate the reward based on the designed reward function. ; According to the reward Update Q function ; And based on the updated Q function Adjust routing strategy The updated formula is as follows: ； in, This represents the updated Q-value after the user's iteration (t+1). Indicates the learning rate. Indicates the discount factor. The index represents the summation index of the action space traversal. This represents the updated policy probability that a user chooses the nth routing action in round t+1. This represents the temperature coefficient of the Softmax strategy.

7. A lightweight, anti-congestion intelligent routing device for satellite constellations based on Stackelberg game theory, characterized in that, The lightweight, anti-blocking intelligent routing device for satellite constellations based on Stackelberg game theory includes: The simulation environment building unit is used to establish a satellite constellation network simulation environment and simulate the dynamic topology of the satellite constellation network. The game modeling unit is used to perform Stackelberg game modeling based on the established satellite constellation network simulation environment. It defines the blocker as the leader who first decides the interference strategy, and the satellite as the follower who observes the blocking behavior and then chooses the transmission path. It also proves that the game has a Stackelberg equilibrium. A satellite constellation intelligent routing model construction unit is used to construct a satellite constellation intelligent routing model resistant to blocking attacks. It includes a routing selection module and an anti-blocking module. The routing selection module is used to generate an available set of routes, and the anti-blocking module makes decisions using a subset of routes generated by the routing selection module as the decision space. The training and optimization unit is used to train and optimize the routing module and the anti-blocking module using a reinforcement learning strategy, wherein the routing module adopts an Actor-Critic architecture and the anti-blocking module adopts the Q-Learning algorithm. The lightweight processing unit is used to perform network lightweighting processing on the trained intelligent routing model, including manual dimensionality reduction of the state space and knowledge distillation of the policy network, and then deploys the lightweight algorithm on satellite nodes.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the lightweight, anti-blocking intelligent routing method for satellite constellations based on Stackelberg game theory as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the lightweight, anti-blocking intelligent routing method for satellite constellations based on Stackelberg game theory as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the lightweight, anti-blocking intelligent routing method for satellite constellations based on Stackelberg game theory as described in any one of claims 1-6.

Images