A long-term strategy constraint method for space-air-ground integrated network

Abstract

This invention provides a long-term strategy constraint method for integrated air-space-ground networks, comprising: constructing an integrated air-space-ground network model; calculating the structural importance weights of each ground node based on ground-side traffic and backhaul communication load; constructing a node interference cost and budget constraint model based on the weights and the total budget; modeling budget scheduling as a Markov decision process, using reinforcement learning to output the budget allocation ratio for each cycle to obtain the available budget for the current cycle; selecting ground nodes in descending order of unit structural benefit ratio using a reverse auction mechanism to obtain an intervention node set and update the remaining budget; calculating the structural influence factor based on the intervention node set to construct a repeated game model between the interferer and the user; and adjusting parameters using a zero-determinant strategy to limit the user's long-term benefit within a target range. This method can accurately identify key ground nodes, easily allocate interference resources rationally in multi-cycle operation, and continuously adjust the user's long-term benefit without significantly affecting the normal operation of the network.

Description

Technical Field

[0001] This invention relates to the fields of cyberspace security and satellite communication technology, and in particular to a long-term strategy constraint method for integrated air-space-ground networks. Background Technology

[0002] With the gradual realization of the 6G communication vision, the Space-Ground Integrated Network (SGIN) achieves seamless global coverage and ubiquitous connectivity by integrating low Earth orbit satellite constellations with terrestrial communication networks. This effectively compensates for the insufficient coverage of traditional terrestrial networks in areas such as oceans, deserts, high altitudes, and remote mountainous regions. In recent years, the rapid deployment of large Low Earth Orbit (LEO) constellations such as Starlink and OneWeb has propelled SGIN from conceptual research to practical application, playing a crucial role in broadband access, aerospace communications, maritime communications, and emergency communications. This network consists of a satellite layer, a ground node layer, and a user layer. Ground nodes bear the responsibility of a large amount of user traffic aggregation and backhaul, making them structurally important. However, due to the high-speed movement of LEO satellites causing dynamic changes in network topology and uneven service density across different regions, some key ground nodes bear a traffic volume far exceeding that of other nodes. For example, network interference with the satellite internet system severely impacted satellite communication services, rendering many terminal devices malfunction. This incident demonstrates that jammers can disrupt communication services not only through traditional network interference methods but also by gaining access to ground control systems or manipulating terminal devices, causing widespread damage to satellite communication infrastructure. Therefore, a systematic study of the long-term strategic security risks in SGIN is of significant theoretical importance and practical necessity.

[0003] In existing technologies, security research for integrated air-space-ground networks mainly focuses on defensive mechanisms, including machine learning-based physical layer authentication, data-driven anomaly detection frameworks, security constraints and zero-trust architectures in resource management processes, and methods such as identifying traffic aggregation nodes and bottleneck links through network traffic distribution analysis. However, existing technologies typically rely on random interference or simple link interference, lacking node importance analysis mechanisms tailored to network structural characteristics, making it difficult to identify critical ground nodes that have the greatest impact on overall network performance. They often assume that the interferer has sufficient resources, ignoring the impact of interference costs and budget constraints on interference decisions, making it difficult to rationally allocate interference resources during multi-cycle network operation. Existing interference methods are mostly one-off destructive acts, such as denial-of-service interference or link interference. These types of interference are easily identified by detection systems and are difficult to continuously affect user communication performance without triggering anomaly detection, making it difficult to continuously adjust long-term user benefits without significantly impacting normal network operation. Summary of the Invention

[0004] The purpose of this invention is to provide a long-term strategy constraint method for integrated air-space-ground networks, which solves the problems of existing technologies that make it difficult to identify key ground nodes that have the greatest impact on the overall network performance, difficult to reasonably allocate interference resources during multi-cycle network operation, and difficult to continuously adjust long-term user benefits without significantly affecting the normal operation of the network.

[0005] To address the aforementioned technical problems, the embodiments of the present invention provide the following technical solutions: The first aspect of this invention provides a long-term strategy constraint method for integrated air-space-ground networks, the method comprising: Construct an integrated air-space-ground network model and obtain the set of ground nodes and user sets; Calculate the structural importance weight of each ground node based on the ground-side traffic carried by each ground node and the backhaul communication load with the satellite; Based on the structural importance weights and the preset total resource budget, a node interference cost and budget constraint model for each surface node is constructed, and the preset total resource budget is initialized to the current remaining budget. The budget scheduling problem is modeled as a Markov decision process, and reinforcement learning is used to output the budget allocation ratio for each period based on the current network state and the current remaining budget, so as to obtain the available budget for the current period. Based on the available budget for the current period, a reverse auction mechanism is used to select ground nodes in descending order of unit structural benefit ratio to obtain the set of intervention nodes for the current period. The remaining budget is then updated under the budget constraint model. The unit structural benefit ratio is the ratio of structural importance weight to node interference cost. Calculate the structural impact factor based on the set of intervention nodes, and construct a repeated game model between the disruptor and the user based on the structural impact factor; In the repeated game model, a zero-determinant strategy is adopted, and the strategy parameters are adjusted according to the structural influence factor so that the long-term user benefits are limited to the target benefit range, thereby achieving resistance to interference in the integrated air-space-ground network.

[0006] A second aspect of the present invention provides a long-term strategy constraint device for an integrated air-space-ground network, the device comprising: The first construction module is used to build an integrated air-space-ground network model and obtain the set of ground nodes and user sets; The weight calculation module is used to calculate the structural importance weight of each surface node based on the ground-side traffic carried by each surface node and the backhaul communication load with the satellite. The second construction module is used to construct the node interference cost and budget constraint model of each surface node according to the structural importance weight and the preset total resource budget, and initialize the preset total resource budget to the current remaining budget; The modeling module is used to model the budget scheduling problem as a Markov decision process and use reinforcement learning to output the budget allocation ratio for each period based on the current network state and the current remaining budget, so as to obtain the available budget for the current period. The selection and update module is used to select ground nodes in descending order of unit structure benefit ratio based on the available budget for the current period using a reverse auction mechanism, to obtain the set of intervention nodes for the current period, and to update the current remaining budget under the budget constraint model. The unit structure benefit ratio is the ratio of the structural importance weight to the node interference cost. The calculation and construction module is used to calculate the structural influence factor based on the set of intervention nodes, and to construct a repeated game model between the disruptor and the user based on the structural influence factor. The adjustment module is used in repeated game models to adopt a zero-determinant strategy and adjust the strategy parameters according to the structural influence factor so that the user's long-term returns are limited to the target return range, thereby achieving resistance to interference in the integrated air-space-ground network.

[0007] Compared to existing technologies, this invention provides a long-term strategy constraint method for integrated air-space-ground networks. It constructs an integrated air-space-ground network model and obtains a set of ground nodes and a set of users. Based on the ground-side traffic carried by each ground node and the backhaul communication load with satellites, it calculates the structural importance weights of each ground node. Based on the structural importance weights and a preset total resource budget, it constructs a node interference cost and budget constraint model for each ground node, and initializes the preset total resource budget to the current remaining budget. It models the budget scheduling problem as a Markov decision process and uses reinforcement learning to output the budget allocation ratio for each period based on the current network state and the current remaining budget. The system obtains the available budget for the current period. Based on the available budget, a reverse auction mechanism is used to select ground nodes in descending order of unit structural benefit ratio to obtain the set of intervention nodes for the current period. The remaining budget is then updated under the budget constraint model, where the unit structural benefit ratio is the ratio of structural importance weight to node interference cost. The structural impact factor is calculated based on the set of intervention nodes, and a repeated game model between the interferer and the user is constructed based on the structural impact factor. In the repeated game model, a zero-determinant strategy is adopted, and the strategy parameters are adjusted according to the structural impact factor to limit the user's long-term benefits within the target benefit range, thereby achieving resistance to interference in the integrated air-space-ground network. In this way, by coupling the calculation of ground-side traffic and backhaul communication load between satellites, the structural importance weight of each node is quantified, which can accurately identify the key ground nodes that have the greatest impact on the overall network performance. At the same time, by modeling the budget scheduling problem as a Markov decision process and using reinforcement learning to dynamically output the allocation ratio, it is easy to rationally allocate interference resources during multi-cycle network operation. In addition, the zero-determinant strategy in the repeated game model constrains the user's long-term payoff within the target range, making it easy for interference behavior to continuously adjust the user's long-term payoff without significantly affecting the normal operation of the network, thus taking into account both concealment and persistence. Attached Figure Description

[0008] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of the invention are illustrated by way of example and not limitation, with the same or corresponding reference numerals denoteing the same or corresponding parts, wherein: Figure 1 A flowchart illustrating a long-term strategy constraint method for an integrated air-space-ground network is shown schematically. Figure 2 A schematic diagram of a two-layer model architecture is shown. Figure 3 The diagram illustrates a comparison of honest pricing experiments. Figure 4 The diagram illustrates a comparison of experiments involving fake quotes. Figure 5 The experimental diagram illustrating the flow rate variation is shown schematically. Figure 6 The diagram illustrates a comparative experiment of multi-slot algorithms; Figure 7 The experimental graph illustrating how returns change over time is shown schematically. Figure 8 The diagram illustrates an experiment showing how revenue varies with the budget. Figure 9 A schematic diagram of a zero-determinant (ZD) adversarial experiment is shown. Figure 10 A diagram illustrating the profit comparison between attackers and defenders is provided. Figure 11 An experimental diagram illustrating the control results under multi-slot conditions is shown. Figure 12 A diagram illustrating the gains and losses of multi-slot attackers and defenders is provided. Figure 13 A schematic diagram of a long-term strategy constraint device for an integrated air-space-ground network is shown. Detailed Implementation

[0009] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the invention and to fully convey the scope of the invention to those skilled in the art.

[0010] It should be noted that, unless otherwise stated, the technical or scientific terms used in this invention should have the ordinary meaning as understood by one of ordinary skill in the art.

[0011] The methods described in the embodiments of the present invention will be explained in detail below.

[0012] Figure 1 A flowchart illustrating a long-term policy constraint method for an integrated air-space-ground network according to an embodiment of the present invention is shown. See [link / reference]. Figure 1 As shown, the long-term strategy constraint method for integrated air-space-ground networks may include: S101. Construct an integrated air-space-ground network model and obtain the set of ground nodes and user sets.

[0013] A space-air-ground integrated network structure is constructed, consisting of a satellite layer, a ground node layer, and a user layer. The satellite layer comprises a low-Earth orbit satellite constellation, providing space communication links; the ground node layer consists of multiple ground stations, handling user traffic aggregation and backhaul; and the user layer comprises multiple ground users. Let the set of ground nodes be denoted as... The user set is User-generated data traffic accesses the network via visible satellites and is ultimately transmitted back to the terrestrial network via a ground node. This step establishes the basic topology of the integrated air-space-ground network, providing network status information for subsequent node importance calculations and attack decisions.

[0014] In this embodiment, an SGIN environment is considered, where the network consists of a low-Earth orbit satellite constellation, ground nodes, and ground users. The satellite layer provides space communication links, while the ground node layer handles user traffic access, aggregation, and backhaul. Users access the network via visible satellites, and their service traffic ultimately enters the ground core network through a specific ground node. Let the set of ground nodes be... ,in This represents the total number of ground nodes. Ground nodes handle user traffic access, aggregation, and backhaul, serving as crucial hubs connecting the satellite layer and the terrestrial core network. Let the user set be... ,in This represents the number of users. Each user accesses the network wirelessly via visible satellites, and their traffic must ultimately enter the terrestrial network through a ground node. Therefore, in the model, the ground node constitutes a mandatory convergence point for user traffic.

[0015] In the SGIN environment, system evolution exhibits a clear separation of time scales. On one hand, the importance of ground nodes adjusts slowly with changes in operational needs and satellite orbits. This structural-level change typically occurs on a minute or even hourly scale, at which the disruptor decides whether to acquire node control. On the other hand, strategic interactions between the disruptor and the user continue to occur on a finer-grained time scale, with both parties choosing actions based on historical states and gaining immediate benefits.

[0016] To characterize this feature, this invention introduces a dual-timescale modeling framework. Let the entire research time domain be divided into... A discrete decision-making cycle (epoch), denoted as , This represents the total number of discrete decision-making cycles. At the beginning of each epoch, the disruptor determines the set of control nodes based on the current network state and budget constraints. , For the first The set of intervention nodes controlled by the disruptor in a discrete decision-making cycle. Within this cycle, the importance of nodes and the control set remain constant; therefore, structural-level decision-making is a slow-timescale process. Within each epoch, there are multiple rounds of policy interaction between the disruptor and the user. Let each epoch contain... Round-robin game interaction, denoted as , For the first In this round-based game interaction, each player chooses an action based on a predetermined strategy and receives an immediate reward. Since the set of node control units remains fixed throughout the entire epoch, changes in reward arise solely from the strategy interaction process. As the number of rounds of the game interaction increases... When the time scale is sufficiently large, the average user revenue within that period tends to a statistical steady-state value. This dual-timescale separation not only aligns with the characteristics of actual network operation but also provides a theoretical basis for constructing subsequent two-layer optimization problems, enabling hierarchical analysis of structural control and revenue realization problems.

[0017] S102. Calculate the structural importance weight of each surface node based on the ground-side traffic carried by each surface node and the backhaul communication load with the satellite.

[0018] Specifically, this includes: calculating the user traffic volume carried by each node; calculating the backhaul communication volume between nodes and satellites; and obtaining the importance weight of each node through a weighted calculation model. The node importance weight is used to measure the structural value of each node in the network.

[0019] Specifically, based on the ground-side traffic carried by each surface node and the backhaul communication load with the satellite, the structural importance weight of each surface node is calculated, including: Step A1: Perform coupled calculation and normalization on the ground-side traffic carried by each ground node in the current period and the backhaul communication load between each ground node and the satellite to obtain the original importance weight of each ground node.

[0020] Step A2: Perform exponential weighted smoothing on the historical weights and original importance weights of the previous period, and use the smoothed result as the structural importance weights of the ground nodes in the current period.

[0021] Specifically, the system calculates the user traffic volume carried by each node; it also calculates the backhaul communication volume between nodes and satellites; and it uses a weighted calculation model to obtain the importance weights of each node. Node importance weights are used to measure the structural value of each node in the network. Within each time period, users select an access node based on visibility and geographical location. Related variables are defined. When users In discrete decision cycle Through nodes The value is 1 when transmitting back traffic, and 0 otherwise, and the constraint is satisfied. Assume the user In discrete decision cycle The resulting traffic demand is Then the ground node In discrete decision cycle The total surface flow carried is: ; in, ground nodes In discrete decision cycle Total ground-side flow carried, Total number of users For users In discrete decision cycle The resulting traffic demand, To represent users In discrete decision cycle Passing or not passing ground nodes Return traffic.

[0022] This quantity characterizes the ground nodes. The aggregated load during this period. Due to differences in population density and business needs across different regions. The nodes exhibit a significantly uneven distribution, and this unevenness is one source of structural vulnerability. In addition to ground-side traffic aggregation, the nodes also bear the backhaul communication load with satellites. Let... Representing ground nodes In discrete decision cycle Total bidirectional transmission volume between the satellite and ground stations. This variable reflects the strength of the space-to-ground coupling. By simultaneously considering ground-side traffic and space-side backhaul load, the comprehensive strategic value of a node within the entire SGIN system can be characterized. An interference cost is defined for each ground node to characterize the resources required for an attacker to gain control of the node. Node interference costs include: basic interference costs; node importance-related costs; and node security heterogeneity factors. The attacker has a total budget constraint and must select a control node within this budget limit during each decision-making cycle.

[0023] Consider an external disruptor whose goal is to gain control of a subset of ground nodes under resource constraints, thereby establishing structural influence. Assume the disruptor operates during the time period... The selected set of control nodes is The disruptor faces budget constraints at each time period, and its control is limited by the scale of available resources. The specific budget constraint model will be given in a later section.

[0024] To ensure the feasibility of the theoretical analysis, this invention adopts the following reasonable abstractions: the satellite layer is not directly attacked, but only acts as a traffic relay; users select only one ground node for backhaul in each time period; the importance of a ground node can be characterized by its combined traffic carrying capacity and backhaul load; the interferer's control over the node can be abstracted as a set selection problem. These assumptions simplify the model complexity without sacrificing structural features, making subsequent two-layer optimization and repeated game theory analysis feasible.

[0025] To characterize the structural influence of different ground nodes in the overall network, both ground-side converged traffic and space-to-ground backhaul load are considered. Let... This represents the theoretical maximum backhaul capacity of a single node, used for normalization processing. Then, the ground node... In discrete decision cycle The original importance weight is defined as ; in, ground nodes In discrete decision cycle The original importance weights, For balance coefficient, The use of a multiplicative coupling approach, rather than a simple weighted summation, has clear structural implications. First, The scale of traffic concentration of a node is a fundamental factor in determining whether it becomes a critical node in the structure. This describes the spatial coupling strength, reflecting its load status in the space-to-ground communication link. The multiplicative structure embodies the amplification effect: when a node bears both high ground traffic and high backhaul load, its strategic value is further amplified. This amplification effect aligns with the actual network operation logic; once a high-traffic, highly coupled node is controlled, it will have a more significant impact on the overall system.

[0026] A simple additive structure cannot reflect this synergistic enhancement effect; a purely multiplicative form may lead to weights being overly sensitive to a single variable. Therefore, this invention employs a multiplicative correction structure dominated by ground flow and modified by spatial load, achieving a balance between stability and expressive power. Since node flow is affected by diurnal variations and satellite coverage dynamics, it may fluctuate in the short term. Directly using instantaneous weights as structural evaluation indicators may lead to oversensitivity of upper-level decisions to short-term fluctuations. To characterize the long-term structural position of nodes in a statistical sense, this invention introduces an exponential smoothing update mechanism: ; in, ground nodes In the next discrete decision cycle The structural importance weight, ground nodes In the current discrete decision cycle Historical weight, This is a smoothing factor.

[0027] This weight will play a crucial role in the subsequent construction of the structural influence factor. This update method has the following property: when... When the weights approach 1, the changes are gradual, emphasizing the long-term structural position; when... When the weights are smaller, the weights respond more quickly to changes in flow. The exponential smoothing mechanism is equivalent to low-pass filtering the importance of nodes, making structural control decisions based on statistically stable structures rather than instantaneous disturbances, thereby enhancing the robustness of upper-level decisions.

[0028] S103. Based on the structural importance weights and the preset total resource budget, construct a node interference cost and budget constraint model for each surface node, and initialize the preset total resource budget to the current remaining budget.

[0029] A node interference cost is defined for each ground node to characterize the resources required for an attacker to gain control of the node. The node interference cost includes: basic interference cost; node importance-related costs; and node security heterogeneity factors. The attacker has a total budget constraint and must select a control node within this budget limit during each decision cycle.

[0030] Specifically, based on the structural importance weights and the preset total resource budget, a node interference cost and budget constraint model for each surface node is constructed, including: Step B1: Determine the node interference cost for each surface node based on the structural importance weight, basic interference cost, importance correlation coefficient, and node security heterogeneity factors.

[0031] Step B2: Based on the preset total resource budget, establish target constraint relationships and use these target constraint relationships as the budget constraint model.

[0032] The target constraint relationship is the constraint that the sum of the allocation budgets for each cycle during the cross-cycle allocation process shall not exceed the preset total resource budget.

[0033] Specifically, considering the practical costs of interference, ground nodes are set up. In discrete decision cycle The control cost is: ; in, ground nodes In discrete decision cycle The cost of node interference, Based on interference cost, The importance correlation coefficient, Due to the heterogeneity of node security, different nodes differ in terms of geographical location, physical protection level, and operational strategies. This is used to characterize these individual differences, making the cost model more realistic. Let the disruptor's preset total resource budget be... And allocate a budget for each epoch. ,satisfy: ; in, The total number of discrete decision cycles. In the first Allocation budget for each discrete decision cycle This is the preset total resource budget.

[0034] In discrete decision cycle Within the control set, the following must be satisfied: ; in, ground nodes In discrete decision cycle The cost of node interference.

[0035] By introducing cost and budget constraints, the structural control problem is transformed into a resource-constrained optimization problem.

[0036] Initialize the remaining budget This means that the preset total resource budget is initialized to the current remaining budget.

[0037] S104. Model the budget scheduling problem as a Markov decision process, and use reinforcement learning to output the budget allocation ratio for each period based on the current network state and the current remaining budget, so as to obtain the available budget for the current period.

[0038] The budget scheduling problem is modeled as a Markov decision process, and a reinforcement learning algorithm is used to dynamically schedule budgets for different time periods. Within each period, based on the current network state and remaining budget, the policy network outputs a budget allocation ratio, thereby determining the available budget for node acquisition in the current period. This step is used to achieve dynamic allocation of attack resources across periods.

[0039] Specifically, the budget scheduling problem is modeled as a Markov decision process, and reinforcement learning is used to output the budget allocation ratio for each period based on the current network state and the current remaining budget, so as to obtain the available budget for the current period, including: Step C1: Set the current network state in the budget scheduling problem as the state space containing the structural influence factor of the previous cycle, the current remaining budget, and the user revenue of the previous cycle. Set the budget allocation ratio as the action space and set the sum of the negative loss function values ​​obtained after the allocation as the reward function to construct a Markov decision process.

[0040] Step C2: Utilize the policy network of reinforcement learning to output the budget allocation ratio for the current period based on the current network state.

[0041] The policy network is trained using a proximal policy optimization algorithm.

[0042] Step C3: Multiply the budget allocation ratio by the current remaining budget to obtain the available budget for the current period.

[0043] Specifically, in the two-layer attack framework, the core objective of the upper-layer decision-making is to dynamically select the set of ground nodes under budget constraints in order to maximize the structural impact capability. Figure 2 A schematic diagram of a two-layer model architecture is shown, such as... Figure 3 As shown, due to the information privacy of control costs at different nodes and the long-term impact of cross-cycle budget allocation, this invention models the upper-level decision-making as a joint mechanism of PPO budget scheduling and DSIC reverse auction. The attack budget allocation problem is modeled as a Markov decision process, and a reinforcement learning algorithm is used to dynamically schedule the attack budget for different time periods. Within each period, based on the current network state and remaining budget, the policy network outputs the budget allocation ratio, thereby determining the attack budget available for node acquisition in the current period. This step is used to achieve dynamic allocation of cross-cycle attack resources. In each discrete decision period... Attackers face two decisions: budget allocation and node selection. Assume the current network state is: ; in, For the first The current network state for each discrete decision cycle. ground nodes In the current discrete decision cycle Historical weight, Indicates the first The current remaining budget for each discrete decision cycle. The disruptor first determines the budget allocation ratio. Corresponding to the actual allocated budget: ; in, In the first Allocation budget for each discrete decision cycle.

[0044] Because the reward feedback varies across different epochs, single-epoch optimization does not guarantee global optimization. This invention models the budget scheduling problem as a Markov Decision Process (MDP). The state is defined as follows: ; in, The structural influence factor of the previous cycle, For the previous cycle, i.e., the first User revenue related quantities for each discrete decision-making cycle.

[0045] Action definition Instant rewards Defined as: ; in, For the first The immediate reward for each discrete decision cycle is the sum of the negative user benefit deviation losses. For users In the The profit deviation loss in each discrete decision cycle.

[0046] The attacker's goal is to maximize the cumulative discount reward: ; in, For the first Discount factor for each discrete decision cycle This is the mathematical expectation symbol.

[0047] in Indicates budget allocation strategy, This is the discount factor. This invention uses the Proximal Policy Optimization (PPO) algorithm to update the policy network parameters. Its pruning objective function is: ; in, Let PPO be the shearing objective function. This represents the probability ratio between the old and new strategies. For the parameters of the policy network, For probability ratios, This is the estimated value of the dominance function. The shear coefficient is... This is the current policy network.

[0048] in, ; in, This represents the probability ratio between the old and new strategies. For the current policy network, This is an old strategy network.

[0049] In summary, top-level decision-making can be expressed as: ; Simultaneously, the single-cycle pre-constraints are satisfied: ; in, ground nodes In the current discrete decision cycle The actual cost of payment.

[0050] Budget allocation is determined by the PPO strategy, while node selection is accomplished by the DSIC reverse auction mechanism; both factors jointly determine the structural impact factor. And it serves as the input parameter for the lower-level profit manipulation problem.

[0051] Real control cost of ground nodes Information may be private. Attackers cannot fully grasp the true security cost of a node, and nodes may engage in strategic bidding behavior. If nodes are selected directly based on estimated costs, information asymmetry may lead to: inflated costs, resource misallocation, and decreased structural control. Therefore, an incentive-compatible mechanism is needed to ensure that honest cost reporting is the optimal strategy for nodes.

[0052] S105. Based on the available budget for the current period, the ground nodes are selected in descending order of unit structural revenue ratio using a reverse auction mechanism to obtain the set of intervention nodes for the current period, and the current remaining budget is updated under the budget constraint model.

[0053] The unit structure benefit ratio is the ratio of the structural importance weight to the node interference cost.

[0054] Under budget constraints, key ground nodes are selected through a reverse auction mechanism. The specific process includes: each node submits a bid based on its interference cost; the unit structure benefit ratio is calculated based on the node's importance weight and the bid; nodes are ranked according to the unit structure benefit ratio; nodes are selected sequentially until the budget is exhausted. This step is used to determine the set of ground nodes controlled by the attacker in the current cycle.

[0055] Specifically, based on the available budget for the current period, a reverse auction mechanism is used to select ground nodes in descending order of unit structural benefit ratio, resulting in the intervention node set for the current period. The remaining budget is then updated under the budget constraint model, including: Step D1: In the reverse auction mechanism, under the incentive compatibility condition, the node interference cost is determined to be equal to the bid submitted by each surface node.

[0056] Step D2: Determine the unit structure revenue ratio of each surface node based on the ratio of the structural importance weight to the bids submitted by each surface node.

[0057] Step D3: Sort the ground nodes in descending order of unit structure benefit ratio to obtain the sorted ground node sequence.

[0058] Step D4: Based on the available budget for the current period, select ground nodes sequentially according to the sorted ground node sequence until the remaining budget is insufficient to select the next ground node. Then, use the selected ground nodes as the intervention node set for the current period.

[0059] Step D5: Determine the actual payment cost for each selected ground node according to the critical payment rule, and deduct the actual payment cost from the remaining budget to update the current remaining budget.

[0060] Specifically, ground nodes With real control costs and submit a quote. Define the unit structure benefit ratio as... ; in, ground nodes In the current discrete decision cycle The unit structure benefit ratio, ground nodes In the current discrete decision cycle The submitted quote.

[0061] The disruptor according to Sort the nodes from largest to smallest, and select nodes in turn until the budget is exhausted to obtain the control set. The payment rules employ a critical payment mechanism: ; in, ground nodes In the current discrete decision cycle The actual payment cost To make ground nodes The critical bid that just disqualifies a node from winning the bid. If a node's bid exceeds this critical value, its unit structure benefit ratio will decrease, causing it to be replaced by other nodes in the ranking.

[0062] To achieve coordinated optimization of cross-cycle budget scheduling and single-cycle node selection, this example combines the PPO budget scheduling strategy with the Dominant Strategy Incentive Compatibility (DSIC) reverse auction mechanism into a unified upper-level joint algorithm, as shown in Algorithm 3.1. At the beginning of each epoch, the attacker first observes the current network state, including the distribution of node importance, the remaining budget size, and the structural impact feedback from the previous cycle. Subsequently, the PPO policy network outputs the budget allocation ratio for this cycle, and obtains the upper limit of the actual available budget accordingly. Under this budget constraint, the attacker executes the DSIC reverse auction, sorts and filters nodes based on the unit cost structural benefit ratio, and determines the set of control nodes for this cycle. Finally, the remaining budget is updated according to the critical payment of the winning node, and the structural impact benefit obtained in this cycle is fed back to the PPO policy for subsequent policy updates.

[0063] Upper-layer node acquisition algorithm based on PPO-DSIC: Input: Total Budget Total number of epochs Weight of each node Price quotes at each node PPO strategy network parameters .

[0064] Output: Budget allocation for each period control node set Structural Influence Factors .

[0065] 1: Initialize the remaining budget Initialize the PPO experience cache; 2: For each epoch Perform the following operations: 3: Construct the current state ; 4: Budget allocation actions output by PPO strategy This yields the upper limit of the budget for this period; 5: Calculate the unit cost-benefit ratio for each candidate node. ; 6: Press Sort the nodes from largest to smallest, and use a greedy approach to try adding candidate nodes one by one until the budget constraint can no longer be satisfied, thus obtaining the winning set. ; 7: Update remaining budget And calculate the attacker's structural influence factor. ; 8: Periodically update the parameters of the PPO policy network and value network using empirical samples.

[0066] S106. Calculate the structural influence factor based on the set of intervention nodes, and construct a repeated game model between the disruptor and the user based on the structural influence factor.

[0067] Within each time period, the disruptor and the user engage in multiple rounds of strategic interaction. In each round: the disruptor chooses an attack strategy; the user chooses a routing strategy or a defense strategy; both parties gain immediate benefits based on the strategy combination. Through multiple rounds of interaction, the system gradually forms a stable long-term profit state.

[0068] Specifically, structural influence factors are calculated based on the set of intervention nodes, and a repeated game model between the disruptor and the user is constructed based on these structural influence factors, including: Step E1: Determine the structural influence factor based on the ratio of the sum of the ground-side flows carried by each ground node in the intervention node set to the total ground-side flows carried by all ground nodes.

[0069] Step E2: Determine the immediate payoff of each user in each round of the game based on the structural influence factor, basic payoff, the preset loss function value corresponding to each user, and the user's own action cost.

[0070] Specifically, step E2 includes: Step E21: Determine the first product of the structural influence factor, user weight coefficient, and the preset loss function value corresponding to each user.

[0071] Step E22: Subtract the first product from the base payout and subtract the user's own action cost to obtain the user's immediate payout in each round of the game.

[0072] Step E3: Determine the immediate payoff of the disruptor in each round of the game based on the structural influence factor, the preset loss function values ​​of all users, and the disruptor's own action cost.

[0073] Specifically, step E3 includes: Step E31: Determine the second product of the structural impact factor and the sum of the preset loss function values ​​of all users.

[0074] Step E32: Subtract the interferer's own action cost from the second product to obtain the interferer's immediate gain in each round of the game.

[0075] Step E4: Based on the user's immediate payoff and the interferer's immediate payoff, construct a repeated game model. The repeated game model is used to conduct multiple rounds of strategy interaction in each cycle.

[0076] Specifically, to characterize the attacker in epochs The overall structural influence capability obtained is defined as the structural influence factor: ; in, For the first Structural influencing factors of discrete decision-making cycles.

[0077] because This can be interpreted as the proportion of traffic controlled by an attack. Subsequent analysis will demonstrate that the achievable range of long-term user benefits depends solely on… And it does not depend on a specific set of control nodes, therefore It can be considered a sufficient statistic of structural control capability. This variable plays a key coupling role between the slow and fast time scales in the two-level optimization framework.

[0078] Determine structural influence factors during the upper-level node acquisition phase. Subsequently, the next objective is to manipulate long-term user gains through a repetitive interaction mechanism, given the structural control capabilities. This invention constructs a gain regulation model based on a zero-determinant (ZD) strategy and provides the reachable interval and closed-form expression.

[0079] Within each epoch, the attacker and user engage in repeated game interactions. Each round of game interaction... Disruptor's choice of action Each user selects an action ,user The instant benefit is defined as: ; in, For the first Users in round-robin games Instant benefits Basic income, User weight coefficient, For the first In round-robin games, the disruptor chooses an action. For the first Each user's action in a round-robin game For the user's own action cost, For the first Structural influencing factors of discrete decision-making cycles The preset loss function value is for each user.

[0080] The immediate benefit for the disruptor is: ; in, For the first The immediate payoff of the disruptor in round-robin games. For the cost of the actions of the disruptor, This is the sum of the loss function values ​​for all users.

[0081] S107. In the repeated game model, a zero-determinant strategy is adopted, and the strategy parameters are adjusted according to the structural influence factor so that the user's long-term benefits are limited to the target benefit range, thereby achieving resistance to interference in the air-space-ground integrated network.

[0082] In repeated game scenarios, the disruptor employs a zero-determinant strategy, adjusting policy parameters to ensure a pre-defined linear relationship (i.e., a linear constraint) between the disruptor's payoff and the user's payoff. Through this mechanism, the disruptor can limit the user's long-term payoff during extended interactions, gradually converging the user's payoff to a target range. This step achieves continuous control over the long-term payoff of network users.

[0083] Specifically, in the repeated game model, a zero-determinant strategy is adopted, and the strategy parameters are adjusted according to the structural influence factor to limit the user's long-term payoff within the target payoff range, including: Step F1: Determine the reachable range of long-term user returns based on structural impact factors, and select the target return range within the reachable range.

[0084] Step F2: Adjust the linear constraint parameters of the zero determinant strategy according to the target profit range so that the long-term profit of the disruptor and the long-term profit of the user satisfy the linear constraint relationship.

[0085] Step F3: Execute the adjusted zero determinant strategy in the repeated game model to limit the user's long-term payoff to the target payoff range through linear constraints.

[0086] Specifically, according to the multiplayer zero determinant strategy theory, in repeated games, an attacker can choose an appropriate strategy to ensure that long-term payoffs satisfy a linear constraint relationship: ; in, The linear constraint coefficient for the long-term gains of the disruptor. This indicates the long-term benefits for the disruptor. Indicates the first Long-term benefits for individual users and This is an adjustable parameter for the disruptor. This relationship indicates that the disruptor can impose constraints on the long-term payoff structure of the system during repeated interactions. When the above formula is used, it can be simplified to: ; Furthermore, if you choose , The disruptor can directly fix a user's earnings: ; in, For the first The long-term average return for each user.

[0087] This relationship holds independently of the user policy. Let the set of feasible ZD policies be defined as follows: .user The reachable set of benefits is defined as: ; in, To influence a given structural factor Below, user The long-term returns can reach the set. The strategies employed by the disruptors.

[0088] Since the stationary distribution set is a convex set, and the returns are related to... Given an affine function, the reachable range of the user's long-term benefit is: ; in, To influence a given structural factor Below is the achievable range of long-term benefits for users. For users With structural influence factor The minimum long-term average return that can be achieved at that time. For users With structural influence factor The maximum long-term average return that can be achieved at that time.

[0089] From the profit expression, we can see that: ; in, For users The immediate benefits depend on the combination of actions. and structural influencing factors , For the combination of actions of the attacker and the user, The loss function depends on the combination of actions taken by both parties. Cost of user actions.

[0090] Its about Monotonically non-increasing. Therefore: ; in, For users The long-term average return.

[0091] Therefore, we can conclude that: ; In other words, an increase in the influencing factor will shift the overall return range downwards. Let the target return range be... Then the discrete decision period The interval deviation loss is: ; in, For the first Interval bias loss for each discrete decision cycle For users The lower limit of the target return range For users The upper limit of the target profit range.

[0092] Therefore, the lower-level optimal value function is: ; in, For the first The optimal value function for the lower-level revenue manipulation problem in a discrete decision cycle. For the first Interval bias loss for each discrete decision cycle.

[0093] because and about Monotonically non-increasing: ; Therefore, the upper layer maximizes This allows for a monotonic reduction of the lower-level bias loss, thus forming a strictly coupled two-layer structure. At this point, the lower-level revenue manipulation model is complete.

[0094] This invention constructs a data-driven simulation environment based on real LEO constellation data and publicly available traffic statistics to systematically evaluate the proposed two-layer interference framework. Experiments include verification of the upper-layer node acquisition mechanism, cross-cycle budget scheduling analysis, verification of lower-layer revenue manipulation, and evaluation of the overall two-layer coupling effect.

[0095] To ensure the realism of the experiment, this example uses the Starlink Shell-1 constellation as the simulation object. Spatial flow distribution is generated based on a population-weighted model. Geographic grid blocks are defined. The basic traffic is Since the original simulation platform did not consider the time tidal effect, this example introduces a day-night modulation function based on Cloudflare Radar data: ; in, For geographic grid blocks In discrete decision cycle Modulated flow weights, For geographic grid blocks Basic traffic, To harmonize world time, For geographic grid blocks longitude, This is a normalized diurnal periodic function. This method introduces real-time fluctuations without altering the spatial distribution. All experimental results are averaged using multiple random seeds.

[0096] This example focuses on the single-cycle node acquisition problem, systematically evaluating the proposed DSIC reverse auction mechanism. The core objective of this experiment is to verify whether, under conditions of budget constraints and incomplete node cost information, the constructed incentive-compatible mechanism can suppress strategic bidding behavior while ensuring allocation efficiency, thereby stably achieving high structural control capabilities.

[0097] To ensure the representativeness of the evaluation results, the experiment was repeated under different budget sizes and node cost distributions, and the results were averaged using multiple random seeds. Comparison methods included the DSIC reverse auction mechanism with critical payoff rules, the First-Price mechanism lacking incentive compatibility, the Posted-Price mechanism based on a fixed price threshold, and the theoretically optimal knapsack solution obtained under complete information conditions.

[0098] Figure 3 The diagram illustrates a comparison of honest pricing experiments. (See attached image) Figure 3 As shown, under the condition of actual node bids, the performance differences between the various mechanisms are small. Experimental results show that the effective structural weights obtained by the proposed DSIC mechanism are highly close to the theoretical optimal solution, indicating that introducing incentive compatibility constraints does not significantly reduce allocation efficiency in the absence of strategic behavior. This result verifies the feasibility of the constructed mechanism in terms of efficiency.

[0099] Further, strategic pricing scenarios are introduced, setting a certain percentage of milestones to inflate the actual cost, such as... Figure 4 As shown, Figure 4 The diagram illustrates a comparison of false bidding experiments. The experiments show that the First-Price mechanism experiences a significant decrease in structural weights when strategic bidding is present, and the efficiency degradation becomes more pronounced with increasing false bidding levels. The Posted-Price mechanism exhibits instability under different cost distributions, easily leading to resource waste or the loss of effective nodes due to inappropriate price threshold selection. In contrast, the DSIC mechanism maintains near-optimal allocation results even with an increased proportion of strategic bids. This demonstrates that the critical pay rule effectively eliminates the incentive for nodes to falsely bid, ensuring structural stability even under conditions of information asymmetry.

[0100] Experiments conducted in a dynamic scenario considering diurnal traffic fluctuations reveal a clear tidal characteristic in node weight changes over time, such as... Figure 5 As shown, Figure 5 The experimental diagram illustrating flow variation is shown. Under these conditions, the DSIC mechanism maintains high structural control capability throughout the day, while the non-excitation-compatible mechanism is more prone to allocation deviations during peak flow periods. This result demonstrates that the proposed mechanism is effective not only under static conditions but also exhibits stability in dynamic weighting environments.

[0101] Overall, the upper-level node acquisition mechanism performs well in terms of efficiency, robustness, and dynamic stability, providing a reliable structural foundation for subsequent cross-cycle budget scheduling and revenue manipulation.

[0102] This example evaluates the impact of budget scheduling strategies on long-term structural control capabilities in a multi-period environment. Unlike single-period node selection, multi-period decision-making requires a trade-off between current gains and future opportunities; therefore, a simple greedy or uniform allocation strategy is unlikely to guarantee global optimality.

[0103] The experiment was set to a 24-hour cycle with a fixed total budget, requiring the attacker to allocate the budget across multiple epochs. Comparison strategies included uniform allocation, concentrated early investment, proportionally decreasing allocation, and the PPO dynamic scheduling strategy proposed in this example.

[0104] Figure 6 The diagram illustrates a comparative experiment of multi-slot algorithms, such as... Figure 6 The experimental results show significant differences in budget allocation patterns among different strategies. The early-stage concentrated investment strategy achieves higher structural weights in the early stages, but due to rapid budget depletion, it cannot continue to acquire key nodes during subsequent high-value periods. While the uniform allocation strategy is stable, it ignores the differences between traffic peaks and troughs, resulting in low resource utilization efficiency. The proportionally decreasing strategy balances budget consumption to some extent, but still lacks precise responsiveness to dynamic traffic changes. In contrast, the PPO scheduling strategy can automatically adjust the budget ratio based on historical revenue feedback. It significantly increases investment during peak traffic periods and proactively reduces budget consumption during low-value periods, thus reserving resources for subsequent critical periods. This phenomenon indicates that the reinforcement learning mechanism successfully captures the correlation between temporal tidal characteristics and structural revenue.

[0105] Figure 7 The experimental graph illustrating the change in returns over time is shown schematically, particularly regarding the cumulative structural control capability index, such as... Figure 7 As shown, PPO scheduling consistently outperforms other strategies throughout the entire cycle, and its advantages become more pronounced as the budget size increases. Figure 8 As shown, Figure 8 The experimental diagram illustrating the variation of revenue with budget demonstrates that the dynamic scheduling mechanism can more effectively utilize budget elasticity to maximize revenue across cycles. Furthermore, repeated experiments under multiple random seeds and different traffic distributions show that the PPO strategy exhibits a relatively consistent scheduling pattern, indicating its stability and generalization ability. In summary, the multi-cycle experiments validate the necessity and effectiveness of the reinforcement learning budget scheduling mechanism in a two-layer attack framework.

[0106] To verify the beneficial effects of this invention, the upper-layer PPO-DSIC budget scheduling mechanism and the lower-layer ZD profit manipulation mechanism are simultaneously enabled in a unified simulation environment to evaluate the overall effect of the two-layer attack framework. Unlike the aforementioned modular verification, this invention focuses on the collaborative performance of the two-layer structure in a long-term dynamic environment, verifying whether the structural control capability and profit manipulation capability form an effective closed loop.

[0107] This study verifies the actual effect of the lower-level ZD payoff manipulation mechanism in a repeated game environment, focusing on whether the long-term payoff can stably converge to the target range, and whether the payoff reach range changes monotonically with the structural influence capability.

[0108] Figure 9 The diagram schematically illustrates a zero-determinant adversarial experiment, such as... Figure 9 As shown, running a repeated game under a fixed structural influence factor, we can observe that the long-term average user payoff fluctuates in the initial stage, but gradually converges to a stable value as the number of rounds increases. This stable value is consistent with the theoretically derived target payoff, indicating that the ZD strategy can maintain a linear payoff constraint even when user strategies change. Further comparative experiments by changing the level of the structural influence factor reveal that the payoff reach range generally shifts downward as the influence factor increases, and the payoff trends for different users are consistent with the theoretical analysis. This verifies the monotonic relationship between payoff and the ability to influence structural factors. Figure 10 The diagram illustrates a comparison of the gains for attackers and defenders. (See attached image) Figure 10 As shown, under prolonged repeated interactions, the fluctuation range of returns gradually decreases, indicating that the Markov chain possesses good long-term stability after reaching a stationary distribution. Combining the lower-level mechanism with the upper-level scheduling reveals that as structural control capabilities improve, the lower-level return deviation significantly decreases, and the return control accuracy further improves. This demonstrates a clear coupling relationship between structural control capabilities and return manipulation capabilities.

[0109] Overall, the results of the lower-level experiments verified the feasibility, stability, and monotonic correlation between the ZD payoff manipulation mechanism and structural influencing factors.

[0110] To demonstrate the overall advantages of the framework, experiments were conducted under the same network environment and budget constraints, comparing four methods: single-cycle greedy node selection, reinforcement learning budget scheduling, ZD payoff control, and the complete two-layer mechanism. Evaluation metrics included long-term average payoff deviation, cumulative structural control capability, and the overall attack objective function value. Long-term payoff deviation measures the cumulative deviation between the user's actual payoff and the target range; cumulative structural control capability reflects the total amount of structural weights acquired by the attacker throughout the cycle; and the overall attack objective function value comprehensively reflects the combined optimization effect of the two layers.

[0111] Figure 11 The experimental results of control under multiple time slots are schematically shown, such as... Figure 11 As shown, the two-layer PPO-ZD mechanism can significantly reduce long-term revenue deviation within a complete 24-hour cycle. The method using only single-cycle greedy node selection lacks cross-cycle budget scheduling capabilities, failing to concentrate resources in a timely manner during peak traffic periods, resulting in significant fluctuations in revenue control performance. While reinforcement learning scheduling without introducing ZD control enhances structural control capabilities, persistent deviations in long-term user revenue still exist. Figure 12 The diagram illustrates a comparison of the gains and losses for multi-slot attackers and defenders, such as... Figure 12 As shown, when only ZD control is used without dynamic budget scheduling, the achievable profit range is narrow due to limited structural influence, and the overall control capability has a clear upper bound. In contrast, the two-layer mechanism continuously keeps the profit fluctuating around the target range throughout the entire cycle, demonstrating more stable and precise control capability.

[0112] Based on the above experimental results, it can be concluded that optimizing either the upper or lower layer alone cannot achieve the optimal attack effect. Only by coupling the PPO-DSIC node acquisition mechanism with the ZD profit manipulation mechanism in a two-layer manner can stable and efficient profit control be achieved in a long-term dynamic environment. The two-layer mechanism establishes a clear monotonic coupling relationship between structural control capability and profit adjustment capability, thus forming a complete attack optimization closed loop.

[0113] The core idea of ​​this invention is to construct a two-layer attack framework consisting of a node acquisition layer and a strategy interaction layer. Under resource constraints, this framework continuously regulates the long-term benefits of network users through key node acquisition and strategic interaction. The key technical points and protected content of this invention mainly include: The invention constructs a two-layer attack framework consisting of a node acquisition layer and a strategy interaction layer. The upper layer is responsible for acquiring key ground nodes under budget constraints, while the lower layer, after gaining control of the nodes, regulates user benefits through strategic interaction. In the node acquisition layer, a reverse auction model is constructed to determine the priority of candidate nodes based on the relationship between node bids and node structural value, and key ground nodes are selected under budget constraints. During node acquisition, reinforcement learning is introduced to dynamically allocate the attack budget, enabling optimized allocation of attack resources across different time periods, thereby improving the efficiency of attack resource utilization. After gaining control of key ground nodes, the strategy interaction between the attacker and the user is modeled as a repeated game process, forming a long-term benefit relationship through multiple rounds of strategy interaction. During the repeated game process, a linear constraint relationship between the attacker's benefit and the user's benefit is established through a zero-determinant strategy, thereby achieving continuous regulation of the user's long-term benefits. Through the aforementioned key technologies, this invention can identify and acquire ground nodes with high structural value under resource-constrained conditions, and achieve a covert and continuous impact on users' long-term benefits through strategic interaction. The method of this invention, by constructing a ground node importance assessment mechanism, a node acquisition mechanism under budget constraints, and a benefit manipulation mechanism based on repeated game theory, enables interference behavior to continuously adjust users' long-term benefits without significantly affecting the normal operation of the network, thereby improving the covertness and persistence of the interference process.

[0114] Based on the above Figure 1As can be seen from the implementation method, this embodiment of the invention constructs an integrated air-space-ground network model and obtains a set of ground nodes and a set of users; it calculates the structural importance weight of each ground node based on the ground-side traffic carried by each ground node and the backhaul communication load with the satellite; based on the structural importance weight and the preset total resource budget, it constructs a node interference cost and budget constraint model for each ground node, and initializes the preset total resource budget to the current remaining budget; it models the budget scheduling problem as a Markov decision process, and uses reinforcement learning to output the budget allocation ratio for each period based on the current network state and the current remaining budget, so as to obtain the available budget for the current period. Budget: Based on the available budget for the current period, a reverse auction mechanism is used to select ground nodes in descending order of unit structural benefit ratio to obtain the set of intervention nodes for the current period. The remaining budget is then updated under the budget constraint model, where the unit structural benefit ratio is the ratio of structural importance weight to node interference cost. The structural impact factor is calculated based on the set of intervention nodes, and a repeated game model between the interferer and the user is constructed based on the structural impact factor. In the repeated game model, a zero-determinant strategy is adopted, and the strategy parameters are adjusted according to the structural impact factor to limit the user's long-term benefits within the target benefit range, thereby achieving resistance to interference in the integrated air-space-ground network. In this way, by coupling the calculation of ground-side traffic and backhaul communication load between satellites, the structural importance weight of each node is quantified, which can accurately identify the key ground nodes that have the greatest impact on the overall network performance. At the same time, by modeling the budget scheduling problem as a Markov decision process and using reinforcement learning to dynamically output the allocation ratio, it is easy to rationally allocate interference resources during multi-cycle network operation. In addition, the zero-determinant strategy in the repeated game model constrains the user's long-term payoff within the target range, making it easy for interference behavior to continuously adjust the user's long-term payoff without significantly affecting the normal operation of the network, thus taking into account both concealment and persistence.

[0115] Based on the same inventive concept, as an implementation of the above-mentioned long-term strategy constraint method for integrated air-space-ground networks, this embodiment of the invention also provides a long-term strategy constraint device for integrated air-space-ground networks. Figure 13 This is a structural diagram of a long-term strategy constraint device for an integrated air-space-ground network according to an embodiment of the present invention. See also... Figure 13 As shown, the device may include: The first construction module 1301 is used to construct an integrated air-space-ground network model and obtain a set of ground nodes and a set of users; The weight calculation module 1302 is used to calculate the structural importance weight of each surface node based on the ground-side traffic carried by each surface node and the backhaul communication load with the satellite. The second construction module 1303 is used to construct a node interference cost and budget constraint model for each surface node based on the structural importance weight and the preset total resource budget, and initialize the preset total resource budget to the current remaining budget; Modeling module 1304 is used to model the budget scheduling problem as a Markov decision process and use reinforcement learning to output the budget allocation ratio for each period based on the current network state and the current remaining budget, so as to obtain the available budget for the current period. The selection and update module 1305 is used to select ground nodes in descending order of unit structure benefit ratio based on the available budget for the current period using a reverse auction mechanism, to obtain the set of intervention nodes for the current period, and to update the current remaining budget under the budget constraint model. The unit structure benefit ratio is the ratio of the structural importance weight to the node interference cost. The calculation and construction module 1306 is used to calculate the structural influence factor based on the set of intervention nodes, and to construct a repeated game model between the interferer and the user based on the structural influence factor. The adjustment module 1307 is used to adopt a zero-determinant strategy in a repeated game model, and adjust the strategy parameters according to the structural influence factor so that the user's long-term returns are limited to the target return range, thereby achieving resistance to interference in the integrated air-space-ground network.

[0116] The weight calculation module 1302 is specifically used to perform coupled calculation and normalization processing on the ground-side traffic carried by each ground node in the current period and the backhaul communication load between each ground node and the satellite to obtain the original importance weight of each ground node; and to perform exponential weighted smoothing on the historical weight and the original importance weight of the previous period, and use the smoothed result as the structural importance weight of each ground node in the current period.

[0117] The second construction module 1303 is specifically used to determine the node interference cost of each surface node based on the structural importance weight, basic interference cost, importance correlation coefficient and node security heterogeneity factors; and to establish a target constraint relationship based on the preset total resource budget, and use the target constraint relationship as the budget constraint model. The target constraint relationship is a constraint relationship in which the sum of the allocation budgets of each cycle in the cross-cycle allocation process does not exceed the preset total resource budget.

[0118] Modeling module 1304 is specifically used to set the current network state in the budget scheduling problem as a state space containing the structural influence factor of the previous period, the current remaining budget, and the user revenue of the previous period; to set the budget allocation ratio as the action space; and to set the sum of the negative loss function values ​​obtained after the allocation as the reward function, so as to construct a Markov decision process. Using a policy network of reinforcement learning, the budget allocation ratio of the current period is output according to the current network state. The policy network is trained using a proximal policy optimization algorithm. The available budget for the current period is obtained by multiplying the budget allocation ratio by the current remaining budget.

[0119] The selection and update module 1305 is specifically used in the reverse auction mechanism to determine the node interference cost as equal to the bids submitted by each ground node under incentive compatibility conditions; to determine the unit structure benefit ratio of each ground node based on the ratio of the structural importance weight to the bids submitted by each ground node; to sort the ground nodes in descending order of unit structure benefit ratio to obtain a sorted ground node sequence; to select ground nodes sequentially according to the available budget for the current period, until the remaining budget is insufficient to select the next ground node, at which point the selected ground nodes are used as the intervention node set for the current period; and to determine the actual payment cost of each selected ground node according to the critical payment rule, and to deduct the actual payment cost from the remaining budget to update the current remaining budget.

[0120] The calculation and construction module 1306 is specifically used to determine the structural influence factor based on the ratio of the sum of ground-side traffic carried by each ground node in the intervention node set to the total ground-side traffic carried by all ground nodes; to determine the immediate payoff of each user in each round of the game based on the structural influence factor, the basic payoff, the preset loss function value corresponding to each user, and the user's own action cost; to determine the immediate payoff of the interferer in each round of the game based on the structural influence factor, the preset loss function value of all users, and the interferer's own action cost; and to construct a repeated game model based on the immediate payoff of the user and the immediate payoff of the interferer, which is used to conduct multiple rounds of strategy interaction in each cycle.

[0121] In the calculation and construction module 1306, the immediate payoff of each user in each round of the game is determined based on the structural influence factor, the basic payoff, the preset loss function value corresponding to each user, and the user's own action cost. This includes: determining the first product of the structural influence factor, the user weight coefficient, and the preset loss function value corresponding to each user; subtracting the first product from the basic payoff and subtracting the user's own action cost to obtain the user's immediate payoff in each round of the game.

[0122] In the calculation and construction module 1306, the immediate payoff of the disruptor in each round of the game is determined based on the structural influence factor, the preset loss function values ​​of all users, and the disruptor's own action cost. This includes: determining the second product of the sum of the structural influence factor and the preset loss function values ​​of all users; and subtracting the disruptor's own action cost from the second product to obtain the disruptor's immediate payoff in each round of the game.

[0123] The adjustment module 1307 is specifically used to determine the reachable range of the user's long-term returns based on the structural influence factors, and select the target return range within the reachable range; adjust the linear constraint parameters of the zero determinant strategy according to the target return range so that the long-term returns of the disruptor and the user satisfy the linear constraint relationship; execute the adjusted zero determinant strategy in the repeated game model to limit the user's long-term returns to the target return range through the linear constraint relationship.

[0124] It should be noted that the above description of the long-term strategy constraint device embodiment for integrated air-space-ground networks is similar to the description of the long-term strategy constraint method embodiment for integrated air-space-ground networks, and has similar beneficial effects. For any technical details not disclosed in the embodiments of the long-term strategy constraint device for integrated air-space-ground networks of this invention, please refer to the description of the long-term strategy constraint method embodiment for integrated air-space-ground networks of this invention for understanding.

[0125] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for long-term policy constraint for space-air-ground integrated network, characterized in that, include: Construct an integrated air-space-ground network model and obtain the set of ground nodes and user sets; The structural importance weights of each ground node are calculated based on the ground-side traffic carried by each ground node and the backhaul communication load with the satellite. Based on the structural importance weights and the preset total resource budget, a node interference cost and budget constraint model for each surface node is constructed, and the preset total resource budget is initialized to the current remaining budget. The budget scheduling problem is modeled as a Markov decision process, and reinforcement learning is used to output the budget allocation ratio for each period based on the current network state and the current remaining budget, so as to obtain the available budget for the current period. Based on the available budget for the current period, a reverse auction mechanism is used to select ground nodes in descending order of unit structure benefit ratio to obtain the set of intervention nodes for the current period. The current remaining budget is then updated under the budget constraint model. The unit structure benefit ratio is the ratio of the structure importance weight to the node interference cost. Calculate the structural influence factor based on the set of intervention nodes, and construct a repeated game model between the disruptor and the user based on the structural influence factor. The zero-determinant strategy is adopted in the repeated game model. The strategy parameters are adjusted according to the structural influence factor so that the user's long-term returns are limited to the target return range, thereby achieving resistance to interference in the integrated air-space-ground network.

2. The method of claim 1, wherein, The calculation of the structural importance weights of each surface node based on the ground-side traffic carried by each node and the backhaul communication load with the satellite includes: The ground-side traffic carried by each ground node in the current period and the backhaul communication load between each ground node and the satellite are coupled and normalized to obtain the original importance weight of each ground node. The historical weights of the previous period and the original importance weights are subjected to exponential weighted smoothing, and the smoothed result is used as the structural importance weights of the surface nodes in the current period. 3.The method for long-term policy constraint for space-air-ground integrated network according to claim 1, wherein, The step of constructing a node interference cost and budget constraint model for each surface node based on the structural importance weights and a preset total resource budget includes: Based on the structural importance weight, basic interference cost, importance correlation coefficient, and node security heterogeneity factors, the node interference cost of each ground node is determined. Based on the preset total resource budget, a target constraint relationship is established, and the target constraint relationship is used as the budget constraint model. The target constraint relationship is a constraint relationship in which the sum of the allocation budgets of each period in the cross-period allocation process does not exceed the preset total resource budget.

4. The method of claim 1, wherein, The budget scheduling problem is modeled as a Markov decision process, and reinforcement learning is used to output the budget allocation ratio for each period based on the current network state and the current remaining budget, so as to obtain the available budget for the current period, including: The current network state in the budget scheduling problem is set as a state space containing the structural influence factor of the previous cycle, the current remaining budget, and the user revenue of the previous cycle. The budget allocation ratio is set as the action space, and the sum of the negative loss function values ​​obtained after the allocation is set as the reward function, so as to construct the Markov decision process. Using the reinforcement learning policy network, the budget allocation ratio for the current period is output according to the current network state. The policy network is trained using a proximal policy optimization algorithm. The available budget for the current period is obtained by multiplying the budget allocation ratio by the current remaining budget.

5. The method of claim 1, wherein, The step of selecting ground nodes in descending order of unit structural benefit ratio based on the available budget for the current period using a reverse auction mechanism to obtain the intervention node set for the current period, and updating the current remaining budget under the budget constraint model, includes: In the reverse auction mechanism, under incentive compatibility conditions, the node interference cost is determined to be equal to the bids submitted by each surface node; The unit structure revenue ratio of each surface node is determined based on the ratio of the structural importance weight to the bid submitted by each surface node. The ground nodes are sorted in descending order of the unit structure benefit ratio to obtain a sorted ground node sequence. Based on the available budget for the current period, ground nodes are selected sequentially according to the sorted ground node sequence until the remaining budget is insufficient to select the next ground node. The selected ground nodes are then used as the intervention node set for the current period. According to the critical payment rule, the actual payment cost of each selected ground node is determined, and the actual payment cost is deducted from the remaining budget to update the current remaining budget.

6. The method of claim 1, wherein, The step of calculating the structural influence factor based on the set of intervention nodes and constructing a repeated game model between the disruptor and the user based on the structural influence factor includes: The structural influence factor is determined based on the ratio of the sum of the ground-side flows carried by each ground node in the intervention node set to the total ground-side flows carried by all ground nodes. Based on the structural influence factors, basic returns, preset loss function values ​​for each user, and the user's own action costs, determine the user's immediate returns in each round of the game. Based on the structural influence factor, the preset loss function values ​​of all users, and the interference's own action cost, the immediate benefit of the interference in each round of the game is determined. Based on the user's immediate payoff and the interferer's immediate payoff, the repeated game model is constructed, which is used to conduct multiple rounds of strategy interaction in each cycle.

7. The method of claim 6, wherein the long-term policy constraint is determined based on a space-air-ground integrated network. The process of determining a user's immediate payoff in each round of the game based on the structural influence factor, basic payoff, preset loss function values ​​for each user, and the user's own action cost includes: Determine the first product of the structural influence factor, the user weight coefficient, and the preset loss function value corresponding to each user; Subtract the first product from the base payout, and subtract the user's own action cost to obtain the user's immediate payout in each round of the game.

8. The method of claim 6, wherein the long-term policy constraint is determined based on a space-air-ground integrated network. The determination of the disruptor's immediate payoff in each round of the game, based on the structural influence factor, the preset loss function values ​​of all users, and the disruptor's own action cost, includes: Determine the second product of the structural influence factor and the sum of the preset loss function values ​​of all users; Subtracting the interferer's own action cost from the second product yields the interferer's immediate gain in each round of the game. 9.The method for long-term policy constraint for space-air-ground integrated network according to claim 1, wherein, The adoption of a zero-determinant strategy in the repeated game model, adjusting strategy parameters according to the structural influence factor to limit the user's long-term returns within a target return range, includes: The achievable range of the user's long-term returns is determined based on the structural influencing factors, and the target return range is selected within the achievable range. Based on the target profit range, adjust the linear constraint parameters of the zero determinant strategy so that the long-term profit of the disruptor and the long-term profit of the user satisfy a linear constraint relationship. In the repeated game model, an adjusted zero determinant strategy is executed, which limits the user's long-term payoff to the target payoff range through the linear constraint relationship.

10. A long-term policy constraint device for space-air-ground integrated network, characterized in that, include: The first construction module is used to build an integrated air-space-ground network model and obtain the set of ground nodes and user sets; The weight calculation module is used to calculate the structural importance weight of each ground node based on the ground-side traffic carried by each ground node and the backhaul communication load with the satellite. The second construction module is used to construct the node interference cost and budget constraint model of each surface node according to the structural importance weight and the preset total resource budget, and initialize the preset total resource budget to the current remaining budget; The modeling module is used to model the budget scheduling problem as a Markov decision process and use reinforcement learning to output the budget allocation ratio for each period based on the current network state and the current remaining budget, so as to obtain the available budget for the current period. The selection and update module is used to select ground nodes in descending order of unit structure benefit ratio according to the available budget for the current period using a reverse auction mechanism, to obtain the set of intervention nodes for the current period, and to update the current remaining budget under the budget constraint model. The unit structure benefit ratio is the ratio of the structure importance weight to the node interference cost. The calculation and construction module is used to calculate the structural influence factor based on the set of intervention nodes, and to construct a repeated game model between the interferer and the user based on the structural influence factor. The adjustment module is used to employ a zero-determinant strategy in the repeated game model, and adjust the strategy parameters according to the structural influence factor so that the user's long-term returns are limited to the target return range, thereby achieving resistance to interference in the integrated air-space-ground network.

Images