A federated learning driven low-orbit satellite network adaptive intrusion response control method

Abstract

This invention relates to network information security, specifically to a federated learning-driven adaptive intrusion response control method for low-Earth orbit satellite networks. The method includes: if the risk of a selected defensive action exceeds a set threshold, the action is suspended, an intent vector is generated, and reported to the cluster head node. The current satellite node then selects an optimal action from a set of low-risk action candidates for execution. Upon receiving the intent vector, the cluster head node selects high-reputation nodes with reputation values ​​exceeding a set threshold to initiate consensus. Each high-reputation node evaluates the intent vector based on its local model and votes. If more than two-thirds of the high-reputation nodes reach consensus, the cluster head node sends permission to execute the suspended action to the generated intent vector and increases the satellite node's reputation. If consensus fails, the suspended action is not allowed, and the satellite node's reputation is decreased. This invention addresses key technical challenges such as difficulty in security policy convergence, misjudgment of single-point high-risk situations, and delayed response times.

Description

Technical Field

[0001] This invention relates to the interdisciplinary fields of network information security, space communication network architecture, and distributed artificial intelligence, specifically to a federated learning-driven adaptive intrusion response control method for low-Earth orbit satellite networks. Background Technology

[0002] As sixth-generation mobile communication (6G) networks gradually evolve towards a space-ground converged architecture, low-Earth orbit (LEO) satellite networks, with their outstanding advantage of wide-area coverage, have become an important space infrastructure supporting the operation of critical services. Current security technologies for satellite networks mostly focus on the "intrusion detection" stage. However, in the "intrusion response" stage, the industry still primarily employs traditional centralized control strategies or relies on single-node edge AI for decision-making. Both of these mainstream approaches have significant drawbacks: Firstly, the centralized response model suffers from unavoidable high latency. Low-Earth orbit satellite networks inherently possess the physical characteristics of frequent topological changes and intermittent satellite-to-ground links. When faced with a sudden network attack, security data must be transmitted back to the ground for processing before the ground platform issues response commands. This process creates a significant defense latency gap, making it difficult to quickly block attacks and mitigate losses. Secondly, relying solely on edge nodes of a single satellite for AI autonomous decision-making carries a high risk of catastrophic misjudgments. Limited by the local observation range of a single satellite and environmental interference such as space radiation, AI models are prone to errors. If the model incorrectly executes high-risk protection operations such as hard rebooting onboard routes or severing inter-satellite links, it can easily cause irreversible systemic service interruptions. Furthermore, the attack traffic data faced by different orbital nodes in a satellite network exhibits a significant non-independent and identically distributed pattern. Due to the characteristics of Distributed (Non-IID), traditional distributed defense systems lack efficient data collaboration mechanisms, resulting in insufficient generalization capabilities of local security models and an inability to effectively cope with various unknown and complex attacks. Based on these issues, there is an urgent need for a distributed intrusion response architecture that can achieve rapid local response in communication-constrained scenarios, possess fault-tolerant and corrective capabilities, and leverage global security experience for collaborative evolution, thereby addressing the core shortcomings of existing technologies. Summary of the Invention

[0003] To address the issue of misjudgment caused by limited local field of view when using AI for intrusion detection on a single-point satellite, and to prevent satellite network outages due to erroneous execution of high-risk actions, this invention proposes a federated learning-driven adaptive intrusion response control method for low-Earth orbit satellite networks, specifically including the following steps:

[0004] The available satellite nodes for the next time step are divided into multiple clusters, and the satellite nodes within each cluster are interconnected. Each cluster elects a cluster head node.

[0005] Satellite nodes construct state vectors, which include alarm features, satellite body status, and defense history context. The state vectors are compressed into state features by a state encoding module consisting of an input layer and two fully connected hidden layers.

[0006] The state features obtain the expected return of the defense effect for each candidate defense action through a defense effect evaluation branch consisting of a fully connected layer;

[0007] Satellite nodes obtain the inherent risk of each defensive action based on the local action risk quantification table, and estimate the energy consumption cost of each defensive action based on the local energy perception evaluator.

[0008] The total value of each action is calculated based on the defense effectiveness assessment branch, the action risk quantification table, and the energy consumption cost of the defense action. The final defense decision is generated based on the total value of the action and the current satellite reputation.

[0009] If the risk of the selected defensive action exceeds the set threshold, the action is suspended, an intent vector is generated and reported to the cluster head node, and the current satellite node selects the optimal action from the low-risk action candidate set to execute.

[0010] After receiving the intent vector, the cluster head selects a high-reputation node with a reputation value greater than a set threshold from within the cluster to initiate consensus. Each high-reputation node evaluates the intent vector based on its local model and votes. If more than two-thirds of the high-reputation nodes reach consensus, the cluster head node sends permission to the satellite node that generated the intent vector to execute the suspension action and increases the reputation of that satellite node. If the consensus fails, the suspension action is not allowed and the reputation of that satellite node is reduced.

[0011] Compared with the prior art, the present invention has the following beneficial effects:

[0012] 1. This invention relies on an on-board layered asynchronous response architecture to decentralize conventional defense actions to the edge of the satellite and directly infer and execute them through the FRL model, thus completely eliminating the high satellite-to-ground round-trip latency caused by traditional cloud-based centralized decision-making;

[0013] 2. The innovative high-risk action suspension and intra-cluster consensus gating mechanism of this invention adds a safety lock to the uninterpretability of AI algorithms at the architecture level, effectively mitigating the risk of network cascading collapse caused by single-point satellite misjudgment, and greatly improving the survivability of satellite communication systems;

[0014] 3. This invention employs a federated learning framework that allows satellites to achieve cloud aggregation by sharing only model parameters without uploading underlying sensitive traffic logs. This not only reduces the bandwidth pressure on satellite-to-ground communication but also integrates adversarial experience from different orbits across the network, significantly improving the model's generalization and immunity to complex and unknown attacks. Attached Figure Description

[0015] Figure 1 This is a schematic diagram illustrating the inference process of a federated learning-driven adaptive intrusion response control method for low-Earth orbit satellite networks according to the present invention.

[0016] Figure 2 This is a schematic diagram of the low Earth orbit satellite network system framework of the present invention;

[0017] Figure 3 This is a flowchart illustrating the specific implementation of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] This invention proposes a federated learning-driven adaptive intrusion response control method for low-Earth orbit satellite networks, such as... Figure 1 Specifically, it includes the following steps:

[0020] The available satellite nodes for the next time step are divided into multiple clusters, and the satellite nodes within each cluster are interconnected. Each cluster elects a cluster head node.

[0021] Satellite nodes construct state vectors, which include alarm features, satellite body status, and defense history context. The state vectors are compressed into state features by a state encoding module consisting of an input layer and two fully connected hidden layers.

[0022] The state features obtain the expected return of the defense effect for each candidate defense action through a defense effect evaluation branch consisting of a fully connected layer;

[0023] Satellite nodes obtain the inherent risk of each defensive action based on the local action risk quantification table, and estimate the energy consumption cost of each defensive action based on the local energy perception evaluator.

[0024] The total value of each action is calculated based on the defense effectiveness assessment branch, the action risk quantification table, and the energy consumption cost of the defense action. The final defense decision is generated based on the total value of the action and the current satellite reputation.

[0025] If the risk of the selected defensive action exceeds the set threshold, the action is suspended, an intent vector is generated and reported to the cluster head node, and the current satellite node selects the optimal action from the low-risk action candidate set to execute.

[0026] After receiving the intent vector, the cluster head selects a high-reputation node with a reputation value greater than a set threshold from within the cluster to initiate consensus. Each high-reputation node evaluates the intent vector based on its local model and votes. If more than two-thirds of the high-reputation nodes reach consensus, the cluster head node sends permission to the satellite node that generated the intent vector to execute the suspension action and increases the reputation of that satellite node. If the consensus fails, the suspension action is not allowed and the reputation of that satellite node is reduced.

[0027] like Figure 2 In the low Earth orbit satellite network system of this invention, satellite nodes are divided into multiple clusters for management. When an intrusion response occurs in a node within a cluster, collaborative confirmation is performed within the cluster. Each cluster communicates with the ground security center through the cluster head satellite, including updating the network parameters corresponding to each cluster based on federated learning.

[0028] In each federated learning communication cycle, this invention constructs a set of topologically stable and reasonably sized collaborative confirmation clusters, and elects the optimal cluster head within each cluster. Its function is to periodically aggregate local models and verify high-risk behaviors in real time.

[0029] The two steps of periodic local model aggregation and real-time consensus verification of high-risk behaviors in this invention have the following time differences:

[0030] During periodic local federated learning aggregation, including local model training for each satellite within the cluster, gradient uploading to the cluster head, and cluster head aggregation, the total time consumption is T. agg ;

[0031] When conducting consensus verification for high-risk operations, if any satellite generates a high-risk defense intent, the cluster head must immediately organize intra-cluster consensus, which takes T seconds. con ;

[0032] Therefore, the connection period of a cluster is T. cluster ≥ max(T agg ,T con )+T margin T margin To include a safety margin for topological reconvergence time and ephemeris prediction error, T is generally used. agg The value is much greater than T con Therefore, T cluster A complete cycle of local federated learning typically takes 5 to 15 minutes in low-Earth orbit satellite constellations.

[0033] Based on the connection period, the definition of a stable connection in this invention is: using satellite ephemeris prediction, in the future T... cluster An inter-satellite link that exists continuously throughout the entire time period and maintains uninterrupted two-way communication can be considered a "stable edge." The cooperative confirmation cluster constructed in this way is located in T... cluster The internal physical topology is constant, which can support model aggregation and consensus verification of high-risk actions that occur at any time.

[0034] To support the continuous iteration of federated learning, the ground center divides the timeline into a series of non-overlapping periodic windows, each with a length of T. cluster Before the start of each periodic window, the ground center re-executes the complete clustering and election process to generate a collaborative confirmation cluster corresponding to that period. The models are connected between periods through global aggregation, that is, the cluster gradient is uploaded to the ground control center by the cluster head satellite. Cluster reorganization caused by changes in satellite geographical location will not affect the learning process.

[0035] As an optional implementation, the present invention provides a process for collaboratively confirming cluster partitioning, which specifically includes the following steps:

[0036] Based on the current ephemeris, extract all inter-satellite links that meet the stable connection conditions in the next cycle and construct an undirected graph. The nodes in the undirected graph are all currently active satellites in orbit.

[0037] Detect all maximal connected components in an undirected graph, where each connected component contains satellites in region T. cluster All of them can be interconnected through single-hop or multi-hop stable paths, providing a physical basis for forming a cooperative confirmation cluster;

[0038] To control the communication overhead of distributed consensus and the convergence quality of local model aggregation, upper and lower limits are imposed on the number of satellites within a cluster. In this embodiment, the lower limit is set to 6 (this embodiment is based on the geometric equation of Earth entity occlusion that enables unobstructed direct line-of-sight communication between adjacent satellites). Given a defined absolutely physically connected topological boundary, solve for N as the lower bound. For the Earth's radius, The altitude of the satellite orbit. The minimum belief angle is 11, and the maximum belief angle is 11 (based on tolerance in this embodiment). Given a Byzantine fault node, using the Byzantine fault tolerance algorithm The complexity of communication signaling interaction should not lead to congestion in the onboard router queue or processor crashes (determining the computational power sensitive extreme value).

[0039] If the number of nodes in a connected component is less than or equal to 11, it is directly designated as a cooperative acknowledgment cluster.

[0040] If the value is greater than 11, the component is split into several subclusters, each with a size within the range of [6, 11]. The splitting strategy can be:

[0041] Minimum Cut Cluster: Using spectral clustering or minimum cut algorithm, the subclusters are cut at relatively weak edges while ensuring internal connectivity, so that each subcluster remains highly stable and connected.

[0042] Core node coverage clustering: In excessively large components, the satellite with the highest degree is selected as the "cluster core" and it and its one-hop neighbors are included in the same sub-cluster until the sub-cluster size reaches 11. The already divided nodes are removed from the graph. This process is repeated until all nodes are clustered. When selecting neighbors, if the number of candidate nodes exceeds the limit, satellites with higher link signal-to-noise ratio and more stable relative motion are selected first.

[0043] The above splitting process is based solely on connection stability, without considering business factors such as data distribution and reputation, to ensure the objectivity of the physical basis of clustering. If a residual set of nodes fails to reach the required size after splitting, it will be merged into a neighboring sub-cluster that already meets the size requirement; if merging is not possible, it will be marked as an "edge weak cluster," allowing it to operate at a size below 6, and triggering an alarm to notify the ground center. High-risk operations of such nodes can be ensured through a subsequent emergency delegation mechanism.

[0044] Within each collaborative confirmation cluster, three indicators of each satellite are comprehensively evaluated, and a comprehensive competency score is obtained through weighted calculation. The satellite with the highest score is automatically selected as the cluster head, responsible for local model aggregation, high-risk intent consensus scheduling, and security gating within that cycle. Specifically, this includes the following steps:

[0045] 1. Score for evenness of flow distribution

[0046] Based on long-term historical security data, the ground control center has compiled a global normal network traffic distribution reference vector Q={q1,q2,…,q}. K}, where q k (k∈{1,2,…, K) represents the feature of the k-th dimension, which includes the proportion of different protocol types, packet length distribution or other traffic features, and is periodically distributed to each satellite.

[0047] The i-th satellite node captures recent local network traffic and calculates its distribution P. i ={p 1,i ,p 2,i ,…,p K,i}, calculate the distribution P i The KL divergence relative to Q is expressed as:

[0048]

[0049] The smaller the divergence value, the closer the local traffic distribution is to the global normal baseline, the lower the probability of the satellite itself being intruded or hijacked, and the more suitable it is to assume the role of cluster head.

[0050] Mapping KL divergence to balance score:

[0051]

[0052] in, The score for the traffic distribution balance of the i-th satellite node; This is the mapping steepness coefficient, typically ranging from 1.0 to 2.0. When... hour, The maximum value is 1; as the degree of traffic anomaly increases, the score monotonically decreases and approaches 0.

[0053] 2. Historical security and credit rating

[0054] The reputation of the i-th satellite node ∈[0,1], this value is maintained uniformly by the ground center, and integrates the satellite's performance in the previous collaborative confirmation process, including: the honesty of consensus verification, whether the defense action produced false alarms / missed alarms, whether it responded to cluster head scheduling in a timely manner, etc.; the reputation is updated using the sliding window exponential decay mean to ensure that recent behavior has a higher weight; the satellite's initial reputation is set to 1.0.

[0055] 3. Scoring based on available computing power and remaining energy.

[0056] Cluster heads need to undertake computationally intensive tasks such as model aggregation and consensus coordination; therefore, the available computing power of the satellite must be considered. and remaining battery energy Both are normalized separately:

[0057]

[0058]

[0059] in, This represents the maximum available computing power among all satellites. The satellite is fully charged.

[0060] In summary, the available computing power and remaining energy score of the i-th satellite node are expressed as follows:

[0061]

[0062] in, , The weighting coefficients for available computing power and remaining power are respectively. This embodiment is set up =0.6、 =0.4, meaning that in this embodiment, computing power is more important than energy.

[0063] 4. Overall competence score of satellite nodes

[0064] In summary, satellite i's overall suitability score is a linear weighted sum of the three indicators:

[0065]

[0066] in, The overall competence score for the i-th satellite node; , , These are the weighting coefficients for the traffic distribution balance score, historical security reputation, available computing power, and remaining energy score, respectively. .

[0067] The weighting coefficients (α, β, γ) are dynamically set by the ground center based on the current overall network security situation and mission requirements, and are injected into each satellite along with the Q-vector before the start of each cycle via the satellite-to-ground link. Typical modes include:

[0068] Normal mode: α=0.4, β=0.4, γ=0.2; This mode balances traffic normality and node reliability, and is suitable for most situations.

[0069] Attack escalation mode: β=0.6, α=0.25, γ=0.15; This mode can be switched when an increase in the frequency of network attacks or an increase in the number of abnormal nodes is detected. This mode significantly increases the reputation weight to ensure that the cluster head is a highly reliable satellite.

[0070] High-performance mode: α=0.5, β=0.2, γ=0.3; This mode, when the security situation is stable, moderately favors traffic balance and resource capacity to optimize model aggregation efficiency.

[0071] When multiple satellites within a cluster have the same highest score, the cluster head node is determined by prioritizing historical security reputation, traffic distribution balance score, available computing power, and remaining energy score. This process includes the following steps:

[0072] Each satellite in the cluster calculates its own traffic distribution balance score, available computing power and remaining energy score, and together with its own reputation obtained from the ground center, it is digitally signed and then broadcast in a flood within the cluster.

[0073] Each satellite independently calculates the score of all members (including itself) within the cluster based on the same weight vector, and sorts them in descending order of value;

[0074] The highest-scoring satellite is automatically selected as the cluster head. If there is a tie, the node with the higher historical security reputation is selected. If the historical security reputation is the same, the node with the higher traffic distribution balance score is selected. If the traffic distribution balance score is still the same, the node with the higher available computing power and remaining energy score is selected.

[0075] The elected cluster head broadcasts an acknowledgment message to the cluster and reports it to the ground center for record-keeping via the satellite-to-ground link. The cluster head is responsible for local model aggregation, cluster consensus scheduling, and security gating within the current cycle.

[0076] This invention designs a federated reinforcement learning policy inference model, which mainly consists of four functional modules:

[0077] The state encoding module consists of an input layer and two fully connected hidden layers (128 neurons per layer, ReLU activation). It is responsible for compressing the original high-dimensional state vector into a compact feature representation for subsequent modules to share.

[0078] The defense effectiveness evaluation branch receives the state encoding and outputs the expected return of the pure defense effectiveness for each candidate defense action through a fully connected layer (64 neurons). The parameters of this branch will participate in cross-node sharing during the global aggregation process of federated learning.

[0079] The safety and resource cost assessment module consists of two parts, both of which are local personalized parameters that are not federated. These include: an action risk quantification table, which pre-calibrates the inherent danger and irreversibility of each action based on domain safety knowledge and is fixed in the model, not adjusted with learning; and an energy-aware evaluator, which in this embodiment is implemented using a small feedforward network to output the estimated energy cost of performing the action under the current resource state based on the input state and action code.

[0080] The reputation-gated action selector, located before the model output, receives the combined total value and the current satellite reputation to generate the final decision.

[0081] When a satellite receives an intrusion alarm, it autonomously constructs a state vector for the current environment. In this embodiment, the state vector includes alarm features (including alarm type, severity level, attack source / target, protocol, etc. encoding), satellite body state (including local parameters such as CPU utilization, memory usage, current network throughput, remaining power ratio, link quality index, etc.), and defense history context (including the encoding of the most recent m actions and their actual effects). The final dimension of the state vector can be defined according to actual deployment requirements, for example, it can be set to 512 dimensions.

[0082] If the state vector s t Input state encoding module to generate compact hidden layer features h t Hidden layer features ht Branch input defense effectiveness evaluation branch and energy perception evaluator:

[0083] In the defense effectiveness evaluation branch, the expected return of the defense effectiveness under each action is output, represented as: {Q eff (s t ,a1), Q eff (s t ,a N ),…, Q eff (s t ,a N )}, where Q eff (s t ,a n ) represents the state vector s t When performing action a n The expected return on the defense effect, n={1,2,…,N}, where N is the total number of predefined security defense actions;

[0084] In the energy-sensing evaluator, the energy-sensing evaluator outputs the energy consumption value for each action.

[0085] This invention calculates the total value of each candidate action based on a decoupled value function:

[0086]

[0087] in, This represents the total value of action a under state vector s; This represents the expected return of the purely defensive effect of action a under state vector s; , This is the balance coefficient; The risk coefficient for action 'a' in the action risk quantification table; This represents the estimated energy cost of action a under state vector s. This calculation automatically reduces the evaluation value of high-risk, high-energy-consuming actions during the inference phase, making the model naturally inclined towards safe and energy-efficient defense strategies.

[0088] This embodiment also incorporates the satellite's current reputation score R∈ [0,1] to further modulate the total value:

[0089]

[0090] in, The total value after modulation; As a gating unit, when the reputation value R of a satellite node is less than a set threshold and the risk coefficient of the action chosen by that node is greater than the set threshold, a modulation coefficient is output to attenuate the total value, and the attenuation magnitude is proportional to the risk level; otherwise, the gating unit does not attenuate the total value. When R is low, for those... Actions exceeding a threshold are subject to additional attenuation, with the attenuation rate proportional to the level of risk. This forces low-reputation satellites to automatically become more conservative in their autonomous decision-making, suppressing their ability to select high-risk operations.

[0091] This embodiment selects the optimal action a* based on the modulated value using an ε-greedy strategy, and then performs a security classification, namely:

[0092] If the risk coefficient of the optimal action a* If the value is less than or equal to the set threshold, the action is considered low-risk; the model directly outputs the action command, the satellite executes it immediately, and at the same time generates corresponding experience records and stores them in the local experience pool for subsequent local model updates;

[0093] If the risk coefficient of the optimal action a* If the value exceeds a set threshold, the action is considered high-risk; in this case, the model will not execute the action, but will instead trigger a unique intent suspension and temporary defense mechanism:

[0094] 1. Suspend action a* at the physical layer, preventing its execution;

[0095] 2. The current state s, the high-risk action to be performed a*, and the values ​​of each action... Information such as distribution and model confidence is packaged into an intent vector and reported to the cluster head via inter-satellite links to request collaborative confirmation.

[0096] 3. The model is redesigned to meet the risk coefficient. Select actions that are less than or equal to the set threshold The largest action, as a temporary defensive action, is executed immediately to contain the spread of the attack during the consensus period, and the experience of this temporary action is also stored in the experience pool.

[0097] In this embodiment, the continuous reputation value R∈[0,1] of each satellite node is calculated and maintained uniformly by the ground control center and periodically sent to each satellite via the satellite-to-ground link. The initial reputation value of all newly inserted satellites or satellites reactivated after a long period of deorbiting is set to 1.0 (i.e., completely trustworthy).

[0098] The reason why the reputation score is managed uniformly by the ground center is: 1. The ground center can integrate multi-source global information to avoid the risk of deception by single-satellite self-assessment; 2. It can form linkage through asynchronous global model aggregation - if the model gradient of a satellite is detected to be abnormal, its reputation score can be directly penalized.

[0099] In this embodiment, the reputation of a satellite node includes at least the following update methods:

[0100] 1. When consensus is reached and a real threat is confirmed, the credibility of the node that initiated the intent vector satellite can be slightly increased, for example, by 0.02; when consensus is rejected, it is determined that there is a false alarm or abuse of high-risk defense strategy, and the credibility of the node that initiated the intent vector satellite can be significantly reduced, for example, by 0.1~0.3.

[0101] 2. If the satellite node successfully defends itself without causing any other negative effects, its reputation score can be slightly increased, for example, by 0.01. If the defense fails, or if the defense strategy causes business interruption, resource waste, or other negative impacts, the satellite node's reputation score will be slightly decreased, for example, by 0.05.

[0102] The Federated Reinforcement Learning Policy Inference Model (FRL-QN) defined in this invention has the following total action value during forward inference:

[0103]

[0104] After the defensive action is executed, the environment provides feedback on three objective physical quantities:

[0105] : Security blocking benefits (e.g., complete blocking: +1.0, partial suppression: +0.3, ineffective: -1.0, wrongful blocking of legitimate business: -0.8).

[0106] Service quality loss (e.g., the mean of normalized increments of latency, throughput, and packet loss rate can be used);

[0107] Actual normalized energy consumption (actual energy consumed / energy budget per use).

[0108] Joint Award for this Invention Defined as a weighted composite of security gains and service costs, energy consumption is not included in this reward but is used separately to update the energy consumption predictor (because energy consumption is the instantaneous cost of an action, not the result of changes in the environmental state caused by the action, and is not suitable as a target for reinforcement learning to predict "future rewards"). Energy consumption has been... Since this is explicitly deducted from the total value, the defense effectiveness evaluation branch only needs to learn the combined reward of security and QoS, expressed as:

[0109]

[0110] in, This is a balancing factor for the reward function, typically set to 0.5, but can also be statically configured according to task requirements. This design ensures... This reflects the overall effectiveness of a defensive action in terms of security and communication quality, with a value range of approximately [value range missing]. Positive values ​​represent net gains, while negative values ​​represent net losses.

[0111] In this invention, low-risk actions are performed normally, or high-risk actions are approved by consensus: the actual actions performed are recorded. High-risk actions are suspended, and temporary low-risk actions are executed instead: Record the actual temporary actions performed. The empirical tuples are uniformly defined as follows:

[0112]

[0113] High-risk intentions that lead to consensus rejection produce no experience because there is no physical execution and no environmental feedback.

[0114] The defense effectiveness evaluation branch uses standard deep Q-learning to update network parameters, with the goal of approximating the network's performance. Over a long period of time, a small training batch is randomly sampled from the experience pool during training. Given the number of samples, calculate the target value for each sample:

[0115]

[0116] in, The target value for the sample; This is the discount factor, with a typical value of 0.95; This indicates that the sample is in the state vector. The action to be performed is The expected return on the defensive effect at that time.

[0117] The loss function is the mean squared error:

[0118]

[0119] in, The loss function for the defense effectiveness evaluation branch; Represents a training batch The number of samples; This indicates that the sample is in the state vector. The action to be performed is The expected return on the defensive effect at that time.

[0120] Energy consumption predictors learn "actions". "How much energy is expected to be consumed under current conditions?" This is a regression problem, not using the Bellman equation. The target value of the energy consumption predictor is the normalized energy consumption actually measured. The loss function for training the energy-aware evaluator is expressed as:

[0121]

[0122] in, The loss function for the energy-sensing evaluator; For the true cost of energy consumption; For the energy-sensing estimator based on state vector Execution of actions Estimated energy consumption cost.

[0123] In this invention, the parameters participating in global aggregation in the local FRL-QN model of each satellite include only the network parameters of the state coding module. Network parameters of the defense effectiveness evaluation branch Each node within a cluster uploads its local gradient to the cluster head, which then aggregates and uploads it to the ground center for global model parameter updates. The update process includes:

[0124] The current cluster head node receives the global model parameters sent by the ground center and records the iteration number of the global model, denoted as . The global model parameters are distributed to satellite nodes within the cluster for training based on local data.

[0125] Before the end of a satellite update cycle, the cluster head collects the local model gradients uploaded by the M satellites within the cluster via inter-satellite links: { }, , This represents the gradient of the local model for the i-th satellite. The latest network parameters for the i-th satellite; These are the historical network parameters of the i-th satellite, typically the most recently received global network parameters;

[0126] The cluster head determines the current security reputation of each satellite. Calculate the cluster weights and perform a weighted average of the collected gradients to obtain the cluster-level gradients:

[0127]

[0128] in, This is the weighted average of the difference data for the j-th cluster. This invention uses reputation as the weight, allowing the learning experience of high-reputation nodes to have a greater weight in local aggregation, suppressing malicious or low-quality gradients that low-reputation nodes may upload, and improving the security of the global model at the source.

[0129] The ground center maintains a global model parameter And record its current version round. Because the communication windows for each cluster head are different, the ground center performs asynchronous aggregation on each arriving cluster-level gradient, specifically including:

[0130] When the cluster-level gradient is received from the cluster head of the j-th cluster At that time, based on the latest records of the j-th cluster... And the latest version of the ground center's current round Calculate the delay number of rounds that this cluster lags behind the current global model:

[0131]

[0132] in, This represents the delay round that the j-th cluster lags behind the current global model.

[0133] To reduce the interference of stale gradients on the global model, a delay decay function is introduced, then:

[0134]

[0135] in, Let be the delay decay factor for the j-th cluster; Based on the learning rate (typically 0.05). The decay exponent (typically 1.5–2.0) is used; this function monotonically decreases as the delay increases, ensuring that fresh gradients have a greater impact on the global model.

[0136] The ground center uses a weighted average to update the global model:

[0137]

[0138] in, Represents global model parameters; A left arrow indicates that the value of the parameter on the right is assigned to the parameter on the right. The baseline model version used by this cluster head when generating local gradients (usually compared to the global model in rounds). (The snapshots are consistent with the timestamps, recorded and compared by the ground center). After the update is completed, the global round... Increment by 1.

[0139] Example 1: Consensus on high-risk actions in standard scenarios is passed and executed.

[0140] This embodiment corresponds to conventional satellite-to-ground communication conditions, with ample satellite resources. High-risk actions are authorized for execution after being verified through cluster consensus. In specific implementation, such as... Figure 3 Specifically, it includes the following steps:

[0141] S1 Dynamic Clustering and Cluster Head Election: Based on real-time ephemeris data, the ground control center groups six adjacent satellites that maintain stable inter-satellite links for the next 15 minutes into a cooperative confirmation cluster. The lower limit of the cluster size (6) is determined by the Earth's occlusion geometry equation, and the upper limit (11) is constrained by the Byzantine fault-tolerant message complexity. When calculating the overall competence score of each satellite, the KL divergence weight coefficient is set to 0.5, the historical reputation weight to 0.4, and the resource score weight to 0.1. The satellite with the highest score is selected as the cluster head.

[0142] S2. Local Inference and High-Risk Action Suspension: The second satellite in the cluster suffers a distributed denial-of-service attack. The local FRL-QN model extracts alarm features and constructs a state vector from the satellite status. After passing through the shared coding layer, the defense effectiveness evaluation branch evaluates the defense returns of each action, the local energy perception estimator predicts energy consumption, and the inherent risk is queried from the action risk quantification table according to the formula. The total value is calculated, and the optimal action, "cut off the backbone laser inter-satellite link," is selected after reputation-gated modulation. The system determines that the risk value of this action exceeds the threshold and is classified as a Class A high-risk action. Therefore, it is suspended at the physical layer, and an intent vector is generated and sent to the cluster head. At the same time, a temporary action, "limit abnormal IP traffic rate to 10Mbps," is reselected from the low-risk action subset and executed immediately.

[0143] S3. Consensus Verification and Authorization: The cluster head selects three high-reputation nodes with a reputation score greater than 0.85 to form a consensus group. Each node independently evaluates the intent vector using its local model and votes on it with weighted averages (weights proportional to reputation score). If two votes pass and one vote opposes, exceeding two-thirds of the total weight, the consensus is passed. The cluster head issues an authorization command to the second satellite, and the second satellite officially executes the disconnection action. Based on this, the ground center adds one positive behavior record to the second satellite, slightly increasing its reputation score.

[0144] S4. Local Model Update: After the link break action is executed, the second satellite obtains physical feedback from the environment: benefits of secure blocking. (Attack completely blocked), service quality loss Actual energy consumption According to the formula Calculate the combined reward. (Consider the experience tuples.) The data is stored in the local experience pool. After sampling, temporal differential reinforcement learning is performed on the defense effect evaluation branch, and supervised regression is performed on the local energy perception evaluator. The action risk quantification table and reputation gating parameters are not included in the update.

[0145] S5. Asynchronous Global Aggregation: The ground center receives the aggregation gradient uploaded by the cluster head during the communication window. The model is calculated to be 3 times behind the global rounds, and the time delay decay function is substituted into the calculation. The aggregated weights are obtained, weighted and fused with the global model, and then distributed to each satellite. The satellites merge the global parameters with their locally retained parameters and proceed to the next cycle.

[0146] Example 2: High-risk actions rejected by consensus in scenarios with limited computing power

[0147] This embodiment addresses a scenario where satellite computing power is limited and communication resources are restricted. High-risk actions are rejected after verification through cluster consensus, triggering a reputation penalty. In specific implementation, such as... Figure 3 Specifically, it includes the following steps:

[0148] S1. Dynamic Clustering and Cluster Head Election: The ground control center divides eight adjacent satellites with stable inter-satellite connectivity durations greater than 20 minutes into a cooperative confirmation cluster. The KL divergence weight is set to 0.8 to prioritize the election of the node with the most normal traffic as the cluster head (due to the higher risk of intrusion when computing power is limited). The first satellite with the highest comprehensive score is elected as the cluster head.

[0149] S2. Local Inference and High-Risk Action Suspension: The third satellite in the cluster suffers a route hijacking attack, and the FRL-QN model outputs a "hard reboot of onboard critical communication transceiver" action. The system determines this action to be an irreversible Class A high-risk action, suspends it at the physical layer, and generates an intent vector to report to the cluster head; the third satellite selects "physical port logical isolation" as a temporary low-risk action and executes it immediately to curb the spread of the attack.

[0150] S3. Consensus Verification and Rejection: The cluster head selects four high-reputation nodes with a reputation score greater than 0.90 for verification. These four nodes evaluate the intent based on the current link status and traffic characteristics, with one vote in favor and three votes against. Since the opposing votes account for more than one-third of the weight, consensus is rejected. The cluster head notifies the third satellite to terminate the high-risk action and roll back the temporary action. The ground center adds one abnormal behavior record to the third satellite in the Beta distribution reputation model, significantly reducing its reputation score. This high-risk intent, since it was not actually executed, does not generate any model training experience.

[0151] S4. Local Model Update: Third Satellite Extracts Physical Feedback After Performing Temporary Low-Risk Actions: Safety Benefits (The attack was only partially suppressed), resulting in a loss of service quality. Actual energy consumption Joint Awards Experience tuples are temporary actions that are actually performed. The labels are stored in the experience pool, and the temporal differential reinforcement learning of the defense effect evaluation branch and the supervised regression update of the local energy perception evaluator are completed according to the same rules as in Example 1.

[0152] S5. Asynchronous Global Aggregation: The ground center receives the gradient of this cluster, lagging by 5 rounds, and substitutes it into the decay function. This completes asynchronous aggregation and model distribution.

[0153] Example 3: Scenario of Direct Execution of Low-Risk Actions

[0154] This embodiment addresses covert data tampering attacks. The model's output action is of low risk, requiring no suspension or consensus; it is executed autonomously on-board and updated in real-time. In specific implementation, such as... Figure 3 Specifically, it includes the following steps:

[0155] S1. Dynamic Clustering and Cluster Head Election: The ground control center divides six adjacent satellites with stable inter-satellite connectivity durations greater than 10 minutes into a cooperative confirmation cluster. The KL divergence weight is set to 0.3, and the resource and reputation weights are increased accordingly to reflect the generally high reputation of this cluster. The satellite with the highest score is elected as the cluster head.

[0156] S2. Local Inference and Direct Execution of Low-Risk Actions: The fourth satellite in the cluster detected an anomaly in the control command, and the FRL-QN model output the action of "forced asymmetric encryption key rotation". The system determined that the risk value of this action was below the threshold, belonging to the C-level low-risk action, and did not trigger intent suspension, but was directly executed physically on the satellite.

[0157] S3 Consensus Verification Skip: Skip the S3 consensus verification and reputation update process because no high-risk actions were generated.

[0158] S4, Local Model Update: Fourth Satellite Extracts Physical Feedback: Security Benefits (Tampering attack completely blocked), service quality loss. Actual energy consumption Joint Awards The experience tuples are stored in the experience pool with the actual action tags, completing the temporal differential reinforcement learning of the defense effect evaluation branch and the supervised regression update of the local energy perception evaluator.

[0159] S5. Asynchronous Global Aggregation: The ground center receives this model, lagging by only 1 round, and substitutes it into the decay function. This completes asynchronous aggregation and model distribution.

[0160] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A federated learning driven low earth orbit satellite network adaptive intrusion response control method, characterized in that, Specifically, the following steps are included: The available satellite nodes for the next time step are divided into multiple clusters, and the satellite nodes within each cluster are interconnected. Each cluster elects a cluster head node. Satellite nodes construct state vectors, which include alarm features, satellite body status, and defense history context. The state vectors are compressed into state features by a state encoding module consisting of an input layer and two fully connected hidden layers. The state features obtain the expected return of the defense effect for each candidate defense action through a defense effect evaluation branch consisting of a fully connected layer; Satellite nodes obtain the inherent risk of each defensive action based on the local action risk quantification table, and estimate the energy consumption cost of each defensive action based on the local energy perception evaluator. The total value of each action is calculated based on the defense effectiveness assessment branch, the action risk quantification table, and the energy consumption cost of the defense action. The final defense decision is generated based on the total value of the action and the current satellite reputation. If the risk of the selected defensive action exceeds the set threshold, the action is suspended, an intent vector is generated and reported to the cluster head node, and the current satellite node selects the optimal action from the low-risk action candidate set to execute. After receiving the intent vector, the cluster head selects a high-reputation node with a reputation value greater than a set threshold from within the cluster to initiate consensus. Each high-reputation node evaluates the intent vector based on its local model and votes. If more than two-thirds of the high-reputation nodes reach consensus, the cluster head node sends permission to the satellite node that generated the intent vector to execute the suspension action and increases the reputation of that satellite node. If the consensus fails, the suspension action is not allowed and the reputation of that satellite node is reduced.

2. The federated learning driven low earth orbit satellite network adaptive intrusion response control method of claim 1, wherein, The satellite node with the highest overall competence score is selected as the cluster head node. The overall competence score of the node is represented as: ； in, The overall competence score for the i-th satellite node; The traffic distribution balance score for the i-th satellite node is denoted as: , This is the mapping steepness coefficient. This represents the local network traffic distribution of the i-th satellite node. KL divergence between the distribution of all satellite network traffic Q as statistically analyzed by the ground center; The historical security reputation of the i-th satellite node; Let the available computing power and remaining energy score of the i-th satellite node be represented as: , The available computing power of the i-th satellite node. Let be the remaining power of the i-th satellite node. , The weighting coefficients for available computing power and remaining power are respectively. ; , , These are the weighting coefficients for the traffic distribution balance score, historical security reputation, available computing power, and remaining energy score, respectively. .

3. The federated learning-driven adaptive intrusion response control method for low-Earth orbit satellite networks according to claim 2, characterized in that, In normal mode, let α=0.4, β=0.4, and γ=0.2; when an increase in network attack frequency or an increase in abnormal nodes is detected, enter attack escalation mode, let β=0.6, α=0.25, and γ=0.15; in high-performance mode, let α=0.5, β=0.2, and γ=0.

3.

4. The federated learning-driven adaptive intrusion response control method for low-Earth orbit satellite networks according to claim 1, characterized in that, The total value of each action is calculated based on the defense effectiveness assessment branch, the action risk quantification table, and the energy consumption cost of the defense action, and is expressed as follows: ； in, This represents the total value of action a under state vector s; This represents the expected return of the purely defensive effect of action a under state vector s; , This is the balance coefficient; The risk coefficient for action 'a' in the action risk quantification table; This represents the estimated energy cost of action a under state vector s.

5. The federated learning-driven adaptive intrusion response control method for low-Earth orbit satellite networks according to claim 4, characterized in that, The total value is modulated using a reputation-based gating module, specifically including: ； in, The total value after modulation; As a gating unit, when the reputation value R of a satellite node is less than a set threshold and the risk coefficient of the action selected by the node is greater than the set threshold, a modulation coefficient is output to attenuate the total value, and the attenuation magnitude is proportional to the risk level; otherwise, the gating unit does not attenuate the total value.

6. A federated learning-driven adaptive intrusion response control method for low-Earth orbit satellite networks according to claim 1 or 5, characterized in that, The optimal action selected using the ε-greedy strategy is either randomly selected with a probability of ε, or the action with the highest total value is selected with a probability of 1-ε, where ε∈[0,1].

7. The federated learning-driven adaptive intrusion response control method for low-Earth orbit satellite networks according to claim 1, characterized in that, The defense effectiveness evaluation branch uses standard deep Q-learning to update network parameters, and the loss function of this branch is expressed as: ； ； ； in, The loss function for the defense effectiveness evaluation branch; Represents a training batch The number of samples; The target value for the sample; This indicates that the sample is in the state vector. The action to be performed is The expected return of the defense effect at that time, in the state vector is Execute action at time The state vector is updated to ; For the total reward function; Discount factor; This indicates that the sample is in the state vector. The actions performed are Expected return on defensive effectiveness at that time; To safely block profits, To reduce service quality This is the balancing factor for the reward function.

8. The federated learning-driven adaptive intrusion response control method for low-Earth orbit satellite networks according to claim 1, characterized in that, The loss function for training the energy-aware evaluator is expressed as: ； in, The loss function for the energy-sensing evaluator; Represents a training batch The number of samples; For the true cost of energy consumption; For the energy-sensing estimator based on state vector Execution of actions Estimated energy consumption cost.

9. A federated learning-driven adaptive intrusion response control method for low-Earth orbit satellite networks according to claim 1, characterized in that, At fixed intervals, the network parameters of the local state coding module and the defense effectiveness evaluation branch are updated through federated learning, specifically including the following steps: Satellite nodes train the network parameters of the local state coding module and the defense effect evaluation branch based on local data, and upload the difference between the trained parameters and the latest received global parameters to the cluster head; The cluster head node performs a weighted average aggregation of the difference data based on the reputation value of each satellite, and then uploads the resulting cluster-level gradient to the ground center. After receiving the cluster-level gradients of each cluster, the ground center calculates the number of iterations that the cluster-level gradient lags behind the current global model parameters, and calculates the delay decay factor based on the number of iterations. The global model parameters are updated based on the decay factor, and the updated global model parameters are then distributed to the corresponding cluster heads. The global model parameter update process includes: ； in, Represents global model parameters; A left arrow indicates that the value of the parameter on the right is assigned to the parameter on the right. Let be the delay decay factor for the j-th cluster; The baseline model version used by the j-th cluster when generating local gradients; The weighted average of the difference data for the j-th cluster is expressed as: M is the number of satellite nodes in the j-th cluster. This represents the reputation value of the i-th satellite node. This represents the difference between the parameters trained on the i-th satellite node and the latest global parameters received.

10. A federated learning-driven adaptive intrusion response control method for low-Earth orbit satellite networks according to claim 1, characterized in that, If consensus is reached, the reputation value of the satellite node that reports the intent vector to the cluster head node is increased; otherwise, the reputation value of the satellite node is decreased. If the satellite node performs the selected action, its reputation value is increased if the defense is successful, and decreased if the defense fails.

Images