A satellite network abnormal traffic detection method based on semi-supervised federated contrast learning and reinforcement learning assisted bidirectional distillation

Abstract

The present application relates to a kind of satellite network abnormal traffic detection methods based on semi-supervised federal contrast learning and reinforcement learning assisted bidirectional distillation, comprising the following steps: S1: using Barlow Twins algorithm based on local unlabeled data on satellite self-supervised learning;S2: each satellite uploads model to ground station and is aggregated;S3: ground model carries out supervised learning and reinforcement learning assisted multi-granularity distillation;S4: using ground model to satellite global model carries out reinforcement learning assisted reverse distillation;S5: by exponential moving average EMA, global model is updated;S6: global model is uploaded to satellite, and local training is carried out again.The present application improves the overall security of low earth orbit satellite network, effectively reduces the attack probability, also achieves the technical effects of reducing annotation dependence, enhancing unlabeled data representation capability, enhancing collaborative training capability, improving multi-satellite collaborative training effect, improving on-board model detection performance and improving model convergence stability.

Description

Technical Field

[0001] This invention belongs to the field of low Earth orbit satellite network security technology, specifically involving a satellite network abnormal traffic detection method based on semi-supervised federated contrastive learning and reinforcement learning-assisted bidirectional distillation. Background Technology

[0002] Satellite networks refer to a global space-based communication infrastructure system built using communication satellites distributed in Earth orbit. With the continuous advancement of 6G mobile communication technology, the future of global communication is gradually evolving towards an integrated space-air-ground network architecture. In this technological context, communication networks centered on massive low-Earth orbit (LEO) satellites are widely considered a key support for achieving seamless global connectivity. Due to their lower orbital altitude, LEO satellites have significant advantages in communication latency, cost control, and spectrum utilization, making them suitable for collaborating with terrestrial networks to build high-capacity, low-latency, and highly reliable communication systems. Currently, satellite networks not only play a crucial role in promoting the widespread adoption of the global internet but also demonstrate significant strategic value in various fields such as scientific research, national defense and security, disaster response, and intelligent transportation. Governments and technology companies worldwide are investing heavily in large-scale LEO satellite constellation construction projects, striving to gain a competitive edge in future global communications. Satellite networks have gradually become an indispensable infrastructure of the modern information society.

[0003] Compared to existing terrestrial networks, while satellite networks are highly open and dynamically changing in topology, these characteristics also expose them to higher security risks. Malicious network attacks are more likely to significantly impact the normal operation of satellite networks, making the deployment of effective malicious traffic detection systems on satellites crucial. However, the massive data volume generated by mega-constellations poses serious challenges to existing centralized processing models, including high latency in satellite-to-ground communication, limited bandwidth, and issues related to data ownership and privacy protection. With the steady improvement of satellite computing power and the strong demand for space-native intelligence, onboard federated learning has become an effective solution to address these existing technical problems. Onboard federated learning enables satellites to jointly construct a globally optimal model through local model training and parameter aggregation without exchanging local data. Based on this, each satellite node only needs to periodically upload model parameters or gradient information, avoiding the high communication overhead of downlink transmission from raw data transmission, thereby alleviating system bottlenecks caused by bandwidth limitations and communication latency. Furthermore, even if some nodes temporarily disconnect from training due to communication interruptions, the overall training process of the model will not be affected.

[0004] However, current spaceborne federated learning still faces significant limitations in addressing the problem of insufficient data labeling. Limited by the lack of ample high-quality labeled data on space, existing supervised learning methods are difficult to implement effectively. Unsupervised learning, relying solely on the intrinsic features of the data for modeling, lacks a clear optimization direction and often struggles to guarantee model accuracy, especially exhibiting unstable performance in complex environments. Therefore, how to fully leverage the advantages of the federated learning paradigm in space environments with extremely scarce labeled data is one of the key technical challenges that needs to be addressed to promote the application and widespread adoption of spaceborne intelligent technologies.

[0005] Chinese invention application CN115834145A discloses a method and system for detecting abnormal satellite traffic based on deep learning. The method includes the following steps: S1, acquiring network traffic; S2, preprocessing and standardizing the format of each generated network traffic stream to visualize it as an image; S3, inputting the network traffic image obtained in S2 into a supervised model and outputting predicted labels; S4, comparing the predicted labels output by various models with the true labels to obtain four indicators: accuracy, recall, precision, and F1 score; S5, generating adversarial examples of network traffic images using an adversarial example generation algorithm to deceive the traffic classification model; S6, using an adversarial example defense model to determine whether perturbations have been added to the network traffic images and defend against malicious traffic. This invention eliminates the need for manual intervention in designing and extracting statistical features, thus reducing labor costs and time. However, since this invention is based on a supervised learning framework, its model training and detection performance largely depend on labeled samples. For satellite network abnormal traffic detection scenarios, it is difficult and costly to obtain on-board labeled samples, making it difficult to solve the problem of data dispersion and difficulty in collaborative utilization in satellite network multi-node deployment scenarios, and unable to solve the technical problem of balancing lightweight deployment and detection performance under the condition of limited on-board resources.

[0006] Chinese invention application CN120857121A discloses a satellite traffic detection method, apparatus, device, storage medium, and program product, belonging to the field of satellite communication technology. The method includes: classifying and detecting anomalies in satellite network traffic based on an isolated forest algorithm model to obtain a first traffic detection result; the anomaly ratio of the isolated forest algorithm model is greater than the actual anomaly ratio of the satellite network, and the first traffic detection result includes a first normal traffic and a first abnormal traffic; classifying and detecting anomalies in the first abnormal traffic based on an X-means algorithm model to obtain a second normal traffic and a second abnormal traffic; the first normal traffic and the second normal traffic are normal traffic data in the satellite network traffic, and the second abnormal traffic is abnormal traffic data in the satellite network traffic. This invention improves the accuracy of satellite traffic detection, reduces the false alarm rate, solves the problem of poor robustness, and can effectively ensure the security of satellite networks. While this invention reduces reliance on labeled data, it remains essentially a traditional unsupervised anomaly detection framework. It primarily relies on anomaly scoring, distance metrics, and clustering to identify anomalies, lacking a deep representation learning mechanism. Therefore, its ability to mine high-level features from unlabeled data remains insufficient, making it difficult to address the challenges of data dispersion across multiple nodes and the difficulty in continuous collaborative model evolution in distributed satellite network scenarios. This invention only improves performance through a two-layer detection approach combining isolated forests and X-means, thus failing to address the issue of balancing lightweight deployment and detection performance under limited onboard resources. Furthermore, this invention still has shortcomings in adaptive optimization of training strategies and stable model updates. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention proposes an anomaly detection method in satellite network scenarios based on bidirectional distillation of large and small models assisted by semi-supervised federated contrastive learning and reinforcement learning. This method fully leverages the value of unlabeled on-orbit data while utilizing a limited number of samples collected and labeled by ground stations to guide and optimize the model, effectively utilizing limited label information and adapting to label-scarce environments. This invention is applicable to models training for anomaly detection in on-orbit environments, where the satellite platform lacks local labeled data, but ground stations can acquire and label a limited amount of historical data. This method aims to effectively utilize limited ground labeling resources to achieve accurate identification and efficient detection of on-board traffic anomalies.

[0008] To achieve the above objectives, the technical solution of the present invention is as follows: A method for detecting abnormal traffic in satellite networks based on semi-supervised federated contrastive learning and reinforcement learning-assisted bidirectional distillation includes the following steps: S1: The Barlow Twins algorithm is used on satellites for self-supervised learning based on local unlabeled data; S2: Each satellite uploads its model to the ground station for aggregation; S3: Supervised learning and reinforcement learning-assisted multi-granularity distillation of the ground model; S4: Reinforcement learning-assisted backdistillation of the satellite global model using the ground model; S5: Update the global model using the Exponential Moving Average (EMA); S6: The global model is uploaded to the satellite and trained locally again.

[0009] Furthermore, S1 specifically includes the following steps: S101: Each satellite performs local window jittering, adds adaptive Gaussian noise and block masking, and shuffles the channels on the local data to obtain two enhanced embeddings. The cross-correlation matrix between them is calculated according to equation (1): ; in: and These represent the two enhanced embedding batches; b Index representing a batch of samples; i and j These represent the dimensions of the network output vector; C It is a square matrix whose size is equal to the dimension of the network output, and its value is between -1 and 1, that is, between perfect negative correlation and perfect positive correlation; S102: Minimize the Barlow Twins loss function calculated by equation (2), and obtain the satellite local model after multiple rounds of local training. : ; in: It is a normal quantity used to balance the importance of the first and second terms in the loss function: It is the cross-correlation matrix output by two identical networks in the batch dimension.

[0010] Furthermore, S2 specifically includes the following steps: Each satellite sends its locally trained model to the ground station. After receiving a certain number of models, the ground station begins aggregation. The aggregation uses the FedAvg algorithm, and its calculation formula is as follows: ; in: Satellite serial number; This refers to the set of satellites participating in the training during the r-th round of training. Let r be the total amount of data in the r-th round; Let k be the amount of data for satellite k.

[0011] Furthermore, S3 specifically includes the following steps: The ground station uses a DenseNet network as a large encoder and a fully connected neural network as a classifier. Supervised learning is performed using labeled data from the server, simultaneously optimizing both the encoder and classifier. The aggregated global model distills the knowledge learned from each satellite and feeds it back to the ground station's large encoder, helping it learn feature representations of the data and overcome the challenge of insufficient data. In supervised learning, the model is typically trained by minimizing the cross-entropy loss function, as shown in the following formula: ; in: It is the number of categories; One-hot encoding of the actual label; For the model to the first The predicted probability of a class; The final loss function is as follows: ; in: This represents the cross-entropy loss calculated using supervised learning; The weights representing the supervised learning loss; The specific method for multi-granularity distillation assisted by reinforcement learning is as follows: S301: Calculating the soft-label distillation loss: In soft-label distillation, the student model learns the soft logits output by the teacher model, instead of relying solely on hard labels. The formula for calculating the soft label loss is as follows: ; in: It is a temperature parameter used to adjust the smoothness of the output distribution; Let represent the Kullback-Leibler divergence, and let represent the probability distribution of using the student network. Approximate Teacher Network Information loss caused by soft tags The larger the divergence value, the stronger the divergence. right The approximation is poor; and These represent the student model and the teacher model respectively for the first... The predicted probabilities of each category; S302: Calculate the intermediate feature distillation loss: Intermediate feature distillation forces the student model to learn features from certain intermediate layers of the teacher model, directly matching intermediate features or learning the transformation relationship between features. However, due to the differences in network structure between the teacher and student models, their corresponding intermediate features often differ in dimensionality and number of channels. Therefore, a feature adapter is introduced between the teacher and student models to transform and align the dimensions and channels of the features, thereby ensuring successful calculation of the feature distillation loss. The calculation formula is as follows: ; in: and These represent the teacher model and the student model in the input sample, respectively. Intermediate features; Indicates feature adapter transformation; Represents the L2 norm; S303: Calculating Consistency Distillation Loss: Consistency distillation first augments the input samples. The model's output for the original samples and the augmented samples should be consistent. Based on this principle, local perturbations are applied to the data samples, and slight Gaussian noise is added. Then, the teacher model and the student model are used to predict the original samples and the augmented samples, respectively. The output of the teacher model to the original samples is taken as the target output. The KL divergence between the soft logits of the outputs is calculated as the consistency score of the student model's prediction to the teacher model. By minimizing this score, the distance between the student model and the teacher model is narrowed. The calculation formula is as follows: ; in: This represents the probability distribution of the teacher model with respect to the original sample; This represents the probability distribution of the student network for augmented samples; S304: Adjust the weights among the three distillation losses calculated in steps S301, S302, and S303 using the PPO algorithm. Specifically, PPO samples a batch of data in each iteration and estimates the dominance function using the old strategy. And optimize the following objective function: ; in: This represents the probability ratio between the old and new strategies; It is a hyperparameter that controls the update magnitude; Indicates at time t The estimated value of the dominance function; The state is a vector composed of accuracy, training progress, and three types of distillation losses during training. The action is a set of weights output by the Actor. The reward is the improvement in loss and accuracy. To ensure the stability of the weights in the early stages of training, a fixed weight approach is adopted. When the model performance does not improve for several consecutive epochs during distillation, reinforcement learning is initiated to dynamically adjust the weights. The weight adjustment process is as follows: ; ; ; ; ; in: Indicates the first t The environmental conditions at the time of the next decision; This represents the current model accuracy. For training progress; and These represent the soft target distillation loss, characteristic distillation loss, and uniform distillation loss, respectively; action This represents the dynamic weights corresponding to the three types of distillation losses, where Control separately and The percentage of total losses; rewards Improvement in accuracy Change in total loss Together they constitute, among which Balance coefficients; policy function The parameter is The Actor network in state Next generation action The conditional probability distribution; and These are the mean and standard deviation of the policy network output, respectively. Standard Gaussian noise; The agent first decides whether to use random exploration or generate actions using an Actor network based on the current state. If it chooses random exploration, it generates weights using a Dirichlet distribution and records a small log probability. Otherwise, it generates action parameters through the Actor network and samples actions using a normal distribution. In addition, it calculates and saves the log probability of the action. After collecting data over a period of time, the state value function was estimated using a Critic network, and the advantage was calculated using generalized advantage estimation, as shown in the following formula: ; in: It is the TD error; It is a discount factor; These are GAE parameters; It is a state value function; Then, Equation (9) is used to update the Actor network; The Critic network updates its state value estimation by minimizing the mean squared error between the predicted value and the actual return. The loss function is as follows: ; in: It is a prediction of the value of the current state; For actual returns; Following the steps above, data is continuously collected during the training loop, the RL agent is updated periodically, the loss weights are dynamically optimized, and the model training effect is improved. S305: Calculate the final distillation loss: Weight the three distillation losses mentioned above to obtain the final distillation loss. The calculation formula is as follows: ; in: These represent the weights corresponding to the soft target distillation loss, feature distillation loss, and consistency distillation loss, respectively.

[0012] Furthermore, the specific method of S4 is as follows: Knowledge distillation is used again to back-transfer the knowledge learned by the large ground station model to the global model, ensuring the lightweight and high performance of the detection model. To achieve this, this invention will stitch a small classifier into the satellite global model, and then use labeled data to perform supervised learning and reinforcement learning-assisted multi-granularity knowledge distillation on the stitched model. The calculation process is the same as described in S3.

[0013] Furthermore, S5 specifically includes the following steps: After model training is complete, the global model is updated using the Exponential Moving Average (EMA). A weighted average is taken between the current global model and the EMA model from the previous round. This method allows the model weights to decrease exponentially over time, with more recent models receiving larger weights, while older models are also given some weight. This helps to retain historical information about the model, preventing drastic oscillations during training and improving the stability of model convergence. The model update formula is as follows: ; in: For the current round's EMA model; This is the global model for the current round; This is the EMA model from the previous round.

[0014] Furthermore, S6 specifically includes the following steps: the updated model will be sent to the satellite via the uplink for the next round of training until the model converges, ultimately realizing a lightweight semi-supervised abnormal traffic detection model of satellite-ground co-evolution.

[0015] Compared with existing deep learning-based methods and systems for detecting abnormal satellite traffic, the advantages of this invention are: First, this invention employs the Barlow Twins algorithm on the satellite side, performing self-supervised learning based on local unlabeled data. This reduces the reliance on manually labeled samples and improves the utilization of unlabeled data, effectively solving the problem of strong label dependence in existing deep learning-based satellite abnormal traffic detection methods and systems.

[0016] Secondly, this invention uploads local models from each satellite to the ground station, aggregates them using FedAvg, and then redistributes the updated global model to the satellites for further training, thus constructing a complete satellite-ground collaborative closed-loop training mechanism. This effectively improves the collaborative utilization capability of multi-satellite distributed data and the generalization performance of the global model.

[0017] Furthermore, this invention introduces supervised learning and reinforcement learning-assisted multi-granularity distillation on the ground side, and further employs back-distillation to transfer the knowledge of the high-performance ground model to the global model on the satellite, thereby balancing lightweight model deployment and detection performance under the condition of limited satellite resources.

[0018] Finally, this invention updates the global model through EMA, which can retain historical information while absorbing the current round of training results, reducing training oscillations and improving the model's convergence stability.

[0019] In summary, compared with existing deep learning-based satellite anomaly traffic detection methods and systems, this invention not only solves the shortcomings of existing technologies in terms of label scarcity adaptability, satellite-ground collaborative training, heterogeneous model knowledge transfer, and training stability, but also achieves technical effects such as reducing label dependence, enhancing collaborative training capabilities, improving on-board model detection performance, and improving model convergence stability.

[0020] Compared with existing satellite traffic detection methods, devices, equipment, storage media, and program products, the advantages of this invention are: First, this invention employs the Barlow Twins algorithm on the satellite side to perform self-supervised learning on local unlabeled data, which can obtain more discriminative deep feature representations, thereby making up for the shortcomings of existing satellite traffic detection methods that rely solely on traditional anomaly detection and cluster analysis.

[0021] Secondly, this invention uploads local models from each satellite to the ground station for FedAvg aggregation, and after the EMA is updated, the global model is distributed to the satellites again for continued training, forming a complete satellite-ground collaborative training closed loop, thereby improving the collaborative training capability of multi-satellite data and the continuous evolution capability of the model.

[0022] Furthermore, this invention introduces supervised learning and reinforcement learning-assisted multi-granularity distillation on the ground side, and further implements back-distillation, so that the knowledge of the high-performance ground model can be effectively transferred to the satellite model, thereby improving the detection performance of the lightweight satellite model.

[0023] Finally, this invention uses EMA to smoothly update the global model, which can reduce fluctuations during training and improve the model's convergence stability; at the same time, combined with reinforcement learning-assisted distillation, it further enhances the adaptability of the training process.

[0024] In summary, compared with existing satellite traffic detection methods, devices, equipment, storage media, and program products, this invention not only solves the shortcomings of existing technical solutions in deep representation learning, satellite-ground collaborative training, heterogeneous model knowledge transfer, and training adaptive optimization and stability, but also achieves technical effects such as enhancing unlabeled data representation capabilities, improving multi-satellite collaborative training effects, improving on-board model detection performance, and enhancing model convergence stability. Attached Figure Description

[0025] Figure 1 This is a flowchart of a preferred embodiment of the satellite network abnormal traffic detection method based on semi-supervised federated contrastive learning and reinforcement learning-assisted bidirectional distillation of the present invention; Figure 2 This is a flowchart of a preferred embodiment of the satellite network anomaly traffic detection method based on semi-supervised federated contrastive learning and reinforcement learning-assisted bidirectional distillation, which is a flowchart of a multi-granularity distillation method assisted by reinforcement learning. Figure 3 This is a schematic diagram of the semi-supervised federated learning principle based on contrastive learning in a preferred embodiment of the satellite network abnormal traffic detection method based on semi-supervised federated contrastive learning and reinforcement learning-assisted bidirectional distillation of the present invention. Figure 4 This is a schematic diagram of the principle of the reinforcement learning-assisted multi-granularity distillation method in a preferred embodiment of the satellite network abnormal traffic detection method based on semi-supervised federated contrastive learning and reinforcement learning-assisted bidirectional distillation of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0027] A method for detecting abnormal traffic in satellite networks based on semi-supervised federated contrastive learning and reinforcement learning-assisted bidirectional distillation, such as... Figure 1 As shown, it includes the following steps: S1: The Barlow Twins algorithm is used on satellites for self-supervised learning based on local unlabeled data; S101: Each satellite performs local window jittering, adds adaptive Gaussian noise and block masking, and shuffles the channels on the local data to obtain two enhanced embeddings. The cross-correlation matrix between them is calculated according to equation (1): ; in: and These represent the two enhanced embedding batches; Index representing a batch of samples; and These represent the dimensions of the network output vector; It is a square matrix whose size is equal to the dimension of the network output, and its value is between -1 and 1, that is, between perfect negative correlation and perfect positive correlation; S102: Minimize the Barlow Twins loss function calculated by equation (2), and obtain the satellite local model after multiple rounds of local training. , ; in: It is a normal quantity used to balance the importance of the first and second terms in the loss function; It is the cross-correlation matrix of the outputs of two identical networks in the batch dimension; S2: Each satellite uploads its locally trained model to the ground station for aggregation: Each satellite sends its locally trained model to the ground station. After receiving a certain number of models, the ground station begins aggregation. The aggregation uses the FedAvg algorithm, and its calculation formula is as follows: ; in: Satellite serial number; This refers to the set of satellites participating in the training during the r-th round of training. Let r be the total amount of data in the r-th round; The amount of data for satellite k; S3: As Figure 2 , Figure 3 , Figure 4As shown, the ground model undergoes supervised learning and reinforcement learning-assisted multi-granularity distillation: the ground station uses a DenseNet network as a large encoder and a fully connected neural network as a classifier. Supervised learning is performed using labeled data from the server, simultaneously optimizing both the encoder and classifier. The aggregated global model distills the knowledge learned from each satellite into the ground station's large encoder, helping it learn feature representations of the data and overcome the data shortage problem. In supervised learning, the model is typically trained by minimizing the cross-entropy loss function, as shown in the following formula: ; in: It is the number of categories; One-hot encoding of the actual label; For the model to the first The predicted probability of a class; The final loss function is as follows: ; in: This represents the cross-entropy loss calculated using supervised learning; The weights represent the supervised learning loss; The specific method for multi-granularity distillation assisted by reinforcement learning is as follows: S301: Calculating the soft-label distillation loss: In soft-label distillation, the student model learns the soft logits output by the teacher model, instead of relying solely on hard labels. The formula for calculating the soft label loss is as follows: ; in: It is a temperature parameter used to adjust the smoothness of the output distribution; Let represent the Kullback-Leibler divergence, and let represent the probability distribution of using the student network. Approximate Teacher Network Information loss caused by soft tags The larger the divergence value, the stronger the divergence. right The approximation is poor; and These represent the student model and the teacher model respectively for the first... The predicted probabilities of each category; S302: Calculate the intermediate feature distillation loss: Intermediate feature distillation forces the student model to learn features from certain intermediate layers of the teacher model, directly matching intermediate features or learning the transformation relationship between features. However, due to the differences in network structure between the teacher and student models, their corresponding intermediate features often differ in dimensionality and number of channels. Therefore, a feature adapter is introduced between the teacher and student models to transform and align the dimensions and channels of the features, thereby ensuring successful calculation of the feature distillation loss. The calculation formula is as follows: ; in: and These represent the teacher model and the student model in the input samples, respectively. Intermediate features; Indicates feature adapter transformation; Represents the L2 norm; S303: Calculating Consistency Distillation Loss: Consistency distillation first augments the input samples. The model's output for the original samples and the augmented samples should be consistent. Based on this principle, local perturbations are applied to the data samples, and slight Gaussian noise is added. Then, the teacher model and the student model are used to predict the original samples and the augmented samples, respectively. The output of the teacher model to the original samples is taken as the target output. The KL divergence between the soft logits of the outputs is calculated as the consistency score of the student model's prediction to the teacher model. By minimizing this score, the distance between the student model and the teacher model is narrowed. The calculation formula is as follows: ; in: This represents the probability distribution of the teacher model with respect to the original sample; This represents the probability distribution of the student network for augmented samples; S304: The PPO algorithm is used to adjust the weights among the three distillation losses calculated in S301, S302, and S303. Specifically, PPO samples a batch of data in each iteration and estimates the dominance function using the old strategy. And optimize the following objective function: ; in: This represents the probability ratio between the old and new strategies; It is a hyperparameter that controls the update magnitude; Indicates at time t The estimated value of the dominance function; The state is a vector composed of accuracy, training progress, and three types of distillation losses during training. The action is a set of weights output by the Actor. The reward is the improvement in loss and accuracy. To ensure the stability of the weights in the early stages of training, a fixed weight approach is adopted. When the model performance does not improve for several consecutive epochs during distillation, reinforcement learning is initiated to dynamically adjust the weights. The weight adjustment process is as follows: ; ; ; ; ; in: Indicates the first t The environmental conditions at the time of the next decision; This represents the current model accuracy. For training progress; and These represent the soft target distillation loss, characteristic distillation loss, and uniform distillation loss, respectively; action This represents the dynamic weights corresponding to the three types of distillation losses, where Control separately and The percentage of total losses; rewards Improvement in accuracy Change in total loss Together they constitute, among which Balance coefficients; policy function The parameter is The Actor network in state Next generation action The conditional probability distribution; and These are the mean and standard deviation of the policy network output, respectively. Standard Gaussian noise; The agent first decides whether to use random exploration or generate actions using an Actor network based on the current state. If it chooses random exploration, it generates weights using a Dirichlet distribution and records a small log probability. Otherwise, it generates action parameters through the Actor network and samples actions using a normal distribution. In addition, it calculates and saves the log probability of the action. After collecting data over a period of time, the state value function was estimated using a Critic network, and the advantage was calculated using generalized advantage estimation, as shown in the following formula: ; in: It is the TD error; It is a discount factor; These are GAE parameters; It is a state value function; Then, Equation (9) is used to update the Actor network; The Critic network updates its state value estimation by minimizing the mean squared error between the predicted value and the actual return. The loss function is as follows: ; in: It is a prediction of the value of the current state; For actual returns; Following the steps above, data is continuously collected during the training loop, the RL agent is updated periodically, the loss weights are dynamically optimized, and the model training effect is improved. S305: Calculate the final distillation loss: Weight the three distillation losses mentioned above to obtain the final distillation loss. The calculation formula is as follows: ; in: These represent the weights corresponding to the soft target distillation loss, feature distillation loss, and consistency distillation loss, respectively. S4: Back-distillation of the satellite global model with reinforcement learning assistance using the ground model: Knowledge distillation technology is used again to back-transfer the knowledge learned by the large ground station model to the global model, ensuring the lightweight and high performance of the detection model. To achieve this, a small classifier will be spliced ​​into the satellite global model, and then the labeled data will be used to perform supervised learning and reinforcement learning-assisted multi-granularity knowledge distillation on the spliced ​​model. The calculation process is the same as that in S3. S5: Update the global model: After the model training is complete, update the global model using the Exponential Moving Average (EMA). This involves a weighted average of the current global model and the previous EMA model. This method allows the model weights to decrease exponentially over time, with more recent models receiving larger weights, while older models are also given some weight. This helps to retain historical information about the model, preventing drastic oscillations during training and improving the stability of model convergence. The formula for updating the model is as follows: ; in: For the current round's EMA model; This is the global model for the current round; This is the EMA model from the previous round; S6: Global model uploaded to satellite for local training again: The updated model will be sent to the satellite via uplink for the next round of training until the model converges, ultimately realizing a lightweight semi-supervised abnormal traffic detection model that achieves satellite-ground co-evolution.

[0028] Compared with existing technologies, this invention not only solves the shortcomings of existing technical solutions in terms of label scarcity adaptability, deep representation learning, satellite-ground collaborative training, heterogeneous model knowledge transfer, and training adaptive optimization and stability, but also achieves technical effects such as reducing label dependence, enhancing the ability to represent unlabeled data, enhancing collaborative training capabilities, improving the effect of multi-satellite collaborative training, improving the detection performance of on-board models, and improving the convergence stability of models.

Claims

1. A method for detecting abnormal traffic in satellite networks based on semi-supervised federated contrastive learning and reinforcement learning-assisted bidirectional distillation, characterized in that, Includes the following steps: S1: The Barlow Twins algorithm is used on satellites for self-supervised learning based on local unlabeled data; S2: Each satellite uploads its model to the ground station for aggregation; S3: Supervised learning and reinforcement learning-assisted multi-granularity distillation of the ground model; S4: Reinforcement learning-assisted backdistillation of the satellite global model using the ground model; S5: Update the global model using the Exponential Moving Average (EMA); S6: The global model is uploaded to the satellite and trained locally again.

2. The satellite network anomaly traffic detection method based on semi-supervised federated contrastive learning and reinforcement learning-assisted bidirectional distillation as described in claim 1, characterized in that, S1 specifically includes the following steps: S101: Each satellite performs local window jittering, adds adaptive Gaussian noise and block masking, and shuffles the channels on the local data to obtain two enhanced embeddings. The cross-correlation matrix between them is calculated according to equation (1): ； in: and These represent the two enhanced embedding batches; b Index representing a batch of samples; i and j These represent the dimensions of the network output vector; C It is a square matrix whose size is equal to the dimension of the network output, and its value is between -1 and 1, that is, between perfect negative correlation and perfect positive correlation; S102: Minimize the Barlow Twins loss function calculated by equation (2), and obtain the satellite local model after multiple rounds of local training. : ； in: It is a normal quantity used to balance the importance of the first and second terms in the loss function; It is the cross-correlation matrix output by two identical networks in the batch dimension.

3. The satellite network anomaly traffic detection method based on semi-supervised federated contrastive learning and reinforcement learning-assisted bidirectional distillation according to claim 1, characterized in that, S2 specifically includes the following steps: Each satellite sends its locally trained model to the ground station. After receiving a certain number of models, the ground station begins aggregation. The aggregation uses the FedAvg algorithm, and its calculation formula is as follows: ； in: Satellite serial number; This refers to the set of satellites participating in the training during the r-th round of training. Let r be the total amount of data in the r-th round; Let k be the amount of data for satellite k.

4. The satellite network anomaly traffic detection method based on semi-supervised federated contrastive learning and reinforcement learning-assisted bidirectional distillation as described in claim 1, characterized in that, S3 specifically includes the following steps: The ground station uses a DenseNet network as a large encoder and a fully connected neural network as a classifier. Supervised learning is performed using labeled data from the server, simultaneously optimizing both the encoder and classifier. The aggregated global model distills the knowledge learned from each satellite and feeds it back to the ground station's large encoder, helping it learn feature representations of the data and overcome the challenge of insufficient data. In supervised learning, the model is typically trained by minimizing the cross-entropy loss function, as shown in the following formula: ； in: It is the number of categories; One-hot encoding of the actual label; For the model to the first The predicted probability of a class; The final loss function is as follows: ； in: This represents the cross-entropy loss calculated using supervised learning; The weights representing the supervised learning loss; The specific method for multi-granularity distillation assisted by reinforcement learning is as follows: S301: Calculating the soft-label distillation loss: In soft-label distillation, the student model learns the softlogits output by the teacher model, instead of relying solely on hard labels. The formula for calculating the soft label loss is as follows: ； in: It is a temperature parameter used to adjust the smoothness of the output distribution; Let represent the Kullback-Leibler divergence, and let represent the probability distribution of using the student network. Approximate Teacher Network Information loss caused by soft tags The larger the divergence value, the stronger the divergence. right The approximation is poor; and These represent the student model and the teacher model respectively for the first... The predicted probabilities of each category; S302: Calculate the intermediate feature distillation loss: Intermediate feature distillation forces the student model to learn features from certain intermediate layers of the teacher model, directly matching intermediate features or learning the transformation relationship between features. However, due to the differences in network structure between the teacher and student models, their corresponding intermediate features often differ in dimensionality and number of channels. Therefore, a feature adapter is introduced between the teacher and student models to transform and align the dimensions and channels of the features, thereby ensuring successful calculation of the feature distillation loss. The calculation formula is as follows: ； in: and These represent the teacher model and the student model in the input sample, respectively. Intermediate features; Indicates feature adapter transformation; Represents the L2 norm; S303: Calculating Consistency Distillation Loss: Consistency distillation first augments the input samples. The model's output for the original samples and the augmented samples should be consistent. Based on this principle, local perturbations are applied to the data samples, and slight Gaussian noise is added. Then, the teacher model and the student model are used to predict the original samples and the augmented samples, respectively. The output of the teacher model to the original samples is taken as the target output. The KL divergence between the soft logits of the outputs is calculated as the consistency score of the student model's prediction to the teacher model. By minimizing this score, the distance between the student model and the teacher model is narrowed. The calculation formula is as follows: ； in: This represents the probability distribution of the teacher model with respect to the original sample; This represents the probability distribution of the student network for augmented samples; S304: The PPO algorithm is used to adjust the weights among the three distillation losses calculated in S301, S302, and S303. Specifically, PPO samples a batch of data in each iteration and estimates the dominance function using the old strategy. And optimize the following objective function: ； in: This represents the probability ratio between the old and new strategies; It is a hyperparameter that controls the update magnitude; Indicates at time t The estimated value of the dominance function; The state is a vector composed of accuracy, training progress, and three types of distillation losses during training. The action is a set of weights output by the Actor. The reward is the improvement in loss and accuracy. To ensure the stability of the weights in the early stages of training, a fixed weight approach is adopted. When the model performance does not improve for several consecutive epochs during distillation, reinforcement learning is initiated to dynamically adjust the weights. The weight adjustment process is as follows: ； ； ； ； ； in: Indicates the first t The environmental conditions at the time of the next decision; This represents the current model accuracy. For training progress; and These represent the soft target distillation loss, characteristic distillation loss, and uniform distillation loss, respectively; action This represents the dynamic weights corresponding to the three types of distillation losses, where Control separately and The percentage of total losses; rewards Improvement in accuracy Change in total loss Together they constitute, among which Balance coefficients; policy function The parameter is The Actor network in state Next generation action The conditional probability distribution; and These are the mean and standard deviation of the policy network output, respectively. Standard Gaussian noise; The agent first decides whether to use random exploration or generate actions using an Actor network based on the current state. If it chooses random exploration, it generates weights using a Dirichlet distribution and records a small log probability. Otherwise, it generates action parameters through the Actor network and samples actions using a normal distribution. In addition, it calculates and saves the log probability of the action. After collecting data over a period of time, the state value function was estimated using a Critic network, and the advantage was calculated using generalized advantage estimation, as shown in the following formula: ； in, It is the TD error; It is a discount factor; These are GAE parameters; It is a state value function; Then, Equation (9) is used to update the Actor network; The Critic network updates its state value estimation by minimizing the mean squared error between the predicted value and the actual return. The loss function is as follows: ； in: It is a prediction of the value of the current state; For actual returns Following the steps above, data is continuously collected during the training loop, the RL agent is updated periodically, the loss weights are dynamically optimized, and the model training effect is improved. S305: Calculate the final distillation loss: Weight the three distillation losses mentioned above to obtain the final distillation loss. The calculation formula is as follows: ； in: These represent the weights corresponding to the soft target distillation loss, feature distillation loss, and consistency distillation loss, respectively.

5. The satellite network anomaly traffic detection method based on semi-supervised federated contrastive learning and reinforcement learning-assisted bidirectional distillation according to claim 1, characterized in that, The specific method of S4 is as follows: Knowledge distillation technology is used again to back-transfer the knowledge learned by the large ground station model to the global model, ensuring the lightweight and high performance of the detection model. To achieve this, a small classifier will be spliced ​​into the satellite global model, and then the spliced ​​model will be subjected to supervised learning and reinforcement learning-assisted multi-granularity knowledge distillation using labeled data. The calculation process is the same as described in S3.

6. The satellite network anomaly traffic detection method based on semi-supervised federated contrastive learning and reinforcement learning-assisted bidirectional distillation according to claim 1, characterized in that, S5 specifically includes the following steps: After the model training is complete, the global model is updated using the Exponential Moving Average (EMA). A weighted average is taken between the current global model and the EMA model from the previous round. This method allows the model weights to decrease exponentially over time, with more recent models receiving larger weights, while older models are also given some weight. This helps to retain historical information about the model to some extent, preventing drastic oscillations during training and improving the stability of model convergence. The model update formula is as follows: ； in, For the current round's EMA model; This is the global model for the current round; This is the EMA model from the previous round.

7. The satellite network anomaly traffic detection method based on semi-supervised federated contrastive learning and reinforcement learning-assisted bidirectional distillation according to claim 1, characterized in that, S6 specifically includes the following steps: the updated model will be sent to the satellite via the uplink for the next round of training until the model converges, and finally a lightweight semi-supervised abnormal traffic detection model of satellite-ground co-evolution will be realized.

Images