Privacy enhancement federated learning method for satellite network intrusion detection

Abstract

The invention provides a privacy enhancement federated learning method for satellite network intrusion detection, and aims to improve data privacy protection and security in a satellite network. According to the method, a differential privacy protection mechanism and a federated distillation framework are organically combined, so that the privacy protection capability and the detection performance of the model are remarkably improved. According to the distributed architecture characteristics of the satellite network, the method adopts a personalized local differential privacy technology, node data are effectively protected before data are uploaded, and the risk of data leakage is remarkably reduced. And in combination with a bidirectional knowledge distillation mechanism of the conditional generative adversarial network, efficient knowledge transmission between the server side and the satellite nodes is realized, and the performance of the global model and the local model is optimized. The method achieves good balance among privacy protection, communication resource utilization and model performance, is suitable for intrusion detection tasks in a complex satellite network environment, and provides an innovative solution for improving the security of a satellite network.

Description

Technical Field

[0001] The present invention relates to the field of satellite network security detection, and in particular to a privacy-enhanced federated learning method for satellite network intrusion detection. Background Art

[0002] With the rapid development of satellite networks, their importance in the fields of communication, navigation and remote sensing monitoring has become increasingly prominent. With its wide-area coverage, flexible deployment and all-weather service capabilities, satellite networks have become an important part of supporting global critical infrastructure. However, due to its special distributed structure, open communication environment and complex interaction with ground networks, satellite networks are facing increasingly severe network intrusion threats, such as signal interference, data tampering, malicious hijacking and denial of service attacks, which will not only lead to service interruptions, but also may cause serious economic losses and security threats.

[0003] Traditional centralized intrusion detection systems rely on uploading data to a central server for analysis and modeling. This approach faces multiple challenges in a satellite network environment. First, data in satellite networks often contain sensitive information, and centralized storage and processing of centralized methods may lead to privacy leakage risks. Second, the communication links of satellite networks usually have high latency and low bandwidth, especially in geosynchronous orbit or deep space missions. Uploading large-scale data will further increase the communication burden. In addition, the single point failure problem of the centralized architecture may cause detection failure at critical moments and cannot adapt to dynamic and complex network attack scenarios.

[0004] In recent years, federated learning has gradually become an important research direction for solving satellite network intrusion detection problems due to its distributed computing and privacy protection characteristics. Federated learning allows clients (such as satellites or ground stations) to train models locally and collaborate by uploading model parameters or gradients, thereby avoiding the centralized storage of raw data. However, existing federated learning methods still face many challenges in practical applications. Federated learning based on differential privacy requires a reasonable allocation of privacy budgets, but static allocation strategies are often difficult to adapt to the dynamic changes in task requirements, resulting in an insufficient balance between privacy protection and model performance. In addition, data distribution in satellite networks may be heterogeneous, and local data characteristics of satellite nodes may vary, which will affect the training effect and convergence speed of the model. At the same time, federated learning relies on frequent model updates, uploads, and downloads, but existing methods fail to effectively optimize communication protocols and data volumes, especially in satellite network environments with limited bandwidth resources. This problem is more prominent. Finally, the high heterogeneity of satellite networks makes the computing power, data distribution, and communication conditions of different nodes significantly different, which puts higher requirements on the stability and performance of federated learning.

[0005] Therefore, the present invention proposes a privacy-enhanced federated learning method for satellite network intrusion detection, which can adaptively optimize the privacy budget and efficiently utilize communication resources. Summary of the invention

[0006] The present invention proposes a privacy-enhanced federated learning method for satellite network intrusion detection. The technical solution adopted includes the following steps:

[0007] Step 1: Parameter initialization and data preprocessing. The server node negotiates with each satellite node to determine their specific requirements for privacy protection, initializes privacy parameters for each satellite node, and each node collects network data. During data transmission, each node independently applies local differential privacy to its data according to the privacy budget.

[0008] Step 2: Use the satellite federated distillation framework to train the satellite network intrusion detection model. The framework is based on an adaptive weight mechanism. The global model of the server node is used as the teacher model, and the local model of the satellite node is used as the student model. Bidirectional knowledge distillation is performed, and the teacher model and the student model transfer knowledge to each other.

[0009] Step 3: Each node applies personalized local differential privacy to evaluate the performance of the global model and the local model to ensure that they meet the accuracy requirements of the model. The privacy budget of each node is dynamically adjusted according to the model performance and privacy protection effect.

[0010] Step 4: Repeat the loop until the model meets the predetermined performance criteria or reaches the predetermined number of training rounds;

[0011] Furthermore, the model deployment in step 2 is characterized by:

[0012] The network intrusion detection model deployed on servers and satellite nodes is a hybrid model that integrates text convolutional neural network and long short-term memory neural network to capture local features and temporal information at the same time;

[0013] Text convolutional neural network mainly extracts local features from the input through convolution operation. Given the input network traffic data, it is converted into a word vector matrix X = [x1, x2, ..., x n ], where each x i For the embedding vector of a word, use the convolution kernel W k Perform convolution operation on the matrix to obtain the feature h tcnn , the formula can be expressed as:

[0014]

[0015] Among them, W k is the convolution kernel, b k is the bias, h tcnnrepresents the local features obtained from the convolution operation, and ReLU(·) represents the linear rectification function;

[0016] Using long short-term memory neural network to process text convolutional neural network extracted features h tcnn , to capture the temporal dependencies of network traffic data and transform the feature output h of the text convolutional neural network tcnn As its input, the LSTM neural network updates its hidden state h at time t t and memory unit C t , the formula can be expressed as:

[0017]

[0018] Among them, h t is the hidden state at the tth moment, C t is the memory unit at the tth moment, o t is the output gate, f i It is the forget gate, t is the input gate, is a candidate memory unit, tanh(·) is the hyperbolic tangent function, and T is the last moment of the input sequence;

[0019] By fusing the local features extracted by the text convolutional neural network with the hidden state of the time series information captured by the long short-term memory neural network at time T, the final classification result o can be obtained, which can be expressed as:

[0020] o=softmax(W f ·[h tcnn ,h T ]+b f ) (3)

[0021] Among them, h T is the hidden state of the LSTM neural network at the final moment, W f is the fusion weight matrix, b f is the bias term, and softmax(·) is the softmax function.

[0022] Furthermore, the personalized local differential privacy method in step 3 is characterized by:

[0023] Personalized local differential privacy allows a certain degree of data analysis and utilization while protecting the privacy of personal data. In personalized local differential privacy, each participating node can have an independent privacy budget instead of uniformly using global privacy parameters.

[0024] When initializing the parameters, the initial privacy budget ε0 and the expected threshold of loss L are set for each satellite node. e, the initial privacy budget ε0 is the initial privacy protection parameter, which is usually small to ensure strong privacy, L e is the expected loss value, used to calculate the feedback function;

[0025] Dynamically adjust the privacy budget ε of the kth satellite node through a feedback mechanism k , where the feedback function is the core of the privacy budget adjustment, by calculating the current loss L t and the expected loss L e The privacy budget is updated by the difference, and the feedback function can be expressed as:

[0026]

[0027] Among them, L t is the model loss of the tth round of training, L e is the desired target loss value, η is the parameter used to control the feedback adjustment amplitude;

[0028] According to the feedback function fb(L t , L e ), dynamically adjust the local privacy budget at time t+1 The formula can be expressed as:

[0029]

[0030] in, is the local privacy budget of the kth satellite node in the tth round of training, and K is the number of satellite nodes.

[0031] Furthermore, the federated distillation method based on the adaptive weight mechanism in step 2 is characterized by:

[0032] A conditional generative adversarial network model is trained on the server side. This model is used to generate training samples that can enhance the local data of the node. The trained model consists of a generator G and a discriminator D. The loss function L for the training of the generator G is G , the formula can be expressed as:

[0033]

[0034] Among them, G(z|c) is the output of the generator under the given condition c, D(G(z|c)) is the output of the discriminator under the given condition information c, z is a random noise vector and z obeys the probability distribution p z (z);

[0035] The discriminator D maximizes the loss L of the discriminator by distinguishing between real samples and generated samples. D , the formula can be expressed as:

[0036]

[0037] Among them, D(d|c) is the probability that the discriminator judges the real sample under the given condition c, d is the real sample and d follows the data distribution p data (d);

[0038] The server sends the trained generator G to each node. After receiving the generator, each node optimizes the local data to obtain the localized generator G′, and combines the enhanced samples generated by the generator with the local data to perform knowledge distillation and model training. The adaptive weight mechanism is used during distillation.

[0039] Before knowledge distillation, each node is evaluated to determine its contribution to the model by evaluating the label quality. The local soft label quality score r of the kth satellite node is k Calculate the weight ω of its local soft label k , local soft label quality score r k According to the local soft label p of node k k，i And the global soft label obtained Definition, its formula can be expressed as:

[0040]

[0041] in, It is expressed as the L2 norm between the local soft label and the global soft label, is the L2 norm of the global soft label, used for normalization, and N is the number of samples;

[0042] The global soft label p used by the server i It is obtained by weighted calculation of all local soft labels received from satellite nodes, and its formula can be expressed as:

[0043]

[0044] Among them, ω m is the global soft label p i The weight of the mth satellite node in can be expressed as:

[0045]

[0046] The kth satellite node obtains the global soft label from the server It is calculated by calculating the local soft labels of other satellite nodes and the local soft labels p′ of the server node model i The weighted calculation is obtained, and the formula can be expressed as:

[0047]

[0048] Among them, ω′ mGlobal soft label The weight of the mth satellite node, ω′0 is the weight of the local soft label of the server node model, and its formula can be expressed as:

[0049]

[0050] Among them, r0 is the quality score of the local soft label of the server node model;

[0051] Each node receives the global soft label and performs knowledge distillation on the public dataset and enhanced dataset according to the global soft label. The distillation loss function between the teacher model and the student model can be expressed as:

[0052]

[0053] Where T is the temperature parameter, C is the number of categories, is the global soft label of the i-th sample of the student model for classification category j on the public dataset and the enhanced dataset, is the predicted output of the student model for the i-th sample on the public dataset and the enhanced dataset for the classification category j;

[0054] The prediction loss function of the student model can be expressed as:

[0055]

[0056] Among them, onehot(·) is the hot encoding function, is the correct label corresponding to the i-th sample in the public dataset and the enhanced dataset;

[0057] Therefore, the loss of training the student model can be expressed as:

[0058] L=αL pl +(1-α)L kl (15)

[0059] Among them, the parameter α is the balance of the student model distillation loss L pl With the prediction loss L kl Parameters.

[0060] Furthermore, the loop in step 4 is iterated until the model meets a predetermined performance standard or reaches a predetermined number of training rounds.

[0061] During this process, it is necessary to dynamically monitor the performance indicators of the global model (such as accuracy and recall) and the consumption of communication resources. If the model performance improvement tends to be stable after multiple iterations and the resource consumption reaches the preset threshold, the training can be terminated in advance to improve the overall efficiency. In addition, considering the limitations of communication bandwidth and computing power, the data transmission strategy and local computing load can be optimized to improve the efficiency of the training process, thereby promoting more efficient collaborative learning.

[0062] The beneficial effects of the present invention include the following three aspects: First, through the personalized local differential privacy mechanism, combined with the dynamic feedback adjustment strategy, the privacy budget is flexibly allocated according to the personalized needs of the nodes, thereby maintaining the efficiency of the model while enhancing data privacy protection. Secondly, the intrusion detection performance is improved. A fusion model based on text convolutional neural network and long short-term memory network is adopted, and combined with two-way knowledge distillation technology, the comprehensive capture of local features and timing information is achieved, and the detection accuracy is significantly improved. In the process of global soft label aggregation, a node weight allocation mechanism is introduced, and high-performance nodes are given higher weights, while the influence of low-performance nodes is appropriately weakened, thereby further improving the convergence effect and overall performance of the model. In addition, the use of communication resources is optimized. By generating enhanced samples through global soft label aggregation and conditional generative adversarial networks, the amount of data transmission is greatly reduced, while the quality of model training is guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is an overall flow chart of the privacy-enhanced federated learning method for satellite network intrusion detection of the present invention.

[0064] Figure 2 This is the personalized differential privacy flow chart of the present invention.

[0065] Figure 3 It is the federal distillation flow chart of the present invention.

[0066] Figure 4 It is a distillation framework diagram of the present invention.

[0067] Figure 5 It is a schematic diagram of the overall framework of the present invention.

[0068] Figure 6 This is a comparison chart of the training accuracy of the model of the present invention.

[0069] Figure 7 It is a comparison chart of the training loss values ​​of the model of the present invention. DETAILED DESCRIPTION

[0070] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0071] The present invention provides a privacy-enhanced federated learning method for satellite network intrusion detection. The present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0072] See also Figure 1 , is a flowchart of a privacy-enhanced federated learning method for satellite network intrusion detection provided by an embodiment of the present invention. Figures 1 to 5 As shown, the method mainly comprises the following steps:

[0073] Step 1: Parameter initialization and data preprocessing. The server node negotiates with each satellite node to determine their specific requirements for privacy protection, initializes privacy parameters for each satellite node, and each node collects network data. During data transmission, each node independently applies local differential privacy to its data according to the privacy budget.

[0074] Step 2: Use the satellite federated distillation framework to train the satellite network intrusion detection model. The framework is based on an adaptive weight mechanism. The global model of the server node is used as the teacher model, and the local model of the satellite node is used as the student model. Bidirectional knowledge distillation is performed, and the teacher model and the student model transfer knowledge to each other.

[0075] Step 3: Each node applies personalized local differential privacy to evaluate the performance of the global model and the local model to ensure that they meet the accuracy requirements of the model. The privacy budget of each node is dynamically adjusted according to the model performance and privacy protection effect.

[0076] Step 4: Repeat the loop until the model meets the predetermined performance criteria or reaches the predetermined number of training rounds;

[0077] Furthermore, the model deployment in step 2 is characterized by:

[0078] The network intrusion detection model deployed on servers and satellite nodes is a hybrid model that integrates text convolutional neural network and long short-term memory neural network to capture local features and temporal information at the same time;

[0079] Text convolutional neural network mainly extracts local features from the input through convolution operation. Given the input network traffic data, it is converted into a word vector matrix X = [x1, x2, ..., x n ], where each xi For the embedding vector of a word, use the convolution kernel W k Perform convolution operation on the matrix to obtain the feature h tcnn , the formula can be expressed as:

[0080]

[0081] Among them, W k is the convolution kernel, b k is the bias, h tcnn represents the local features obtained from the convolution operation, and ReLU(·) represents the linear rectification function;

[0082] Using long short-term memory neural network to process text convolutional neural network extracted features h tcnn , in order to capture the temporal dependencies of network traffic data, the feature output h of the text convolutional neural network tcnn As its input, the LSTM neural network updates its hidden state h at time t t and memory unit C t , the formula can be expressed as:

[0083]

[0084] Among them, h t is the hidden state at the tth moment, C t is the memory unit at the tth moment, o t is the output gate, f t It is the forget gate, t is the input gate, is a candidate memory unit, tanh(·) is the hyperbolic tangent function, and T is the last moment of the input sequence;

[0085] By fusing the local features extracted by the text convolutional neural network with the hidden state of the time series information captured by the long short-term memory neural network at time T, the final classification result o can be obtained, which can be expressed as:

[0086] o=softmax(W f ·[h tcnn ,h T ]+b f ) (3)

[0087] Among them, h T is the hidden state of the LSTM neural network at the final moment, W f is the fusion weight matrix, b f is the bias term, softmax(·) is the softmax function;

[0088] In specific implementation, when processing network traffic data, the convolution kernel size can be dynamically adjusted to adapt to different types of attack traffic features, further improving the ability to extract local features. Standardize or normalize the input data to eliminate the impact of data distribution differences on model performance. To further optimize detection performance, an attention mechanism can be introduced in model fusion so that the model can dynamically focus on important features and improve the learning effect of the model.

[0089] Furthermore, the personalized local differential privacy method in step 3 is characterized by:

[0090] Personalized local differential privacy allows a certain degree of data analysis and utilization while protecting the privacy of personal data. In personalized local differential privacy, each participating node can have an independent privacy budget instead of uniformly using global privacy parameters.

[0091] When initializing the parameters, the initial privacy budget ε0 and the expected threshold of loss L are set for each satellite node. e , the initial privacy budget ε0 is the initial privacy protection parameter, which is usually small to ensure strong privacy, L e is the expected loss value, used to calculate the feedback function;

[0092] Dynamically adjust the privacy budget ε of the kth satellite node through a feedback mechanism k , where the feedback function is the core of the privacy budget adjustment, by calculating the current loss L t and the expected loss L e The privacy budget is updated by the difference, and the feedback function can be expressed as:

[0093]

[0094] Among them, L t is the model loss of the tth round of training, L e is the desired target loss value, η is the parameter used to control the feedback adjustment amplitude;

[0095] According to the feedback function fb(L t , L e ), dynamically adjust the local privacy budget of the kth satellite node at time t+1 The formula can be expressed as:

[0096]

[0097] in, is the local privacy budget of the kth satellite node in the tth round of training, where K is the number of satellite nodes;

[0098] In specific implementation, nodes can customize privacy budgets according to task priorities. For example, a higher privacy budget is allocated to data with higher sensitivity and a lower budget is allocated to ordinary traffic data to optimize overall performance. A noise adjustment method based on time series is introduced to dynamically evaluate the privacy needs of nodes and reasonably allocate privacy budgets.

[0099] Furthermore, the federated distillation method based on the adaptive weight mechanism in step 2 is characterized by:

[0100] A conditional generative adversarial network model is trained on the server side. This model is used to generate training samples that can enhance the local data of the node. The trained model consists of a generator G and a discriminator D. The loss function L for the training of the generator G is G , the formula can be expressed as:

[0101]

[0102] Among them, G(z|c) is the output of the generator under the given condition c, D(G(z|c)) is the output of the discriminator under the given condition information c, z is a random noise vector and z obeys the probability distribution p z (z);

[0103] The discriminator D maximizes the loss L of the discriminator by distinguishing between real samples and generated samples. D , the formula can be expressed as:

[0104]

[0105] Among them, D(d|c) is the probability that the discriminator judges the real sample under the given condition c, d is the real sample and d follows the data distribution p data (d);

[0106] The server sends the trained generator G to each node. After receiving the generator, each node optimizes the local data to obtain the localized generator G′, and combines the enhanced samples generated by the generator with the local data to perform knowledge distillation and model training. The adaptive weight mechanism is used during distillation.

[0107] Before knowledge distillation, each node is evaluated to determine its contribution to the model by evaluating the label quality. The local soft label quality score r of the kth satellite node is k Calculate the weight ω of its local soft label k , local soft label quality score r k According to the local soft label p of node k k，i And the global soft label obtained Definition, its formula can be expressed as:

[0108]

[0109] in, It is expressed as the L2 norm between the local soft label and the global soft label, is the L2 norm of the global soft label, used for normalization, and N is the number of samples;

[0110] The global soft label p used by the server i It is obtained by weighted calculation of all local soft labels received from satellite nodes, and its formula can be expressed as:

[0111]

[0112] Among them, ω m is the global soft label p i The weight of the mth satellite node in can be expressed as:

[0113]

[0114] The kth satellite node obtains the global soft label from the server It is calculated by calculating the local soft labels of other satellite nodes and the local soft labels p′ of the server node model i The weighted calculation is obtained, and the formula can be expressed as:

[0115]

[0116] Among them, ω′ m Global soft label The weight of the mth satellite node in ω′0 is the weight of the local soft label of the server node model, and its formula can be expressed as:

[0117]

[0118] Among them, r0 is the quality score of the local soft label of the server node model;

[0119] Each node receives the global soft label and performs knowledge distillation on the public dataset and enhanced dataset according to the global soft label. The distillation loss function between the teacher model and the student model can be expressed as:

[0120]

[0121] Where T is the temperature parameter, C is the number of categories, is the global soft label of the i-th sample of the student model for classification category j on the public dataset and the enhanced dataset, is the predicted output of the student model for the i-th sample on the public dataset and the enhanced dataset for the classification category j;

[0122] The prediction loss function of the student model can be expressed as:

[0123]

[0124] Among them, onehot(·) is the hot encoding function, is the correct label corresponding to the i-th sample in the public dataset and the enhanced dataset;

[0125] Therefore, the loss of training the student model can be expressed as:

[0126] L=ɑL pl +(1-α)L kl (15)

[0127] Among them, the parameter ɑ is the balanced student model distillation loss L pl With the prediction loss L kl Parameters;

[0128] In specific implementation, the enhanced samples generated by the conditional generative adversarial network can be added with a diversity evaluation mechanism to remove low-quality samples and ensure the reliability and representativeness of the training data. In the aggregation process of global soft labels, a node weight allocation mechanism is introduced. High-performance nodes are given higher weights, and low-performance nodes have their influence on global soft labels reduced.

[0129] Furthermore, the loop in step 4 is iterated until the model meets the predetermined performance criteria or reaches the predetermined number of training rounds:

[0130] During this process, it is necessary to dynamically monitor the performance indicators of the global model (such as accuracy and recall) and the consumption of communication resources. If the model performance improvement tends to be stable after multiple iterations and the resource consumption reaches the preset threshold, the training can be terminated in advance to improve the overall efficiency. In addition, considering the limitations of communication bandwidth and computing power, the data transmission strategy and local computing load can be optimized to improve the efficiency of the training process, thereby promoting more efficient collaborative learning.

[0131] Depend on Figure 6 and Figure 7 It can be seen that the model algorithm of the present invention has shown superior performance to traditional federated learning in terms of both loss value and accuracy. The final accuracy of the algorithm of the present invention is 93.37%, which is 4.13% higher than the model without adaptive weight mechanism and 11.53% higher than the traditional federated learning algorithm. In addition, the algorithm of the present invention has a rapid improvement in accuracy and a significant reduction in loss value within the first 10 rounds, a faster convergence speed, and less fluctuation in the later stage of training, showing higher stability.

[0132] The present invention proposes a privacy-enhanced federated learning method for satellite network intrusion detection, which organically combines the differential privacy protection mechanism and the federated distillation framework, significantly improving the privacy protection capability and detection performance of the model. In view of the distributed architecture characteristics of satellite networks, this method adopts personalized local differential privacy technology to effectively protect node data before uploading data, significantly reducing the risk of data leakage. Combined with the bidirectional knowledge distillation mechanism of the conditional generative adversarial network, efficient knowledge transfer between the server and satellite nodes is achieved, and the performance of the global model and the local model is optimized. This method achieves a good balance between privacy protection, communication resource utilization and model performance, is suitable for intrusion detection tasks in complex satellite network environments, and provides an innovative solution for improving the security of satellite networks.

Claims

1. A privacy-enhanced federated learning method for satellite network intrusion detection, characterized by: Step 1: Parameter initialization and data preprocessing. The server node negotiates with each satellite node to determine their specific requirements for privacy protection, initializes privacy parameters for each satellite node, and each node collects network data. During data transmission, each node independently applies local differential privacy to its data according to the privacy budget. Step 2: Use the satellite federated distillation framework to train the satellite network intrusion detection model. Based on the adaptive weight mechanism, the global model of the server node is used as the teacher model, and the local model of the satellite node is used as the student model. Bidirectional knowledge distillation is performed, and the teacher model and the student model transfer knowledge to each other. Step 3: Each node applies personalized local differential privacy to evaluate the performance of the global model and the local model to ensure that they meet the accuracy requirements of the model. The privacy budget of each node is dynamically adjusted according to the model performance and privacy protection effect. Step 4: The loop iterates until the model meets the predetermined performance criteria or reaches the predetermined number of training rounds.

2. The network intrusion detection model according to claim 1, characterized in that: The network intrusion detection model deployed on the server and satellite nodes is a hybrid model that integrates the text convolutional neural network and the long short-term memory neural network to capture both local features and temporal information. The text convolutional neural network mainly extracts local features from the input through convolution operations. Given the input network traffic data, it is converted into a word vector matrix X = [x1, x2, ..., x n ], where each x i For the embedding vector of a word, use the convolution kernel W k Perform convolution operation on the matrix to obtain the feature h tcnn , the formula can be expressed as: Among them, W k is the convolution kernel, b k is the bias, h tcnn represents the local features obtained from the convolution operation, and ReLU(·) represents the linear rectification function; Using long short-term memory neural network to process text convolutional neural network extracted features h tcnn , to capture the temporal dependencies of network traffic data and transform the feature output h of the text convolutional neural network tcnn As its input, the LSTM neural network updates its hidden state h at time t t and memory unit C t , the formula can be expressed as: Among them, h t is the hidden state at the tth moment, C t is the memory unit at the tth moment, o t is the output gate, f t It is the forget gate, t is the input gate, is a candidate memory unit, tanh(·) is the hyperbolic tangent function, and T is the last moment of the input sequence; By fusing the local features extracted by the text convolutional neural network with the hidden state of the time series information captured by the long short-term memory neural network at time T, the final classification result o can be obtained, which can be expressed as: o=softmax(W f ·[h tcnn ,h T ]+b f ) (3) Among them, h T is the hidden state of the LSTM neural network at the final moment, W f is the fusion weight matrix, b f is the bias term, and softmax(·) is the softmax function.

3. The personalized local differential privacy method according to claim 1, characterized in that: Personalized local differential privacy allows a certain degree of data analysis and utilization while protecting the privacy of personal data. In personalized local differential privacy, each participating node can have an independent privacy budget instead of uniformly using global privacy parameters. When initializing the parameters, the initial privacy budget ε0 and the expected threshold of loss L are set for each satellite node. e , the initial privacy budget ε0 is the initial privacy protection parameter, which is usually small to ensure strong privacy, L e is the expected loss value, used to calculate the feedback function; Dynamically adjust the privacy budget ε of the kth satellite node through a feedback mechanism k , where the feedback function is the core of the privacy budget adjustment, by calculating the current loss L t and the expected loss L e The privacy budget is updated by the difference, and the feedback function can be expressed as: Among them, L t is the model loss of the tth round of training, and η is a parameter used to control the feedback adjustment amplitude; According to the feedback function fb(L i , L e ), dynamically adjust the local privacy budget of the kth satellite node at time t+1 The formula can be expressed as: in, is the local privacy budget of the kth satellite node in the tth round of training, and K is the number of satellite nodes.

4. The federated distillation method based on the adaptive weight mechanism according to claim 1, characterized in that: A conditional generative adversarial network model is trained on the server side. This model is used to generate training samples that can enhance the local data of the node. The trained model consists of a generator G and a discriminator D. The loss function L for the training of the generator G is G , the formula can be expressed as: Among them, G(z|c) is the output of the generator under the given condition c, D(G(z|c)) is the output of the discriminator under the given condition information c, z is a random noise vector and z obeys the probability distribution p z (z); The discriminator D maximizes the discriminator loss L by distinguishing between real samples and generated samples. D , the formula can be expressed as: Among them, D(d|c) is the probability that the discriminator judges the real sample under the given condition c, d is the real sample and d follows the data distribution p data (d); The server sends the trained generator G to each node. After receiving the generator, each node optimizes the local data to obtain the localized generator G′, and combines the enhanced samples generated by the generator with the local data to perform knowledge distillation and model training. The adaptive weight mechanism is used during distillation. Before knowledge distillation, each node is judged by evaluating its label quality to determine its contribution to the model. The local soft label quality score r of the kth satellite node is k Calculate the weight ω of its local soft label k , local soft label quality score r k According to the local soft label p of node k k,i And the global soft label obtained Definition, its formula can be expressed as: in, It is expressed as the L2 norm between the local soft label and the global soft label, is the L2 norm of the global soft label, used for normalization, and N is the number of samples; The global soft label p used by the server i It is obtained by weighted calculation of all local soft labels received from satellite nodes, and its formula can be expressed as: Among them, ω m is the global soft label p i The weight of the mth satellite node in can be expressed as: The kth satellite node obtains the global soft label from the server It is calculated by calculating the local soft labels of other satellite nodes and the local soft labels p′ of the server node model i The weighted calculation is obtained, and the formula can be expressed as: Among them, ω′ m Global soft label The weight of the mth satellite node in ω′0 is the weight of the local soft label of the server node model, and its formula can be expressed as: Among them, r0 is the quality score of the local soft label of the server node model; Each node receives the global soft label and performs knowledge distillation on the public dataset and enhanced dataset according to the global soft label. The distillation loss function between the teacher model and the student model can be expressed as: Where T is the temperature parameter, C is the number of categories, is the global soft label of the i-th sample of the student model for classification category j on the public dataset and the enhanced dataset, is the predicted output of the student model for the i-th sample on the public dataset and the enhanced dataset for the classification category j; The prediction loss function of the student model can be expressed as: Among them, onehot(·) is the hot encoding function, is the correct label corresponding to the i-th sample in the public dataset and the enhanced dataset; Therefore, the loss of training the student model can be expressed as: L=ɑL pl +(1-α)L kl (15) Among them, the parameter ɑ is the balanced student model distillation loss L pl With the prediction loss L kl Parameters.

Images