Collaborative intrusion detection method for heterogeneous industrial network and electronic device

Abstract

The application relates to the technical field of industrial internet security, and specifically provides a collaborative intrusion detection method for a heterogeneous industrial network and an electronic device. The collaborative intrusion detection method comprises the following steps: obtaining local industrial network intrusion data and preprocessing; performing feature extraction on input time sequence features to obtain convolution features, and complementarily fusing the convolution features and the input time sequence features; performing long time sequence dependence modeling; weighting a hidden state sequence to obtain a context feature vector; outputting a local original classification score based on the context feature vector and a final time hidden state; updating model parameters based on a local self-distillation total loss; after the local self-adaptive gradual self-distillation stage is completed, performing a personalized knowledge distillation stage and updating the model parameters based on a personalized joint loss function. The application enables the client to continuously refine and utilize local effective knowledge during the federal training process, and improves the stability and adaptability of the local model under the condition of heterogeneous data.

Description

Technical Field

[0001] This application relates to the technical field of industrial internet security, and more specifically, to a collaborative intrusion detection method and electronic device for heterogeneous industrial networks. Background Technology

[0002] In related technologies, common industrial network intrusion detection methods are supervised training methods based on machine learning. However, since industrial network traffic data involves sensitive information such as core production process parameters of enterprises, it is difficult for different industrial enterprises to directly share raw data. This results in a shortage of labeled samples for training and difficulty for models to obtain sufficient multi-scenario training data, thus limiting the detection accuracy and generalization ability of single-node detection models. Summary of the Invention

[0003] This application aims to address at least the technical problems existing in the prior art or related technologies, such as data silos in industrial network intrusion detection leading to difficulties in cross-node knowledge collaboration, insufficient ability of single-node detection models to identify minority attacks, and limited generalization ability of existing federated learning methods in heterogeneous scenarios.

[0004] Therefore, the first aspect of this application proposes a collaborative intrusion detection method for heterogeneous industrial networks.

[0005] The second aspect of this application proposes a collaborative intrusion detection method for heterogeneous industrial networks.

[0006] A third aspect of this application proposes an electronic device.

[0007] In view of this, the first aspect of this application provides a collaborative intrusion detection method for heterogeneous industrial networks. The collaborative intrusion detection method is applied to client nodes in a federated learning framework. The collaborative intrusion detection method includes: acquiring local industrial network intrusion data and preprocessing it to obtain input temporal features in a unified format; using a temporal convolutional network to extract multi-scale local temporal features from the input temporal features to obtain convolutional features, and using a hybrid pooling mechanism to complementarily fuse the convolutional features with the input temporal features to obtain local temporal fusion features; inputting the local temporal fusion features into a long short-term memory network for long temporal dependency modeling, and outputting a hidden state sequence and a final hidden state; using a sparse attention mechanism to weight the hidden state sequence to obtain a context feature vector; concatenating the context feature vector and the final hidden state to output a local original classification score representing the prediction result; performing a local adaptive progressive self-distillation stage and updating the model parameters based on the local self-distillation total loss; after the local adaptive progressive self-distillation stage is completed, acquiring the globally shared classification score issued by the server, performing a personalized knowledge distillation stage, and updating the model parameters based on a personalized joint loss function.

[0008] This application proposes a collaborative intrusion detection method for heterogeneous industrial networks. Through multi-scale dilated convolution and residual connection design of temporal convolutional networks, it effectively captures short-term abnormal fluctuations and local attack patterns in industrial traffic while maintaining parameter efficiency. Simultaneously, this application updates memory units and calculates hidden states through the synergistic effect of the input gate, forget gate, candidate memory states, and output gate of a long short-term memory network. This effectively characterizes the dynamic evolution of industrial traffic over a longer time range. Through the complementary fusion of max pooling and average pooling in the hybrid pooling mechanism, this application retains fine-grained and effective discriminative information while compressing redundant features. Through the design of sparsemax normalization in the sparse attention mechanism, this application can concentrate attention weights on the few key moments with the highest discriminative power for intrusion detection, resetting the attention weights of a large number of redundant background states to zero. The synergistic effect of temporal convolutional networks, hybrid pooling, long short-term memory networks, and sparse attention mechanisms enables the local model to exhibit superior overall detection performance on both the natural gas pipeline dataset and the UNSW-NB15 dataset. This application also employs a local adaptive progressive self-distillation stage, where the client leverages its own historical prediction knowledge to guide current training. This stage is completely independent of server-side information, protecting local data privacy. A personalized knowledge distillation stage introduces globally shared classification scores from the server, using the importance weights of client nodes to construct a personalized teacher distribution by weighting the globally shared classification scores element-wise, achieving selective absorption of globally shared knowledge. Furthermore, this application utilizes a two-stage sequential training strategy—first local self-distillation, then personalized distillation—to absorb global knowledge. Within a federated learning framework, this achieves collaborative modeling and personalized adaptation of cross-node attack detection knowledge, enabling it to adapt to complex industrial environments with heterogeneous client data distribution.

[0009] In some technical solutions of this application, local industrial network intrusion data is acquired and preprocessed to obtain input time-series features in a unified format, including: using one-hot encoding to expand the character features in the local industrial network intrusion data and converting the character features into numerical features; standardizing the numerical features converted by one-hot encoding and the original numerical features in the local industrial network intrusion data to determine the standardized numerical features; and using the min-max normalization method to map the standardized numerical features to the interval between 0 and 1 to obtain input time-series features in a unified format.

[0010] In some technical solutions of this application, a temporal convolutional network is used to extract multi-scale local temporal features from the input temporal features to obtain convolutional features. This includes: using a one-dimensional fully convolutional network to extract features from the input temporal features, and employing a padding strategy to ensure that the outputs of multiple convolutional layers are aligned with the input temporal features in the temporal dimension, thus determining the aligned temporal features; using causal convolution to constrain the aligned temporal features, ensuring that the output at the current time depends only on the current and historical time inputs, thus determining the causally constrained temporal features; using a dilated convolutional structure to introduce a dilation factor into the causally constrained temporal features, expanding the network's receptive field without increasing the parameter scale, capturing multi-scale local temporal patterns, and determining the dilated convolutional temporal features; and using residual blocks to construct cross-layer shortcut connections, directly transmitting the input temporal features to the deep network, fusing them with the dilated convolutional temporal features, and determining the convolutional features.

[0011] In some technical solutions of this application, a hybrid pooling mechanism is used to complementarily fuse convolutional features with input temporal features to obtain local temporal fusion features. This includes: fusing input temporal features with convolutional features through a skip connection mechanism to obtain a fusion feature matrix; performing max pooling and average pooling operations on the fusion feature matrix respectively, and applying hyperbolic tangent activation to the pooling results to obtain max pooling features and average pooling features; comparing the corresponding position values ​​of max pooling features and average pooling features element by element: taking the minimum value between max pooling features and average pooling features at each position where max pooling features are greater than or equal to average pooling features, and taking the average value between max pooling features and average pooling features at each position where max pooling features are less than average pooling features, to obtain local temporal fusion features.

[0012] In some technical solutions of this application, a sparse attention mechanism is used to weight the hidden state sequence to obtain a context feature vector, including: calculating the intermediate representation vector of each time step in the hidden state sequence in the attention space, and determining the intermediate representation vector of each time step; calculating the dot product of each intermediate representation vector and the preset learnable context vector to determine the original attention score of each time step relative to the intrusion detection target; sparsely processing the original attention score through a sparse normalization function to obtain a sparse attention weight vector; and weighting and summing the hidden state sequence based on the sparse attention weight vector to obtain the context feature vector.

[0013] In some technical solutions of this application, the local adaptive progressive self-distillation stage includes: softening the local original classification score according to the distillation temperature coefficient to obtain the predicted distribution of the current training round; linearly fusing the historical predicted distribution of the previous training round with the real label based on the progressive coefficient that increases linearly with the training round to construct progressive soft labels; calculating the relative entropy between the predicted distribution of the current training round and the progressive soft labels, and normalizing the relative entropy using the sum of the task loss of the current training round and the task loss of the previous training round as the normalization denominator to obtain the adaptive distillation loss; adding the task loss of the current training round to the adaptive distillation loss to obtain the total local self-distillation loss, and updating the model parameters based on the total local self-distillation loss.

[0014] In some technical solutions of this application, the personalized knowledge distillation stage includes: using the importance weights of the client node to perform element-wise weighting of the global shared classification score, and combining the distillation temperature coefficient to perform normalized exponential transformation to construct a personalized teacher distribution; calculating the relative entropy between the student model prediction distribution and the personalized teacher distribution, and using the sum of the task loss of the student model and the task loss corresponding to the personalized teacher distribution as the normalization denominator to normalize the relative entropy to obtain the adaptive personalized distillation loss; summing the task loss of the student model and the adaptive personalized distillation loss to obtain the personalized joint loss function, and updating the model parameters based on the personalized joint loss function.

[0015] The second aspect of this application provides a collaborative intrusion detection method for heterogeneous industrial networks. This method is applied to server nodes within a federated learning framework. The collaborative intrusion detection method includes: receiving local raw classification scores uploaded by each client node, where the local raw classification scores are output by the client after detecting intrusion data in the local industrial network based on a local intrusion detection model; calculating the category distribution information entropy of each client's local dataset and the maximum mean difference between the local data distribution of each client and the distribution of the public dataset; generating a first aggregated weight component based on the category distribution information entropy and a second aggregated weight component based on the maximum mean difference; weighting and summing the first and second aggregated weight components according to a balance coefficient to generate dynamic aggregated weights corresponding to each client node; using the dynamic aggregated weights corresponding to each client node to weighted aggregate all received local raw classification scores to generate a globally shared classification score; and distributing the globally shared classification score to each client node.

[0016] In some technical solutions of this application, the first aggregated weight component is used to characterize the degree of class distribution balance of the client's local data, and the second aggregated weight component is used to characterize the degree of distribution difference between the client's local data and the public dataset.

[0017] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements a collaborative intrusion detection method for heterogeneous industrial networks as described in any of the above technical solutions.

[0018] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0019] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is one of the flowcharts illustrating a collaborative intrusion detection method for heterogeneous industrial networks according to an embodiment of this application; Figure 2 This is a second schematic flowchart of a collaborative intrusion detection method for heterogeneous industrial networks according to an embodiment of this application; Figure 3 This is a schematic block diagram of an electronic device according to an embodiment of this application; Figure 4 This is the third flowchart illustrating a collaborative intrusion detection method for heterogeneous industrial networks according to an embodiment of this application. Detailed Implementation

[0020] To better understand the above-mentioned objectives, features, and advantages of this application, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0021] Many specific details are set forth in the following description in order to provide a full understanding of this application. However, this application may also be implemented in other ways different from those described herein. Therefore, the scope of protection of this application is not limited to the specific embodiments disclosed below.

[0022] The following reference Figures 1 to 4 This application describes a collaborative intrusion detection method and electronic device for heterogeneous industrial networks according to some embodiments.

[0023] like Figure 1 As shown, this application provides a cooperative intrusion detection method for heterogeneous industrial networks. The cooperative intrusion detection method is applied to client nodes within a federated learning framework. The cooperative intrusion detection method includes: Step 102: Obtain local industrial network intrusion data and preprocess it to obtain input time-series features in a unified format; Step 104: Use a temporal convolutional network to extract multi-scale local temporal features from the input temporal features to obtain convolutional features, and use a hybrid pooling mechanism to complement and fuse the convolutional features with the input temporal features to obtain local temporal fusion features; Step 106: Input the local temporal fusion features into the long short-term memory network to perform long temporal dependency modeling, and output the hidden state sequence and the hidden state at the last time step; Step 108: Use a sparse attention mechanism to weight the hidden state sequence to obtain the context feature vector; Step 110: Concatenate the context feature vector and the hidden state at the last time step to output the local original classification score used to represent the prediction result; Step 112: Perform the local adaptive progressive self-distillation stage and update the model parameters based on the total local self-distillation loss; Step 114: After the local adaptive progressive self-distillation stage is completed, obtain the global shared classification score issued by the server, perform the personalized knowledge distillation stage, and update the model parameters based on the personalized joint loss function.

[0024] This application proposes a collaborative intrusion detection method for heterogeneous industrial networks. Through multi-scale dilated convolution and residual connection design of a temporal convolutional network, it effectively captures short-term abnormal fluctuations and local attack patterns in industrial traffic while maintaining parameter efficiency. Simultaneously, this application updates memory units and calculates hidden states through the synergistic effect of the input gate, forget gate, candidate memory states, and output gate of a long short-term memory network. This effectively characterizes the dynamic evolution of industrial traffic over a longer time range. Through the complementary fusion of max pooling and average pooling in the hybrid pooling mechanism, this application retains fine-grained and effective discriminative information while compressing redundant features. Through the design of sparsemax normalization in the sparse attention mechanism, this application can concentrate attention weights on the few key moments with the highest discriminative power for intrusion detection, resetting the attention weights of a large number of redundant background states to zero. The synergistic effect of the above four modules (temporal convolutional network, hybrid pooling, long short-term memory network, and sparse attention mechanism) enables the local model to exhibit superior overall detection performance on both the natural gas pipeline dataset and the UNSW-NB15 dataset. This application also employs a local adaptive progressive self-distillation stage, where the client leverages its own historical prediction knowledge to guide current training. This stage is completely independent of server-side information, protecting local data privacy. A personalized knowledge distillation stage introduces globally shared classification scores from the server, using the importance weights of client nodes to construct a personalized teacher distribution by element-wise weighting of the globally shared classification scores, achieving selective absorption of globally shared knowledge. Furthermore, this application utilizes a two-stage sequential training strategy—first local self-distillation, then personalized distillation—to absorb global knowledge. This achieves collaborative modeling and personalized adaptation of cross-node attack detection knowledge within a federated learning framework, adapting to complex industrial environments with heterogeneous client data distributions. The local adaptive progressive self-distillation stage includes fusing historical prediction distributions from the previous training round with real labels to construct soft labels. The personalized knowledge distillation stage includes using the importance weights of client nodes to construct a personalized teacher distribution by element-wise weighting of the globally shared classification scores.

[0025] Specifically, this application first acquires and preprocesses local industrial network intrusion data to obtain input time-series features in a unified format. Industrial network intrusion data typically contains numerical features and character features, which differ in value range, data format, and semantic expression. The preprocessing includes three specific operations.

[0026] First, a one-hot encoding method is used to expand the character-type features and convert them into numerical features, thus transforming character-type features that cannot be directly used for numerical calculations into numerical features that the model can process. Second, the numerical features are standardized to reduce the impact of differences in the distribution range of different features on model training. The standardization formula is: ; in, Represents the original value of the feature. This represents the average value of the feature across the sample set. It represents the standard deviation.

[0027] This formula transforms the distribution of different features by subtracting the mean from the original feature values ​​and then dividing by the standard deviation, thus converting the distribution of each feature into a standard normal distribution with a mean of 0 and a standard deviation of 1. Third, the min-max normalization method is used to map the features to the [0,1] interval. The normalization formula is: ; Where x represents the normalized feature value of the final input model. and These represent the minimum and maximum values ​​of the feature after standardization. This formula linearly compresses the feature values ​​to the [0,1] interval, allowing different features to be compared and calculated on the same order of magnitude. This preprocessing eliminates differences in the value range and data type of different features, preventing gradient update imbalances caused by different feature scales during model training and laying a data foundation for subsequent feature extraction.

[0028] This application utilizes a Temporal Convolutional Network (TCN) to extract multi-scale local temporal features from the input temporal features to obtain convolutional features. A TCN is a network structure used for multi-scale local temporal feature extraction of industrial flow sequences, enhancing the model's ability to perceive short-term abnormal fluctuations and local attack patterns. The TCN employs a one-dimensional fully convolutional network to extract features from the input sequence and introduces a padding strategy in the convolution operation to ensure that the output sequence after multiple convolution operations remains aligned with the input sequence in the temporal dimension, thus guaranteeing that the output at each time step corresponds to the input at the same time step. The TCN uses causal convolution to avoid data leakage, ensuring consistent temporal constraints in both the training and actual prediction phases, preventing future information from leaking into the past and causing inconsistencies between training and inference. The TCN employs a dilated convolution structure, introducing a dilation factor in the convolution operation, causing the convolution kernel to sample the input sequence at set intervals, thereby expanding the network's receptive field without significantly increasing the parameter size. TCN employs residual blocks to establish fast connections between different network layers, enabling input features to directly participate in deep network computations and fuse with convolutional layer outputs, thereby enhancing feature representation capabilities and reducing model training difficulty. Through the aforementioned multi-scale dilated convolution and residual connection design of TCN, this application can effectively capture short-term abnormal fluctuations and local attack patterns in industrial traffic while maintaining parameter efficiency, improving the model's sensitivity to rapidly emerging attack behaviors.

[0029] This application employs a hybrid pooling mechanism to complementarily fuse convolutional features with input temporal features, i.e., fuzzy fusion, to obtain local temporal fused features. Fuzzy fusion is achieved through element-wise comparison-based hybrid fusion or adaptive hybrid fusion. The hybrid pooling mechanism, on the other hand, combines feature fusion with hybrid pooling to fuse and reorganize the original input information and deep convolutional features, compressing redundant features while preserving fine-grained effective information.

[0030] Specifically, the convolutional feature representation obtained through the TCN network is denoted as h(x). The original input and the convolutional features are fused through a skip connection mechanism to obtain a new fused feature representation. The specific formula is as follows: ; in The input is a time series. In this formula... This represents the transpose of the input sequence. This indicates that the input sequence is multiplied by the convolutional features, allowing the temporal information in the original input to directly interact with the deep abstract features extracted by the convolution. This is done by fusing the feature matrix. Pooling is performed on a local window to reduce redundant information, lower the complexity of subsequent modeling, and retain the main temporal patterns that are crucial for intrusion detection. Max pooling is introduced, specifically as follows: ; in This indicates that the maximum value within a defined local window is taken as the output of that window, where tanh represents the hyperbolic tangent activation function. Max pooling captures the most significant activation response within the local window, preserving the main temporal peak patterns that are most discriminative for intrusion detection. Simultaneously, average pooling is applied to the fused features, specifically: ; Here, avg pooling represents taking the average of all values ​​within a defined local window as the output of that window. Average pooling can preserve the overall trend information within the local window, avoiding neglecting the global context by focusing excessively on extreme values. The two pooling results are jointly modeled using a fuzzy fusion mechanism, with the specific formula as follows: ; When the max pooling result is significantly higher than the average pooling result, it indicates the presence of a prominent attack signal peak within the local window. In this case, taking the minimum value can suppress the overfitting risk caused by excessively high activation values ​​while preserving significant features. When the difference between the two is not significant, it can balance the information contribution of local extrema and global trends. Through the synergistic design of max pooling, average pooling, and fuzzy fusion, this application effectively preserves fine-grained discriminative information while compressing redundant features and reducing the complexity of subsequent modeling, avoiding the information loss or feature distortion problems caused by a single pooling method.

[0031] Local temporal fusion features are input into a Long Short-Term Memory (LSTM) network for long-term temporal dependency modeling, outputting a sequence of hidden states and the final hidden state. LSTM is a network structure used to characterize the dynamic evolution of industrial traffic over a longer time period, fully expressing the persistence, stages, and contextual relevance of attack behavior. LSTM uses the following input gate: ; Controlling the extent to which new information is written to memory cells, among which The output vector of the input gate. This represents the Sigmoid activation function (which maps the input to probability values ​​between 0 and 1). and This is the weight matrix. For the current input, This is the hidden state from the previous moment. This is a bias term. Forget gate: Controls the degree to which information from the previous memory unit is retained, among which This is the output vector of the forget gate.

[0032] Candidate memory states: Generate new candidate information for the current time step, where tanh is the hyperbolic tangent activation function (mapping the input to the range of -1 to 1). Output gate: ; The extent to which information from the control memory unit is output to the hidden state, where This is the output vector of the output gate. LSTM uses the above gate signal to update the memory cells at the current time step, with the specific formula:

[0033] in, This indicates element-wise multiplication. This represents the updated state of the memory cell at the current moment. This represents the state of the memory unit at the previous moment. The formula means: the memory information at the previous moment... Multiply by the Gate of Oblivion Obtain the preserved historical information, plus the input gate With candidate memory states The product of the two parts yields the newly added information at the current time step, and the sum of the two parts is the current memory cell state. Finally, the LSTM calculates the hidden state at the current time step based on the updated memory state. That is, output gate With the process Element-wise multiplication of the activated memory cell states. Through the gating mechanism of LSTM and the memory cell update design, this application can effectively characterize the dynamic evolution of industrial flow over a long time range, fully expressing the persistence, stage, and contextual relevance of attack behavior, and making up for the shortcomings of TCN in modeling ultra-long temporal dependencies.

[0034] This application utilizes a sparse attention mechanism to weight the hidden state sequence to obtain a context feature vector. The sparse attention mechanism suppresses redundant background states through sparse weight allocation, focuses the model's attention on key moments that are more indicative of attacks, and enhances the model's ability to recognize minority class attacks and weak anomaly patterns. Specifically, it computes the intermediate representation at time step t. ;in Let be the representation vector of the t-th time step in the attention space. For attention transformation matrix, This is the bias vector. A learnable context vector is introduced. This is used to measure the correlation between each time step and the final detection target. The original attention score at time step t is calculated as follows: ,in, Let be a scalar, representing the importance score of the t-th time step relative to the current classification task. Sparsemax is used to sparsely normalize the original scores to obtain the sparse attention weight vector. , Sparsemax is a normalization function that outputs a sparse probability distribution—meaning that only a subset of time steps have positive attention weights, while the weights at the remaining time steps are precisely zero. Compared to the traditional Softmax normalization function (which outputs a dense distribution where all positions are positive), Sparsemax forces the model to focus on the few most relevant time steps, eliminating interference from redundant background states. After obtaining the sparse attention weights, the hidden state sequence output by the LSTM is weighted and summed to obtain the context vector. Where T is the total number of time steps, Let be the sparse attention weights at time step t. Let be the hidden state at time step t. This formula means that the hidden state at each time step is multiplied by its corresponding attention weight and then summed. The higher the weight (i.e., the more concentrated the attention), the greater the contribution of the time step to the context vector. Through the above Sparsemax sparse normalization design, this application can concentrate attention weights on the few key moments most discriminative for intrusion detection, resetting the attention weights of a large number of redundant background states to zero, thereby effectively suppressing the interference of redundant background states on detection decisions. The sparse attention allocation mechanism is particularly suitable for industrial network intrusion detection scenarios. In industrial traffic data, a large number of time steps are in a normal state, with only a few moments containing abnormal attack signals. Sparse attention can automatically identify and focus on these key moments, thereby enhancing the model's ability to identify minority attacks and weak anomaly patterns.

[0035] This application concatenates the context feature vector and the final hidden state to output the local raw classification score representing the prediction result. The feature z is then input into a fully connected classification layer to complete the intrusion detection discrimination. The formula for calculating z is... Where c is the context vector, The state is hidden at the end of the time frame. This indicates that the context vector and the hidden state at the final time step are concatenated along the vector dimension. The local raw classification score refers to the raw score vector (logits) output by the fully connected layer of the classification network, which has not undergone Softmax normalization and is used to represent the model's prediction confidence for the input sample in each attack category. By jointly encoding global context information and local state information at the current time step, this application can simultaneously consider the overall attack pattern of the sequence and the immediate abnormal signal at the current time step during discrimination, thereby improving the accuracy of detection decisions.

[0036] This application performs a local adaptive progressive self-distillation stage and updates model parameters based on the total loss from local self-distillation. The local adaptive progressive self-distillation stage (APS-KD) is an adaptive progressive self-distillation and model update process. In APS-KD, the input sample is set as x, its corresponding hard label is yi (i.e., the true class label), and the model output logits vector is denoted as z(x). The prediction distribution P(x) is obtained through a softmax transformation. A temperature coefficient is introduced during the knowledge distillation process. right( >0) The prediction distribution is softened to obtain the temperature-scaled prediction probability. ,in Let logits represent the output of sample x in the i-th class, exp be the natural exponential function, and the denominator be the sum of the exponential values ​​of all classes. Temperature coefficient. Used to control the smoothness of the predicted distribution. A larger value indicates a smoother predicted distribution, smaller probability differences between categories, and richer implicit inter-class similarity information, which is beneficial for the transfer of dark knowledge during knowledge distillation. In APS-KD, the predicted distribution of the model in the (t-1)th round... With real labels Perform linear fusion to construct progressive soft tags, the expression of which is: ,in The asymptotic coefficient controls the degree to which historical knowledge is involved in the current training. To ensure the model initially focuses on the true labels and gradually increases the confidence level of its historical predictions later, the asymptotic coefficient... It grows linearly with the number of training epochs, and its calculation formula is: Where T represents the total number of training rounds in the local self-distillation phase. This represents the upper bound of the asymptotic coefficient in the final training round. The formula means that the asymptotic coefficient starts at 0 and gradually increases with the number of training rounds t, reaching a preset upper bound in the final round. Define adaptive distillation loss. It is used to measure the degree of difference between predicted distributions. This indicates the task loss in the previous training round. This represents the task loss for the current training round. The formula means: divide the KL divergence between the current round's prediction distribution and the previous round's prediction distribution by the sum of the task losses for both rounds. When the historical prediction quality is high (small task loss), the influence of distillation loss is relatively increased to promote knowledge transfer; when the historical prediction quality is low (large task loss), the influence of distillation loss is relatively decreased to avoid misleading the current training with low-quality historical predictions. Combining the task loss and adaptive distillation loss, the total loss function for the client is formed. Using gradient descent based on Update the parameters of the local model on the client. This stage does not rely on server-side information at all, protecting local data privacy. Through the design of the above adaptive progressive self-distillation mechanism, this application enables the client to continuously refine and utilize effective local knowledge during federated training, improving the stability and adaptability of the local model under heterogeneous data conditions.

[0037] After completing the local adaptive progressive self-distillation stage, this application obtains the globally shared classification score issued by the server, performs the personalized knowledge distillation stage, and updates the model parameters based on the personalized joint loss function. The personalized knowledge distillation (PKD) stage is a process of personalized knowledge distillation federated learning and joint loss update, utilizing importance weights. Shared classification scores globally Combined, to construct a personalized teacher distribution ,in This is the distillation temperature coefficient. This represents the element-wise multiplication of importance weights with the globally shared classification score. Importance weights reflect the degree of importance of different feature dimensions to the client's local detection task. By multiplying the importance weights element-wise with the globally shared classification score, selective enhancement or suppression of globally shared knowledge is achieved, strengthening feature dimensions important to the local task and weakening feature dimensions irrelevant or negatively correlated to the local task. The adaptive distillation loss is then calculated. ,measure and The differences between them, among which This is the predicted distribution for the student model (i.e., the predicted distribution for the client's local model). For personalized teacher distribution, and These represent the task losses for student model prediction and personalized teacher distribution, respectively. The formula implies that the KL divergence is calculated by dividing by... + To achieve dynamic adjustment of distillation intensity, the influence of distillation loss is relatively increased when the task loss of both the student and teacher models is low to promote knowledge transfer, and relatively decreased when the task loss of either model is high to avoid misleading low-quality prediction distributions. and This forms the joint loss function for the PKD stage. Using gradient descent based on The parameters of the client's local model are updated to complete the personalized knowledge distillation process. Through the design of the above-mentioned personalized knowledge distillation mechanism, this application realizes the selective absorption of globally shared knowledge, enabling each client to select the most valuable parts for local detection from global knowledge based on the distribution characteristics of its local data and task requirements. This achieves an effective balance between globally shared knowledge and local personalized needs within the federated framework.

[0038] In summary, this application presents an industrial network intrusion detection model, TLSA, based on TCN-LSTM-Sparse Attention. By integrating multi-scale temporal feature extraction, long-term temporal dependency modeling, and a sparse attention enhancement mechanism, it improves the accuracy and robustness of single-node industrial network intrusion detection. Specifically, TCN's multi-scale dilated convolution and residual connections can capture short-term abnormal fluctuations and local attack patterns in industrial traffic; the complementary fusion of max pooling and average pooling in the hybrid pooling mechanism compresses redundant features while retaining fine-grained and effective discriminative information; LSTM's gating mechanism and memory unit update can characterize the dynamic evolution of industrial traffic over a longer time range; and the Sparsemax sparse attention mechanism can concentrate attention weights on the few key moments that are most discriminative for intrusion detection, enhancing the model's ability to identify minority attacks and weak anomaly patterns.

[0039] In one embodiment, taking the natural gas pipeline dataset and the UNSW-NB15 dataset as examples, Table 1 shows the natural gas pipeline dataset and Table 2 shows the UNSW-NB15 dataset. Table 1, the natural gas pipeline dataset, includes seven types of industrial attack samples: normal traffic and simple malicious response injection attacks, complex malicious response injection attacks, malicious state command injection attacks, malicious parameter command injection attacks, malicious function command injection attacks, denial-of-service attacks, and reconnaissance attacks, corresponding to typical industrial control network attack scenarios. Table 2, the UNSW-NB15 dataset, covers nine types of network attacks: normal traffic and fuzzing, analytical attacks, backdoors, denial-of-service attacks, vulnerability exploitation, general cryptographic attacks, reconnaissance, malicious code injection, and worms, covering a variety of common network intrusion modes. Both are used together to verify the intrusion detection performance of the method in this application under different scenarios.

[0040] Table 1

[0041] Table 2

[0042] Based on the synergistic effect of the four modules mentioned above, the TLSA model achieved detection performance of 94.12% accuracy, 79.10% precision, 87.00% recall, 81.89% F1 score, and 1.02% false positive rate on the natural gas pipeline dataset, and 86.29% accuracy, 72.12% precision, 75.60% recall, 72.19% F1 score, and 1.84% false positive rate on the UNSW-NB15 dataset, as shown in Tables 3 and 4 below. Table 3

[0043] Table 4

[0044] As shown in Tables 3 and 4, this application outperforms the CNN-BiLSTM and LSTM-AE comparison methods. The client-adaptive progressive self-distillation mechanism (APS-KD) in the KDFed personalized federated learning framework, based on knowledge distillation, enables clients to continuously refine and utilize effective local knowledge during federated training, improving the stability and adaptability of local models under heterogeneous data conditions. The dynamically weighted personalized knowledge distillation method (PKD) comprehensively considers the differences in knowledge contributions and local task requirements among different clients, performing differentiated fusion and selective absorption of shared knowledge, thus improving KDFed's personalized modeling capabilities and overall detection performance in complex industrial heterogeneous scenarios. In large-scale heterogeneous federated scenarios (the number of clients expanded from 4 to 30), the KDFed algorithm in this application achieves higher detection accuracy and lower false positive rate compared to FedAvg, FedProx, and FD-IDS methods, demonstrating better intrusion detection performance and adaptability to industrial scenarios.

[0045] In some embodiments of this application, local industrial network intrusion data is acquired and preprocessed to obtain input time-series features in a unified format, including: expanding character features in the local industrial network intrusion data using one-hot encoding to convert character features into numerical features; standardizing the numerical features converted by one-hot encoding and the original numerical features in the local industrial network intrusion data to determine the standardized numerical features; and mapping the standardized numerical features to the 0 to 1 interval using the min-max normalization method to obtain input time-series features in a unified format.

[0046] The above embodiments define the specific implementation method for acquiring and preprocessing local industrial network intrusion data. This application employs one-hot encoding to expand character features in the local industrial network intrusion data, converting them into numerical features. One-hot encoding is a method for converting categorical features into numerical features. For character features with multiple category values, a binary feature dimension is created for each category, where only the dimension corresponding to the sample's category has a value of 1, and the other dimensions have values ​​of 0. This converts character features that cannot be directly used for numerical calculations into numerical features that can be processed by the model. This application standardizes the numerical features converted by one-hot encoding and the original numerical features in the local industrial network intrusion data to determine the standardized numerical features. Standardization refers to using the aforementioned standardization formula. The process involves subtracting the mean of the feature from the sample set and then dividing by the standard deviation, transforming the feature's distribution into a standard normal distribution with a mean of 0 and a standard deviation of 1. This eliminates the adverse effects of differences in the value ranges of different features on model training. This application employs the min-max normalization method to map the standardized numerical features to the 0-1 interval, obtaining input time-series features in a unified format. Min-max normalization refers to using the aforementioned normalization formula... The feature value is linearly compressed to the [0,1] interval by subtracting the minimum value of the feature from all samples and then dividing by the difference between the maximum and minimum values ​​of the feature. One-hot encoding converts character-type features to numerical features, standardization eliminates the dimensional differences between different features, and min-max normalization maps features uniformly to the 0-1 interval. This combination of three preprocessing methods transforms heterogeneous raw industrial network data into a standardized format suitable for deep learning models, providing high-quality input data for subsequent feature extraction and model training in temporal convolutional networks.

[0047] In one embodiment, for numerical features with severely skewed value distributions, a logarithmic transformation or Box-Cox transformation can be performed before standardization to improve the distribution shape, thereby further enhancing the model's ability to model long-tailed distribution features.

[0048] In some embodiments of this application, a temporal convolutional network is used to extract multi-scale local temporal features from the input temporal features to obtain convolutional features. This includes: using a one-dimensional fully convolutional network to extract features from the input temporal features, and using a padding strategy to ensure that the output of the multi-layer convolutional network is aligned with the length of the input temporal features in the time dimension, thereby determining the aligned temporal features; using causal convolution to constrain the aligned temporal features so that the output at the current time depends only on the input at the current and historical time points, thereby determining the causally constrained temporal features; using a dilated convolutional structure to introduce a dilation factor into the causally constrained temporal features, thereby expanding the receptive field of the network without increasing the parameter scale, capturing multi-scale local temporal patterns, and determining the dilated convolutional temporal features; and using residual blocks to construct cross-layer shortcut connections to directly pass the input temporal features to the deep network, where they are fused with the dilated convolutional temporal features to determine the convolutional features.

[0049] The above embodiments define the specific implementation method for multi-scale local temporal feature extraction using temporal convolutional networks. This application employs a one-dimensional fully convolutional network to extract features from the input temporal features, and uses a padding strategy to ensure that the output of multiple convolutional layers is aligned with the input temporal features in the temporal dimension, thus determining the aligned temporal features. A one-dimensional fully convolutional network refers to a network structure where the convolutional kernels only slide in the temporal dimension and the network does not contain fully connected layers, allowing it to accept input sequences of arbitrary length. The padding strategy involves adding zero values ​​to both ends of the input sequence to ensure that the output sequence after convolution is of the same length as the input sequence in the temporal dimension. This guarantees that the output at each time step corresponds to the input at the same time step, providing an aligned temporal representation for subsequent element-wise feature fusion and attention weighting. This application uses causal convolution to constrain the aligned temporal features, ensuring that the output at the current time depends only on the current and historical inputs, thus determining the causally constrained temporal features. Causal convolution means that the output at the current moment depends only on the input at the current and past moments, and not on the input at future moments. This ensures that the model meets consistent temporal constraints during both the training and actual prediction phases, preventing future information from leaking into the past and causing inconsistencies between training and inference, thus guaranteeing the reliability of the intrusion detection model in online real-time detection scenarios. This application employs a dilated convolution structure, introducing a dilation factor into the temporal features after causal constraints. This expands the network's receptive field without increasing the parameter size, capturing multi-scale local temporal patterns and determining the temporal features after dilated convolution. Dilated convolution introduces a dilation factor into the convolution operation, causing the convolution kernel to sample the input sequence at intervals specified by the dilation factor. This exponentially expands the network's receptive field, enabling the model to capture local temporal patterns at different time scales without significantly increasing the number of parameters, improving the model's ability to perceive various attack behaviors with large duration differences. This application uses residual blocks to construct cross-layer shortcut connections, directly passing the input temporal features to the deep network, where they are fused with the temporal features after dilated convolution to determine the convolutional features. Residual blocks are network modules that directly pass input features across layers via shortcut connections and fuse them element-wise with the output of convolutional layers. This allows input features to directly participate in deep network computations, thereby improving feature representation capabilities and reducing model training difficulty, while avoiding the gradient vanishing problem in deep networks. Through the collaborative design of one-dimensional fully convolutional networks, causal convolutions, dilated convolutions, and residual blocks, this application effectively captures short-term abnormal fluctuations and local attack patterns in industrial traffic while maintaining parameter efficiency.

[0050] In some embodiments of this application, a hybrid pooling mechanism is used to complementaryly fuse convolutional features with input temporal features to obtain local temporal fusion features. This includes: fusing input temporal features with convolutional features through a skip connection mechanism to obtain a fusion feature matrix; performing max pooling and average pooling operations on the fusion feature matrix respectively, and applying hyperbolic tangent activation to the pooling results to obtain max pooling features and average pooling features; comparing the corresponding position values ​​of the max pooling features and average pooling features element by element: taking the minimum value between the max pooling features and average pooling features at each position where the max pooling features are greater than or equal to the average pooling features, and taking the average value between the max pooling features and average pooling features at each position where the max pooling features are less than the average pooling features, to obtain local temporal fusion features.

[0051] In the above embodiments, this application specifies a concrete implementation method for complementary fusion using a hybrid pooling mechanism. This application fuses input temporal features with convolutional features through a skip connection mechanism to obtain a fused feature matrix. The skip connection mechanism refers to directly transferring features from the original input layer or shallow network to the deep network through cross-layer connections, allowing the original input information to bypass the complex transformations of intermediate layers and directly participate in the final feature representation. The fusion formula for skip connections is as follows: Where x is the input time series, The features are convolutional. This application performs max pooling and average pooling operations on the fused feature matrix, and applies hyperbolic tangent activation to the pooling results, yielding max pooling features and average pooling features respectively. Max pooling refers to taking the maximum value within a set local window as the output of that window. The max pooling formula is... It can capture the most significant activation responses within a local window, preserving the main temporal peak patterns that are most discriminative for intrusion detection. Average pooling refers to taking the average of all values ​​within a set local window as the output of that window. The average pooling formula is... This approach preserves the overall trend information within a local window, avoiding the neglect of global context due to excessive focus on extreme values. Hyperbolic tangent activation involves inputting the pooling result into the hyperbolic tangent function tanh, mapping the values ​​to the range of -1 to 1, introducing nonlinear transformation capabilities, and enabling the model to learn more complex feature representations. This application compares the corresponding values ​​of the max-pooling feature and the average-pooling feature element-wise: at each position where the max-pooling feature is greater than or equal to the average-pooling feature, the minimum value between the max-pooling feature and the average-pooling feature is taken; at each position where the max-pooling feature is less than the average-pooling feature, the average value between the max-pooling feature and the average-pooling feature is taken, resulting in local temporal fusion features. Element-wise comparison refers to comparing and taking values ​​at each position in the feature matrix separately, rather than comparing the entire matrix as a whole. Complementary fusion is also known as fuzzy fusion, and the fuzzy fusion rule corresponds to the aforementioned formula. By employing a collaborative design that combines four aspects—maximum pooling to capture local saliency, average pooling to preserve global trends, hyperbolic tangent activation to introduce nonlinearity, and element-wise fuzzy fusion to achieve adaptive trade-offs—this application effectively preserves fine-grained discriminative information while compressing redundant features and reducing the complexity of subsequent modeling, thus avoiding information loss or feature distortion caused by a single pooling method.

[0052] In one embodiment, the local window size and stride used in the hybrid pooling operation can be adaptively configured according to the sampling frequency of industrial traffic data. For industrial scenarios with high sampling frequency, a larger window is used to cover more temporal context information, while for industrial scenarios with low sampling frequency, a smaller window is used to avoid excessive smoothing that leads to the loss of fine-grained attack features.

[0053] In some embodiments of this application, a sparse attention mechanism is used to weight the hidden state sequence to obtain a context feature vector, including: calculating the intermediate representation vector of each time step in the hidden state sequence in the attention space, and determining the intermediate representation vector of each time step; calculating the dot product of each intermediate representation vector and a preset learnable context vector to determine the original attention score of each time step relative to the intrusion detection target; sparsely processing the original attention score through a sparse normalization function to obtain a sparse attention weight vector; and weighting and summing the hidden state sequence based on the sparse attention weight vector to obtain the context feature vector.

[0054] In the above embodiments, the specific implementation method of obtaining the context feature vector using the sparse attention mechanism is defined. This application calculates the intermediate representation vector in the attention space for each time step in the hidden state sequence, determining the intermediate representation vector for each time step. The intermediate representation vector refers to the vector representation obtained by mapping the hidden state of the LSTM to the attention space through linear transformation and nonlinear activation; the calculation formula is the same as described above. Correspondingly, among which Let be the representation vector of the t-th time step in the attention space. For attention transformation matrix, The bias vector provides a feature representation in the attention space for subsequent attention score calculation. This application calculates the dot product of each intermediate representation vector and a predefined learnable context vector to determine the original attention score for each time step relative to the intrusion detection target. The learnable context vector is a parameter vector automatically optimized through backpropagation during model training, used to measure the correlation between each time step and the final detection target. The original attention score calculation formula is the same as described above. Correspondingly, a higher dot product value indicates a stronger correlation between that time step and the intrusion detection target. This application uses a sparse normalization function to sparsify the original attention score, obtaining a sparse attention weight vector. The sparse normalization function (Sparsemax) is a normalization function whose output is a sparse probability distribution; that is, in all time steps, only a portion of the time steps have positive attention weights, while the weights of the remaining time steps are precisely set to zero. The sparsification formula is the same as described above. Correspondingly, compared to the traditional Softmax normalization function, the sparse normalization function forces the model to focus on the few most relevant time steps, eliminating interference from redundant background states. This application uses a weighted summation of the hidden state sequence based on the sparse attention weight vector to obtain the context feature vector. The weighted summation formula is the same as described above. Correspondingly, by sparsifying the original attention score using a sparse normalization function, this application can concentrate attention weights on a few key moments that are most discriminative for intrusion detection, resetting the attention weights of a large number of redundant background states to zero, thereby effectively suppressing the interference of redundant background states on detection decisions. This sparse attention allocation mechanism is particularly suitable for industrial network intrusion detection scenarios, where a large number of time steps in industrial traffic data are in a normal state, with only a few moments containing abnormal attack signals. Sparse attention can automatically identify and focus on these key moments, thereby enhancing the model's ability to identify minority attacks and weak anomaly patterns.

[0055] In one embodiment, the sparsity of the output of the sparse normalization function can be indirectly controlled by a temperature coefficient. When the temperature coefficient is low, the output of the sparse normalization function is more concentrated (i.e., more sparse), and when the temperature coefficient is high, the output is smoother (i.e., relatively dense). By adjusting the temperature coefficient, the concentration of attention can be flexibly controlled to adapt to the differences in the sparsity of attack signals in different industrial scenarios.

[0056] In some embodiments of this application, the local adaptive progressive self-distillation stage includes: softening the local original classification score according to the distillation temperature coefficient to obtain the predicted distribution of the current training round; linearly fusing the historical predicted distribution of the previous training round with the real labels based on the progressive coefficient that increases linearly with the training round to construct progressive soft labels; calculating the relative entropy between the predicted distribution of the current training round and the progressive soft labels, and normalizing the relative entropy using the sum of the task loss of the current training round and the task loss of the previous training round as the normalization denominator to obtain the adaptive distillation loss; adding the task loss of the current training round to the adaptive distillation loss to obtain the total local self-distillation loss, and updating the model parameters based on the total local self-distillation loss.

[0057] In the above embodiments, this application specifies a concrete implementation method for performing the local adaptive progressive self-distillation stage. This application softens the original local classification scores based on the distillation temperature coefficient to obtain the prediction distribution for the current training epoch. The temperature scaling formula is... correspond, Where τ is the distillation temperature coefficient (τ>0). Let logits represent the output of sample x in the i-th category. The distillation temperature coefficient is a parameter used to control the smoothness of the predicted distribution. The larger the temperature coefficient value, the smoother the predicted distribution, the smaller the probability difference between categories, and the richer the implicit inter-class similarity information, which is beneficial to the transfer of dark knowledge during the knowledge distillation process. This application constructs progressive soft labels by linearly fusing the historical predicted distribution of the previous training round with the real labels based on a progressive coefficient that increases linearly with the training round. The progressive soft label formula is the same as described above. Correspondingly, among which These are asymptotic coefficients, and the formula for calculating asymptotic coefficients is the same as described above. Correspondingly, the asymptotic coefficient increases linearly with the number of training epochs, allowing the model to focus on the supervision signal of the real labels in the early stages of training to quickly establish basic detection capabilities, and gradually improve the confidence of its own historical predictions in the later stages of training to achieve continuous refinement of local effective knowledge. This application calculates the relative entropy between the prediction distribution of the current training epoch and the asymptotic soft label, and uses the sum of the task loss of the current training epoch and the task loss of the previous training epoch as the normalization denominator to normalize the relative entropy, thus obtaining the adaptive distillation loss. The adaptive distillation loss formula is the same as described above. Correspondingly, KL represents the KL divergence (relative entropy), which measures the degree of difference between two probability distributions. This indicates the task loss in the previous training round. This represents the task loss of the current training round. The normalized denominator dynamically adjusts the distillation intensity by dividing the KL divergence value by the sum of the task losses from the previous two rounds. When the historical prediction quality is high (small task loss), the influence of the distillation loss is relatively increased to promote knowledge transfer; when the historical prediction quality is low (large task loss), the influence of the distillation loss is relatively decreased to avoid being misled by low-quality historical predictions. This application adds the task loss of the current training round to the adaptive distillation loss to obtain the total local self-distillation loss, and updates the model parameters based on the total local self-distillation loss. The formula for the total local self-distillation loss is the same as described above. Correspondingly, by using a linear increase in the progressive coefficient, the training initially focuses on the true labels, while in the later stages, it focuses on the model's own historical knowledge. By using a normalized denominator, the distillation intensity is dynamically adjusted. This collaborative design enables the client to continuously refine and utilize local effective knowledge during federated training, thereby improving the stability and adaptability of the local model under heterogeneous data conditions.

[0058] In one embodiment, the distillation temperature coefficient can be adjusted in conjunction with the asymptotic coefficient. A lower distillation temperature is used in the early stage of training to retain more real label information, and the distillation temperature is gradually increased in the later stage of training to utilize more dark knowledge information, thereby further improving the knowledge transfer efficiency of the self-distillation stage.

[0059] In some embodiments of this application, the personalized knowledge distillation stage includes: weighting the global shared classification score element-wise using the local importance weights of the client nodes, and performing a normalized exponential transformation in conjunction with the distillation temperature coefficient to construct a personalized teacher distribution; calculating the relative entropy between the student model prediction distribution and the personalized teacher distribution, and normalizing the relative entropy using the sum of the task loss of the student model and the task loss corresponding to the personalized teacher distribution as the normalization denominator to obtain the adaptive personalized distillation loss; summing the task loss of the student model and the adaptive personalized distillation loss to obtain the personalized joint loss function, and updating the model parameters based on the personalized joint loss function.

[0060] In the above embodiments, this application specifies the concrete implementation method for performing the personalized knowledge distillation stage. The globally shared classification scores are weighted element-wise using the local importance weights of the client nodes, and a normalized exponential transformation is performed using the distillation temperature coefficient to construct a personalized teacher distribution. Importance weights To characterize the importance of different feature dimensions to the local detection task on the client side, the feature importance score can be calculated and normalized based on the mutual information between each feature dimension and the detection results in the local client dataset, or it can be calculated and normalized based on the gradient magnitude of each feature dimension during the training of the local client model. The formula for personalized teacher distribution is the same as described above: ; in, This is the distillation temperature coefficient. This represents the element-wise multiplication of importance weights with the globally shared classification score. By multiplying importance weights element-wise with the globally shared classification score, selective enhancement or suppression of globally shared knowledge is achieved, strengthening feature dimensions important to the local task and weakening feature dimensions irrelevant or negatively correlated to the local task. The normalized exponential transformation, also known as the Softmax transformation, converts the weighted score vector into a probability distribution, forming a personalized teacher distribution. This application calculates the relative entropy between the student model's prediction distribution and the personalized teacher distribution, and uses the sum of the student model's task loss and the task loss corresponding to the personalized teacher distribution as the normalization denominator to normalize the relative entropy, obtaining the adaptive personalized distillation loss. The formula for the adaptive personalized distillation loss is the same as described above: ; in, Predict the distribution for the student model (i.e., the predicted distribution for the client's local model). For the task loss of the student model, The task loss corresponds to the personalized teacher distribution. The student model refers to the client-side local intrusion detection model. Dynamic adjustment of the distillation intensity is achieved by dividing the relative entropy by the sum of the two aforementioned task losses. When the task losses of both the student and teacher models are low, the influence of the distillation loss increases relatively to promote knowledge transfer; conversely, when the task loss of either model is high, the influence of the distillation loss decreases relatively to avoid misleading model training due to low-quality prediction distributions. This application sums the task loss of the student model with the adaptive personalized distillation loss to obtain the personalized joint loss function, and updates the model parameters based on this function. The formula for the personalized joint loss function is the same as described above. Correspondingly, by weighting the globally shared classification scores element-by-element, selective absorption of global knowledge is achieved. By normalizing the denominator, the distillation intensity is dynamically adjusted. This collaborative design enables each client to select the most valuable parts of the global knowledge for local detection based on the distribution characteristics of its local data and task requirements. Under the federated framework, an effective balance between globally shared knowledge and local personalized needs is achieved.

[0061] In one embodiment, the importance weights can be recalculated and updated based on the gradient information of the local dataset using the current model parameters before each round of personalized distillation, enabling the importance weights to evolve dynamically with the model training process and further improving the adaptability of knowledge selection in the personalized distillation stage.

[0062] like Figure 2As shown, embodiments of this application provide a cooperative intrusion detection method for heterogeneous industrial networks. This cooperative intrusion detection method is applied to server nodes within a federated learning framework. The cooperative intrusion detection method includes: Step 202: Receive the local raw classification scores uploaded by each client node. The local raw classification scores are output by the client after detecting local industrial network intrusion data based on the local intrusion detection model. Step 204: Calculate the category distribution information entropy of each client's local dataset and the maximum mean difference between the distribution of each client's local data and the distribution of the public dataset; Step 206: Generate the first aggregate weight component based on the category distribution information entropy, generate the second aggregate weight component based on the maximum mean difference, and weight the first aggregate weight component and the second aggregate weight component according to the balance coefficient to generate the dynamic aggregate weight corresponding to each client node. Step 208: Using the dynamic aggregation weights corresponding to each client node, all received local raw classification scores are weighted and aggregated to generate a globally shared classification score. Step 210: Distribute the globally shared classification score to each client node.

[0063] This application protects a collaborative intrusion detection method for heterogeneous industrial networks applied to server nodes within a federated learning framework. This application receives local raw classification scores uploaded by each client node. These local raw classification scores are output by the client after detecting intrusion data in the local industrial network based on its local intrusion detection model. The local raw classification score refers to the raw score vector (logits) output by the fully connected classification layer of the client's local model, without Softmax normalization. This application calculates the class distribution information entropy of each client's local dataset and the maximum mean difference between the distribution of each client's local data and the distribution of the public dataset. (Class distribution information entropy) In information theory, this is an indicator used to measure the degree of uncertainty in a probability distribution. A lower value indicates a more unbalanced distribution of samples across different categories in the client's local dataset, and a greater scarcity of local knowledge for the client. Maximum Mean Discrepancy (MMD) is another indicator. This is a non-parametric distance metric used to measure the degree of difference between two distributions; a larger maximum mean difference indicates a more unique data distribution for that client. This application generates a first aggregated weight component based on the information entropy of the category distribution and a second aggregated weight component based on the maximum mean difference. The first and second aggregated weight components are then weighted and summed according to a balance coefficient to generate the dynamic aggregated weight for each client node. The formula for the dynamic aggregated weight is: ; Where β is the balance coefficient, used to adjust the relative contribution ratios of the information entropy component and the maximum mean difference component in the aggregation weight. This application utilizes the dynamic aggregation weights corresponding to each client node to perform weighted aggregation on all received local raw classification scores to generate a globally shared classification score. The weighted aggregation formula is: ; Where N is the total number of client nodes. For the local original category score uploaded by the i-th client, The dynamic aggregation weight corresponds to the i-th client. Weighted aggregation refers to multiplying the local original classification score of each client by the dynamic aggregation weight corresponding to that client, summing the results, and then dividing by the sum of all weights to obtain the weighted average global shared classification score. This application distributes the global shared classification score to each client node. Through a collaborative design involving four aspects—measuring client data scarcity through category distribution information entropy, measuring client data uniqueness through the maximum mean difference, generating dynamic aggregation weights by weighted summation of two components, and using dynamic aggregation weights to achieve differentiated weighted aggregation—this application, compared to the FedAvg algorithm's simple average aggregation of all clients, achieves differentiated knowledge fusion. This allows clients with more unique data distributions and higher knowledge contribution value to gain greater influence in global aggregation, thereby improving the generalization performance of the federated framework in heterogeneous industrial scenarios. Experimental results show that the method in this application achieves higher detection accuracy and lower false positive rate compared to FedAvg, FedProx, and FD-IDS methods in large-scale heterogeneous federated scenarios. The public dataset is the reference dataset held by the server.

[0064] In some embodiments of this application, the first aggregated weight component is used to characterize the degree of class distribution balance of the client's local data, and the second aggregated weight component is used to characterize the degree of distribution difference between the client's local data and the public dataset.

[0065] In the above embodiment, the functions of the two components of the dynamically aggregated weight generated by the server are defined. The first aggregated weight component is used to characterize the degree of class distribution balance of the client's local data. The degree of class distribution balance refers to the measure of whether the distribution of samples of each class in the client's local dataset is uniform. When the number of samples of each class is similar, the distribution is balanced; when the number of samples of some classes is significantly greater than that of other classes, the distribution is unbalanced. The first aggregated weight component is determined by the class distribution information entropy. To reflect the aforementioned balance, a lower category distribution information entropy indicates a more unbalanced category distribution, meaning the client's local knowledge is scarcer and has a higher supplementary value to the global model. The second aggregation weight component characterizes the degree of distribution difference between the client's local data and the public dataset. The degree of distribution difference refers to the magnitude of the difference between the client's local data distribution and the overall distribution of the public dataset. A larger difference indicates a more unique industrial network environment for the client (e.g., specific types of industrial protocols, specific attack patterns, or specific data acquisition environments), and its local knowledge has a higher contribution value to the generalization ability of the global model in similar scenarios. The second aggregation weight component uses the maximum mean difference... To reflect the degree of distributional differences, a larger maximum mean difference indicates a more significant distributional difference. By using the first aggregation weight component to characterize the balance of category distribution and the second aggregation weight component to characterize the degree of data distributional differences, this application enables the server to comprehensively consider both the scarcity and uniqueness of data from each client during global aggregation. This achieves a more comprehensive and reasonable aggregation weight allocation than a single-dimensional evaluation, ultimately using a balance coefficient... The weighted sum of the two components yields the aforementioned result: ; In one embodiment, the value of the balance coefficient β can be adaptively configured based on the main sources of data heterogeneity in the industrial network scenario. When the heterogeneity mainly stems from an imbalance in the proportion of various attack samples, the weight of the information entropy component is increased; when the heterogeneity mainly stems from differences in data distribution across different industrial scenarios, the weight of the maximum mean difference component is increased, thereby further enhancing the adaptability of the dynamic aggregation weight under different heterogeneous scenarios. The balance coefficient is a preset coefficient, i.e., a pre-defined hyperparameter.

[0066] like Figure 3 As shown in the embodiments of this application, an electronic device 300 is provided, including a memory 302, a processor 304, and a computer program stored in the memory 302 and executable on the processor 304. When the processor 304 executes the computer program, it implements the collaborative intrusion detection method for heterogeneous industrial networks as described above. Therefore, the electronic device 300 possesses all the beneficial effects of the aforementioned collaborative intrusion detection method for heterogeneous industrial networks.

[0067] like Figure 4 As shown, Figure 4 This is a flowchart illustrating a collaborative intrusion detection method for heterogeneous industrial networks. The overall process starts from the initial node, and the specific steps include: Step 402, data preprocessing, which involves formatting and standardizing the values ​​of the raw industrial network intrusion data. Step 404: Input TCN, that is, input the preprocessed data into the temporal convolutional network for multi-scale local temporal feature extraction; Next, step 406 is executed to perform hybrid pooling, completing feature fusion and redundant information compression; Step 408: Input the data into the Long Short-Term Memory network to characterize long-term temporal dependencies; Step 410: Input sparse attention to strengthen the weights of key anomalous features; Step 412: Perform federated learning. After completing local feature modeling, proceed to federated learning and carry out cross-node collaborative training. Step 414: Whether the learning has met the standard. That is, after each round of training, the judgment step checks whether the learning has met the standard. If it has not met the standard, it returns to the federated learning step to continue iterative optimization. If it has met the standard, it executes step 416 to save the learning results. The final output is the intrusion detection classification result corresponding to step 418.

[0068] In the claims, description, and accompanying drawings of this application, the terms "one embodiment," "some embodiments," "specific embodiment," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of this application. In the claims, description, and accompanying drawings of this application, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0069] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A collaborative intrusion detection method for heterogeneous industrial networks, characterized in that, The collaborative intrusion detection method is applied to client nodes in a federated learning framework, and the collaborative intrusion detection method includes: Local industrial network intrusion data is acquired and preprocessed to obtain input time-series features in a unified format; A temporal convolutional network is used to extract multi-scale local temporal features from the input temporal features to obtain convolutional features. Then, the convolutional features are complementary and fused with the input temporal features through a hybrid pooling mechanism to obtain local temporal fusion features. The local temporal fusion features are input into a long short-term memory network to perform long temporal dependency modeling, and the hidden state sequence and the hidden state at the last time step are output. The hidden state sequence is weighted using a sparse attention mechanism to obtain a context feature vector; The context feature vector and the final hidden state are concatenated to output the local raw classification score representing the prediction result; Perform a local adaptive progressive self-distillation phase and update the model parameters based on the total local self-distillation loss; After the local adaptive progressive self-distillation stage is completed, the global shared classification score issued by the server is obtained, the personalized knowledge distillation stage is performed, and the model parameters are updated based on the personalized joint loss function.

2. The collaborative intrusion detection method for heterogeneous industrial networks according to claim 1, characterized in that, The process of acquiring and preprocessing local industrial network intrusion data to obtain input time-series features in a unified format includes: One-hot encoding is used to expand the character features in the local industrial network intrusion data, and the character features are converted into numerical features. The numerical features converted by one-hot encoding and the original numerical features in the local industrial network intrusion data are standardized to determine the standardized numerical features. The min-max normalization method is used to map the standardized numerical features to the 0 to 1 interval, resulting in input time-series features in a unified format.

3. The collaborative intrusion detection method for heterogeneous industrial networks according to claim 1, characterized in that, The step of using a temporal convolutional network to extract multi-scale local temporal features from the input temporal features to obtain convolutional features includes: A one-dimensional fully convolutional network is used to extract features from the input temporal features, and a padding strategy is used to ensure that the output of the multi-layer convolutional network is aligned with the input temporal features in the time dimension, thereby determining the aligned temporal features. Causal convolution is used to constrain the aligned temporal features so that the output at the current time depends only on the input at the current and historical times, thus determining the causally constrained temporal features; A dilated convolution structure is adopted, and a dilation factor is introduced into the temporal features after the causal constraint. This expands the receptive field of the network without increasing the parameter scale, captures multi-scale local temporal patterns, and determines the temporal features after dilated convolution. Residual blocks are used to construct cross-layer shortcut connections, which directly transmit the input temporal features to the deep network and fuse them with the temporal features after dilated convolution to determine the convolutional features.

4. The collaborative intrusion detection method for heterogeneous industrial networks according to claim 1, characterized in that, The step of using a hybrid pooling mechanism to complementarily fuse the convolutional features with the input temporal features to obtain local temporal fusion features includes: The input temporal features are fused with the convolutional features through a skip connection mechanism to obtain a fused feature matrix; Max pooling and average pooling operations are performed on the fused feature matrix respectively, and hyperbolic tangent activation is applied to the pooling results to obtain the corresponding max pooling features and average pooling features. The corresponding position values ​​of the max pooling feature and the average pooling feature are compared element by element: at each position where the max pooling feature is greater than or equal to the average pooling feature, the minimum value between the max pooling feature and the average pooling feature is taken; at each position where the max pooling feature is less than the average pooling feature, the average value between the max pooling feature and the average pooling feature is taken to obtain the local temporal fusion feature.

5. The collaborative intrusion detection method for heterogeneous industrial networks according to claim 1, characterized in that, The step of using a sparse attention mechanism to weight the hidden state sequence to obtain a context feature vector includes: Calculate the intermediate representation vector in the attention space for each time step in the hidden state sequence, and determine the intermediate representation vector for each time step; Calculate the dot product of each intermediate representation vector with a preset learnable context vector to determine the raw attention score for each time step relative to the intrusion detection target; The original attention score is sparsified by a sparse normalization function to obtain a sparse attention weight vector. The hidden state sequence is weighted and summed based on the sparse attention weight vector to obtain the context feature vector.

6. The collaborative intrusion detection method for heterogeneous industrial networks according to claim 1, characterized in that, The execution of the local adaptive progressive self-distillation phase includes: The original local classification score is softened based on the distillation temperature coefficient to obtain the predicted distribution for the current training round. Based on the progressive coefficient that increases linearly with the training rounds, the historical predicted distribution of the previous training round is linearly fused with the real labels to construct progressive soft labels. Calculate the relative entropy between the prediction distribution of the current training round and the progressive soft label, and normalize the relative entropy by using the sum of the task loss of the current training round and the task loss of the previous training round as the normalization denominator to obtain the adaptive distillation loss. The task loss of the current training round is added to the adaptive distillation loss to obtain the total local self-distillation loss, and the model parameters are updated based on the total local self-distillation loss.

7. The collaborative intrusion detection method for heterogeneous industrial networks according to claim 1, characterized in that, The personalized knowledge distillation stage includes: The global shared classification score is weighted element-wise using the local importance weights of the client nodes, and a normalized exponential transformation is performed in conjunction with the distillation temperature coefficient to construct a personalized teacher distribution. Calculate the relative entropy between the student model prediction distribution and the personalized teacher distribution, and use the sum of the task loss of the student model and the task loss corresponding to the personalized teacher distribution as the normalization denominator to normalize the relative entropy, thereby obtaining the adaptive personalized distillation loss. The task loss of the student model is summed with the adaptive personalized distillation loss to obtain the personalized joint loss function, and the model parameters are updated based on the personalized joint loss function.

8. A collaborative intrusion detection method for heterogeneous industrial networks, characterized in that, The collaborative intrusion detection method is applied to the server node in a federated learning framework, and the collaborative intrusion detection method includes: Receive the local raw classification scores uploaded by each client node. The local raw classification scores are output by the client after detecting local industrial network intrusion data based on the local intrusion detection model. Calculate the category distribution information entropy of each client's local dataset, and the maximum mean difference between the distribution of each client's local data and the distribution of the public dataset; A first aggregated weight component is generated based on the category distribution information entropy, and a second aggregated weight component is generated based on the maximum mean difference. The first aggregated weight component and the second aggregated weight component are weighted and summed according to the balance coefficient to generate the dynamic aggregated weight corresponding to each client node. Using the dynamic aggregation weights corresponding to each client node, all received local raw classification scores are weighted and aggregated to generate a globally shared classification score; The globally shared classification score is distributed to each client node.

9. The collaborative intrusion detection method for heterogeneous industrial networks according to claim 8, characterized in that, The first aggregated weight component is used to characterize the degree of class distribution balance of the client's local data, and the second aggregated weight component is used to characterize the degree of distribution difference between the client's local data and the public dataset.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the collaborative intrusion detection method for heterogeneous industrial networks as described in any one of claims 1 to 9.

Images