Encrypted traffic anomaly detection method and system based on attention mechanism

Abstract

The invention relates to the technical field of encrypted traffic of a large-scale data set, in particular to an encrypted traffic anomaly detection method and system based on an attention mechanism. Obtaining packet data, and carrying out data preprocessing to obtain preprocessed data; constructing an auto-encoder model, wherein the auto-encoder model comprises a multi-head attention mechanism enhanced bidirectional gating cycle unit encoding layer and a self-attention mechanism enhanced bidirectional long-short-term memory network decoding layer; and carrying out encryption traffic anomaly detection on the preprocessed data by using the auto-encoder model. The multi-head attention mechanism enhanced bidirectional gating circulation unit and the self-attention mechanism enhanced bidirectional long-short-term memory network are creatively combined, so that the time sequence characteristics are effectively extracted, and the front-back relevance of the sequence data is kept. Meanwhile, model training is carried out by adopting a composite loss function containing business associated loss, and the generalization ability of the model is improved by combining cross validation and regularization technologies.

Description

Technical Field

[0001] The present invention relates to the field of large-scale data set encrypted traffic technology, and in particular to an attention mechanism-based encrypted traffic anomaly detection method and system. Background Art

[0002] With the development of the times, encrypted traffic has become the mainstream trend of communication, but at the same time, it is also used to cover up malicious attacks to evade detection by opponents. Especially in cyber attacks, those well-designed malicious encryption anomaly attacks against security systems have brought unprecedented challenges to the stable operation of the power grid. At present, the power grid system not only lacks a complete early warning mechanism for identifying malicious encrypted traffic, but also urgently needs to develop more efficient, rapid and accurate identification methods. These are urgent needs and security issues that need to be solved. Only in this way can we respond to and effectively solve information security issues in a timely manner.

[0003] Traditional malicious encrypted traffic detection technologies mostly rely on the analysis of common payload content, such as deep packet inspection, which is usually rule-driven. For example, Van et al. used the plaintext information exposed in encrypted communications by identifying specific patterns of encrypted traffic and performing fingerprint matching. This method relies heavily on the rule base. However, with the continuous evolution of encryption algorithms, this rule-based method has revealed its inherent limitations in practice and is difficult to adapt to the evolving encryption algorithm environment. Therefore, exploring algorithms in the field of machine learning has become a key research direction for detecting malicious encrypted traffic. Chen et al. proposed an optimized density peak clustering algorithm (DPC-GSMND), which significantly improved the accuracy of identifying encrypted malicious traffic while reducing computational complexity. Nevertheless, the training of this model still relies on the support of large-scale sample data. Wang et al. developed the Deep-Full Range (DFR) algorithm, which combines three technologies: CNN, LSTM, and stacked autoencoder (SAE) to extract spatial features, temporal features, and encoding features, respectively. Although the DFR algorithm has made some attempts in identifying encrypted malicious traffic, its performance still has room for improvement.

[0004] In general, we still lack an efficient, practical, and comprehensive detection system for dealing with encrypted traffic in the power grid system. This defect not only restricts our ability to monitor the security of encrypted communications in real time, but also limits the effective identification and response to potential malicious activities in complex network environments. Therefore, the development of a detection system that integrates advanced algorithms, can adapt to the development of encryption technology, and has high operability cannot be ignored in order to ensure the security of power grid information. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention is proposed.

[0006] Therefore, the problem to be solved by the present invention is how to solve the significant limitations in dealing with the dynamics and complexity of encrypted traffic: these technologies are often based on static rule settings and are difficult to adapt to the ever-changing data characteristics and encryption modes in encrypted communications, resulting in poor results in identifying new or mutated malicious encrypted traffic.

[0007] There is an inefficiency problem when processing large-scale data sets: Traditional detection systems often face the challenges of high computing resource consumption and slow processing speed when processing massive data, which is particularly prominent in network security monitoring scenarios with high real-time requirements.

[0008] In the encrypted traffic identification system of power grid, the research and application are relatively blank: in the encrypted traffic identification system, the construction of early warning and response mechanism is still in its infancy. How to achieve early detection and rapid response to potential threats is an urgent problem to be solved in the field of encrypted traffic identification of power grid.

[0009] In summary, this patent proposes a solution, the core of which is to use a comprehensive feature extraction method, and combine the multi-head attention mechanism and the bidirectional gated recurrent unit (BiGRU), namely BiMAGRU, to extract time series features. In addition, the bidirectional long short-term memory network (BiLSTM) enhanced by the self-attention mechanism, namely BiSALSTM, is used to combine the two to construct AE to achieve efficient recognition of encrypted traffic.

[0010] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0011] In a first aspect, an embodiment of the present invention provides an encrypted traffic anomaly detection method based on an attention mechanism, which includes obtaining pap data and performing data preprocessing on the pap data to obtain preprocessed data;

[0012] The autoencoder model includes a bidirectional gated recurrent unit encoding layer enhanced by a multi-head attention mechanism and a bidirectional long short-term memory network decoding layer enhanced by a self-attention mechanism;

[0013] The autoencoder model is used to perform encrypted traffic anomaly detection on the preprocessed data.

[0014] As a preferred solution of the encrypted traffic anomaly detection method based on the attention mechanism of the present invention, the data preprocessing includes:

[0015] Clean the pacp data and delete duplicate, damaged, and incompletely captured traffic information;

[0016] Performing data truncation and padding on the cleaned data to keep the input data length consistent;

[0017] The data are normalized.

[0018] As a preferred solution of the encrypted traffic anomaly detection method based on the attention mechanism described in the present invention, it also includes a data feature extraction step:

[0019] Calculate the chi-square statistic between categorical features and labels;

[0020] Evaluate the mutual information score between numerical features and labels;

[0021] Features with mutual information scores exceeding 0.01 and chi-square statistic P values ​​less than 0.1 were selected;

[0022] The classification features that are not eliminated are transformed by the hash function.

[0023] As a preferred solution of the encrypted traffic anomaly detection method based on the attention mechanism of the present invention, wherein: the multi-head attention mechanism enhanced bidirectional gated recurrent unit encoding layer includes:

[0024] Incorporate multi-head attention mechanisms into candidate hidden states in the forward and backward passes;

[0025] Derive query, key, and value vectors from candidate hidden states via linear transformations;

[0026] Calculate the attention score and normalize it through the activation function to get the attention weight;

[0027] The outputs of each attention head are concatenated and transformed through a linear layer to obtain the attention output;

[0028] The attention output is combined with the original forward and backward hidden states to update the final hidden state through a gating mechanism.

[0029] As a preferred solution of the encrypted traffic anomaly detection method based on the attention mechanism of the present invention, wherein: the self-attention mechanism enhanced bidirectional long short-term memory network decoding layer includes:

[0030] For each time step, compute forward and reverse queries, keys, and values;

[0031] Calculate the attention score and the attention output by the values;

[0032] Update the hidden state using the attention output;

[0033] Concatenate the forward and reverse hidden states.

[0034] As a preferred solution of the encrypted traffic anomaly detection method based on the attention mechanism described in the present invention, it also includes a supervised fitting step:

[0035] Use cross entropy as loss function;

[0036] Perform cross validation;

[0037] Update the gradient using the stochastic gradient descent algorithm;

[0038] L2 regularization and dropout function are used to prevent the model from overfitting.

[0039] As a preferred solution of the encrypted traffic anomaly detection method based on the attention mechanism described in the present invention, it also includes model evaluation and tuning steps:

[0040] Evaluate the performance of the autoencoder model on an independent validation set;

[0041] Adjust model parameters based on indicators such as accuracy, F1 score, and cross entropy loss function;

[0042] Optimize learning rate and network structure.

[0043] In a second aspect, an embodiment of the present invention provides an encrypted traffic anomaly detection system based on an attention mechanism, which includes a data preprocessing module, obtains pap data, and performs data preprocessing on the pap data to obtain preprocessed data;

[0044] A construction module is used to construct an autoencoder model, wherein the autoencoder model includes a bidirectional gated recurrent unit encoding layer enhanced by a multi-head attention mechanism and a bidirectional long short-term memory network decoding layer enhanced by a self-attention mechanism;

[0045] The model early warning module uses the autoencoder model to perform encrypted traffic anomaly detection on the preprocessed data.

[0046] In a third aspect, an embodiment of the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program instructions are executed by the processor, the steps of the encrypted traffic anomaly detection method based on the attention mechanism as described in the first aspect of the present invention are implemented.

[0047] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program instructions are executed by a processor, the steps of the encrypted traffic anomaly detection method based on the attention mechanism as described in the first aspect of the present invention are implemented.

[0048] The beneficial effects of the present invention are as follows: This research scheme designs a large-scale data set encrypted traffic anomaly detection mechanism, which is optimized in terms of performance and applicability to adapt to the key characteristics of power industry data, such as high real-time, strong periodicity, and complex correlation. This not only ensures the security of industry data, but also provides a solid guarantee for the stable operation of the system.

[0049] In terms of realizing encrypted traffic anomaly detection, this study adopted the following strategies: First, the feature extraction process was streamlined and optimized, successfully retaining key information while significantly reducing computational complexity, providing high-quality data input for subsequent deep learning models. Secondly, a diversified attention mechanism was introduced to effectively improve the model's ability to capture key features in encrypted traffic, enabling the model to simultaneously identify and focus on important information in different spatial dimensions. Furthermore, this study innovatively combined the bidirectional gated recurrent unit enhanced by the multi-head attention mechanism with the bidirectional long short-term memory network enhanced by the self-attention mechanism, which not only effectively extracted the temporal features, but also maintained the before-and-after correlation of the sequence data, thereby more accurately identifying abnormal patterns in encrypted traffic and greatly improving the classification accuracy of the model. In addition, by adopting cross-validation and regularization techniques, this study further enhanced the generalization ability and robustness of the model, providing strong support for data security in the power industry.

[0050] (1) Filling technical gaps and leading innovation. This invention integrates advanced deep learning technologies and models to build an efficient encrypted traffic anomaly detection model and system, effectively filling the technical gap. The implementation of this technical solution significantly improves the intelligent early warning capability of cryptographic threats in the information system of the power industry, and provides a new technical path and solution for the industry's information security protection.

[0051] (2) Breakthrough in technical difficulties and solution to industry pain points. The present invention directly addresses the problem that has long plagued the industry, namely, how to effectively detect anomalies in encrypted traffic, a problem that has not been satisfactorily resolved. Through the technical solution of the present invention, the identification and intelligent early warning of encrypted traffic attacks in complex network environments are realized, providing strong technical support for network security protection in the power industry and even in a wider range of fields. This breakthrough not only reflects the innovativeness of the present invention, but also demonstrates its remarkable results in solving industry pain points.

[0052] In summary, this encoder encrypted traffic anomaly detection method that combines a bidirectional gated recurrent unit enhanced by a multi-head attention mechanism and a bidirectional long short-term memory network enhanced by a self-attention mechanism can efficiently and accurately process large-scale data sets, realize abnormal traffic detection, and ensure data accuracy and consistency. At the same time, the model has generalization capabilities and is not only applicable to the power industry, but can also play an important role in other industries and fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0054] Figure 1 This is a flowchart of the encrypted traffic anomaly detection method based on the attention mechanism;

[0055] Figure 2 A computer device diagram for an encrypted traffic anomaly detection method based on an attention mechanism;

[0056] Figure 3 This is a schematic diagram of the autoencoder architecture of the encrypted traffic anomaly detection method based on the attention mechanism;

[0057] Figure 4 This is a schematic diagram of the internal structure of the encoding layer of the encrypted traffic anomaly detection method based on the attention mechanism;

[0058] Figure 5 This is a schematic diagram of the internal structure of the decoding layer of the encrypted traffic anomaly detection method based on the attention mechanism;

[0059] Figure 6 Schematic diagram of the intelligent early warning system for cryptographic threats based on the encrypted traffic anomaly detection method based on the attention mechanism. DETAILED DESCRIPTION

[0060] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0061] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0062] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.

[0063] Example 1

[0064] Reference Figure 1-2, which is the first embodiment of the present invention, and provides an encrypted traffic anomaly detection method based on an attention mechanism, comprising:

[0065] S100: Acquire pacp data, and perform data preprocessing on the pacp data to obtain preprocessed data;

[0066] In the embodiment of the present application, the pacp data mainly includes the following aspects: data packet header information (including timestamp, length, etc.), protocol type information, data flow statistical characteristics (such as traffic size, number of packets, etc.), and encryption-related characteristic data (such as cipher suites, certificate information, etc.).

[0067] Specifically, the collection of pacp data can be achieved in the following ways:

[0068] Use network sniffing tools (such as Wireshark) to directly capture real-time traffic on the network interface; collect traffic data from the mirror port of network devices (such as switches and routers); collect data at key nodes through dedicated network probe devices;

[0069] In an optional embodiment, the system dynamically adjusts the data collection strategy according to actual needs. For example, during periods of heavy network load, sampling can be used to collect complete data every fixed time window (such as 5 minutes). During low-peak periods such as nighttime, full data collection can be performed. This flexible collection strategy ensures the representativeness of the data and avoids the waste of storage resources.

[0070] In another optional embodiment, the system can also be configured with a multi-layer collection mechanism. A lightweight collection program is used at the edge layer to collect only basic traffic characteristics, while a more comprehensive collection module is deployed at the core node to obtain richer protocol details and encryption parameters. This layered collection method not only ensures the real-time performance of the system, but also provides detailed information required for in-depth analysis when necessary.

[0071] It should be noted that the following points need special attention during the data collection process: Clock synchronization: ensure the time synchronization of all collection nodes to facilitate subsequent data correlation analysis; Collection integrity: For long connections such as TCP streams, ensure that the complete session process is captured; Storage format: Use a unified pcap format for storage, and perform necessary compression and backup.

[0072] S101: Data preprocessing includes:

[0073] Clean the pacp data and delete duplicate, damaged, and incompletely captured traffic information;

[0074] Perform data truncation and padding on the cleaned data to keep the input data length consistent;

[0075] Normalize the data.

[0076] In the embodiment of the present application, the data cleaning process adopts a multi-level filtering mechanism:

[0077] First level: filtering based on packet header integrity; second level: filtering based on protocol compliance; third level: filtering based on business logic.

[0078] Specifically, the session reorganization process includes the following steps:

[0079] Extract five-tuple information (source IP, destination IP, source port, destination port, protocol type); group data packets according to the five-tuple; sort the data packets within the group according to the TCP sequence number; handle abnormal situations such as data packet retransmission and loss.

[0080] In an optional embodiment, the system uses a sliding window mechanism for feature extraction. The window size can be dynamically adjusted according to actual needs (such as 30 seconds, 1 minute, etc.). In each window, the system calculates the following features:

[0081] Basic statistical features: number of packets, number of bytes, average packet length, etc.; timing features: packet interval, burstiness, etc.; encryption features: frequency of use of cipher suites, length of certificate chains, etc.;

[0082] It should be noted that the following key points need to be paid attention to during the entire preprocessing process: real-time requirements: the delay of preprocessing should not exceed the threshold set by the system (such as 100ms); resource usage: reasonably allocate CPU and memory resources to avoid affecting other system functions; exception handling: there must be a complete error handling mechanism for data that cannot be processed normally.

[0083] S102: Also includes the data feature extraction step: calculating the chi-square statistic between the classification feature and the label; evaluating the mutual information score between the numerical feature and the label; screening the features whose mutual information score exceeds 0.01 and whose chi-square statistic P value is less than 0.1; transforming the classification features that are not eliminated through the hash function.

[0084] S200: Build an autoencoder model, which includes a bidirectional gated recurrent unit encoding layer enhanced by a multi-head attention mechanism and a bidirectional long short-term memory network decoding layer enhanced by a self-attention mechanism;

[0085] In the embodiment of the present application, the construction process of the encoder layer is as follows:

[0086] Input layer configuration: Feature dimension: set the input dimension according to the preprocessed data features; batch size: set to a power of 2 between 32 and 128; sequence length: determined according to the time window size.

[0087] BiGRU layer configuration: Hidden layer size: usually set to 64-256; Number of layers: 2-3 stacked layers; Activation function: tanh or ReLU.

[0088] Multi-head attention configuration: Number of heads: usually set to 4-8; Dimension of each head: total dimension divided by the number of heads; Attention score calculation: using scaled dot product attention mechanism;

[0089] In an optional embodiment, the encoder may further add the following optimization mechanism:

[0090] Residual connection: adding cross-layer connections in deep networks to alleviate the gradient vanishing problem; LayerNormalization: adding layer normalization after each sublayer to stabilize the training process; Dropout: adding dropout after the multi-head attention layer to prevent overfitting;

[0091] The construction of the decoder layer is equally important, including:

[0092] BiLSTM configuration:

[0093] Cell type: You can choose basic LSTM or LSTM with peephole connection; State size: usually matches the encoder output dimension; Forget gate bias: initialized to 1 to promote long-term memory retention;

[0094] Self-attention mechanism configuration:

[0095] Attention matrix calculation: using additive attention or multiplicative attention; temperature parameter: used to adjust the softening degree of attention distribution; mask mechanism: using masks during training to prevent seeing future information.

[0096] It should be noted that special attention should be paid to the following aspects during model construction:

[0097] Gradient clipping: Set an appropriate clipping threshold to prevent gradient explosion; parameter initialization: Use Xavier or He initialization method; optimizer selection: It is recommended to use the Adam optimizer and set the learning rate to 0.001.

[0098] S201: The multi-head attention mechanism-enhanced bidirectional gated recurrent unit encoding layer includes:

[0099] Incorporate multi-head attention mechanisms into the forward and backward priority hidden states;

[0100] Derive query, key, and value vectors from candidate hidden states via linear transformations;

[0101] Calculate the attention score and normalize it through the activation function to get the attention weight;

[0102] The outputs of each attention head are concatenated and transformed through a linear layer to obtain the attention output;

[0103] The attention output is combined with the original forward and backward hidden states to update the final hidden state through a gating mechanism.

[0104] S202: The self-attention mechanism enhanced bidirectional long short-term memory network decoding layer includes:

[0105] For each time step, compute forward and reverse queries, keys, and values;

[0106] Calculate the attention score and attention output by value;

[0107] Update the hidden state using the attention output;

[0108] Concatenate the forward and reverse hidden states.

[0109] S203: Also includes supervised fitting steps:

[0110] Use cross entropy as loss function;

[0111] Perform cross validation;

[0112] Update the gradient using the stochastic gradient descent algorithm;

[0113] L2 regularization and dropout function are used to prevent the model from overfitting.

[0114] S300: Use the autoencoder model to perform encrypted traffic anomaly detection on the preprocessed data.

[0115] S301: Also includes model evaluation and tuning steps:

[0116] Evaluate the performance of the autoencoder model on an independent validation set;

[0117] Adjust model parameters based on indicators such as accuracy, F1 score, and cross entropy loss function;

[0118] Optimize learning rate and network structure.

[0119] In the embodiment of the present application, the anomaly detection process includes the following steps:

[0120] Data entry processing:

[0121] Batch preparation: grouping data into batch sizes; Feature scaling: ensuring that all features are in a similar numerical range; Sequence padding: padding sequences of unequal lengths;

[0122] Model forward propagation:

[0123] Encoding stage: extract features through BiMAGRU; attention calculation: calculate multi-head attention weights; decoding stage: reconstruct the sequence through BiSALSTM.

[0124] Anomaly score calculation:

[0125] Reconstruction error calculation: use MSE or MAE to measure the reconstruction quality; threshold judgment: determine the abnormal threshold based on statistics or machine learning methods; abnormal marking: mark samples exceeding the threshold as abnormal.

[0126] In an optional embodiment, the system can also implement the following advanced functions:

[0127] Adaptive Threshold:

[0128] Use sliding windows to calculate dynamic thresholds; adjust thresholds based on factors such as time and load; and support multi-dimensional threshold combination judgments.

[0129] Abnormal classification:

[0130] Classify detected anomalies into different types (such as DDoS, data leakage, etc.); assign risk levels to each anomaly type; and generate detailed anomaly description reports.

[0131] Real-time response mechanism:

[0132] Immediately trigger alarms for high-risk anomalies; support automated response operations (such as blocking, current limiting, etc.); and record the execution effects of response operations.

[0133] It should be noted that the following points should be noted when performing anomaly detection:

[0134] Detection delay: The time from data input to output results should be controlled within an acceptable range; False alarm control: Balance the accuracy and recall of detection by adjusting parameters; Explainability: Provide necessary explanations for the detection results.

[0135] In addition, the system should also include a complete evaluation and feedback mechanism:

[0136] Performance evaluation: real-time calculation of indicators such as accuracy, recall, and F1 score; monitoring of performance indicators such as detection delay and resource usage; and regular generation of evaluation reports.

[0137] Model update: Automatically adjust model parameters based on detection results; regularly retrain the model with new data; maintain model version and update history.

[0138] Operation and maintenance support: Provide detailed operation logs; support backup and recovery of model parameters; provide fault diagnosis and processing tools.

[0139] In actual deployment, this system adopts a distributed architecture:

[0140] Data collection node:

[0141] Deployed at the edge of the network; responsible for the collection and preliminary processing of raw data; transmitting data to the central node through the message queue.

[0142] Computing nodes: Use GPU clusters to accelerate model operations; use load balancing strategies to distribute computing tasks; support horizontal expansion to improve processing capabilities.

[0143] Storage nodes: Distributed storage of raw data and test results: Support fast retrieval and analysis of data; Implement disaster recovery backup of data;

[0144] The technical solution of this embodiment achieves efficient anomaly detection of encrypted traffic by innovatively combining the multi-head attention mechanism and the bidirectional recurrent neural network. The system not only has significant improvements in accuracy, but also has good scalability and practicality. Through meticulous engineering implementation and optimization, the stable operation of the solution in the actual environment is ensured.

[0145] Furthermore, this embodiment also provides an encrypted traffic anomaly detection system based on an attention mechanism, comprising:

[0146] The data preprocessing module obtains the pap data and performs data preprocessing on the pap data to obtain the preprocessed data;

[0147] Build a module to build an autoencoder model, which includes a bidirectional gated recurrent unit encoding layer enhanced by a multi-head attention mechanism and a bidirectional long short-term memory network decoding layer enhanced by a self-attention mechanism;

[0148] The model early warning module uses the autoencoder model to perform encrypted traffic anomaly detection on the preprocessed data.

[0149] In summary, by using a bidirectional gated recurrent unit enhanced by a multi-head attention mechanism as the encoding layer and combining a bidirectional long short-term memory network enhanced by a self-attention mechanism as the decoding layer, the innovative architecture achieves bidirectional capture and deep analysis of temporal features. This structure can not only maintain the correlation between sequence data, but also break through the limitation of traditional methods that rely only on a single feature, enabling the model to adaptively identify new and mutated malicious encrypted traffic, greatly improving the detection accuracy.

[0150] By introducing the dual screening mechanism of chi-square statistics and mutual information score in the feature extraction step, accurate screening of encrypted traffic features is achieved. This method significantly reduces the computational complexity while retaining key information, providing high-quality data input for subsequent deep learning models and solving the problem of low efficiency of traditional methods in processing large-scale data.

[0151] By designing a composite loss function that includes business-related losses and combining cross-validation and L2 regularization techniques, the overfitting problem of traditional models when processing large-scale data sets is solved. This method not only improves the generalization ability of the model, but also achieves efficient processing of massive data while ensuring detection accuracy, meeting the needs of real-time monitoring scenarios.

[0152] The overall solution adopts a distributed architecture design, deploying lightweight collection programs on edge nodes and complete functional modules on core nodes, realizing a multi-level adaptive data collection and processing mechanism. This architecture improves the scalability and robustness of the system while ensuring the real-time response capability of the system, providing a new technical solution for network security protection in the power industry.

[0153] Example 2

[0154] Reference Figure 1 - Figure 6 , which is the second embodiment of the present invention, provides an encrypted traffic anomaly detection method based on the attention mechanism. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0155] In view of the problems existing in the prior art, the present invention proposes an autoencoder (AE) architecture consisting of BiMAGRU and BiLSTM, aiming to contribute an innovative solution and system to the field of encrypted traffic anomaly detection on power grid platforms.

[0156] The autoencoder consists of two parts: the encoder and the decoder. The encoder uses a neural network architecture, such as a convolutional neural network (CNN) or a recurrent neural network (RNN), to achieve a fine mapping of the input data and convert it into a low-dimensional feature space. In essence, it is an efficient compression of the data, aiming to extract the core features of the data while reducing its complexity. Correspondingly, the decoder is responsible for reconstructing and mapping the low-dimensional feature vector output by the encoder back to the complete spatial dimension of the original data through a series of network layers. Its structure is symmetrical with the encoder, and it decompresses the data to ensure the restoration and reconstruction of the original information.

[0157] Within this framework, the solution of the present invention cleverly utilizes BiMAGRU's bidirectional information capture capabilities for sequence data, as well as BiLSTM's deep analysis of temporal features and contextual association enhancement of the self-attention mechanism. This method not only improves the ability to extract and reconstruct features of encrypted traffic, but also significantly improves the performance and robustness of the detection system in identifying potential malicious activities through its unique network structure design. To further optimize model performance, the solution of the present invention also uses cross-validation to improve the accuracy of data analysis, uses regularization technology to prevent the model from being overly complex, and uses an early stopping strategy to effectively avoid the risk of overfitting. The combined use of these technologies ensures the practical value of this model in the application of encrypted traffic anomaly detection in smart grids. Figure 1 A specific schematic diagram of the model of the present invention

[0158] The present invention is implemented in such a way that a bidirectional gated recurrent unit (BiMAGRU) enhanced by a multi-head attention mechanism serves as an encoding layer, and a bidirectional long short-term memory network (BiSALSTM) enhanced by a self-attention mechanism serves as a decoding layer, which comprises:

[0159] Step 1: Preprocess the data and perform systematic cleaning, balancing, and normalization on the pacp raw data to ensure data quality and model training efficiency;

[0160] Step 2: Extract and select features from the processed data, and encode the data to remove data features irrelevant to the protocol;

[0161] Step 3: Build an autoencoder model (AE): Build a bidirectional gated recurrent unit with multi-head attention enhancement in the encoding layer, build a bidirectional long short-term memory network with self-attention enhancement in the decoding layer, and initialize the model parameters. The combination of these two structures aims to improve the model's ability to extract and reconstruct complex features;

[0162] Step 4: Use cross entropy as the loss function and perform cross validation to measure the difference between the input data and the reconstructed data;

[0163] Step 5: Iteratively train the AE model and use regularization techniques and early stopping strategies to optimize the model’s fitting effect;

[0164] Step 6: Evaluate the performance of the AE model on an independent validation set and adjust parameters based on accuracy, loss and other indicators to improve the accuracy and generalization ability of the model and ensure the effectiveness of the model on new data;

[0165] Furthermore, the process of step 1 is specifically as follows: cleaning, balancing and normalizing pacp data. Clean irrelevant network messages that are not suitable for encrypted malicious traffic detection research, delete duplicate, damaged, and incompletely captured traffic flows or information; use data truncation and padding to keep the length of input data consistent; filter out non-encrypted traffic; to ensure data quality and model training efficiency. The goal of this step is to eliminate noise in the data, unify the data format, balance the data distribution, and scale the data range to provide a standardized and consistent data set for the input of subsequent models.

[0166] Furthermore, the process of step 2 is specifically as follows: calculating the chi-square statistic between the categorical features and the labels to quantify the correlation between the features and the target variable; secondly, evaluating the mutual information score between the numerical features and the labels to capture the information dependency between them. Through this series of calculations, those significant features with mutual information scores exceeding 0.01 and chi-square statistic P values ​​less than 0.1 are screened out, ensuring that the feature set used for model training is both representative and predictive; finally, the categorical features that are not eliminated are transformed by a hash function.

[0167] Furthermore, the process of step 3 is specifically as follows: first, a bidirectional gated recurrent unit structure enhanced by a multi-head attention mechanism is constructed in the encoding layer of the AE model to enhance the model's ability to capture bidirectional information of sequence data; in the decoding layer, a bidirectional long short-term memory network structure enhanced by a self-attention mechanism is constructed to deeply analyze temporal features and strengthen contextual associations. Then, the initial parameters of the model are set, and appropriate neural network parameters are selected, such as the number of layers, the number of neurons in each layer, and the type of activation function; in addition, other hyperparameters of the AE model, such as the learning rate and batch size, need to be determined. The formula for the bidirectional gated recurrent unit enhanced by the multi-head attention mechanism in the encoding layer is as follows, and the internal schematic diagram is shown in Figure 4 The bidirectional gated recurrent unit enhanced by the multi-head attention mechanism in the present invention uses the multi-head attention mechanism in both the forward and reverse candidate hidden states. In the multi-head attention mechanism, each head will calculate a set of query, key and value vectors, and these vectors in the present invention are obtained from the candidate hidden states through linear transformation. Each head calculates the attention score and normalizes it through the activation function to obtain the attention weight. Then each head performs weighted summation of the value vector by the attention weight to obtain the weighted output. Then the outputs of all heads are spliced ​​and transformed through another linear layer to obtain the final attention output. Finally, the attention output is combined with the original forward and reverse hidden states, and the final hidden state is updated through the gating mechanism. The specific formula is

[0168] ,

[0169] The same is true for the BiSALSTM of the decoding layer. The internal schematic diagram is as follows Figure 3 , the formula is as follows,

[0170] Q t =x t W q , K t =x t W k , V t =x t W v ,

[0171]

[0172] where α t is the attention score, Q is the query, K is the key and V is the value, d k Dimension of the key vector, z t Self-attention output

[0173] Furthermore, the process of step 4 is specifically as follows: using cross entropy as the loss function, the cross entropy function is defined as:

[0174] At the same time, the model is cross-validated. First, the data is divided into k subsets of equal size. In each training, k-1 are used as the training set and the rest as the validation set. The cross entropy loss is calculated and the above steps are repeated until each fold is used as a validation set once. The average value of multiple times is calculated as the final measure of model performance to evaluate the model generalization ability.

[0175] Furthermore, the process of step 5 is specifically as follows: using the stochastic gradient descent algorithm to update the gradient, the weights and biases of the encoder and decoder, and repeating this process until each layer reaches a convergence state; using L2 regularization and dropout function to prevent the model from overfitting; tracking the loss on the validation set, and when it no longer improves significantly within a certain number of training cycles, stopping training and optimizing the generalization ability of the model.

[0176] Furthermore, the process of step 6 is as follows: after the model is trained, the AE model is carefully evaluated on an independent validation set. Key indicators include accuracy, F1 score, and cross entropy loss function. Based on these key indicator results, model parameters are adjusted, such as optimizing the learning rate and network structure, to improve the accuracy and generalization ability of the model.

[0177] The present invention also provides an autoencoder (AE) system consisting of a bidirectional gated recurrent unit (BiMAGRU) enhanced by a multi-head attention mechanism and a bidirectional long short-term memory network (BiSALSTM) enhanced by a self-attention mechanism, such as Figure 4 ,include:

[0178] Data preprocessing module: used to extract, clean, balance and normalize data to provide standardized and consistent data sets for subsequent model input.

[0179] Data feature extraction module: used to screen and extract data features and perform data encoding at the same time, aiming to ensure that the final model training feature set is both representative and predictive.

[0180] Encoder building module: A bidirectional gated recurrent unit model enhanced by a multi-head attention mechanism is constructed as the encoding layer, aiming to enhance the model's ability to capture deep semantic information of sequence data, thereby improving the model's expressiveness and robustness in complex tasks.

[0181] Decoder building module: A bidirectional long short-term memory network model enhanced by a self-attention mechanism is constructed in the decoder to deepen the model's understanding of the implicit information in the sequence data.

[0182] Supervised fitting module: Iteratively trains the autoencoder model through operations such as gradient update, uses cross entropy as the loss function, and cross-validates the model.

[0183] Model evaluation and tuning module: Regularization and early stopping mechanisms are added, and hyperparameters are adjusted to improve the accuracy and generalization ability of the model.

[0184] Model result warning module: Apply the model to specific power grid data, and issue corresponding pop-up windows or red-mark intercept warnings for abnormal classification results.

[0185] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0186] Figure 1 The present invention is a flowchart of applying the multi-head attention mechanism enhanced bidirectional gated recurrent unit (BiMAGRU) and self-attention mechanism enhanced bidirectional long short-term memory network (BiSALSTM) autoencoder to the intelligent early warning of password threats in the power industry information system, and the steps specifically include:

[0187] S1: Data preprocessing: extraction, cleaning, balancing and normalization of pacp data, eliminating noise in the data, unifying the data format, balancing the data distribution, and scaling the data range to provide a standardized and consistent data set for the input of subsequent models.

[0188] S2: Data feature extraction: Calculate the chi-square statistic and mutual information score of the features, screen out significant features, and transform the categorical features that are not eliminated through the hash function. This is to ensure that the feature set used for model training is both representative and predictive.

[0189] S3: Encoder construction: A bidirectional gated recurrent unit model enhanced by a multi-head attention mechanism is constructed, which cleverly integrates the multi-head attention mechanism into the candidate hidden states of both the forward and reverse directions. Specifically, the model first aggregates the candidate hidden states of each head under the framework of the multi-head attention mechanism and calculates their query, key, and value vectors respectively. Subsequently, for each attention head, its corresponding attention score is calculated and normalized by the softmax function to obtain the attention weight. Next, each attention head performs a weighted sum operation on the corresponding value vector according to the obtained weight to form a weighted output. These outputs from different heads are concatenated together in sequence and transformed by an additional linear layer to obtain the final attention output. Finally, this attention output is fused with the original forward and reverse hidden states, and the final hidden state is updated through the gating mechanism. It aims to enhance the model's ability to capture deep semantic information of sequence data, thereby improving the model's expressiveness and robustness in complex tasks.

[0190] S4: Decoder construction: Construct a bidirectional long short-term memory network model enhanced by the self-attention mechanism. The model calculates the forward and reverse queries, keys and values ​​for each time step; then calculates the attention score and attention output zt through the above values; then uses zt to update the hidden state; and finally concatenates the forward and reverse hidden states. The purpose is to deepen the model's understanding of the implicit information in the sequence data.

[0191] S5: Supervised fitting: The stochastic gradient descent algorithm updates the gradient, the weights and biases of the encoder and decoder, and repeats this process until each layer reaches convergence. At the same time, cross entropy is used as the loss function, and the model is cross-validated to optimize the model parameters and evaluate the performance of the model on different data subsets, thereby improving the robustness and accuracy of the model in practical applications.

[0192] S6: Model evaluation and tuning: Use indicators such as F1 score and cross entropy loss function to adjust model parameters, such as optimizing learning rate and network structure, to improve the accuracy and generalization ability of the model.

[0193] S7: Model result warning: Substitute the trained model into the new data set and output the classification result. If the classification result is abnormal, a data pop-up warning will be issued to display information about the abnormal data point, such as the index, timestamp or other relevant attributes of the data point, and mark the data in red to intercept it when necessary.

[0194] It should be noted that the implementation strategy of the present invention is flexible and diverse, and can be specifically implemented through hardware facilities, software programming, or a clever combination of the two. At the hardware level, specific functions can be completed with the help of professional logic circuits; at the software level, the relevant program code can be stored in various storage media and efficiently executed by a microprocessor or customized hardware through a precise instruction set. A person of ordinary skill in the art can understand that the above-mentioned devices and methods can be implemented in the form of computer executable instructions and / or embedded processor control codes, which can be stored in a variety of carrier media, such as disks, CDs, DVD-ROMs, or stored in programmable memories such as read-only memories (firmware), or transmitted through optical or electronic signal carriers. The devices and components involved in the present invention can be implemented by hardware circuits constructed by semiconductor technologies such as ultra-large-scale integrated circuits, gate arrays, logic chips, transistors, etc., or by programmable hardware devices such as field programmable gate arrays and programmable logic devices. In addition, these functions can also be implemented by software programs executed by various processors, or in the form of a combination of hardware circuits and software, such as firmware.

[0195] Example 3

[0196] This embodiment also provides a computer device, which is suitable for a method for detecting anomalies in encrypted traffic based on an attention mechanism, and includes a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement a forced oscillation detection and positioning method for a distribution network as proposed in the above embodiment.

[0197] This embodiment further provides a storage medium on which a computer program is stored. When the program is executed by a processor, a forced oscillation detection and positioning method for a distribution network is implemented as proposed in the above embodiment.

[0198] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0199] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0200] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0201] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0202] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0203] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for detecting anomalies in encrypted traffic based on an attention mechanism, characterized by: The method comprises obtaining pap data and performing data preprocessing on the pap data to obtain preprocessed data; Constructing an autoencoder model, the autoencoder model comprising a bidirectional gated recurrent unit encoding layer enhanced by a multi-head attention mechanism and a bidirectional long short-term memory network decoding layer enhanced by a self-attention mechanism; The autoencoder model is used to perform encrypted traffic anomaly detection on the preprocessed data.

2. The method for detecting anomalies in encrypted traffic based on the attention mechanism as claimed in claim 1, characterized in that: The data preprocessing includes: Clean the pacp data and delete duplicate, damaged, and incompletely captured traffic information; Performing data truncation and padding on the cleaned data to keep the input data length consistent; The data are normalized.

3. The method for detecting anomalies in encrypted traffic based on the attention mechanism as claimed in claim 2, characterized in that: It also includes the steps of data feature extraction: calculating the chi-square statistic between categorical features and labels; evaluating the mutual information score between numerical features and labels; screening features with mutual information scores exceeding 0.01 and chi-square statistic P values ​​less than 0.1; and transforming the categorical features that are not eliminated through a hash function.

4. The method for detecting anomalies in encrypted traffic based on the attention mechanism as claimed in claim 3, characterized in that: The multi-head attention mechanism enhanced bidirectional gated recurrent unit encoding layer includes: Incorporate multi-head attention mechanisms into candidate hidden states in the forward and backward passes; Derive query, key, and value vectors from candidate hidden states via linear transformations; Calculate the attention score and normalize it through the activation function to get the attention weight; The outputs of each attention head are concatenated and transformed through a linear layer to obtain the attention output; The attention output is combined with the original forward and backward hidden states to update the final hidden state through a gating mechanism.

5. The method for detecting anomalies in encrypted traffic based on the attention mechanism as claimed in claim 4, characterized in that: The self-attention mechanism enhanced bidirectional long short-term memory network decoding layer includes: for each time step, calculating forward and reverse queries, keys and values; calculating attention scores and attention outputs through the values; using the attention outputs to update hidden states; and concatenating forward and reverse hidden states.

6. The method for detecting anomalies in encrypted traffic based on the attention mechanism as claimed in claim 5, characterized in that: It also includes supervised fitting steps: using cross entropy as the loss function; performing cross validation; updating the gradient using the stochastic gradient descent algorithm; and using L2 regularization and dropout functions to prevent model overfitting.

7. The method for detecting anomalies in encrypted traffic based on the attention mechanism as claimed in claim 6, characterized in that: Model evaluation and tuning steps are also included: Evaluate the performance of the autoencoder model on an independent validation set; Adjust model parameters based on indicators such as accuracy, F1 score, and cross entropy loss function; Optimize learning rate and network structure.

8. An attention-based encrypted traffic anomaly detection system, based on the attention-based encrypted traffic anomaly detection method according to any one of claims 1 to 7, characterized in that: It also includes a data preprocessing module, which obtains pap data and performs data preprocessing on the pap data to obtain preprocessed data; A construction module is used to construct an autoencoder model, wherein the autoencoder model includes a bidirectional gated recurrent unit encoding layer enhanced by a multi-head attention mechanism and a bidirectional long short-term memory network decoding layer enhanced by a self-attention mechanism; The model early warning module uses the autoencoder model to perform encrypted traffic anomaly detection on the preprocessed data.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the encrypted traffic anomaly detection method based on the attention mechanism are implemented as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the encrypted traffic anomaly detection method based on the attention mechanism according to any one of claims 1 to 7 are implemented.

Images