Network intrusion traffic detection method, system, device and storage medium

Abstract

The application discloses a network intrusion flow detection method, system, device and storage medium, and is applied to power grid flow detection. The method proposes a pre-training strategy based on a mask-reconstruction and an improved single-center loss function. In the pre-training stage, a SupMAE strategy is adopted, network flow is processed through an encoder, a decoder and a classification head, and reconstruction and classification losses are calculated to optimize the model. The improved stable single-center loss function enhances the detection capability and generalization performance of the model on unknown abnormal data. The training process is divided into stages to set parameters such as an optimizer and a learning rate. The experimental results on different data sets are better than those of existing methods, the network intrusion flow can be effectively detected, the detection precision and efficiency are improved, reliable protection is provided for network security, and the method is suitable for intrusion detection systems in various network environments.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network security, and particularly relates to a network intrusion flow detection method, system, device and storage medium. BACKGROUND

[0002] With the wide popularity and deep application of computer networks, network space has penetrated into the whole business chain of the power system. As a national critical information infrastructure, the digitalization process of the smart grid deeply accesses the Internet key devices such as SCADA systems, PMU synchronous measurement devices, and new energy station controllers, forming a new attack surface of "physical-information-digital" ternary fusion. According to statistics, 73% of network attacks on the power industry involve cross-attack of OT and IT systems, which prolongs the average recovery time by 4.2 times compared with traditional IT system attacks. This extensive application scenario provides attackers with more attack entry points and potential targets. On the one hand, common security threats intensify, attackers exploit multi-dimensional vulnerabilities to launch compound attacks, mainly including protocol-level attacks, service-layer attacks, and data-chain attacks. In terms of protocol-level attacks, attackers exploit the time-sensitive network (TSN) characteristics of IEC 61850 GOOSE / SV messages to construct malformed messages with legal syntax but violating power business logic, and the traditional rule-based detection engine has a low recognition rate for such semantic attacks. In terms of service-layer attacks, attackers launch distributed denial-of-service (DDoS) attacks by impersonating HPLC communication modules, and a regional power grid once formed a Botnet due to smart meter terminals being controlled, resulting in a significant decrease in the throughput of the electricity information collection system. In terms of data-chain attacks, attackers inject adversarial samples into digital twin training data, causing a significant increase in the misjudgment rate of LSTM flow detection models in a provincial dispatching center for hidden channel flows. The characteristics of the power industry further amplify network security risks, and attackers exploit network protocol vulnerabilities, operating system defects, and application security weaknesses to launch security attacks through various means such as malware implantation, phishing, and distributed denial-of-service attacks (DDoS). At the same time, as these attack methods continue to evolve, they become increasingly complex and stealthy, often combining multiple technologies, making accurate intrusion detection a highly challenging task. Once the intrusion is not timely prevented, it may lead to data leakage, service interruption, and other serious consequences, posing a serious threat to the smooth operation of the smart grid and greatly damaging the credibility of security services such as data confidentiality, integrity, and availability.

[0003] The existing rule template-based detection method (such as CN202311304667.6) performs static matching or field verification on the header field of the industrial protocol (Modbus, IEC61850, DNP3, etc.), and cannot deeply analyze the application layer semantics. Deep learning technology currently faces many difficulties in the application of power grid intrusion detection systems. For example, the traditional LSTM model (such as CN202411339260.1) has a gradient vanishing problem in long-period traffic analysis of the power grid. The main reason is that LSTM can handle long-term dependencies in sequences to some extent, but as the sequence length increases, the decay problem in the internal information transmission process gradually becomes prominent. Early information gradually weakens its influence on subsequent decisions after being transmitted through multiple time steps, making it difficult to effectively preserve long-term information and thus performing poorly in capturing long-range dependencies. For example, when detecting distributed attacks that span a long time interval, LSTM may not be able to accurately correlate the traffic patterns before and after, thereby affecting the overall detection ability of complex attack patterns. The Transformer model, with its powerful self-attention mechanism and multi-head attention mechanism, has achieved great success in natural language processing and other fields, providing new ideas and methods for power network intrusion detection. However, when applying it to power network intrusion detection, the Transformer also faces many challenges. On the one hand, it has high demands on data volume and computing resources, requiring a large amount of training data to fully learn the complex patterns in network traffic, and consuming a large amount of computing time and hardware resources during training. General training methods are difficult to meet its needs. On the other hand, due to the complexity of the power grid features, the data types in network intrusion detection are extremely diverse, with various abnormal traffic emerging and new attack methods constantly appearing. The patterns are diverse and difficult to predict. When facing such a complex and variable data environment, the Transformer model is difficult to effectively detect abnormal traffic, especially those with new attack features or small differences from normal traffic patterns, limiting its application in actual network security detection. These shortcomings affect the accuracy and timeliness of power network security detection, so further optimization is needed to improve detection efficiency and practicality. SUMMARY

[0004] The present application provides a network intrusion traffic detection method, system, device and storage medium, which uses mask pre-training technology and improved single-center loss to accelerate the training of the intrusion detection model and the detection ability of abnormal data, improves the precision and efficiency of network intrusion detection, and enhances network security.

[0005] Technical solution: In a first aspect, a network intrusion traffic detection method comprises the following steps:

[0006] After preprocessing the collected network traffic data, the training set and the test set are divided;

[0007] The training set data is masked according to the specified mask ratio, and the data after the mask operation is sent to the encoder based on the Transformer architecture. The feature vector is extracted by using the multi-head self-attention mechanism and the feedforward neural network layer. The feature vector output by the encoder is predicted by the classification head, and compared with the real class label to calculate the classification loss.

[0008] The feature vector extracted by the encoder is sent to the decoder based on the Transformer architecture for inverse transformation or inverse operation, and the decoding result of the traffic is output, and the reconstruction loss between the decoded traffic data and the original data is calculated.

[0009] The pre-training total loss function is constructed by weighted sum of the reconstruction loss and the classification loss, and the encoder and decoder model are pre-trained by using the AdamW optimizer combined with the cosine annealing decay learning rate scheduling strategy.

[0010] Adjust the AdamW optimizer parameters, and combine the cosine annealing decay learning rate scheduling strategy to retrain the encoder using the training set traffic data. After training to the specified round, the stable single center loss sSCL is calculated. The normal network traffic sample set is selected from the training batch, and the average Euclidean distance between the sample features and the normal traffic center point is calculated to obtain sSCL. Combine sSCL with the classification loss to form the fine-tuning total loss function, and continue training.

[0011] After the model training is completed, the test set is used for evaluation. The model set that meets the requirements after evaluation is integrated into the network security detection related system of digital power grid. The real-time network traffic data is received as input, and the detection result is output.

[0012] Further, the collected network traffic data is preprocessed, including:

[0013] Data format conversion and storage are performed according to specified requirements;

[0014] Data cleaning to remove noise and error data;

[0015] Numerical value processing of symbolic features and normalization processing of numerical features.

[0016] Further, the data after the mask operation is sent to the encoder based on the Transformer architecture. The feature vector is extracted by using the multi-head self-attention mechanism and the feedforward neural network layer, including:

[0017] The input network traffic data is divided into fixed-size blocks, and each block is converted into a vector representation through an embedding layer while adding position encoding to preserve the position information of the data;

[0018] The multi-head self-attention mechanism is used to extract feature relationships in the data from multiple dimensions of the network traffic, and a feedforward neural network layer is used to perform nonlinear transformation on the extracted features to output key features representing the network traffic data.

[0019] Further, the classification loss is measured by cross-entropy loss, and the reconstruction loss is measured by mean square error.

[0020] Further, the parameter settings of the AdamW optimizer are adjusted according to the training requirements of different stages. In the pre-training stage, according to the selected base learning rate, weight decay value and momentum parameter, combined with the warm-up stage setting in the cosine annealing decay learning rate scheduling strategy, the learning rate change of the optimizer in the initial stage and the subsequent training process is reasonably allocated, so that the model can converge stably when distributed training on multiple GPUs;

[0021] In the fine-tuning stage, in addition to adjusting the base learning rate and momentum parameter, a layer-wise learning rate decay mechanism is introduced, and according to the set warm-up number of rounds and total number of rounds, the corresponding learning rate scheduling strategy is used to make the AdamW optimizer better adapt to the model optimization requirements in this stage.

[0022] Further, the calculation formula of the stable single-center loss sSCL is:

[0023]

[0024] Ω nat is the set of normal network traffic samples in the current batch, n nat is the number of samples in the set, f i is the feature representation of sample x i , and C is the center point of normal network traffic.

[0025] The fine-tuning total loss function is as follows:

[0026] L total =L cls +λL sSCL

[0027] Where L cls is the classification loss of the classification head, and the hyperparameter λ is used to balance the key role of the two influences.

[0028] Further, the method uses a random scaling and cropping data augmentation method to randomly scale the size and crop part of the content of the input data in its original dimension in the pre-training stage.

[0029] Data augmentation optimization is performed in the fine-tuning stage using one or more of the following techniques:

[0030] Random transformation of input data using random augmentation operations to diversify;

[0031] Smooth the real label of the training sample using the label smoothing technique;

[0032] Use resampling techniques to address data set imbalance;

[0033] The training process applies a random inactivation technique, randomly discarding some neurons.

[0034] In the second aspect, a network intrusion traffic detection system comprises:

[0035] A data preparation module is used to preprocess the collected network traffic data and divide it into training and test sets;

[0036] The classification module is used to mask the training set data according to the specified mask ratio, and the masked data is fed into the encoder based on the Transformer architecture, and the multi-head self-attention mechanism and the feedforward neural network layer are used to extract the feature vector. The classification head is used to predict the traffic category of the feature vector output by the encoder, and the real category label is compared to calculate the classification loss;

[0037] The reconstruction module is used to feed the feature vector extracted by the encoder into the decoder based on the Transformer architecture to perform inverse transformation or inverse operation, output the decoding result of the traffic, and calculate the reconstruction loss between the decoded traffic data and the original data;

[0038] The pre-training module is used to construct a pre-training total loss function by weighting and summing the reconstruction loss and the classification loss, and use the AdamW optimizer combined with the cosine annealing decay learning rate scheduling strategy to pre-train the encoder and decoder model;

[0039] The fine-tuning module is used to adjust the AdamW optimizer parameters and combine the cosine annealing decay learning rate scheduling strategy to retrain the encoder using the training set traffic data. After training to the specified round, the stable single-center loss sSCL is calculated, the normal network traffic sample set is selected from the training batch, and the average Euclidean distance between the sample features and the normal traffic center point is calculated to obtain sSCL. sSCL is combined with the classification loss to form a fine-tuning total loss function for further training.

[0040] The deployment and application module is used to evaluate the model after training is completed using the test set, and the model that meets the requirements is integrated into the digital power grid network security detection related system. Through receiving real-time network traffic data as input, the detection result is output.

[0041] In a third aspect, the present application also provides a computer device, comprising: a processor; a memory; and a computer program stored in the memory and configured to be executed by the processor, the computer program, when executed by the processor, implements the steps of the network intrusion traffic detection method according to the first aspect of the present application.

[0042] In a fourth aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a processor, implements the steps of the network intrusion traffic detection method according to the first aspect of the present application.

[0043] In a fifth aspect, the present application also provides a computer program product comprising a computer program, the computer program, when executed by a processor, implements the steps of the network intrusion traffic detection method according to the first aspect of the present application.

[0044] Advantages: Compared with the prior art, the present application has the following advantages:

[0045] (1) The pre-training strategy based on mask-reconstruction accelerates the convergence speed of the model during the training process, enabling the model to learn network traffic data features more efficiently.

[0046] (2) The improved single-center loss function improves the model's ability to detect unknown abnormal data, enhances the model's sensitivity to detecting abnormalities in complex network environments, and effectively deals with new attacks and abnormal traffic patterns.

[0047] (3) Accurate detection of network intrusion traffic helps to discover and prevent network attacks in a timely manner, protecting network systems and user data security, and is of great significance to maintaining network security. The method of the present application performs outstandingly in accuracy, precision, recall rate and other indicators, has obvious advantages compared with existing methods, and can provide more reliable and efficient solutions for network security protection. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 is a flowchart of the power digital network intrusion traffic detection method according to the embodiment of the present application;

[0049] Figure 2 is an encoder architecture based on a mask-reconstruction detector;

[0050] Figure 3 is a schematic diagram of the multi-head attention mechanism in the encoder. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0052] To enable the mask-reconstruction and single-center loss improved network intrusion traffic detector of the present application to be effectively applied in practice, its implementation will be described in detail below, including specific operation steps, data processing process, and the cooperative working mechanism of each component.

[0053] Implementation environment preparation:

[0054] 1. Hardware configuration

[0055] GPU computing resources: Prepare 8 GPUs to support large-scale parallel computing of the model during the training process. These GPUs will work together to accelerate the training process of the model, improve computing efficiency, and ensure that the model can quickly process a large amount of network traffic data.

[0056] Memory and storage devices: Equipped with sufficient memory to store training data, model parameters, and intermediate calculation results. At the same time, a large-capacity storage device (such as a hard disk or a solid-state disk) is needed to save the training data set, the pre-trained model, and various log files and checkpoints during the training process.

[0057] 2. Software environment setup

[0058] Deep learning framework: Choose PyTorch as the deep learning framework, use its powerful functions and rich tool library to build and train the model. PyTorch provides efficient tensor computation, automatic differentiation, and flexible neural network modules, making it easy to implement complex model structures and algorithms in this invention.

[0059] Related dependent library installation: Install the dependent libraries related to PyTorch, such as numpy and pandas libraries for data processing and analysis, opencv-python library for image processing and data augmentation, and sklearn library for model evaluation index calculation, etc. Ensure that the versions of these libraries are compatible to ensure the stable operation of the entire system.

[0060] Data collection and preprocessing

[0061] 1. Network traffic data collection

[0062] Data collection point setting: When building an effective network intrusion traffic detection system, the comprehensiveness and accuracy of network traffic data are crucial. Therefore, data collection points need to be carefully deployed at key network nodes. Routers and switches, as core devices in the network, are ideal locations for data collection. Professional traffic collection equipment can be deployed on these key nodes.

[0063] 1) Hardware flow probe. By deploying hardware devices to monitor network traffic in depth, it has high-speed processing capability, can obtain detailed traffic data in real time without affecting the normal operation of the network, and capture all data packets flowing through the node, including the header information and payload content of the data packet.

[0064] 2) Use network flow monitoring tool software to collect traffic data. By installing the tool on servers or other suitable devices in the network, configure the corresponding monitoring rules to monitor and collect network traffic within a specific range.

[0065] Through fine configuration during deployment, it can effectively ensure that the collected data not only covers normal business running network traffic, but also effectively and accurately captures possible intrusion traffic, such as malicious scanning, attack attempts and other abnormal network behaviors. The collected data should have sufficient breadth and depth to fully reflect the true situation of network activity and provide a solid data foundation for subsequent analysis and detection.

[0066] Data format conversion and storage: The raw network traffic data collected usually has complex and diverse formats, which is difficult to directly use for subsequent processing and analysis. Therefore, it needs to be converted into a format suitable for model processing. Common conversion formats include pcap format, which is one of the standard formats in the field of network packet capture, and can save the original information of the data packet, including the timestamp, source address, destination address, protocol type, port number and packet content. Another commonly used conversion format is a text-based log format, which records network traffic data in a structured text form, and stores the key information of each data packet in a specific format and field, which is convenient for data query and management. In terms of data storage, reasonable data classification and archiving rules need to be formulated. According to the collection time, source IP address range, protocol type and other dimensions, the data can be classified and stored in different directories or file systems. At the same time, a perfect data indexing mechanism is established to quickly locate and retrieve specific data subsets. For long-term storage data, data backup and recovery strategies need to be considered to ensure data security and availability, providing efficient and convenient support for subsequent data management and use.

[0067] 2. Data preprocessing operations

[0068] Data cleaning: Remove duplicate records, error data and outliers in network traffic data to ensure the quality of data input into the encoder and avoid the influence of these interference factors on the accuracy of subsequent feature extraction.

[0069] Symbol feature numericalization: Network traffic data contains various symbolic features, such as protocol types of power grid, application layer protocols, etc. These symbolic features cannot be directly processed by the model and need to be converted into digital codes. For this purpose, establishing a mapping table is an effective method. For protocol types, common ones such as TCP, UDP, HTTP, HTTPS, iec103, iec104, goose, etc. are assigned unique integer identifiers. For example, TCP can be mapped to 1, UDP to 2, HTTP to 3, HTTPS to 4, iec103 to 5, iec104 to 6, goose to 7, etc. For application layer protocols, they are also coded according to their types and frequencies. In this way, when the model reads the data, it can convert the symbolic features into corresponding digital codes through the mapping table, and then perform subsequent calculations and analysis. This numerical processing method enables the model to understand and process these originally symbolic features, and incorporates them into the model's learning process, improving the model's understanding of network traffic features and helping to more accurately detect network intrusion behavior.

[0070] Data normalization: Normalize the numerical features (such as port number, packet length, etc.) in the data set collected by the digital power grid network security, and map them to a specific interval range (such as [0, 1]). The minimum-maximum normalization formula is used: where x is the original data, x ′ is the normalized data, min(x) and max(x) are the minimum and maximum values of the feature in the training set, respectively. This helps improve the training effect and generalization ability of the model, enabling the model to better handle data of different scales.

[0071] 3. Data set division

[0072] The preprocessed dataset is divided into training set, validation set and test set according to the proportion of 70%, 15% and 15%. Ensure that the training set has sufficient rich data information, so that the model can fully learn various patterns and features of network traffic. The training set is the main source of data for model learning. Through the learning of a large amount of normal and abnormal network traffic data in the training set, the model continuously adjusts its parameters to improve the recognition ability of intrusion traffic. The validation set plays a key monitoring role in the training process. After each iteration period of model training, the validation set data is used to evaluate the model, and the loss value, accuracy and other performance indicators are calculated. By observing the changes of these indicators on the validation set, whether the model has overfitting or underfitting problem can be found in time, and the model's hyperparameters such as learning rate and regularization coefficient are adjusted accordingly. The test set is used to evaluate the generalization ability and detection accuracy of the model. After the model training is completed, the test set data is used to comprehensively test the model, and the results can truly reflect the performance of the model on unseen data. Through accurate detection of test set data, the model's detection ability for unknown intrusion traffic in actual network environment is evaluated, ensuring that the model has good generalization performance and can be effectively applied to actual network security detection scenarios.

[0073] Model pre-training stage

[0074] 1. Mask-reconstruction based pre-training strategy implementation

[0075] In the field of network intrusion detection, Vision Transformer (ViT) faces challenges when processing network security data. Self-supervised pre-training techniques such as Masked Autoencoder (MAE) have application prospects, but still have limitations. MAE mainly focuses on optimizing the relationship between local data streams, lacking a comprehensive understanding of the entire network environment, which means it may not effectively identify complex attack patterns that require a global perspective. To solve this problem, SupMAE introduces network traffic labels such as known malicious activity markers, which can guide the model to learn more representative global features, thereby improving its accuracy in actual intrusion detection tasks. Compared with traditional MAE methods, SupMAE exhibits higher training efficiency, achieving high detection accuracy with fewer training rounds, greatly reducing the time cost of model training. This invention introduces SupMAE training strategy, adopts mask-reconstruction training framework, and refers to Figure 1The framework includes an encoder, a decoder and a classification head. The encoder is used for initial analysis and feature extraction of the input network traffic information, converting network traffic data into a processable feature form, mining key features such as traffic size, frequency, flow direction and protocol-related information, etc. The decoder "reconstructs" the partially masked or missing traffic patterns according to the visible traffic features extracted by the encoder, maps the features back to the original representation form through conversion and calculation operations, calculates the reconstruction loss to measure the reconstruction accuracy, and optimizes the parameters through back propagation. The classification head determines whether the network traffic belongs to intrusion behavior according to the features extracted by the encoder, performs classification prediction through global pooling and multilayer perceptron, introduces normalization method and activation function, and calculates classification loss to optimize the classification head and the whole model parameters.

[0076] Specifically, in the pre-training strategy based on mask-reconstruction, the mask operation is one of the key steps. For the input network traffic data, mask processing is performed according to a mask ratio of 75%. The network traffic data is regarded as a combination of elements with specific structure and characteristics, such as a data packet sequence containing source IP address, destination IP address, port number, protocol type, packet length, transmission timestamp, etc. Mask operation randomly masks 75% of these elements, indicating that this part of data is hidden in this input. Such mask method makes the model need to infer the information of the masked part according to the remaining 25% visible data during the training process, so as to promote the model to learn more representative and generalizable feature representation.

[0077] The input data after mask operation is sent to the encoder. When constructing the network intrusion traffic detector based on mask-reconstruction and single-center loss improvement, the construction and configuration of the encoder and decoder in the SupMAE training framework are one of the core links. The present application uses the PyTorch framework to construct the encoder and decoder modules, fully utilizes its advantages in programming convenience and rich library support, and strong functions in deep learning calculation and model construction.

[0078] Encoder Architecture and Configuration: When building the network intrusion flow detector based on the mask-reconstruction and single-center loss improvement, the encoder in the SupMAE training framework is constructed based on the Transformer architecture, which has unique advantages in processing network flow data. The encoder plays a key role in preliminary analysis and feature extraction throughout the detection process. It treats the complex network flow data received as a special kind of input information, analogous to pixel information in images, and mines key features from these seemingly chaotic flow data. These features cover multiple aspects of flow, such as flow size, which reflects the amount of data transmitted by the network; flow frequency, which reflects the frequency of data transmission; flow direction, which indicates the direction of data transmission; and protocol-related information, which is crucial for understanding the application layer protocol and network interaction rules to which the flow belongs. By extracting these key features, the encoder converts the original network flow data into a form that the model can understand and process, providing a solid data foundation for subsequent operations. When SupMAE is used for network intrusion detection, the encoder performs in-depth analysis on the pre-processed network flow data (including data cleaning to remove noise and error data, format conversion to adapt to model input requirements, etc.), extracts features closely related to intrusion detection, and these features play an indispensable key role in subsequent reconstruction and classification tasks, directly affecting the accuracy of the model's judgment on the nature of the flow.

[0079] Reference Figure 2 First, the input network flow data is pre-processed to adapt to the input requirements of the Transformer. After pre-processing, the network flow data is divided into fixed-size blocks, and each block is converted to a vector representation through an embedding layer, while adding position encoding to preserve the position information of the data. The core of the Transformer-based encoder is the multi-head self-attention mechanism. According to the characteristics of network flow data, the number of heads of the multi-head self-attention mechanism is set reasonably (for example, set to 8 heads). Each head can focus on different parts of the input data, so it can capture the feature relationships in the flow data from multiple angles. For example, when processing network flow data containing multiple protocols, different heads can focus on different aspects of features such as protocol type, source and destination address, port number, and packet content, effectively extracting complex patterns in network flow data. Figure 3As shown, after the multi-head self-attention mechanism, a feedforward neural network layer is followed. This layer is used to further nonlinearly transform the features processed by the self-attention mechanism, enhancing the expression ability of the model. The parameter settings of the feedforward neural network layer need to be adjusted according to the data characteristics and model requirements. For example, set the appropriate number of hidden layer neurons (such as 1024), and select the appropriate activation function (such as GELU) to introduce the nonlinear characteristics. Through multiple experiments and adjustments, the structure and parameters of the feedforward neural network layer are determined, so that the encoder can better learn and represent the key features in the network traffic data, providing high-quality feature vectors for subsequent decoding and classification tasks.

[0080] Decoder design and implementation: The core task of the decoder is to "reconstruct" the partially masked or missing traffic patterns based on the visible traffic features extracted by the encoder. In complex network environments, network traffic may have partial feature missing or tampered due to various reasons, such as network failure may cause packet loss, and attack interference may change the normal pattern of traffic. The role of the decoder is to use the visible normal traffic features provided by the encoder to try to restore these possible abnormalities through a series of complex conversion and calculation operations to judge the integrity and normality of the traffic. It first performs a series of complex mathematical transformations on the features output by the encoder, mapping them back to the original representation of network traffic, thereby achieving prediction and repair of missing or abnormal traffic parts.

[0081] The decoder in the present application is also based on PyTorch, and the design of the decoder is closely related to the encoder and is the inverse process. It is also based on the Transformer architecture, corresponding to the encoder, using the multi-head self-attention mechanism, the number of heads and the dimension of the features are smaller than the encoder, and the number of layers is also less, to speed up the pre-training process. In the network intrusion detection scenario where SupMAE is applied, the decoder will carefully process the visible traffic features filled with mask markers (representing missing traffic parts), and after a series of complex decoding operations, the features will be converted to a form similar to the original traffic data through a specific mapping function, generating the reconstructed traffic pattern.

[0082] Classification head construction and optimization: The classification head is responsible for determining whether the network traffic belongs to an intrusion behavior based on the features extracted by the encoder. It utilizes the analysis results of the encoder on network traffic features to classify the traffic through specific classification algorithms, determining whether it is normal traffic or abnormal intrusion traffic. The classification head consists of a global pooling layer and a multi-layer perceptron (MLP). The global pooling layer is used to aggregate the features output by the encoder to obtain a fixed-length global feature representation. The MLP contains multiple fully connected layers, with appropriate activation functions (such as ReLU) used between each layer to introduce non-linear transformations and enhance the model's expression ability. In the last layer of the MLP, according to the requirements of the classification task (binary classification of normal traffic and intrusion traffic), two output nodes are set, and the softmax function is used for probability normalization to obtain the classification prediction result. When constructing the classification head, the performance of the classification head is optimized by adjusting hyperparameters such as the number of neurons in the fully connected layer, learning rate, and regularization parameters. For example, using L2 regularization technology, a suitable regularization coefficient (such as 0.01) is set to prevent model overfitting; by adjusting the learning rate through experiments, a learning rate value (such as 0.001) is found that makes the model converge quickly and has stable performance. At the same time, techniques such as cross-validation are used to evaluate the performance of the classification head under different hyperparameter combinations, and the optimal hyperparameter configuration is selected to improve the classification accuracy of the classification head on network traffic, ensuring that the model can accurately identify intrusion traffic and provide reliable protection for network security.

[0083] Reconstruction loss calculation: During pre-training, for each training sample, the mask operation is a key pre-step for reconstruction loss calculation. The mask operation randomly masks a certain proportion (such as 75%) of elements in the original network traffic data, and these masked elements constitute the target part that the model needs to recover in the subsequent reconstruction process. For example, for a network traffic data sample containing source IP address, destination IP address, port number, protocol type, packet length, etc., the mask operation may randomly mask some packet length values or port numbers. The partially masked input data is input into the encoder-decoder model for reconstruction. The reconstruction loss between the reconstructed traffic data and the original data is calculated, and mean square error (MSE) is used as the loss metric standard. Through the backpropagation algorithm, the reconstruction loss is propagated from the decoder to the encoder layer by layer to update the parameters of the model. During backpropagation, the loss gradient calculated can reflect the influence of each parameter on the reconstruction loss. According to these gradient information, the weights and bias parameters of each layer in the encoder and decoder are adjusted, so that the model can learn the internal patterns and rules of network traffic data and improve the detection ability of abnormal traffic.

[0084] Classification loss calculation and adjustment: The classification loss is calculated using the cross-entropy loss function. For each training sample, the model's predicted probability distribution of traffic classes is compared with the true class label, and the cross-entropy loss is calculated. For each training sample, the model first processes the input network traffic data through the previously constructed encoder-decoder structure and classification head to predict the probability distribution of different classes, such as [prob_normal, prob_intrusion], where prob_normal represents the predicted probability of the sample being normal traffic, and prob_intrusion represents the predicted probability of being intrusion traffic. Then the predicted probability distribution is compared with the true class label. The true class label is known, for example, 0 represents normal traffic and 1 represents intrusion traffic. According to the calculation formula of the cross-entropy loss function, the cross-entropy loss between the predicted probability and the true label is calculated. However, in the network intrusion detection scenario, the network traffic class distribution often has the characteristics of imbalance. For example, in the actual network environment, the number of normal traffic may be much larger than the number of intrusion traffic. This imbalance can cause the model to tend to learn better for the normal traffic class with more quantity during the training process, while ignoring the feature learning of the intrusion traffic class, thereby affecting the model's detection ability for intrusion traffic. In order to solve this problem, different weights are assigned to different classes to balance the contribution of each class in the loss calculation. By minimizing the classification loss, the model can accurately distinguish between normal network traffic and intrusion traffic, learn the key features related to the class, and improve the discrimination ability for different types of traffic.

[0085] Overall loss weighted sum and optimization: The calculation of the overall loss is one of the core links of model training, which is to combine the reconstruction loss (Reconstruction Loss) and the classification loss (Classification Loss) according to the set weight parameters (λ rec and λ cls) is obtained by weighted summation. This process aims to enable the model to balance the learning focus between the reconstruction task and the classification task during training, thereby better adapting to the complexity of the network intrusion detection task. After determining the overall loss function, the model parameters are updated using the AdamW optimizer. The AdamW optimizer has significant advantages in deep learning model optimization. Its parameter settings are a base learning rate of 1.5e-4, a weight decay value of 0.05, and momentum parameters β1 = 0.9 and β2 = 0.95. The role of the weight decay value is to prevent model overfitting. It applies a certain degree of decay penalty to the model weights, so that the model does not excessively rely on specific patterns in the training data during the learning process, making the model more concise and having stronger generalization ability. Momentum parameters β1 and β2 play a key role in accelerating convergence during gradient descent. β1 is used to calculate the first moment estimate of the gradient, which enables the model to consider historical information of previous gradients when updating parameters, thereby maintaining a certain inertia in the parameter update direction, avoiding frequent changes in the update direction due to random fluctuations in the gradient, and accelerating convergence. β2 is used to calculate the second moment estimate of the gradient, which helps to adjust the adaptive update of the learning rate, allowing the model to reasonably adjust the learning rate according to the gradient changes at different training stages, further improving convergence efficiency. Throughout the training process, a cosine annealing decay learning rate scheduling strategy is adopted, which is of great significance to the stable convergence and performance improvement of the model. During the 400 rounds of pre-training, the first 20 rounds are the warm-up stage. In the warm-up stage, the learning rate starts from a very small value and gradually increases to the base learning rate of 1.5e-4. This gradual learning rate growth approach allows the model to start exploring the parameter space with a relatively gentle step at the beginning of training, avoiding the problem of excessive parameter update caused by an excessively large initial learning rate, such as causing the model parameters to oscillate violently near the optimal solution or even diverge, failing to converge to a stable optimal solution. As the training rounds increase, the learning rate gradually decays according to the cosine function. This decay method allows the learning rate to gradually approach zero in the later stages of training, allowing the model to make fine adjustments with smaller steps as it approaches the optimal solution, avoiding oscillation near the optimal solution, and ensuring that the model can stably converge to the global optimal solution, effectively avoiding the dilemma of falling into a local optimal solution. Through such a carefully designed optimization process, the model can achieve a delicate balance between learning local features (through the reconstruction task) and global features (for the classification task) of network traffic, comprehensively improving the overall performance of network intrusion detection, making it have stronger, more accurate, and more efficient intrusion traffic detection capabilities, providing solid and reliable protection for network security.

[0086] Model fine-tuning stage:

[0087] The fine-tuning stage only uses the encoder for feature extraction.

[0088] 1. Parameter adjustment and optimization

[0089] During the model fine-tuning phase, the AdamW optimizer continues to play a key role, but to better adapt to the specific needs of the fine-tuning task, its parameters are carefully adjusted. The base learning rate is adjusted from 1.5e-4 in the pre-training phase to 1e-3. This adjustment is based on the comprehensive consideration of model convergence speed and accuracy in the fine-tuning phase. In the fine-tuning process, the model has already learned certain network traffic feature representations in the pre-training phase, so the learning rate can be appropriately increased to enable the model to adapt more quickly to new data sets and task requirements in the early stages of fine-tuning, accelerating the learning process of the target data set features. However, a too high learning rate may lead to unstable situations in the model on the new data set, such as excessive parameter update amplitude, missing the optimal solution, or even causing the model to diverge. Therefore, after multiple experiments and analysis, the base learning rate of 1e-3 is determined to be a relatively appropriate value, which can ensure the rapid convergence of the model in the fine-tuning phase and maintain the stability of the model.

[0090] The weight decay value remains unchanged at 0.05, which continues to play an important role in preventing model overfitting in the fine-tuning phase. When facing new network intrusion detection tasks, although the data sets are different, the model may still face the risk of overfitting, especially in the fine-tuning process, the model may overfit to the specific patterns of the fine-tuning data set. By keeping the weight decay value unchanged, the model will punish the weights to a certain extent during the learning process, avoid large weights, make the model more concise and generalizable, so as to better handle unseen data and improve the model's application ability in actual network environment.

[0091] The adjustment of the momentum parameters is also an important part of the optimization in the fine-tuning phase. The momentum parameter β1 is adjusted from 0.9 in the pre-training phase to 0.9, and β2 is adjusted from 0.95 to 0.999. β1 is used to calculate the first moment estimate of the gradient, and is kept at 0.9 during fine-tuning, so that the model can continue to consider the historical information of the previous gradient when updating the parameters, maintain a certain inertia, and help the model converge faster on the fine-tuning dataset. The adjustment of β2 to 0.999 is used to calculate the second moment estimate of the gradient, which allows the model to more accurately track the changes in the gradient during the fine-tuning phase, especially when dealing with smaller batch data, and better adjust the learning rate, making the model more stable and efficient in learning during the fine-tuning process. In addition, the introduction of a layer-wise learning rate decay of 0.65 is an important strategy in the fine-tuning phase. In deep learning models, the parameters of different layers have different importance to the learning and representation ability of the model. As the depth of the network increases, the parameters close to the input layer tend to learn more general and basic features, while the parameters close to the output layer focus more on specific task feature learning. In the fine-tuning process, in order to make the model better adapt to the specific task requirements of the target dataset, it is necessary to adjust the learning rate of different layers differently. By introducing layer-wise learning rate decay, in the early stage of fine-tuning, the parameters close to the input layer will be updated with a relatively large learning rate, quickly adapting to the overall feature distribution of the new dataset; as the training progresses, the learning rate gradually decays, and the parameters close to the output layer will be fine-tuned with a relatively small learning rate in the later stage, focusing on optimizing specific feature representations related to the target task. This way of adjusting the learning rate layer by layer can make the model learn the features of the target dataset more accurately during the fine-tuning process, improve the adaptability of the model to the specific network intrusion detection task, and enhance the fitting ability of the model to the target dataset, so that it can more accurately detect network intrusion traffic in practical applications.

[0092] Learning rate scheduling strategy adjustment: In the fine-tuning phase, the learning rate scheduling strategy still adopts the cosine annealing decay strategy, but the warm-up rounds and total training rounds are adjusted to better adapt to the characteristics of the fine-tuning task. The total training rounds are adjusted to 100, and the warm-up rounds are adjusted from 20 rounds in the pre-training phase to 5 rounds. This adjustment is based on the differences between the fine-tuning dataset and the pre-training dataset, as well as the state of the model after pre-training. In the early stage of fine-tuning, the model already has certain initial parameter values, and does not need a long warm-up phase to slowly increase the learning rate. A shorter warm-up round can make the model enter the normal training state relatively quickly at the beginning of fine-tuning, avoiding wasting too much computing resources and training time in the warm-up phase. During the warm-up process, the learning rate gradually increases from a small value to the base learning rate 1e-3, allowing the model to smoothly transition from the pre-training state to the fine-tuning state, avoiding instability caused by an excessively large initial learning rate.

[0093] 2. Improved single-center loss function application

[0094] Single-center loss (SCL) principle: Network traffic data is highly dynamic, and abnormal patterns are constantly evolving. Existing methods have difficulty effectively distinguishing normal and abnormal traffic features. Single-center loss (SCL) focuses on compressing the intra-class distance of normal network traffic and increasing the inter-class difference with abnormal traffic. This allows the network to learn more discriminative feature representations in the feature space, improving traffic differentiation and reducing false positives and false negatives. Given a network traffic dataset, samples are embedded in a vector space by a neural network. SCL sets a center point for normal network traffic, and the loss function is defined as

[0095]

[0096] where M nat represents the average Euclidean distance between the normal network traffic representation and the center point (c) in a batch of training data, and M man represents the average Euclidean distance between the abnormal network traffic representation and the center point c. By minimizing the distance of normal traffic to the center point and ensuring that the distance of abnormal traffic is at least one margin greater than that of normal traffic, the distinction between normal and abnormal traffic is enhanced.

[0097] Stable single-center loss (sSCL) improvement: While SCL can help improve model performance within the domain, it may lead to a relaxed classification decision surface and decreased generalization performance. The sSCL used in this invention only narrows the distance of normal network traffic in the feature space, improving intra-class compactness to enhance generalization ability. The specific sSCL calculation method is as follows: During the 100 epoch fine-tuning training of the model, when the training reaches the set epoch number (E S = 20), the stable single-center loss (sSCL) is started. In each training batch, the model will make a detailed distinction of the data, selecting the samples belonging to normal network traffic to form a set Ω nat . Each sample x i in this set has its unique feature representation f(x i ), which is obtained after processing by the previous encoder and contains information about the network traffic in multiple dimensions, such as traffic size, frequency, flow direction, and protocol-related features. When calculating sSCL, the feature representation f(x i ) of the normal network traffic samples in the current batch is compared with the pre-set normal network traffic center point c. The Euclidean distance between them is calculated, with the formula |f i -C|2, which measures the deviation of each sample feature from the center point. Then, the sum of these distances for all samples in the set Ω nat is calculated and divided by the number of samples nnat , to get the average Euclidean distance, i.e., the value of sSCL. The formula of sSCL is where Ω nat is the set of normal network traffic samples in the current batch, n nat is the number of samples in the set, f(x i ) is the feature representation of sample x i , and c is the center point of normal network traffic. The center point is the center point of all normal traffic features in the training set, which can help further distinguish normal traffic samples and abnormal traffic samples and enhance the detection ability of the model.

[0098] By calculating sSCL in this way, the focus of model training is directed to narrowing the distance of normal network traffic in the feature space. In the feature space, normal network traffic samples may originally be distributed relatively dispersedly. By minimizing sSCL, the model will promote these samples to converge towards the center point, so that the feature representation of normal traffic is more compact, and the intra-class difference gradually decreases. The improvement of this intra-class compactness helps to enhance the generalization ability of the model. When the model faces new, unseen normal network traffic, since it has learned the compact normal traffic feature pattern, it can more accurately identify and process these traffics, reduce the possibility of misjudgment, and thus better adapt to the complex and variable network environment, improve the stability and accuracy of the identification of normal traffic patterns.

[0099] Optimizing the model by combining with cross-entropy loss:

[0100] Combining sSCL with cross-entropy loss is a key strategy to optimize the performance of the model. Cross-entropy loss focuses on the mapping of samples to discrete label space in model training. It measures the difference between the probability distribution of traffic categories predicted by the model and the true class label, and promotes the model to learn the classification boundary that can accurately distinguish different classes of traffic. While sSCL directly acts on feature embedding, optimizing the distribution of normal and abnormal traffic from the perspective of feature space. The combination of the two forms the total loss function L total = L cls + λL sSCLwhere the hyper-parameter λ plays a key role in balancing the influence of both, which is set to 5e-4 in the present invention. During the training process, adjusting the value of λ can flexibly control the degree of influence of sSCL and cross-entropy loss on model learning. When λ is larger, it means that the proportion of sSCL in the total loss function increases, and the model will pay more attention to optimizing the distribution of normal and abnormal traffic in the feature space. The model will strive to make the normal traffic more closely clustered in the feature space, while pushing the abnormal traffic away from the distribution area of normal traffic, thereby improving the detection ability of unknown abnormal data. For example, when facing new, complexly disguised intrusion traffic, the model can better identify the differences between it and normal traffic through the optimized feature space distribution, and timely discover potential security threats. On the contrary, when λ is smaller, the contribution of cross-entropy loss in the total loss function is relatively larger, and the model relies more on cross-entropy loss for classification boundary learning. At this time, the model will focus on adjusting the parameters in the classification head to improve the ability to accurately judge the traffic category, and ensure that accurate classification decisions can be made under known traffic patterns. Through this combination, the model achieves the dual goals of classification boundary learning (through softmax loss) and feature space optimization (through sSCL). In complex network intrusion detection scenarios, network traffic not only has a complex class distribution, but also new attack methods are emerging, and attack patterns are increasingly diversified. This combination enables the model to accurately distinguish between normal and intrusion traffic while deeply understanding the internal structure and distribution rules of traffic in the feature space, thereby comprehensively improving the comprehensive performance of the model in complex network intrusion detection scenarios and effectively dealing with various complex network security challenges, providing reliable security protection for network systems.

[0101] 3. Data augmentation strategy application

[0102] Random augmentation operation implementation: During the fine-tuning phase, random augmentation operations are applied to network traffic data processing as an important data augmentation method. The core purpose is to increase the richness of training data by diversifying random transformations on input data, so as to prompt the model to learn more robust feature representation to better cope with various intrusion traffic patterns in complex and variable network environment. The data augmentation method using random augmentation is used to randomly transform the input network traffic data. For example, adding a small amount of noise, randomly cropping or padding the traffic data, etc. The diversity of training data is increased, so that the model can learn more robust feature representation. When implementing these random augmentation operations, the degree of augmentation must be strictly controlled. Excessive change of the original features of the data may cause the model to fail to learn the true traffic pattern, thereby affecting the learning effect. For example, if the added noise is too large, it may completely mask the features of the original data, making it difficult for the model to recover the effective information from the noise; excessive cropping may destroy the key structure and feature relationship in the data, causing the model to learn the wrong pattern. Therefore, through multiple experiments and analysis, according to the characteristics of the data set and the performance of the model, appropriate augmentation parameters such as noise standard deviation, cropping ratio, etc. are determined to find a balance between increasing data diversity and preserving the original data features, and to ensure that random augmentation operations can effectively improve the performance of the model.

[0103] Other technical assistance optimization: label smoothing technology plays an important role in the fine-tuning stage. In actual network traffic data, there may be some uncertainty or ambiguity in the label, especially for some boundary conditions or difficult to distinguish traffic types. Label smoothing technology can alleviate the problem of model overfitting by smoothing the true label of the training sample to a certain extent, and improve the generalization ability of the model. Specifically, for binary classification problems (normal traffic and intrusion traffic), if the true label is 0 (normal traffic), it will be adjusted to a small number close to 0 (such as 0.1), and the label of the other class (intrusion traffic) will be adjusted to the corresponding small number close to 1 (such as 0.9); vice versa. This processing method makes the model not rely too much on the accurate label value in the training process, but learns the probability distribution of the label, so as to better adapt to the possible label ambiguity in the actual network and improve the classification accuracy of the model when facing unknown traffic. Resampling technology is an effective means to deal with the problem of unbalanced data set. In network intrusion detection data set, normal traffic usually accounts for the majority, while intrusion traffic is relatively small. This imbalance will cause the model to learn the characteristics of normal traffic in the training process, and ignore the characteristics of intrusion traffic, thereby affecting the detection ability of the model to intrusion traffic. Resampling technology adjusts the class distribution of the data set by oversampling the minority class samples (intrusion traffic samples) or undersampling the majority class samples (normal traffic samples), so that the model can better learn the characteristics of each class of samples. Adopting mixup and cutmix technology, linear combination or cutting and splicing different samples, further enriching the distribution of training data, enhancing the adaptability of the model to complex data patterns. In addition, in the training process, the random inactivation technology is applied, and part of the neurons are randomly discarded to prevent the coadaptation between neurons, so that the neurons cannot rely too much on the output of other neurons, so as to promote the neurons to learn more independent and representative features. In this way, the model can more flexibly adjust the activation state of the neurons when facing different network traffic patterns, improve the detection ability of the model to various intrusion traffic patterns, and enhance the stability and reliability of the model in complex network environment. By comprehensively applying these data enhancement and optimization technologies, the performance of the model in the fine-tuning stage is improved, so that it can better cope with various intrusion traffic patterns in the actual network environment.

[0104] Model evaluation and application:

[0105] 1. Model evaluation index calculation

[0106] Precision, recall calculation: After the model training is completed, use the test set to evaluate the model. Calculate the accuracy, precision and recall of the model on the test set. Accuracy reflects the proportion of correctly predicted samples in the total samples, precision measures the proportion of truly abnormal samples in the predicted abnormal samples, and recall represents the proportion of correctly predicted abnormal samples in the actual abnormal samples. Through these indicators, the performance of the model is comprehensively evaluated, and the ability of the model to distinguish between normal network traffic and intrusion traffic is judged.

[0107] F1 score calculation and comprehensive evaluation: Calculate the F1 score, which is the harmonic mean of precision and recall, providing an indicator for comprehensive evaluation of model performance. F1 score can avoid the one-sidedness that may occur when considering precision or recall alone. When both precision and recall are high, F1 score will also be high. By analyzing the F1 score and the specific values of accuracy, precision, and recall, the overall performance of the model in network intrusion detection tasks is evaluated, and it is determined whether the model meets the needs of practical applications.

[0108] 2. Model deployment and application in actual network environment

[0109] System integration and interface development: Integrate the trained model into the network security detection related system of digital power grid, develop the corresponding interface, so that the model can receive real-time network traffic data as input and output detection results. Ensure that the model is seamlessly integrated with existing network infrastructure, realize real-time monitoring and intrusion detection of network traffic.

[0110] Real-time monitoring and alarm mechanism: In the actual network environment, the model monitors real-time network traffic. When detecting suspected intrusion traffic, according to the set threshold and decision rules, timely alarm signals are sent. Alarm information should include the characteristics of intrusion traffic, possible attack types and related network connection information, etc., so that network administrators can quickly take measures to respond, such as blocking attack sources, strengthening network protection, etc.

[0111] Continuous optimization and update: With the changes in network environment and the emergence of new attack methods, collect new network traffic data regularly, retrain and optimize the model, update the parameters and knowledge base of the model, so that the model can adapt to the changing network security threats and maintain good detection performance. At the same time, according to the feedback and needs in practical application, improve the structure and algorithm of the model, further improve the accuracy and efficiency of the model.

[0112] Through the above implementation, the network intrusion flow detector based on the mask-reconstruction and single-center loss improved network of the application can effectively operate in the actual network environment, realizes high-precision detection of network intrusion flow, and provides reliable guarantee for network security. In the implementation process, the parameters and configuration of the model need to be flexibly adjusted according to the specific network environment and demand, so as to achieve the best detection effect.

[0113] The application also provides a network intrusion flow detection system, comprising:

[0114] A data preparation module is configured to divide a training set and a test set after preprocessing the collected network flow data.

[0115] A classification module is configured to perform mask processing on the training set data according to a specified mask proportion, send the data after mask operation to an encoder based on a Transformer architecture, extract a feature vector by using a multi-head self-attention mechanism and a feedforward neural network layer, perform flow category prediction on the feature vector output by the encoder by using a classification head, compare with the real category label, and calculate a classification loss.

[0116] A reconstruction module is configured to send the feature vector extracted by the encoder to a decoder based on the Transformer architecture, perform inverse transformation or inverse operation, output a decoding result of the flow, and calculate a reconstruction loss between the decoded flow data and the original data.

[0117] A pre-training module is configured to construct a pre-training total loss function by weighted sum of the reconstruction loss and the classification loss, and pre-train the encoder and the decoder model by using an AdamW optimizer combined with a cosine annealing decay learning rate scheduling strategy.

[0118] A fine-tuning module is configured to adjust the AdamW optimizer parameters, combine the cosine annealing decay learning rate scheduling strategy, retrain the encoder by using the training set flow data, calculate a stable single-center loss sSCL after training to a specified round, pick out a normal network flow sample set from the training batch, calculate the average Euclidean distance between the sample feature and the normal flow center point to obtain the sSCL, combine the sSCL and the classification loss to form a fine-tuning total loss function, and continue training.

[0119] A deployment and application module is configured to evaluate by using the test set after the model training is completed, integrate the model meeting the demand into a network security detection related system of a digital power grid, receive real-time network flow data as input, and output a detection result.

[0120] The present invention also provides a computer device, comprising: a processor; a memory; and a computer program, the computer program being stored in the memory and configured to be executed by the processor, wherein the computer program, when executed by the processor, implements the steps of the network intrusion traffic detection method as described above.

[0121] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the network intrusion traffic detection method described above.

[0122] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the network intrusion traffic detection method described above.

[0123] This invention is described with reference to a flowchart of a method according to embodiments of the invention. It should be understood that each step in the flowchart and combinations thereof can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose or special-purpose device, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the device, embedded processor, or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 A component that specifies a function in one or more processes.

[0124] These computer program instructions may also be stored in a computer-readable storage medium that enables a boot device, embedded processor, or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction components, which are implemented in a process Figure 1 The function specified in one or more processes.

[0125] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 Steps of a specified function in one or more processes.

Claims

1. A network intrusion traffic detection method, characterized by, The method comprises the following steps: After preprocessing the collected network traffic data, the training set and the test set are divided; The training set data is processed according to the specified mask proportion, the data after the mask operation is sent to the encoder based on the Transformer architecture, the feature vector is extracted by using the multi-head self-attention mechanism and the feedforward neural network layer, the feature vector output by the encoder is predicted by using the classification head, and the classification loss is calculated by comparing with the real class label; The feature vector extracted by the encoder is sent to the decoder based on the Transformer architecture to perform inverse transformation or inverse operation, and the decoding result of the traffic is output, and the reconstruction loss between the decoded traffic data and the original data is calculated; The pre-training total loss function is constructed by weighted sum of the reconstruction loss and the classification loss, and the encoder and decoder model are pre-trained by using the AdamW optimizer combined with the cosine annealing decay learning rate scheduling strategy; The AdamW optimizer parameters are adjusted, and the cosine annealing decay learning rate scheduling strategy is combined to retrain the encoder using the training set traffic data, and the stable single-center loss sSCL is calculated after training to the specified round, the normal network traffic sample set is selected from the training batch, the average Euclidean distance between the sample feature and the normal traffic center point is calculated to obtain sSCL, and the sSCL and the classification loss are combined to form the fine-tuning total loss function for continuous training. After the model training is completed, the test set is used for evaluation, and the model set meeting the requirements is integrated into the network security detection related system of the digital power grid, the real-time network traffic data is received as input, and the detection result is output.

2. The method of claim 1, wherein, The collected network traffic data is preprocessed, including: Data format conversion and storage are performed according to specified requirements; Data cleaning is performed to remove noise and error data; Numerical value processing is performed on symbol features, and normalization processing is performed on numerical features.

3. The method of claim 1, wherein, The data after the mask operation is sent to the encoder based on the Transformer architecture, the feature vector is extracted by using the multi-head self-attention mechanism and the feedforward neural network layer, including: The input network traffic data is divided into fixed-size blocks, and each block is converted into a vector representation through an embedding layer, and position encoding is added to retain the position information of the data; The multi-head self-attention mechanism is used to extract the feature relationship in the data from multiple dimensions of the network traffic, the extracted features are nonlinearly transformed by the feedforward neural network layer, and the key features representing the network traffic data are output.

4. The method of claim 1, wherein, The classification loss is measured by using cross-entropy loss; the reconstruction loss is measured by using mean square error.

5. The method of claim 1, wherein, The parameter settings of the AdamW optimizer are adjusted according to the training requirements in different stages, in the pre-training stage, according to the selected base learning rate, weight decay value and momentum parameter, combined with the warm-up stage setting in the cosine annealing decay learning rate scheduling strategy, the learning rate change of the optimizer in the initial stage and the subsequent training process is reasonably allocated, so that the model can stably converge when distributed training on multiple GPUs; In the fine-tuning stage, in addition to adjusting the base learning rate and momentum parameters, a layer-by-layer learning rate decay mechanism is introduced, and according to the set warm-up rounds and total rounds, the corresponding learning rate scheduling strategy is used to make the AdamW optimizer better adapt to the model optimization requirements in this stage.

6. The method of claim 1, wherein, The stable single-center loss sSCL calculation formula is: Ω nat is a set of normal network traffic samples in the current batch, n nat is the number of samples in the set, f i is the feature representation of sample x i , and C is the center point of normal network traffic. The fine-tuning total loss function is as follows: L total = L cls + λL sSCL where L cls is the classification loss of the classification head, and the hyperparameter λ balances the key role of both influences.

7. The method of claim 1, wherein, In the pre-training stage, the random scaling and cropping data enhancement method is used to randomly scale the size and crop part of the content of the input data in its original dimension. In the fine-tuning stage, one or more of the following techniques are used for data enhancement optimization: Random augmentation operations are used to diversify the random transformation of input data; Label smoothing techniques are used to smooth the true labels of training samples; Resampling techniques are used to solve data set imbalance; Randomly inactivate some neurons during the training process.

8. A network intrusion traffic detection system characterized by, Comprise: A data preparation module is used to preprocess the collected network traffic data and divide it into a training set and a test set; A classification module is used to mask the training set data according to the specified mask ratio, and the masked data is fed into an encoder based on the Transformer architecture to extract feature vectors using multi-head self-attention mechanisms and feedforward neural network layers. The classification head is used to predict the traffic class of the feature vector output by the encoder and compare it with the true class label to calculate the classification loss. A reconstruction module is used to feed the feature vectors extracted by the encoder into a decoder based on the Transformer architecture for inverse transformation or inverse operation to output the decoding result of the traffic and calculate the reconstruction loss between the decoded traffic data and the original data. A pre-training module is used to construct a pre-training total loss function by weighting and summing the reconstruction loss and classification loss, and use the AdamW optimizer combined with the cosine annealing decay learning rate scheduling strategy to pre-train the encoder and decoder models. A fine-tuning module is used to adjust the AdamW optimizer parameters and use the training set traffic data to retrain the encoder using the cosine annealing decay learning rate scheduling strategy. After training to the specified round, the stable single-center loss sSCL is calculated, the normal network traffic sample set is selected from the training batch, and the average Euclidean distance between the sample features and the normal traffic center point is calculated to obtain sSCL. The sSCL and the classification loss are combined to form a fine-tuning total loss function for further training. A deployment and application module is used to evaluate the model using the test set after the model training is completed, integrate the model that meets the requirements into the network security detection related system of the digital power grid, and output the detection result by receiving real-time network traffic data as input.

9. A computer device, comprising: Comprise: a processor; a memory; and a computer program stored in the memory and configured to be executed by the processor, which, when executed by the processor, implements the steps of the network intrusion traffic detection method of any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the network intrusion traffic detection method of any one of claims 1-7.

Images