Network attack detection method fusing momentum contrast learning and Transform

Abstract

The invention discloses a network attack detection method fusing momentum contrast learning and Transform, and the method comprises the steps: collecting a large amount of untagged data of an industrial system network, carrying out the unsupervised learning through employing the momentum contrast learning and fusing the Transform, extracting the high-dimensional and long-sequence features in the data, constructing a feature embedding space with high discrimination, and carrying out the recognition of a network attack through employing the momentum contrast learning and the Transform. The method comprises the following steps of: performing dimension reduction on high-dimensional features by using a principal component analysis method, taking the features subjected to dimension reduction as input of a BDSCAN density clustering algorithm, performing clustering classification, optimizing parameters in the DBSCAN algorithm through a grey wolf optimization algorithm, outputting optimal algorithm parameters through iteration, and ending a program until a detection performance requirement is met. The method can be better suitable for a scene with high-dimensional and complex data, is suitable for a condition that the data has no label, improves the overall network detection accuracy and robustness, and enhances the network detection practicability and adaptability.

Description

Technical Field

[0001] The present invention belongs to the technical fields of information security detection, network security, and complex industrial control system security, and particularly relates to a network attack detection method integrating momentum contrast learning and Transformer. Background Art

[0002] Network attack detection technology is a key technology for the security control of industrial control systems at present. With the continuous development of industrial control systems towards informatization and networking, the harm of network attacks to industrial systems is increasing day by day. Attackers can cause production equipment to malfunction or shut down by tampering with or forging control instructions. In severe cases, key equipment can be damaged. Attackers can also invade the network to carry out a denial-of-service (DoS) attack or intercept and tamper with communication data, resulting in data transmission interruption or information being tampered with, affecting the normal operation of the system. These two types of attacks will disrupt the production process, cause economic losses, and even trigger safety accidents, bringing problems such as difficult measurement of the output of industrial control systems under network attacks and unreliable data.

[0003] To cope with the harm brought by network attacks to industrial control systems (ICS), the current main network attack detection technologies include industrial firewall technology, intrusion prevention system (IPS), behavior analysis system, etc. Industrial firewall technology can filter and control the traffic in and out of the ICS network and can identify and intercept malicious traffic targeting specific ICS protocols (such as Modbus, DNP3, etc.). The intrusion prevention system (IPS) can monitor and prevent potential attacks on industrial control systems in real time. By deeply analyzing network traffic, detecting known attack features or abnormal behaviors, it can automatically take preventive measures. The behavior analysis system can automatically identify unauthorized operation behaviors or attack activities by constructing a baseline of normal operations and monitoring abnormal changes in system behaviors in real time.

[0004] Although there are many network attack detection technologies on the market, due to the relatively late start of the technology in this field in China, there are still problems such as low detection accuracy, detection being subject to the randomness and uncertainty of network attacks, and high false alarms and high missed alarms often occurring in behavior-based intrusion detection in the current domestic industrial system security control. To improve the accuracy of network detection in industrial control systems, it is of great significance to carry out research on network attack technology based on momentum contrast learning and Transformer for industrial system security control in order to solve the above problems. Summary of the Invention

[0005] In view of the problems existing in the existing network attack detection technologies, such as low detection accuracy and lack of data labels, the present invention proposes a network attack detection method that combines momentum contrast learning and Transformer. By collecting a large amount of unlabeled data from industrial system networks, using momentum contrast learning to fuse Transformer for unsupervised learning, extracting high-dimensional and long-sequence features in the data, constructing a highly discriminative feature embedding space, using the principal component analysis method to reduce the dimension of the high-dimensional features, and taking the dimension-reduced features as the input of the BDSCAN density clustering algorithm for clustering classification. The parameters in the DBSCAN algorithm are optimized through the gray wolf optimization algorithm, and the optimal algorithm parameters are output iteratively until the detection performance requirements are met and the program ends.

[0006] The technical solution adopted by the present invention is: a network attack detection method that combines momentum contrast learning and Transformer, which includes the following steps:

[0007] S1: Collect an industrial network data set with high dimensions, complexity, and lack of labels, and preprocess the data set to construct a data set with a time series structure and containing time series features;

[0008] S2: Based on the preprocessed data set, use the Transformer model with a multi-head attention learning mechanism as the encoder in momentum contrast learning to extract high-dimensional and long-sequence features in the network control traffic;

[0009] S3: Perform unsupervised learning through the momentum contrast learning algorithm, construct positive sample pairs and negative sample queues, calculate the contrast learning loss function, and by minimizing the loss function, construct a discriminative embedding space to pull similar traffic closer and push different traffic farther away, so that the model learns highly discriminative feature representations;

[0010] S4: Analyze the high-dimensional and long-sequence features in the extracted network control traffic, use the principal component analysis method for dimensionality reduction, perform linear dimensionality reduction on the high-dimensional embedding, remove the noise and redundancy in the features, enhance the accuracy and effectiveness of the data features, and improve the efficiency of identifying and classifying network attacks in the next step;

[0011] S5: Based on the dimension-reduced data feature set, perform unsupervised clustering and anomaly detection through the DBSCAN density clustering algorithm, classify normal control traffic and known attack traffic, and mark the outliers as potential unknown attack candidates to detect potential unknown attacks;

[0012] S6: To improve the classification and detection capabilities of the DBSCAN algorithm, use the gray wolf optimization algorithm for parameter optimization. Through continuous iterative optimization, the optimal parameters are returned to step S5 to meet the detection performance indicators and achieve good network attack detection performance.

[0013] Furthermore, in the step S1, the complex data preprocessing includes: based on the connection order of the data traffic, converting each field of the traffic into a vector and then concatenating them into a sequence for input, constructing a dataset with a time series structure, and the original feature expression of each connection data traffic is:

[0014] x i ∈R din , i = 1, …, T

[0015] where: x i is the original input feature of the i-th connection sample, d in is the original feature dimension, R is the real number field, and T is the number of data samples.

[0016] Furthermore, in the step S2, based on the self-attention mechanism of Transformer, the encoder has a stronger global perception ability to extract better long sequence data features. The Key encoder in the momentum contrast learning performs momentum update to fit the training of the multi-parameter Transformer model.

[0017] Even further, the design of the Transformer model includes:

[0018] S2-1: Design an input embedding layer. Through the embedding operation, form an embedding matrix for all connection sequences to map the original features into a high-dimensional space for subsequent modeling. The expressions of the embedding operation and the embedding matrix are:

[0019] e i = W e x i + b e

[0020]

[0021] where: is the embedding representation of the i-th connection sample, is the weight matrix that maps the input to the embedding space of Transformer, is the embedding matrix, b e is the bias vector, d model is the unified embedding dimension in the Transformer model, and d in is the original feature dimension;

[0022] S2-2: Design a position encoding unit to inject position information into the sequence. Use sine-cosine position encoding, and its expression is:

[0023]

[0024] where: PE(pos,2i) Denotes the encoded value of the 2i-th dimension in the vector at position pos, PE (pos,2i+1) Denotes the encoded value of the (2i + 1)-th dimension in the vector at position pos, i represents the index in the feature dimension, pos represents the index of the position in the sequence, d model Denotes the embedding dimension;

[0025] The expression for adding positional encoding to the embedding matrix is:

[0026]

[0027] S2-3: Design a multi-head self-attention mechanism layer, linearly map each element in the sequence, and construct the three core matrices Query, Key, and Value in the attention mechanism. The expression is:

[0028]

[0029] Where: Is the embedding representation with positional encoding added, Are the weight matrices for generating Query, Key, and Value respectively, Are the Query, Key, and Value matrices respectively, T is the sequence length, d k Is the dimension of each attention head, d model Is the embedding dimension;

[0030] Calculate the attention score. The calculation formula is:

[0031]

[0032] Where: QK T ∈R T×T Is the scoring matrix, indicating the matching degree of each position to all other positions, Is the scaling factor to avoid the gradient vanishing due to too large inner product value. softmax(·) is the normalization function to obtain the attention weights, V is the value vector, and the final attention output is obtained through weighted combination;

[0033] Perform multi-head concatenation and projection. The expression is:

[0034] head i =Attention(Q i ,K i ,V i )

[0035] MSA(E)=Concat(head1,...,head h )WO

[0036] where: head i represents the output of the i-th attention head, h represents the number of attention heads, represents the final output of the multi-head attention, represents the linear projection matrix after output concatenation;

[0037] S2-4: Design residual connections and layer normalization to avoid gradient vanishing and improve training stability. Its expression is:

[0038] X1 = LayerNorm(E + MSA(E))

[0039] where: E is the input embedding or the output of the previous layer, MSA(E) is the output of the multi-head attention, and LayerNorm(·) represents independent standardization for each sample;

[0040] S2-5: Design a feed-forward network that independently acts on each time step in the sequence using an MLP. Its expression is:

[0041] FFN(x) = max(0, xW1 + b1)W2 + b2

[0042] where: is the weight matrix of the first layer, is the bias term of the first layer, is the weight matrix of the second layer, is the bias term of the second layer, d ff = 4d model , max(0, ·) is the ReLU activation function;

[0043] After adding the residual connection, the expression is:

[0044] X2 = LayerNorm(X1 + FFN(X1))

[0045] S2-6: Perform average pooling to output the semantic representation of the entire connected sequence. Its expression is:

[0046]

[0047] where: is the global representation vector of the sequence for subsequent momentum contrast learning projection, X 2,i is the vector representation of the i-th connection at the output of the last layer, and T is the sequence length;

[0048] S2-7: Use an MLP as a projection head to project z into the contrast learning space for subsequent momentum contrast learning. Its expression is:

[0049]

[0050] Among them: is the comparison feature vector projected into the low-dimensional space.

[0051] Furthermore, the step S3 includes:

[0052] S3-1: Design the Query encoder and Key encoder in momentum contrast learning, and output the Query and Key representations respectively. The expressions are:

[0053] x q = Aug1(x)

[0054] x k = Aug2(x)

[0055]

[0056] Among them: x q and x k are two augmented forms of the same sequence x, is the feature vector output by the encoder Query, is the feature vector output by the encoder Key, θ q and θ k are the update parameters of the encoder Query and encoder Key respectively;

[0057] S3-2: Construct the positive sample pairs and negative sample queue in momentum contrast learning. The positive sample pairs are composed of a pair of query samples and positive samples constitute. The query samples and positive samples are two augmented forms of the same sequence. The negative sample queue is composed of the Key feature vectors of other sequences. During each backpropagation update, the current sample is pushed into the negative sample queue. When the maximum preset queue length is exceeded, the "earliest" is pushed out;

[0058] S3-3: Calculate the contrast loss function. By minimizing the loss function, similar sequence samples are pulled closer, and different sample sequences are pushed farther away, so that the model learns discriminative feature representations. The expression of the loss function used is:

[0059]

[0060] Among them: τ is the temperature parameter, adjusting the sharpness of the function, and K is the number of negative samples.

[0061] Even further, θ q is updated through backpropagation, θ kUpdate θ by momentum method k The update formula is as follows:

[0062] θ k = m·θ k +(1 - m)·θ q , where m ∈ [0, 1).

[0063] Furthermore, the steps of the principal component analysis dimensionality reduction method used in step S4 include:

[0064] S4-1: Decentralize the data, and the formula used is:

[0065]

[0066] where: N is the number of samples, z i is the i-th high-dimensional feature vector, is the mean vector of all sample features, Z is the feature matrix of all samples, and Z c is the centered feature matrix;

[0067] S4-2: Calculate the covariance matrix, and the formula used is:

[0068]

[0069] S4-3: Perform eigenvalue decomposition, and the formula used is:

[0070]

[0071] where: Λ = diag(λ1, λ2,..., λ d ) is the eigenvalue matrix, V = [v1, v2,..., v d is the eigenvector with each column being the covariance matrix, and v i represents the i-th principal component direction;

[0072] S4-4: Perform principal component projection, and the formula used is:

[0073] Z PCA = Z c V[:, 1:d′]

[0074] where: Z PCA is the representation of all samples in the low-dimensional principal component space, d′ is the dimensionality reduction target dimension, and V[:, 1:d′] represents intercepting the first d′ principal component vectors.

[0075] Furthermore, step S5 includes:

[0076] S5-1: Set the radius ε and the density threshold MinPts. ε represents the neighborhood range, and MinPts represents the minimum number of points;

[0077] S5-2: Define density reachability:

[0078] For any point p, if the number of points in its neighborhood ≥ MinPts, then p is a core point, and the expression is:

[0079] If

[0080] N ε (p) = {q ∈ R d′ ||p - q|| ≤ ε}

[0081] And

[0082] N ε (p) ≥ MinPts

[0083] Then p is a core point;

[0084] S5-3: Define the clustering rule to identify normal traffic, known attack traffic, and unknown attack traffic.

[0085] Furthermore, the clustering rule is:

[0086] The neighborhoods of core points are connected to form clusters;

[0087] Points that fall within the neighborhood of a core point but are not core points themselves are classified as border points;

[0088] Points that are neither core nor within the neighborhood of any core point are noise points;

[0089] Clusters and their border points are normal traffic or known attack traffic, while noise points are potential unknown attack traffic.

[0090] Furthermore, in the step S6, the steps of using the grey wolf optimization algorithm for parameter optimization are:

[0091] S6-1: Perform initialization operations, set the grey wolf population size, ε range, MinPts range, and the maximum number of iterations T;

[0092] S6-2: Construct individual solutions, where each individual X i represents a parameter combination, and the expression is:

[0093] X i = (ε i , MinPts i )

[0094] S6-3: Calculate the fitness of each individual according to the fitness function, and the fitness function is:

[0095]

[0096] Where: ai Denote the sample X i The average distance from the sample X to the samples in the same cluster, b i Denote the sample X i The average distance from the sample X to the nearest cluster sample. The closer the calculated value of the Silhouette(·) function is to 1, the higher the fitness;

[0097] S6-4: Based on the calculation of the fitness function, select the current optimal individual, i.e., the individual X with the highest fitness α , the sub-optimal individual X β , the third sub-optimal individual X δ , and the remaining individuals are ordinary grey wolves X ω ;

[0098] S6-5: Update the positions of the ordinary grey wolves, that is, update the parameter combination according to the following update formula:

[0099]

[0100] Where: is the distance vector, is the coefficient, linearly decreasing from 2 to 0, controlling the balance between exploration and exploitation, is the random vector;

[0101] S6-6: Return the parameters ∈ and MinPts optimized by the grey wolf optimization algorithm to step S5, and observe the performance of network attack detection according to the set termination conditions until the requirements for network attack detection performance are met.

[0102] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention effectively addresses the uncertainties existing in industrial control systems by adopting the innovative technology of combining momentum contrast learning with Transformer, broadens the application scenarios of network attack detection, is more applicable to scenarios with high-dimensional and complex data compared with the prior art, and is suitable for the case of unlabeled data, improving the overall network detection accuracy and robustness, and enhancing the practicality and adaptability of network detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0103] Figure 1 is the overall flowchart of the network attack detection method proposed by the present invention.

[0104] Figure 2 is the structure of the Transformer model of the present invention.

[0105] Figure 3 is the principle block diagram of the momentum contrast learning of the present invention.

[0106] Figure 4This is the principle block diagram of the DBSCAN density clustering algorithm of the present invention. Detailed implementation mode

[0107] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0108] As Figures 1 to 4 , a network attack detection method integrating momentum contrast learning and Transformer, which includes the following steps:

[0109] Step1: Collect an industrial network data set with high-dimensional complexity and lack of labels, preprocess the complex data, and construct a data set with a time series structure and containing time series features.

[0110] Among them, the complex data preprocessing includes, based on the connection order of the data traffic, converting each field of the traffic into a vector and then splicing them into a sequence for input, constructing a data set with a time series structure, and the original feature expression of each connection data traffic is:

[0111]

[0112] Among them: x i is the original input feature of the i-th connection sample, and d in is the original feature dimension.

[0113] Step2: Based on the preprocessed data set, use a Transformer model with a multi-head attention learning mechanism as the encoder in momentum contrast learning to extract high-dimensional and long-sequence features in the network control traffic.

[0114] In this step, the Transformer model is used as the encoder in momentum contrast learning. Based on the self-attention mechanism of the Transformer, the encoder has a stronger global perception ability and can extract better long-sequence data features. The Key encoder in momentum contrast learning performs momentum updates, which is suitable for training a Transformer model with multiple parameters. The combination of momentum contrast learning and Transformer can achieve good unsupervised learning.

[0115] The design of the Transformer model includes:

[0116] S2-1: Design an input embedding layer. Through the embedding operation, all connection sequences are formed into an embedding matrix to map the original features into a high-dimensional space for subsequent modeling. The embedding operation and the expression of the embedding matrix are:

[0117] e i = W e x i + b e

[0118]

[0119] Wherein: is the embedding representation of the i-th connection sample, is the weight matrix, mapping the input to the embedding space of the Transformer, is the embedding matrix, b e is the bias vector, d model is the unified embedding dimension in the Transformer model, d in is the original feature dimension.

[0120] S2-2: Design a position encoding unit to inject position information into the sequence. Use sine-cosine position encoding, and its expression is:

[0121]

[0122] Wherein: PE (pos,2i) represents the encoding value of the 2i-th dimension in the vector at position pos, PE (pos,2i+1) represents the encoding value of the (2i + 1)-th dimension in the vector at position pos, i represents the index in the feature dimension, pos represents the index of the position in the sequence, d model represents the embedding dimension.

[0123] The expression for adding the position encoding to the embedding matrix is:

[0124]

[0125] S2-3: Design a multi-head self-attention mechanism layer to linearly map each element in the sequence and construct the three core matrices Query, Key, and Value in the attention mechanism. Its expression is:

[0126]

[0127] Wherein: is the embedding representation with the position encoding added, are the weight matrices for generating Query, Key, and Value respectively, are the Query, Key, and Value matrices respectively, T is the sequence length, d k is the dimension of each attention head, d model is the embedding dimension.

[0128] Calculate the attention score, and the calculation formula is:

[0129]

[0130] Among them: QK T ∈R T×T is the scoring matrix, representing the matching degree of each position to all other positions. is the scaling factor to avoid the vanishing gradient caused by too large inner product value. softmax(·) is the normalization function to obtain the attention weights. V is the value vector, and the final attention output is obtained through weighted combination.

[0131] Perform multi-head concatenation and projection, and its expression is:

[0132] head i = Attention(Q i , K i , V i )

[0133] MSA(E) = Concat(head1,..., head h )W O

[0134] Among them: head i represents the output of the i-th attention head, h represents the number of attention heads, represents the final output of the multi-head attention, represents the linear projection matrix after output concatenation.

[0135] S2-4: Design the residual connection and layer normalization to avoid the vanishing gradient and improve the training stability. Its expression is:

[0136] X1 = LayerNorm(E + MSA(E))

[0137] Among them: E is the input embedding or the output of the previous layer, MSA(E) is the output of the multi-head attention, and LayerNorm(·) represents independent standardization processing for each sample.

[0138] S2-5: Design the feed-forward network, and use MLP to act independently on each time step in the sequence. Its expression is:

[0139] FFN(x) = max(0, xW1 + b1)W2 + b2

[0140] Among them: is the first-layer weight matrix, is the first-layer bias term, is the second-layer weight matrix, is the second-layer bias term, d ff = 4d model , and max(0, ·) is the ReLU activation function.

[0141] After adding the residual connection, the expression becomes:

[0142] X2 = LayerNorm(X1 + FFN(X1))

[0143] S2-6: Perform average pooling to output the semantic representation of the entire connection sequence, and its expression is:

[0144]

[0145] Where: is the global representation vector of the sequence, used for subsequent momentum contrast learning projection, X 2,i is the vector representation output by the i-th connection at the last layer, and T is the sequence length.

[0146] S2-7: Use an MLP as the projection head to project z into the contrast learning space for subsequent momentum contrast learning, and its expression is:

[0147]

[0148] Where: is the contrast feature vector projected into the low-dimensional space.

[0149] Step3: Perform unsupervised learning through the momentum contrast learning algorithm, construct positive sample pairs and negative sample queues, calculate the contrast learning loss function, and by minimizing the loss function, construct a discriminative embedding space to pull similar flows closer and push different flows farther away, so that the model learns discriminative feature representations. Specifically, it includes:

[0150] S3-1: Design the Query encoder and Key encoder in momentum contrast learning, and output the Query and Key representations respectively. The expressions are:

[0151] x q = Aug1(x)

[0152] x k = Aug2(x)

[0153]

[0154] Where: x q and x k are two augmented forms of the same sequence x, is the feature vector output by the encoder Query, is the feature vector output by the encoder Key, θ q and θ k are the update parameters of the encoder Query and the encoder Key respectively.

[0155] Specifically: θ q Update θ through backpropagation k Update θ through momentum method k The update formula is:

[0156] θ k = m·θ k + (1 - m)·θ q , m ∈ [0, 1)

[0157] S3-2: Construct positive sample pairs and negative sample queues in momentum contrast learning. The positive sample pair consists of a pair of query samples and positive samples constitute. The query sample and the positive sample are two augmented forms of the same sequence. The negative sample queue is composed of Key feature vectors of other sequences. During each backpropagation update, the current sample is pushed into the negative sample queue. When the maximum preset queue length is exceeded, the "earliest" is pushed out.

[0158] S3-3: Calculate the contrast loss function. By minimizing the loss function, similar sequence samples are pulled closer, and different sample sequences are pushed farther away, enabling the model to learn discriminative feature representations. The expression of the loss function used is:

[0159]

[0160] where: τ is the temperature parameter, adjusting the sharpness of the function, and K is the number of negative samples.

[0161] Step4: Analyze the high-dimensional complex features in the extracted network control, use the principal component analysis method for dimensionality reduction, perform linear dimensionality reduction on the high-dimensional embedding, remove noise and redundancy in the high-dimensional complex features, enhance the accuracy and effectiveness of data features, and improve the efficiency of the next step of identifying and classifying network attacks.

[0162] Among them, the steps of the principal component analysis dimensionality reduction method include:

[0163] S4-1: Decentralize the data. The formula used is:

[0164]

[0165] where: N is the number of samples, z i is the i-th high-dimensional feature vector, is the mean vector of all sample features, Z is the feature matrix of all samples, and Z c is the centered feature matrix.

[0166] S4-2: Calculate the covariance matrix using the formula:

[0167]

[0168] S4-3: Perform eigenvalue decomposition using the formula:

[0169]

[0170] where: Λ = diag(λ1, λ2,..., λ d ) is the eigenvalue matrix, V = [v1, v2,..., v d is the eigenvector with each column being the eigenvector of the covariance matrix, and v i represents the direction of the i-th principal component.

[0171] S4-4: Perform principal component projection using the formula:

[0172] Z PCA = Z c V[:, 1:d′]

[0173] where: Z PCA is the representation of all samples in the low-dimensional principal component space, d′ is the target dimension of dimensionality reduction, and V[:, 1:d′] represents the first d′ principal component vectors intercepted.

[0174] Step5: Based on the data feature set after dimensionality reduction, perform unsupervised clustering and anomaly detection through the DBSCAN density clustering algorithm, classify normal control traffic and known attack traffic, and detect potential unknown attacks by marking outliers as unknown attack candidates. Specifically, it includes:

[0175] S5-1: Set the radius ε and density threshold MinPts. ε represents the neighborhood range, and MinPts represents the minimum number of points.

[0176] S5-2: Define density reachability:

[0177] For any point p, if the number of points in its neighborhood ≥ MinPts, then p is a core point, and the expression is:

[0178] If

[0179] N ε (p) = {q ∈ R d′ ∣ ||p - q|| ≤ ε}

[0180] and

[0181] N ε (p) ≥ MinPts

[0182] then p is a core point.

[0183] S5-3: Define the clustering rules to identify normal traffic, known attack traffic, and unknown attack traffic. The clustering rules are as follows:

[0184] The neighborhoods of core points are connected to each other to form clusters;

[0185] Points that fall within the neighborhood of a core point but are not core points themselves are classified as border points;

[0186] Points that are neither core points nor within the neighborhood of any core point are noise points.

[0187] Clusters and their border points are normal traffic or known attack traffic, while noise points are potential unknown attack traffic.

[0188] Step6: To improve the classification and detection capabilities of the DBSCAN algorithm, the Grey Wolf Optimization algorithm is used for parameter optimization. Through continuous iterative optimization, the optimal parameters are returned to Step S5 to meet the detection performance indicators and achieve good network attack detection performance.

[0189] Among them, the steps for parameter optimization using the Grey Wolf Optimization algorithm are as follows:

[0190] S6-1: Perform initialization operations, set the size of the Grey Wolf population, the range of ε, the range of MinPts, and the maximum number of iterations T. S6-2: Construct individual solutions. Each individual X i represents a parameter combination, and the expression is:

[0191] X i =(ε i , MinPts i )

[0192] S6-3: Calculate the fitness of each individual according to the fitness function. The fitness function is:

[0193]

[0194] where: a i represents the average distance between sample X i and samples in the same cluster, b i represents the average distance between sample X i and samples in the nearest cluster. The closer the calculated value of the Silhouette(·) function is to 1, the higher the fitness.

[0195] S6-4: Based on the calculation of the fitness function, select the current optimal individual, that is, the individual X α with the highest fitness, the sub-optimal individual X β , the third-best individual X δ , and the remaining individuals are ordinary Grey Wolves X ω .

[0196] S6-5: Update the position of the ordinary grey wolf, that is, update the parameter combination according to the following update formula:

[0197]

[0198] Where: is the distance vector, is the coefficient, linearly decreasing from 2 to 0, controlling the balance between exploration and exploitation, is the random vector.

[0199] S6-6: Return the parameters ∈ and MinPts optimized by the grey wolf optimization algorithm to step S5, map the clustering labels to the external true labels, calculate performance metrics such as precision, recall, and F1-score, and observe the performance of network attack detection until the network attack detection performance requirements are met. The calculation formulas for the performance metrics are as follows:

[0200]

[0201] Where: Precision represents the precision rate, measuring the proportion of samples identified as attacks that are actually attacks; Recall represents the recall rate, measuring the proportion of actual attacks detected; F1 is the F1-score, representing the weighted harmonic mean of precision and recall, taking into account false positives and false negatives; TP, FP, and FN represent the correctly detected attacks, misjudging normal traffic as attacks, and undetected attack traffic, respectively.

[0202]

[0203] Where: FAR represents the false positive rate, measuring the proportion of normal traffic misjudged as attacks; FNR represents the false negative rate, measuring the proportion of real attacks missed; TN represents the correctly detected normal traffic.

[0204] The basic principle, main features, and advantages of the present invention have been clarified in the above content. Those skilled in the art should understand that the above embodiments are only used to illustrate the present invention and do not constitute a limitation on its protection scope. Any technical solutions implemented by equivalent replacement and other means based on the concept of the present invention should be covered within the protection scope of the present invention. The parts not detailed in the present invention belong to the prior art or can be realized with the aid of the prior art.

Claims

1. A network attack detection method that integrates momentum contrastive learning and Transformer, characterized in that It includes the following steps: S1: Collect industrial network datasets that are high-dimensional, complex, and lack labels, and preprocess the datasets to construct datasets with a time series structure and containing time series features; S2: Based on the preprocessed datasets, use a Transformer model with a multi-head attention learning mechanism as the encoder in momentum contrast learning to extract high-dimensional, long sequence features in network control traffic; S3: Conduct unsupervised learning through the momentum contrast learning algorithm, construct positive sample pairs and negative sample queues, calculate the contrast learning loss function, and by minimizing the loss function, construct a discriminative embedding space to pull similar traffic closer and push different traffic farther away, enabling the model to learn discriminative feature representations; S4: Analyze the high-dimensional, long sequence features in the extracted network control traffic, use the principal component analysis method for dimensionality reduction, perform linear dimensionality reduction on the high-dimensional embedding, eliminate noise and redundancy in the features, enhance the accuracy and effectiveness of data features, and improve the efficiency of identifying and classifying network attacks in the next step; S5: Based on the dimensionality-reduced data feature set, conduct unsupervised clustering and anomaly detection through the DBSCAN density clustering algorithm, classify normal control traffic and known attack traffic, and mark outliers as potential unknown attack candidates to detect potential unknown attacks; S6: To improve the classification and detection capabilities of the DBSCAN algorithm, use the Grey Wolf Optimization algorithm for parameter optimization. Through continuous iterative optimization, return the optimal parameters to step S5 to meet the detection performance indicators and achieve good network attack detection performance.

2. The network attack detection method integrating momentum contrastive learning and Transformer according to claim 1, characterized in that, In the above step S1, the complex data preprocessing includes: based on the connection order of data traffic, convert each field of the traffic into a vector and then splice them into a sequence for input, and construct a dataset with a time series structure. The original feature expression of each connection data traffic is: x i ∈R din ,i = 1, …, T where: x i is the original input feature of the i-th connection sample, d in is the original feature dimension, R is the real number field, and T is the number of data samples.

3. The network attack detection method integrating momentum contrastive learning and Transformer according to claim 1, characterized in that, In the above step S2, based on the self-attention mechanism of the Transformer, the encoder has stronger global perception ability to extract better long sequence data features. The Key encoder in momentum contrast learning performs momentum update to suit the training of the multi-parameter Transformer model.

4. A network attack detection method integrating momentum contrastive learning and Transformer according to claim 1 or 3, characterized in that, The design of the Transformer model includes: S2-1: Design an input embedding layer. Through the embedding operation, form an embedding matrix for all connection sequences, realize the mapping of original features to a high-dimensional space for subsequent modeling. The expressions of the embedding operation and the embedding matrix are: e i = W e x i + b e Wherein: is the embedded representation of the i-th connection sample, is the weight matrix that maps the input to the embedding space of the Transformer, is the embedding matrix, b e is the bias vector, d model is the unified embedding dimension in the Transformer model, d in is the original feature dimension; S2-2: Design a position encoding unit to inject position information into the sequence, and use sine-cosine position encoding. Its expression is: Where: PE (pos,2i) represents the encoded value of the 2i-th dimension in the vector at position pos, and PE (pos,2i+1) represents the encoded value of the (2i + 1)-th dimension in the vector at position pos, i represents the index in the feature dimension, pos represents the index of the position in the sequence, and d model represents the embedding dimension; The expression for adding the position encoding to the embedding matrix is: S2-3: Design a multi-head self-attention mechanism layer to linearly map each element in the sequence and construct the three core matrices Query, Key, and Value in the attention mechanism. Their expressions are: Wherein: is the embedding representation with positional encoding, are the weight matrices for generating Query, Key, and Value respectively, are the Query, Key, and Value matrices respectively, T is the sequence length, d k is the dimension of each attention head, d model is the embedding dimension; Calculate the attention score. The calculation formula is: where: QK T ∈R T×T is the scoring matrix, representing the matching degree of each position to all other positions, is the scaling factor to avoid the vanishing gradient caused by too large inner product value. softmax(·) is the normalization function to obtain the attention weights. V is the value vector, and the final attention output is obtained through weighted combination; Conduct multi-head splicing and projection. Its expression is: head i = Attention(Q i , K i , V i ) MSA(E) = Concat(head1,...,head h )W O Where: head i represents the output of the i-th attention head, h represents the number of attention heads, represents the final output of the multi-head attention, represents the linear projection matrix after output concatenation; S2-4: Design residual connection and layer normalization to avoid gradient disappearance and improve training stability. Its expression is: X1 = LayerNorm(E + MSA(E)) Where: E is the input embedding or the output of the previous layer, MSA(E) is the output of the multi-head attention, and LayerNorm(·) represents independent normalization for each sample; S2-5: Design a feed-forward network, and use MLP to act independently on each time step in the sequence. Its expression is: FFN(x) = max(0, xW1 + b1)W2 + b2 Wherein: is the first-layer weight matrix, is the first-layer bias term, is the second-layer weight matrix, is the second-layer bias term, d ff = 4d model , and max(0, ·) is the ReLU activation function; After adding the residual connection, the expression is: X2 = LayerNorm(X1 + FFN(X1)) S2-6: Perform average pooling to output the semantic representation of the entire connected sequence. Its expression is: Wherein: is the global representation vector of the sequence, which is used for subsequent momentum contrast learning projection, X 2,i is the vector representation of the output of the i-th connection in the last layer, and T is the sequence length; S2-7: Use MLP as a projection head to project z into the contrast learning space for subsequent momentum contrast learning. Its expression is: Wherein: is the comparison feature vector projected into the low-dimensional space.

5. The network attack detection method integrating momentum contrastive learning and Transformer according to claim 1, characterized in that, The step S3 includes: S3-1: Design the Query encoder and Key encoder in momentum contrast learning, and output the Query and Key representations respectively. The expression is: x q = Aug1(x) x k = Aug2(x) where: x q and x k are two enhanced forms of the same sequence x, is the feature vector output by the encoder Query, is the feature vector output by the encoder Key, θ q and θ k are the update parameters of the encoder Query and the encoder Key, respectively; S3-2: Construct positive sample pairs and negative sample queues in momentum contrastive learning. The positive sample pairs are composed of a pair of query samples and positive samples . The query samples and positive samples are two augmented forms of the same sequence. The negative sample queue is composed of the Key feature vectors of other sequences. During each backpropagation update, the current sample is pushed into the negative sample queue. When the maximum preset queue length is exceeded, the "earliest" is pushed out. S3-3: Calculate the contrast loss function. By minimizing the loss function, similar sequence samples are pulled closer, and different sample sequences are pushed farther apart, so that the model can learn discriminative feature representations. The expression of the loss function used is: Where: τ is the temperature parameter to adjust the sharpness of the function, and K is the number of negative samples.

6. The network attack detection method integrating momentum contrastive learning and Transformer according to claim 5, characterized in that θ q Update θ through backpropagation k Update θ through momentum k The update formula is: θ k = m·θ k + (1 - m)·θ q , where m ∈ [0, 1).

7. A network attack detection method that integrates momentum contrast learning and Transformer according to claim 1, characterized in that The steps of the principal component analysis dimensionality reduction method used in the step S4 include: S4-1: Decentralize the data. The formula used is: Where: N is the number of samples, z i is the i-th high-dimensional feature vector, is the mean vector of all sample features, Z is the feature matrix of all samples, Z c is the feature matrix after centering; S4-2: Calculate the covariance matrix. The formula used is: S4-3: Perform eigenvalue decomposition. The formula used is: where: Λ = diag(λ1, λ2,..., λ d ) is the eigenvalue matrix, V = [v1, v2,..., v d is the eigenvector whose each column is the covariance matrix, v i represents the direction of the i-th principal component; S4-4: Perform principal component projection. The formula used is: Z PCA = Z c V[:, 1:d′] where: Z PCA is the representation of all samples in the low-dimensional principal component space, d′ is the target dimension of dimensionality reduction, and V[:, 1:d′] represents intercepting the first d′ principal component vectors.

8. The network attack detection method integrating momentum contrast learning and Transformer according to claim 1, characterized in that, The step S5 includes: S5-1: Set the radius ε and the density threshold MinPts. ε represents the neighborhood range, and MinPts represents the minimum number of points; S5-2: Define density reachability: For any point p, if the number of points in its neighborhood ≥ MinPts, then p is a core point. The expression is: If N ε (p) = {q ∈ R d′ | ||p - q|| ≤ ε} And N ε (p) ≥ MinPts Then p is a core point; S5-3: Define the clustering rules to identify normal traffic, known attack traffic, and unknown attack traffic.

9. The network attack detection method integrating momentum contrast learning and Transformer according to claim 8, wherein, The clustering rules are: The neighborhoods of core points are connected to form clusters; Points that fall in the neighborhood of a core point but are non-core points themselves are classified as boundary points; Points that are neither core nor in the neighborhood of any core point are noise points; Clusters and their boundary points are normal traffic or known attack traffic, and noise points are potential unknown attack traffic.

10. A network attack detection method integrating momentum contrastive learning and Transformer according to claim 1, characterized in that, In the step S6, the steps of optimizing parameters using the grey wolf optimization algorithm are: S6-1: Perform initialization operations, set the grey wolf population size, ε range, MinPts range, and the maximum number of iterations T; S6-2: Construct an individual solution, where each individual X i represents a parameter combination, and the expression is: X i =(ε i , MinPts i ) S6-3: Calculate the fitness of each individual according to the fitness function. The fitness function is: Where: a i represents the average distance of sample X i from the samples in the same cluster, and b i represents the average distance of sample X i from the nearest cluster sample. The closer the calculated value of the Silhouette(·) function is to 1, the higher the fitness; S6-4: Based on the calculation of the fitness function, select the current optimal individual, i.e., the individual X with the highest fitness α , the sub-optimal individual X β , the third-optimal individual X δ , and the remaining individuals are ordinary grey wolves X ω ; S6-5: Update the positions of ordinary grey wolves, that is, update the parameter combination according to the following update formula: Wherein: is a distance vector, is a coefficient that linearly decreases from 2 to 0 to control the balance between exploration and exploitation, is a random vector; S6-6: Return the parameters ∈ and MinPts optimized by the grey wolf optimization algorithm to step S5, and observe the performance of network attack detection according to the set termination conditions until the network attack detection performance requirements are met.

Images