Industrial internet unknown traffic identification method and system based on self-supervised learning and prototype network

Abstract

The invention discloses an industrial internet unknown traffic identification method and system based on self-supervised learning and a prototype network, and belongs to the technical field of network security and traffic identification. Processing the original industrial internet traffic to obtain a standardized feature vector; an improved InfoNCE contrast learning mechanism self-supervision pre-training network is constructed, the improved InfoNCE contrast learning mechanism is adopted for self-supervision pre-training on label-free data, and semantic prototypes maintained in the self-supervision stage serve as a system initial prototype set; performing semantic classification and open set identification on the new sample through the trained self-supervised pre-training network; and performing semantic prototype expansion on the identified unknown sample, and performing index moving average updating on the known category sample to correct the semantic prototype in real time. According to the method, efficient feature learning, unknown protocol identification and model continuous increment optimization of industrial internet traffic can be realized without manual annotation.

Description

Technical Field

[0001] This invention belongs to the field of network security and traffic identification technology, specifically relating to a method and system for identifying unknown traffic in the industrial internet based on self-supervised learning and prototype networks, used for protocol identification and abnormal traffic detection in the industrial internet environment. Background Technology

[0002] With the development of the Industrial Internet, more and more industrial equipment and control systems are being connected to the network, resulting in highly interconnected production and manufacturing systems and a complex heterogeneous communication environment. These devices use a variety of communication protocols, including standardized industrial protocols such as Modbus, DNP3, and IEC104, as well as numerous proprietary or custom protocols. The traffic in the network environment is diverse and dynamically changing. The Industrial Internet contains a large amount of normal business traffic, but also constantly emerging security threat traffic, such as new types of malware or abnormal communication behavior. Traditional network traffic identification methods mainly rely on supervised learning or rule bases. These methods require a large amount of manually labeled data and can only identify known types of traffic. They cannot promptly identify newly emerging unknown traffic categories, posing significant security risks. Furthermore, the limited collection of traffic data and high labeling costs in industrial scenarios further exacerbate the difficulty of model training. Currently, there is a lack of a traffic identification solution that can combine open-set identification and incremental learning with limited or even unlabeled data. Therefore, there is an urgent need for a new method that can effectively identify unknown traffic and possess self-learning capabilities without requiring a large number of labeled samples, in order to improve traffic security management and anomaly detection capabilities in the Industrial Internet environment. Summary of the Invention

[0003] Purpose of the invention: This invention provides a method for identifying unknown protocols and abnormal traffic in the Industrial Internet based on self-supervised learning and prototype networks, which can identify unknown protocols and abnormal traffic in the Industrial Internet under conditions of tag scarcity.

[0004] Technical solution: To achieve the above objectives, the technical solution adopted by this invention is as follows:

[0005] A method for identifying unknown traffic in the industrial internet based on self-supervised learning and prototype networks includes the following steps:

[0006] Step 1: Perform session-level aggregation on the raw industrial internet traffic and form a session sequence by combining it with a timeout segmentation strategy.

[0007] Step 2: Extract features from the obtained conversation sequence to construct feature vectors, and process them to obtain standardized feature vectors.

[0008] Step 3: Construct an improved InfoNCE contrastive learning mechanism self-supervised pre-training network. This improved network uses contrastive learning to represent standardized feature vectors under unlabeled data conditions, generating positive and negative sample pairs using data augmentation techniques. A lightweight encoder extracts deep features. Semantic prototype constraints are introduced on top of traditional InfoNCE. During training, a set of dynamic semantic prototypes is maintained in the feature space, and a similarity constraint between sample features and their most similar semantic prototype is added to the loss function, making sample features within the same semantic cluster more tightly aggregated.

[0009] Step 4: Based on maintaining the semantic prototype during the training process of the self-supervised pre-trained network of the improved InfoNCE contrastive learning mechanism, generate and maintain the initial semantic prototype and dynamically update it during the online phase.

[0010] Step 5: Perform semantic classification and open set recognition on new samples using the pre-trained, improved InfoNCE contrastive learning mechanism self-supervised pre-trained network.

[0011] Step 6: For samples identified as unknown, semantic prototype expansion is performed; for samples of known categories, exponential moving average is used to update and correct the semantic prototype in real time.

[0012] The loss function for the improved InfoNCE contrastive learning mechanism's self-supervised pre-trained network is:

[0013]

[0014] in, Indicates the total loss. Indicates the number of samples in the batch. Indicates sample eigenvectors, Representation and Sample Matched positive sample semantic prototype, Indicates the first An adaptive temperature coefficient for a semantic prototype. ,in Based on the base temperature hyperparameter, The standard deviation of the feature distribution within this prototype cluster. This represents the adaptive penalty weight for hard-to-distinguish negative samples, when the sample Non-matching prototypes The similarity is higher than a preset threshold hour, ,otherwise , The total number of categories representing semantic prototypes. Indicates the first A semantic prototype Represents the balance coefficient. and This represents any two distinct semantic prototype vectors in the prototype set.

[0015] Preferred method in step 5 includes:

[0016] Step 51, after the new sample x is input, calculate The encoder extracts its feature representation and uses cosine distance Calculate the similarity distance with each semantic prototype, where x represents the new sample. Indicates sample The normalized feature vector extracted by the encoder This represents the feature extraction function of the lightweight encoder. Representing sample features and the Cosine distance between semantic prototypes Indicates the first A semantic prototype vector.

[0017] Step 52: Select the minimum similarity distance The corresponding semantic prototype is used as the candidate classification result. When the minimum similarity distance is less than the set similarity threshold, the sample is determined to match the corresponding prototype; otherwise, it is determined to be an unknown sample or an abnormal sample.

[0018] Preferably, the similarity threshold is calculated using an adaptive threshold calculation mechanism, based on the set of maximum similarity samples within the training phase or the recent window. Calculate its mean and standard deviation Then the similarity threshold is: ,in, is the similarity threshold, and k is the adjustment coefficient.

[0019] Preferably, in step 2, statistical features, distribution features, temporal features, and behavioral features are extracted from each session and constructed into a feature vector. The feature vector is then standardized using a normalization module to obtain a standardized feature vector.

[0020] Preferred statistical features include session duration, total number of packets, total number of bytes, average packet length, packet length variance, number of uplink packets, number of downlink packets, and uplink / downlink ratio. Distribution features include the percentage of packet length histograms. Temporal features include the arrival time interval of the first 10 packets; sessions with fewer than 10 packets are padded with zeros or interpolated, and missing intervals are padded with the mean or interpolation. Missing intervals can be replaced by the mean or variance. Behavioral features include the number of packet direction switching, the maximum number of consecutive one-way packets, and the average duration of one-way packets.

[0021] Preferred method: Step 6 includes: for samples determined to be unknown, adaptively generating new semantic prototypes based on unsupervised clustering or statistical distribution results, or combining manual verification information to verify the effectiveness of the new clusters, thereby achieving continuous expansion of prototypes and incremental adaptation of models.

[0022] For samples with known categories, perform an exponential moving average update.  The semantic prototype is corrected in real time, whereby... This represents the semantic prototype vector corresponding to the known category that needs to be updated. This represents the momentum coefficient for prototype updates. This indicates a new sample that has been determined to belong to that known category. eigenvectors.

[0023] Preferred method: Step 1 includes the following steps:

[0024] Step 11: Collect raw network data packets in real time from the industrial network interface or mirror port, and filter the raw network data packets. The raw network data packets include Ethernet frames and IP datagrams in their payloads. Perform protocol filtering on the collected raw network data packets, retaining only valid packets in IPv4 and IPv6 formats, and discarding broadcast packets and non-IP layer control packets.

[0025] Step 12: Perform session-level aggregation on the filtered raw network packets. Using the five-tuple information of both communicating parties—source IP address, destination IP address, source port number, destination port number, and protocol number—as the basic aggregation key, the packets are initially grouped. For TCP traffic, the start and end boundaries of the session are determined by detecting the SYN and FIN flags to ensure the integrity of session segmentation. For connectionless or short-interaction protocols, a time window mechanism is introduced based on the five-tuple, setting an adjacent packet interval threshold. When the threshold is exceeded, the packets are automatically segmented into new sessions to avoid cross-session mixing. For industrial control protocols with request-response structures, request-response pairs are preferentially used as the session segmentation unit to maintain the semantic consistency of communication behavior.

[0026] Step 13: After completing the session segmentation, extract the basic metadata of each session and write it into the session index table to form a session sequence. The basic metadata includes the total number of packets, the number of uplink and downlink packets, the start and end times, the duration, the number of bytes, the average packet length, and the protocol type.

[0027] Another objective of this invention is to provide an industrial internet unknown traffic identification system based on self-supervised learning and prototype networks, used to execute the aforementioned industrial internet unknown traffic identification method based on self-supervised learning and prototype networks, including a session aggregation module, a feature extraction and processing module, a self-supervised pre-training module, a prototype learning module, an open set identification and incremental update module, and an output module, wherein:

[0028] The session aggregation module is used to perform session-level aggregation on raw industrial internet traffic and form session sequences by combining timeout segmentation strategies.

[0029] The feature extraction and processing module is used to extract features from the obtained session sequence to construct feature vectors, and then process them to obtain standardized feature vectors.

[0030] The self-supervised pre-training module is used to construct an improved InfoNCE contrastive learning mechanism self-supervised pre-training network. This improved network uses contrastive learning to represent standardized feature vectors under unlabeled data conditions, generating positive and negative sample pairs using data augmentation techniques and extracting deep features through a lightweight encoder. Semantic prototype constraints are introduced on top of traditional InfoNCE. During the training phase, a set of dynamic semantic prototypes is maintained in the feature space, and a similarity constraint between sample features and their most similar semantic prototype is added to the loss function, making sample features within the same semantic cluster more tightly aggregated.

[0031] The prototype learning module is used to generate and maintain the initial semantic prototype and dynamically update it during the online phase, based on the semantic prototype maintained during the training process of the self-supervised pre-trained network of the improved InfoNCE contrastive learning mechanism.

[0032] The open set identification and incremental update module is used to perform semantic classification and open set identification on new samples through a self-supervised pre-trained network based on the improved InfoNCE contrastive learning mechanism. For samples identified as unknown, semantic prototype expansion is performed; for samples of known categories, exponential moving average updates are performed to correct the semantic prototype in real time.

[0033] The output module is used to output the identified semantic classification.

[0034] Another object of the present invention is to provide an electronic device comprising: at least one processor, at least one memory, and a communication interface. The processor, memory, and communication interface communicate with each other. The memory stores program instructions executable by the processor, which invokes the program instructions to execute the described method for identifying unknown traffic in the Industrial Internet based on self-supervised learning and prototype networks.

[0035] Compared with the prior art, the present invention has the following advantages:

[0036] By combining an improved self-supervised contrastive learning algorithm and a few-shot prototype encoder, this invention achieves efficient encoding and identification of various traffic types in the industrial internet environment, and can incrementally update the model when unknown categories are detected. This invention enables efficient feature learning, unknown protocol identification, and continuous incremental model optimization of industrial internet traffic without manual annotation, and is suitable for real-time traffic monitoring and anomaly detection scenarios in industrial settings. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the overall process of the method of the present invention;

[0038] Figure 2 A schematic diagram of the structure of a self-supervised pre-trained network for improving the InfoNCE contrastive learning mechanism. Detailed Implementation

[0039] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these examples are for illustrative purposes only and are not intended to limit the scope of the invention. After reading this invention, any modifications of the invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0040] Example

[0041] This embodiment provides a method for identifying unknown traffic in the Industrial Internet based on self-supervised learning and prototype networks. First, the collected industrial network traffic data is preprocessed and features are constructed. Then, an improved InfoNCE self-supervised contrastive learning mechanism is used to train a feature encoder, obtaining a discriminative and generalizable traffic representation under unlabeled conditions. Based on this, the system adaptively aggregates semantic prototypes according to sample feature similarity, enhances feature aggregation capabilities through semantic prototype constraints, and dynamically updates the prototypes using an exponential moving average strategy to reflect changes in data distribution. During the identification phase, the model calculates the similarity between the test sample and each semantic prototype. When the maximum similarity is below an adaptive threshold, the sample is determined to be unknown. For samples determined to be unknown, the system generates new semantic prototypes and updates the model through unsupervised clustering and self-learning mechanisms. Figure 1 As shown, it includes the following steps:

[0042] Step 1: Perform session-level aggregation on the raw Industrial Internet traffic and form a session sequence using a timeout segmentation strategy. Extract statistical features, distribution features, temporal features, and behavioral features from the raw Industrial Internet traffic to construct a feature vector, which will be used as input for the subsequent self-supervised feature learning stage.

[0043] This step involves collecting and preprocessing network traffic data in the industrial internet environment, segmenting the raw packets, and extracting features such as packet size, time interval, and protocol type. Data augmentation is then applied to the feature vectors to generate multi-view samples, which serve as input features for the model to construct the self-supervised learning dataset required for subsequent training. The specific methods include the following steps:

[0044] Step 11: Collect raw network data packets in real time from the industrial network interface or mirror port, and filter the raw network data packets. The raw network data packets include Ethernet frames and IP datagrams in their payloads. Perform protocol filtering on the collected raw network data packets, retaining only valid packets in IPv4 and IPv6 formats, and discarding broadcast packets and non-IP layer control packets.

[0045] Step 12: Perform session-level aggregation on the filtered raw network packets. Using the five-tuple information of both communicating parties—source IP address, destination IP address, source port number, destination port number, and protocol number—as the basic aggregation key, the packets are initially grouped. For TCP traffic, the start and end boundaries of the session are determined by detecting the SYN and FIN flags to ensure the integrity of session segmentation. For connectionless or short-interaction protocols, the system introduces a time window mechanism based on the five-tuple, setting an interval threshold of 30 seconds between adjacent packets. When this threshold is exceeded, the packets are automatically segmented into new sessions to avoid cross-session mixing. For industrial control protocols with request-response structures, request-response pairs are preferentially used as the session segmentation unit to maintain the semantic consistency of communication behavior.

[0046] Step 13: After completing session segmentation, extract the basic metadata of each session and write it into the session index table to form a session sequence, providing a retrieval entry point for the subsequent feature extraction module. The basic metadata includes the total number of packets, the number of uplink and downlink packets, the start and end times, the duration, the number of bytes, the average packet length, and the protocol type.

[0047] Step 2 involves extracting features from the obtained session sequences to construct feature vectors, and then processing these vectors to obtain standardized feature vectors. The feature vectors are then normalized and missing value processing is performed to ensure comparability of features within a unified dimensional framework and to eliminate data gaps caused by incomplete data collection or unknown protocols, thereby improving the stability of feature input. Normalization and missing value processing map features from different units of data to the same numerical range; missing values ​​caused by incomplete data collection or equipment malfunctions are filled in.

[0048] For each session, statistical features, distribution features, temporal features, and behavioral features are extracted and constructed into a 128-dimensional feature vector. The feature vector is then standardized using a normalization module to obtain a standardized feature vector. Statistical features include session duration, total number of packets, total number of bytes, average packet length, packet length variance, number of uplink packets, number of downlink packets, and uplink / downlink ratio. Distribution features include the packet length histogram percentage, with eight dimensions used to represent the distribution across different packet lengths. Temporal features include the arrival time interval of the first 10 packets; sessions with fewer than 10 packets are padded with zeros or interpolated, and missing intervals can be filled using the mean or interpolation. Behavioral features include the number of packet direction switches, the maximum number of consecutive one-way packets, and the average duration of one-way packets.

[0049] The feature vectors are normalized and then standardized. All feature dimensions are standardized using the Z-score method, with each dimension being [value missing]. During the training phase, the system calculates the mean and standard deviation based on the training set samples, and maintains consistent normalization parameters during the deployment phase. For sessions with insufficient or missing features, the system uses the median for imputation. The normalization process employs Z-score standardization, mapping features of different dimensions to the same numerical range; missing value handling includes imputation using the mean, median, mode, or 0 to reduce noise interference with model training.

[0050] Step 3: Construct an improved InfoNCE contrastive learning mechanism self-supervised pre-training network. This network uses contrastive learning to achieve general traffic representation on standardized feature vectors under unlabeled data conditions. Data augmentation techniques are used to generate positive and negative sample pairs, and a lightweight encoder extracts deep features. Semantic prototype constraints are introduced on top of traditional InfoNCE. During training, a set of dynamic semantic prototypes is maintained in the feature space, and a similarity constraint between sample features and their most similar semantic prototypes is added to the loss function, making sample features within the same semantic cluster more tightly aggregated. The improved InfoNCE contrastive learning mechanism is used for self-supervised pre-training on unlabeled data. Augmented sample pairs are generated through random masking, temporal perturbation, and pruning, and feature representations are extracted by a lightweight encoder and projection head. The encoder parameters are optimized using the improved InfoNCE loss with semantic prototype constraints, causing samples to cluster in the feature space around corresponding semantic prototypes, thus obtaining feature representations with stronger discriminative and generalizable properties. The semantic prototypes are dynamically maintained in a non-gradient manner during training.

[0051] Based on the samples obtained in steps 1 and 2, an improved InfoNCE self-supervised contrastive learning mechanism is used to train the feature encoder. This mechanism introduces semantic prototype constraints on top of traditional InfoNCE. During the training phase, a set of dynamic semantic prototypes is maintained in the feature space, and a similarity constraint between sample features and their most similar semantic prototypes is added to the loss function, making sample features within the same semantic cluster more tightly aggregated. By minimizing the improved contrastive loss function, a traffic feature encoding model with high discriminativeness and stability is obtained.

[0052] The structure of the improved InfoNCE contrastive learning mechanism self-supervised pre-trained network is as follows:

[0053] like Figure 2 As shown, the improved InfoNCE contrastive learning mechanism's self-supervised pre-trained network adopts a Siamese network architecture, mainly including the following core components: a data augmentation module, which performs random masking and temporal jittering on the input session sequence x to generate two augmented views. and The input network uses positive sample pairs. A lightweight encoder with shared parameters employs a two-layer one-dimensional convolutional neural network (1D-CNN). The first layer, with a kernel size of 5, 64 channels, and a stride of 1, is used to extract local temporal features. The second layer, with a kernel size of 3, 128 channels, and a stride of 1, is used to extract high-level semantic features. Pooling layers map variable-length sequences to fixed-length 128-dimensional feature vectors z using global max pooling. The left and right encoders share the same set of parameters. The algorithm updates simultaneously via backpropagation. A semantic prototype library stores semantic prototype vectors for each category. A density estimation module calculates the feature variance within prototype clusters in real time and outputs it to the loss function for temperature adjustment. Non-gradient updates are used; prototype vectors do not participate in gradient descent but instead undergo momentum updates. An improved loss calculation core receives feature vectors z and prototypes. The final improved contrastive loss L is calculated by combining adaptive temperature and hard example weights, driving the encoder to learn a highly discriminative flow representation.

[0054] The method for improving the self-supervised pre-trained network of the InfoNCE contrastive learning mechanism includes the following steps:

[0055] (1) A loss optimization mechanism based on semantic prototype constraints is introduced to enhance the aggregation ability of samples in the feature space. During the training phase, the module first performs data augmentation operations on unlabeled data samples to construct positive and negative sample pairs. Augmentation methods include randomly masking session features in the feature dimension, adding Gaussian noise in the temporal sequence, and randomly pruning continuous segments of the session to simulate traffic disturbances under different network conditions.

[0056] (2) The encoding network adopts a two-layer lightweight one-dimensional convolutional neural network structure. The first layer has a kernel size of 5 and 64 convolutional channels; the second layer has a kernel size of 3 and 128 convolutional channels. Each convolutional layer is equipped with a batch normalization layer and an activation layer. The activation function is preferably a linear rectified function. The encoded output is global pooled to obtain an embedding vector of length 128.

[0057] (3) The pre-trained projection head network is used for feature compression, employing a two-layer fully connected structure to map the embedding vector from 128 dimensions to 64 dimensions, and further to a 32-dimensional representation space. The Adam optimizer is preferred for optimization, with a learning rate of 0.001 and a batch size of 256. To improve training stability, the module dynamically samples negative samples during feature comparison and sets a temperature parameter τ to adjust the smoothness of the sample similarity distribution, thereby ensuring the model's contrast constraint effect. The preferred temperature parameter τ is 0.1.

[0058] (4) Adopting an improved loss function Train the model and improve the loss function by using a contrastive loss term. With semantic prototype constraint terms This composition can shorten the distance between positive sample pairs and widen the distance between negative sample pairs, resulting in a general and robust feature representation. To address the limitations of traditional fixed temperature parameters in adapting to protocol clusters with varying tightness, and the difficulty in distinguishing highly similar proprietary protocols, a new contrast loss term is defined, and two improved parameters are introduced into the formula: a density-aware adaptive temperature coefficient. Difficulty example mining adaptive weights Density-sensing adaptive temperature coefficient The system calculates the number in real time. Standard deviation of feature distribution within each prototype cluster For tightly distributed, strongly rule-based protocols, reducing By shrinking the boundary, for loosely distributed edge length agreements, increase To improve generalization ability. Adaptive weights for difficult example mining. For obfuscation protocols targeting head similarity, when the similarity between a sample and a non-matching prototype exceeds a threshold, a certain condition is set. This forces the model to amplify the penalty for negative samples in the denominator, compelling it to extract finer-grained features. This is a prototype constraint term, a regularization term, based solely on the prototype set. The computation ensures that prototypes are evenly distributed in the feature space, preventing prototype overlap and maintaining the spacing between different semantic clusters. The module combines two losses using a weighted summation method to balance the self-supervised contrastive constraint and the semantic prototype distribution constraint. The loss function is only used to optimize the encoder parameters θ; semantic prototypes are maintained during the training phase through a non-gradient momentum update mechanism.

[0059]

[0060]

[0061] ;

[0062] The loss function of the improved InfoNCE contrastive learning mechanism self-supervised pre-trained network is then:

[0063]

[0064] in, Indicates the total loss. Indicates the number of samples in the batch. Indicates sample eigenvectors, Representation and Sample Matched positive sample semantic prototype, Indicates the first An adaptive temperature coefficient for a semantic prototype. ,in Based on the base temperature hyperparameter, The standard deviation of the feature distribution within this prototype cluster. This represents the adaptive penalty weight for hard-to-distinguish negative samples, when the sample Non-matching prototypes The similarity is higher than a preset threshold hour, ,otherwise , The total number of categories representing semantic prototypes. Indicates the first A semantic prototype Represents the balance coefficient. and This represents any two distinct semantic prototype vectors in the prototype set.

[0065] By minimizing the comprehensive loss function, the model can learn highly discriminative and stable feature representations. The encoder obtained after training has general traffic representation capabilities and can be used as the basis for feature extraction in subsequent few-shot classification and open-set recognition stages.

[0066] Loss on encoder parameters Calculate and update the gradient, prototype They do not participate in gradient updates; they are updated between training iterations via a non-gradient mechanism (see step 4).

[0067] A self-supervised pre-training mechanism based on an improved InfoNCE contrastive learning approach is employed on unlabeled data. Enhanced sample pairs are generated through random masking, temporal perturbation, and pruning, and feature representations are extracted using a lightweight encoder and projection head. The encoder parameters are optimized using an improved InfoNCE loss incorporating semantic prototype constraints, causing samples to cluster in the feature space around corresponding semantic prototypes. This results in more discriminative and generalizable feature representations, with the semantic prototypes dynamically maintained in a non-gradient manner during training.

[0068] Self-supervised pre-training uses a lightweight neural network as the encoder, specifically a 1D-CNN encoder, to extract session-level feature embeddings. A small multilayer perceptron (MLP) is used as the projection head to map the embedding vectors to the contrastive learning space for constraint optimization.

[0069] Self-supervised pre-training includes the following processes: adding small Gaussian noise, random jitter, partial feature masking, or truncating feature sequences to the same traffic session to construct positive sample pairs, and randomly sampling different traffic sessions to form negative sample pairs;

[0070] Self-supervised pre-training employs an improved InfoNCE loss function, calculating the similarity between a sample embedding and its most similar semantic prototype as a positive sample pair, and the similarity between the sample and other semantic prototypes as a negative sample pair. The loss function minimizes the embedding distance of positive sample pairs and maximizes the embedding distance of negative sample pairs, while introducing a prototype distribution constraint term to maintain the separation of each semantic prototype in the feature space, prevent prototype collapse, and thus improve the model's ability to distinguish unknown traffic.

[0071] Step 4: Based on the semantic prototypes maintained during the training process of the improved InfoNCE contrastive learning mechanism's self-supervised pre-trained network, an initial semantic prototype is generated and maintained, and dynamically updated during the online phase. The semantic prototypes maintained in the self-supervised phase serve as the system's initial prototype set. The system forms a semantic prototype structure based on the encoder's feature representation and sample similarity. During operation, an exponential moving average strategy is used to dynamically update the prototypes, allowing them to adaptively adjust with new sample features. The entire prototype generation and update process is completed automatically without manual annotation.

[0072] Based on the encoder trained in step 3, the system utilizes its feature representation on unlabeled data to adaptively aggregate features based on their similarity to form an initial semantic prototype. This initial semantic prototype can be obtained by performing K-means or momentum clustering on the embedding vectors, with each cluster center corresponding to a semantic prototype. During online operation, when a new sample is received, the system dynamically updates the semantic prototype using an exponential moving average strategy, gradually aligning the prototype with the new sample to achieve continuous adaptive feature distribution. The entire process requires no manual annotation; the semantic prototype is generated and continuously refined by the model.

[0073] Step 4 specifically includes the following steps:

[0074] (1) At the beginning or early stage of training, K-means clustering can be performed on a batch of unlabeled samples based on the current output of the encoder to initialize the semantic prototype. Alternatively, it can be randomly initialized and then converged through online updates. Each semantic prototype... It is represented by the mean or momentum center embedded within the cluster.

[0075] (2) During training, the prototypes are updated non-gradiently according to batch statistics or momentum update strategies. This update is a non-backpropagation operation. For example, momentum update is used for the k-th prototype.  , This represents the prototype update rate during training, ranging from 0.001 to 0.1. This update is a non-backpropagation operation, making smooth corrections to the prototype based solely on the encoder's current output, ensuring that the prototype evolves with the feature distribution.

[0076] (3) During the online phase of system deployment, when a new sample is received... If a sample is determined to match the k-th semantic prototype, then an exponential moving average is used to correct the prototype online.  This update method allows the prototype to gradually adapt to the time-varying characteristics of the data distribution without retraining the encoder. (4) The generation and updating of the semantic prototype are both feature aggregation processes generated by the model itself and do not depend on manual labels. The prototype is used for loss term calculation during the training phase to guide the encoder's learning; the prototype set at the end of training can be used as the initial prototype for the online phase to continue to be maintained and expanded.

[0077] The prototype network computation of semantic prototypes specifically includes: initializing the prototype with a batch of unlabeled samples at the beginning or early stage of training in step 3; concurrently updating the prototype using a non-gradient momentum update strategy during the training process in step 3; and dynamically updating the prototype using an exponential moving average strategy during the online operation phase of system deployment.

[0078] The semantic prototypes maintained during the self-supervised phase serve as the initial prototype set of the system. The system forms a semantic prototype structure based on the encoder feature representation and sample similarity. During operation, the system uses an exponential moving average strategy to dynamically update the prototypes, allowing them to adapt to new sample features. The entire prototype generation and update process is completed automatically without manual annotation.

[0079] Step 5: The new samples are semantically classified and open-set identified using a pre-trained, improved InfoNCE contrastive learning mechanism self-supervised pre-trained network. This is used to identify unknown samples and perform prototype expansion. The encoder and semantic prototype network are used to perform semantic classification and open-set identification on new traffic. When the distance between a new sample and all semantic prototypes exceeds a threshold, it is determined to be an unknown protocol or an unknown anomaly and is temporarily stored for subsequent analysis.

[0080] Based on the trained encoder and semantic prototypes, similarity matching and open set identification are performed on new traffic samples. Specifically, this includes: extracting feature representations of the traffic samples to be tested using the trained encoder, calculating the distance between them and each semantic prototype; when the minimum distance is less than a threshold, it is determined to be the corresponding category; if all distances exceed the threshold, it is determined to be an unknown protocol or an unknown anomaly.

[0081] The encoder and semantic prototype network are used to perform semantic classification and open set recognition on new traffic. When the distance between a new sample and all semantic prototypes exceeds a threshold, it is determined to be an unknown protocol or an unknown anomaly and is temporarily stored for subsequent analysis.

[0082] Step 5 includes the following methods:

[0083] Step 51, after the new sample x is input, calculate The encoder extracts its feature representation and uses cosine distance Calculate the similarity distance with each semantic prototype, where x represents the new sample. Indicates sample The normalized feature vector extracted by the encoder This represents the feature extraction function of the lightweight encoder. Representing sample features and the Cosine distance between semantic prototypes Indicates the first A semantic prototype vector.

[0084] Step 52: Select the minimum similarity distance The corresponding semantic prototype is used as the candidate classification result. The corresponding matching similarity is then calculated. When the minimum similarity distance is greater than the set similarity threshold, the sample is determined to match the corresponding prototype; otherwise, it is determined to be an unknown sample or an abnormal sample.

[0085] To improve the robustness of unknown sample determination, the similarity threshold is calculated using an adaptive threshold calculation mechanism. The similarity threshold is not manually specified but automatically determined based on the statistical distribution of similarity between a sample and its nearest semantic prototype in the feature space. This is based on the set of maximum similarity samples within the training phase or recent window. Calculate its mean and standard deviation Then the similarity threshold is: ,in, is the similarity threshold, and k is the adjustment coefficient.

[0086] When the maximum similarity of a new sample is lower than the threshold, it is judged as an unknown or abnormal sample; when the similarity is higher than the threshold, it is considered to match the corresponding semantic prototype.

[0087] Samples identified as unknown are temporarily stored and automatically grouped using the DBSCAN clustering algorithm; a manual verification interface is supported, and if a new category is confirmed, the cluster center is added to the model.

[0088] Step 6: For samples identified as unknown, semantic prototype expansion is performed. For samples of known categories, an exponential moving average is used to update and correct the semantic prototype in real time. The final output includes: semantic cluster labels, unknown sample identifiers, and confidence scores. An incremental update mechanism is implemented: for samples identified as belonging to known categories, the corresponding semantic prototypes are adjusted through a dynamic update strategy to make the prototypes more closely aligned with the new sample embeddings; for newly confirmed unknown samples, clustering or manual verification is performed to generate new semantic prototypes and incorporate them into the model, achieving self-learning and rapid adaptation of the model.

[0089] It supports an incremental update mechanism. For samples identified as belonging to a known category, a dynamic adjustment strategy is used to update the category prototype, gradually bringing the prototype representation closer to the embedding of new samples of that category. For samples identified as unknown, temporary storage and cluster analysis are performed. If manually verified as belonging to a new category, the average feature vector of that category is calculated and added to the prototype library as a new prototype. Through this mechanism, the model can achieve self-learning and rapid adaptation in long-term operation without retraining the entire network.

[0090] Step 6 includes the following methods: For samples determined to be unknown, new semantic prototypes are adaptively generated based on unsupervised clustering or statistical distribution results, or the validity of the new clusters is verified by combining manual review information, thereby achieving continuous expansion of prototypes and incremental adaptation of the model. For samples determined to be unknown, the sample features are temporarily stored in a cache, and a density-based clustering algorithm is periodically triggered to detect new categories. The DBSCAN method is preferred for clustering. After clustering, the average features of the newly formed sample clusters are calculated as new category prototypes and updated in the prototype library.

[0091] For samples with known categories, perform an exponential moving average update.  The semantic prototype is corrected in real time, whereby... This represents the semantic prototype vector corresponding to the known category that needs to be updated. This indicates a very small coefficient, such as 0.01; A vector representing a new sample. This mechanism allows the model to gradually incorporate newly confirmed samples in online deployment without retraining the entire model.

[0092] For samples recognized as known classes, the corresponding semantic prototypes are adjusted through a dynamic update strategy to make the prototypes closer to the embedding of the new samples; after clustering or manual review of newly confirmed unknown samples, new semantic prototypes are generated and incorporated into the model to achieve the self-learning and rapid adaptation of the model.

[0093] Step 7, output the recognition result and alarm information, and transmit the classification label, confidence score, and unknown sample identifier to the security policy engine or network protection system for automated security policy adjustment and traffic response. Transmit the classification label, confidence score, and unknown sample identifier to the security policy engine or network protection system for automated security policy adjustment and traffic response. The output module includes a result formatting unit and an interface transmission unit. The former is responsible for generating structured data including timestamp, session identifier, category prediction, and confidence, and the latter is used to interface with the industrial firewall or security control center to achieve real-time联动防护.

[0094] In another embodiment of the present invention, an industrial Internet unknown traffic recognition system based on self-supervised learning and prototype network is provided for performing the industrial Internet unknown traffic recognition method based on self-supervised learning and prototype network, including a session aggregation module, a feature extraction and processing module, a self-supervised pre-training module, a prototype learning module, an open-set recognition and incremental update module, and an output module, where:

[0095] The session aggregation module is used to perform session-level aggregation on the original industrial Internet traffic, combine the timeout splitting strategy to form a session sequence, and construct the basic data for subsequent analysis.

[0096] The feature extraction and processing module is used to extract features from the obtained session sequence to construct a feature vector and process it to obtain a standardized feature vector. The feature extraction and processing module is used to extract statistical, distribution, temporal, and behavioral features from each session data after session aggregation, construct it into a fixed-length feature vector, perform normalization processing, and fill in the missing values caused by incomplete or abnormal acquisition;

[0097] The self-supervised pre-training module is used to construct an improved InfoNCE contrastive learning mechanism self-supervised pre-training network. This network uses contrastive learning to achieve a general traffic representation on standardized feature vectors under unlabeled data conditions. Data augmentation techniques are used to generate positive and negative sample pairs, and a lightweight encoder extracts deep features. Semantic prototype constraints are introduced on top of traditional InfoNCE. During training, a set of dynamic semantic prototypes is maintained in the feature space, and a similarity constraint between sample features and their most similar semantic prototype is added to the loss function, making sample features within the same semantic cluster more tightly aggregated. The self-supervised pre-training module is used to achieve a general traffic representation under unlabeled data conditions through contrastive learning. The module uses data augmentation techniques to generate positive and negative sample pairs, extracts deep features through a lightweight encoder, and optimizes using an improved InfoNCE loss function. During training, the model minimizes the distance between positive samples and maximizes the distance between negative samples, while introducing a semantic prototype constraint term, enabling the encoder to learn embedding representations with stronger discriminative and generalizable capabilities. The self-supervised pre-training module uses contrastive learning to learn a general traffic representation on unlabeled data and improves feature aggregation capabilities through the semantic prototype constraint mechanism.

[0098] The prototype learning module generates and maintains an initial semantic prototype, dynamically updating it during the online phase, based on the semantic prototype maintained during the training of the self-supervised pre-trained network of the improved InfoNCE contrastive learning mechanism. After obtaining the pre-trained encoder, the prototype learning module adaptively generates a semantic prototype based on sample feature similarity and continuously updates it according to new samples during online operation. The initial prototype is generated by calculating the cluster centers of sample embeddings in the feature space, and an exponential moving average strategy is used to dynamically adjust the prototype to reflect real-time changes in data distribution. The entire prototype generation and updating process does not rely on manual annotation; the system can automatically adjust the semantic prototype structure based on feature aggregation results. The prototype learning module adaptively aggregates semantic prototypes based on feature similarity and dynamically updates the prototypes during system operation using an exponential moving average mechanism.

[0099] The open-set identification and incremental update module is used to perform semantic classification and open-set identification on new samples using a self-supervised pre-trained network based on the improved InfoNCE contrastive learning mechanism. For samples identified as unknown, semantic prototype expansion is performed; for samples of known categories, exponential moving average updates are used to correct the semantic prototype in real time. The open-set identification and incremental update module is used to match known samples and identify unknown samples based on the similarity between the sample and each semantic prototype. By calculating the similarity between the sample and each prototype and comparing it with a set threshold, a match is determined when the similarity is higher than the threshold; if all similarities are lower than the threshold, the sample is considered unknown or an anomaly. For samples determined to be unknown, the system generates new prototypes through unsupervised clustering or model adaptation mechanisms, and expands and updates the model as needed, combining manual review results to achieve continuous self-learning and incremental adaptation. The open-set identification and incremental update module is used to match known samples and identify unknown samples based on the similarity between the sample and each semantic prototype, and after confirmation, generates new semantic prototypes through adaptive clustering, achieving model self-learning and incremental adaptation.

[0100] The output module is used to output the identified semantic classification. It also provides structured encapsulation of the identification results. This process generates the identification results and feeds them back to the security policy engine. The module encapsulates the model's output classification labels, confidence scores, and unknown sample identifiers, and outputs them in a structured form to the industrial firewall or security management system. This enables alarm linkage, policy adjustment, and real-time response, forming a complete intelligent protection closed loop. The module output includes timestamps, session identifiers, classification labels, confidence scores, and unknown sample identifiers. After unified formatting, the output results are sent to the security policy engine or network protection system via an application programming interface (API) for automatic alarm triggering, policy adjustment, and defense response, achieving end-to-end intelligent traffic security management. The output module includes a result formatting unit and an interface unit. The result formatting unit generates output records containing timestamps, session identifiers, category labels, confidence scores, and alarm status. The interface unit outputs these records to the security policy engine, log system, or industrial firewall module.

[0101] In another embodiment of the present invention, an electronic device is provided, comprising: at least one processor, at least one memory, and a communication interface. The processor, memory, and communication interface communicate with each other. The memory stores program instructions executable by the processor, which invokes the program instructions to execute the described industrial internet unknown traffic identification method based on self-supervised learning and prototype networks.

[0102] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for identifying unknown traffic in the industrial internet based on self-supervised learning and prototype networks, characterized in that, Includes the following steps: Step 1: Perform session-level aggregation on the raw industrial internet traffic and form a session sequence by combining it with a timeout segmentation strategy; Step 2: Extract features from the obtained conversation sequence to construct feature vectors, and process them to obtain standardized feature vectors; Step 3: Construct an improved InfoNCE contrastive learning mechanism self-supervised pre-training network. By improving the InfoNCE contrastive learning mechanism self-supervised pre-training network, a general traffic representation is achieved by contrastive learning on standardized feature vectors under unlabeled data conditions. Data augmentation techniques are used to generate positive and negative sample pairs, and deep features are extracted through a lightweight encoder. Semantic prototype constraints are introduced on the basis of traditional InfoNCE. During the training phase, a set of dynamic semantic prototypes is maintained in the feature space, and a similarity constraint between sample features and their most similar semantic prototypes is added to the loss function, so that sample features within the same semantic cluster are more tightly aggregated. Step 4: Based on maintaining the semantic prototype during the training process of the self-supervised pre-trained network of the improved InfoNCE contrastive learning mechanism, generate and maintain the initial semantic prototype and dynamically update it during the online phase. Step 5: Perform semantic classification and open set recognition on new samples using the pre-trained, improved InfoNCE contrastive learning mechanism self-supervised pre-trained network; Step 6: For samples identified as unknown, semantic prototype expansion is performed; for samples of known categories, exponential moving average is used to update and correct the semantic prototype in real time.

2. The method for identifying unknown traffic in the industrial internet based on self-supervised learning and prototype networks according to claim 1, characterized in that: The loss function for the improved InfoNCE contrastive learning mechanism's self-supervised pre-trained network is: in, Indicates the total loss. Indicates the number of samples in the batch. Indicates sample eigenvectors, Representation and Sample Matched positive sample semantic prototype, Indicates the first An adaptive temperature coefficient for a semantic prototype. ,in Based on the base temperature hyperparameter, The standard deviation of the feature distribution within this prototype cluster. This represents the adaptive penalty weight for hard-to-distinguish negative samples, when the sample Non-matching prototypes The similarity is higher than a preset threshold hour, ,otherwise , The total number of categories representing semantic prototypes. Indicates the first A semantic prototype Represents the balance coefficient. and This represents any two distinct semantic prototype vectors in the prototype set.

3. The method for identifying unknown traffic in the industrial internet based on self-supervised learning and prototype networks according to claim 2, characterized in that: Step 5 includes the following methods: Step 51, after the new sample x is input, calculate The encoder extracts its feature representation and uses cosine distance Calculate the similarity distance with each semantic prototype, where x represents the new sample. Indicates sample The normalized feature vector extracted by the encoder This represents the feature extraction function of the lightweight encoder. Representing sample features and the Cosine distance between semantic prototypes Indicates the first A semantic prototype vector; Step 52: Select the minimum similarity distance The corresponding semantic prototype is used as the candidate classification result; when the minimum similarity distance is less than the set similarity threshold, the sample is determined to match the corresponding prototype; otherwise, it is determined to be an unknown sample or an abnormal sample.

4. The method for identifying unknown traffic in the industrial internet based on self-supervised learning and prototype networks according to claim 3, characterized in that: The similarity threshold is calculated using an adaptive thresholding mechanism, based on the set of maximum similarity samples within the training phase or the most recent window. Calculate its mean and standard deviation Then the similarity threshold is: ,in, is the similarity threshold, and k is the adjustment coefficient.

5. The method for identifying unknown traffic in the industrial internet based on self-supervised learning and prototype networks according to claim 4, characterized in that: In step 2, statistical features, distribution features, temporal features, and behavioral features are extracted for each session and constructed into a feature vector. The feature vector is then standardized by the normalization module to obtain a standardized feature vector.

6. The method for identifying unknown traffic in the industrial internet based on self-supervised learning and prototype networks according to claim 5, characterized in that: Statistical characteristics include session duration, total number of packets, total number of bytes, average packet length, packet length variance, number of uplink packets, number of downlink packets, and uplink / downlink ratio; distribution characteristics include the percentage of packet length histograms; temporal characteristics include the arrival time interval of the first 10 data packets, with sessions having fewer than 10 packets padded with zeros or interpolated, and missing intervals padded with the mean or interpolated value, which can be replaced by the mean or variance; behavioral characteristics include the number of packet direction switching, the maximum number of consecutive one-way packets, and the average duration of one-way packets.

7. The method for identifying unknown traffic in the industrial internet based on self-supervised learning and prototype networks according to claim 6, characterized in that: Step 6 includes the following methods: for samples determined to be unknown, new semantic prototypes are adaptively generated based on unsupervised clustering or statistical distribution results, or the validity of new clusters is verified by combining manual verification information, thereby achieving continuous expansion of prototypes and incremental adaptation of models. For samples with known categories, perform an exponential moving average update.  The semantic prototype is corrected in real time, whereby... This represents the semantic prototype vector corresponding to the known category that needs to be updated. This represents the momentum coefficient for prototype updates. This indicates a new sample that has been determined to belong to that known category. eigenvectors.

8. The method for identifying unknown traffic in the industrial internet based on self-supervised learning and prototype networks according to claim 7, characterized in that: Step 1 includes the following steps: Step 11: Collect raw network data packets in real time from the industrial network interface or mirror port, and filter the raw network data packets; the raw network data packets include Ethernet frames and IP datagrams in their payloads. Perform protocol filtering on the collected raw network data packets, retaining only valid packets in IPv4 and IPv6 formats, and discarding broadcast packets and non-IP layer control packets. Step 12: Perform session-level aggregation on the filtered raw network packets; use the five-tuple information of the communicating parties, namely source IP address, destination IP address, source port number, destination port number, and protocol number, as the basic aggregation key to initially group the packets; for TCP traffic, the start and end boundaries of the session are determined by detecting the SYN and FIN flags to ensure the integrity of session segmentation; for connectionless or short-interaction protocols, a time window mechanism is introduced on the basis of the five-tuple, setting an adjacent packet interval threshold, and automatically segmenting into a new session when the adjacent packet interval threshold is exceeded to avoid cross-session mixing; for industrial control protocols with request-response structures, "request-response pairs" are preferred as the session segmentation unit to maintain the consistency of communication behavior semantics; Step 13: After completing the session segmentation, extract the basic metadata of each session and write the basic metadata into the session index table to form a session sequence. The basic metadata includes the total number of packets, the number of uplink and downlink packets, the start and end time, the duration, the number of bytes, the average packet length, and the protocol type.

9. An industrial internet unknown traffic identification system based on self-supervised learning and prototype networks, characterized in that, The method for identifying unknown traffic in the industrial internet based on self-supervised learning and prototype networks as described in claim 1 includes a session aggregation module, a feature extraction and processing module, a self-supervised pre-training module, a prototype learning module, an open set identification and incremental update module, and an output module, wherein: The session aggregation module is used to perform session-level aggregation on raw industrial internet traffic and form a session sequence by combining timeout segmentation strategy. The feature extraction and processing module is used to extract features from the obtained session sequence to construct feature vectors, and then process them to obtain standardized feature vectors. The self-supervised pre-training module is used to construct an improved InfoNCE contrastive learning mechanism self-supervised pre-training network. By improving the InfoNCE contrastive learning mechanism self-supervised pre-training network, a general traffic representation is achieved by contrastive learning on standardized feature vectors under unlabeled data conditions. Data augmentation techniques are used to generate positive and negative sample pairs, and a lightweight encoder is used to extract deep features. On the basis of traditional InfoNCE, semantic prototype constraints are introduced. During the training phase, a set of dynamic semantic prototypes is maintained in the feature space, and a similarity constraint between sample features and their most similar semantic prototypes is added to the loss function, so that sample features within the same semantic cluster are more tightly aggregated. The prototype learning module is used to generate and maintain the initial semantic prototype and dynamically update it during the online phase, based on maintaining the semantic prototype during the training process of the self-supervised pre-trained network of the improved InfoNCE contrastive learning mechanism. The open set recognition and incremental update module is used to perform semantic classification and open set recognition on new samples through a self-supervised pre-trained network with a trained improved InfoNCE contrastive learning mechanism; for samples identified as unknown, semantic prototype expansion is performed; and for samples of known categories, exponential moving average is performed to update and correct the semantic prototype in real time. The output module is used to output the identified semantic classification.

10. An electronic device, characterized in that, include: At least one processor, at least one memory, and a communication interface; The processor, memory, and communication interface communicate with each other; The memory stores program instructions that can be executed by the processor, which calls the program instructions to execute the industrial internet unknown traffic identification method based on self-supervised learning and prototype networks as described in any one of claims 1-8.

Images