Open set encrypted traffic classification method and system based on structured fine-tuning pre-trained model

Abstract

This invention provides an open-set encrypted traffic classification method and system based on a structured fine-tuning pre-trained model, comprising: preprocessing the original encrypted traffic into a lexical sequence that can be processed by a language model; fine-tuning the pre-trained language model using the lexical sequence: performing structured guidance operations on the lexical sequence such as header field rearrangement and / or flow packet segmentation and representation to guide the model to learn the structured semantics of the traffic; extracting aggregated feature vectors of the traffic sequence to be classified using the fine-tuned model; and based on the feature vectors, using an open-set classification algorithm based on distance metrics to determine whether it belongs to an unknown category, and if not, further determining its specific known category. This invention can guide the model to deeply understand the protocol and packet sequence structure of encrypted traffic, thereby achieving both high-precision classification of known traffic and effective identification of unknown traffic in open-set scenarios.

Description

Technical Field

[0001] This invention relates to the field of computer network technology, and in particular to an open set encrypted traffic classification method and system based on a structured fine-tuning pre-trained model. Background Technology

[0002] With the rapid development of information technology and the widespread adoption of internet applications, the network has become deeply integrated into all aspects of social production and life. At the same time, the network environment has become increasingly complex due to the sheer number and variety of applications, posing a severe challenge to the effective governance of cyberspace. Network traffic identification and classification technology, as the foundation of network supervision, security defense, quality of service assurance, and network optimization, has the core task of analyzing network data flows to determine their application type, content nature, and source intent, thereby ensuring the protection of normal business operations and the timely detection of abnormal behavior. Traditional traffic classification methods, such as port-based identification and deep packet inspection (DPI) techniques, were once widely used in traffic billing, policy routing, and security monitoring. However, with the increasing demands for privacy and security from users, encrypted communication technologies, represented by transport layer security protocols (TLS), have become the mainstream of internet traffic. While encryption technology effectively protects communication content, it also renders traditional classification methods that rely on plaintext payload characteristics almost ineffective, creating significant blind spots in network management and security analysis.

[0003] To address this challenge, research in academia and industry has shifted its focus to extracting valuable features from encrypted traffic using machine learning and deep learning techniques. These methods typically treat traffic data (such as packet byte sequences and stream statistics) as input to machine learning models, learning inherent patterns in the data to achieve classification. In recent years, breakthroughs in natural language processing have led to the emergence of large-scale pre-trained language models based on the Transformer architecture, demonstrating powerful sequence modeling and feature extraction capabilities. Inspired by this, researchers have begun to treat encrypted traffic data (especially its byte sequences) as a special kind of language and process it using pre-trained language models. Existing work has shown that this approach can capture deep, context-dependent semantic patterns in encrypted traffic, achieving significant results in closed-set classification tasks.

[0004] However, existing encrypted traffic classification schemes based on pre-trained models still face several fundamental shortcomings in practical deployment. First, at the feature learning level, most methods indiscriminately concatenate the raw byte stream of the entire data packet (including the protocol header and encrypted payload) into a flat sequence input model. This approach ignores the inherent hierarchical structural semantics of network protocols themselves. For example, each field in the TCP / IP header has a clear independent meaning and fixed format. Mixing the structured header information with highly randomized encrypted payloads may lead to the model's inability to efficiently distinguish discrete patterns, affecting the discriminative power and generalization ability of the feature representation. Second, at the flow sequence modeling level, a network session is usually composed of multiple data packets that interact in a time sequence, containing important dynamic evolution information. Existing schemes usually simply concatenate the byte representations of multiple packets end to end, failing to explicitly characterize the boundaries and temporal dependencies between "packets," losing the inherent packet-level structural information of the flow, which is not conducive to the model's understanding of complete communication behavior. Finally, and most importantly, most existing research focuses on closed-set classification scenarios, that is, assuming that all categories in the test data have appeared in the training set. However, the real-world network environment is dynamic, with new applications, protocols, and malware variants constantly emerging. In open-set scenarios, models not only need to accurately classify known categories but also must be able to identify new categories not seen during training. Current methods typically optimize closed-set classification accuracy during model fine-tuning, lacking specific design for the core requirements of open-set recognition. This results in a feature space that is often not compact enough within known classes and has blurred boundaries between classes, limiting the performance of subsequent open-set classifiers (such as those relying on distance metrics) and making it difficult to reliably distinguish between known and unknown traffic. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide an open set encrypted traffic classification method and system based on a structured fine-tuning pre-trained model to solve the problems of insufficient full utilization of protocol structure semantics in feature extraction, lack of structured modeling for traffic sequence, and insufficient discriminative power of feature space in open set scenarios.

[0006] On one hand, this invention provides a method for classifying open-set encrypted traffic based on a structured fine-tuning pre-trained model, the method comprising: The raw encrypted network traffic data to be classified is preprocessed to convert the raw encrypted network traffic data into a lexical sequence that can be processed by a language model; wherein, the preprocessing includes organizing data packets according to network session streams and encoding the byte content of multiple data packets in each session stream into a continuous character sequence; The pre-trained language model is fine-tuned using the pre-processed lexical sequence. During the fine-tuning process, a structured guidance operation is performed on the lexical sequence. The structured guidance operation is used to guide the language model to learn the structured semantics of network traffic, including at least one of header field rearrangement operation and flow packet segmentation and representation operation. The header field rearrangement operation includes identifying and shuffling the order of subsequences corresponding to network protocol header fields in the lexical sequence. The flow packet segmentation and representation operation includes inserting separators between lexical subsequences corresponding to different data packets and assigning different segment embedding identifiers to lexicals belonging to different data packets. The preprocessed traffic word sequence to be classified is input into the fine-tuned language model, and the aggregated feature vector generated by the language model for the entire input sequence is extracted as the feature representation of the traffic. Based on the aggregated feature vector, an open set classification algorithm based on distance metric is used to calculate whether the encrypted network traffic data to be classified belongs to an unknown category; if it is determined that it does not belong to an unknown category, its known specific category is further determined.

[0007] In some embodiments, encoding the byte content of multiple data packets in each session stream into a continuous sequence of characters includes: Convert the raw byte content of each data packet into a hexadecimal string; The hexadecimal string is divided into two tuples of adjacent characters as a basic unit to form the continuous character sequence.

[0008] In some embodiments, the preprocessing further includes: A vocabulary is constructed for the continuous character sequence, and the character sequence is divided into a word sequence using a sub-word-based segmentation algorithm.

[0009] In some embodiments, the pre-trained language model is a generative pre-trained Transformer model.

[0010] In some embodiments, the structured bootstrapping operation includes the header field rearrangement operation and the stream packet segmentation and representation operation.

[0011] In some embodiments, the open set classification algorithm based on distance metric is the K-nearest neighbor distance algorithm, and the determination of whether it belongs to an unknown category includes: calculating the distance between the aggregated feature vector and the known category prototype or the known category sample, and comparing the distance with a preset threshold.

[0012] In some embodiments, further determining the known specific category to which it belongs includes: The aggregated feature vector is input into a closed-set classifier to obtain the probability that the encrypted network traffic data to be classified belongs to each known category, and the category with the highest probability is taken as the classification result.

[0013] In some embodiments, when organizing data packets by network session stream, only the first preset number of data packets in each session stream are extracted for the encoding operation, wherein the preset number is an integer greater than or equal to 1.

[0014] On the other hand, the present invention provides an open set encrypted traffic classification system based on a structured fine-tuning pre-trained model, including a processor, a memory, and a computer program / instructions stored in the memory. The processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of any of the methods described above.

[0015] On the other hand, the present invention provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the method as described in any of the above.

[0016] This invention provides a method and system for classifying open set encrypted traffic based on a structured fine-tuning pre-trained model, which has the following beneficial effects: This invention, by introducing a structured, guided fine-tuning mechanism, infuses pre-trained language models with a profound understanding of the intrinsic structure of encrypted network traffic. The model overcomes the limitations of traditional methods that treat traffic as a flat sequence of bytes, automatically learning and extracting highly discriminative feature representations from both protocol semantics and packet timing dimensions. This enables the model to form more compact clusters of known classes and clearer inter-class boundaries in the feature space.

[0017] Based on the optimized feature representation described above, the open set classification mechanism of this invention exhibits excellent practicality and robustness. It can efficiently and reliably identify unknown traffic categories that did not appear during training, significantly improving the model's adaptability in dynamically changing real-world network environments and the effectiveness of security monitoring. Simultaneously, while maintaining high classification accuracy for known application traffic, this method achieves keen perception of novel or abnormal traffic, providing strong technical support for refined network management and security protection.

[0018] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0019] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings: Figure 1 This is a schematic diagram illustrating the steps of an open set encrypted traffic classification method based on a structured fine-tuning pre-trained model in one embodiment of the present invention.

[0021] Figure 2 This is an overall flowchart of an open set encrypted traffic classification method based on a structured fine-tuning pre-trained model in one embodiment of the present invention.

[0022] Figure 3 This is a detailed flowchart of an open set encrypted traffic classification method based on a structured fine-tuning pre-trained model, according to an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0024] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0025] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0026] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0027] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0028] It should be emphasized here that the step markers mentioned below are not a limitation on the order of the steps, but should be understood as meaning that the steps can be executed in the order mentioned in the embodiments, or in a different order than in the embodiments, or several steps can be executed simultaneously.

[0029] To address the shortcomings of existing technologies, such as insufficient utilization of protocol structural semantics in feature extraction, lack of structured modeling for traffic sequence, and inadequate discriminative power of the feature space in open set scenarios, this invention provides an open set encrypted traffic classification method based on a structured fine-tuning pre-trained model. Figure 1 As shown, the method includes the following steps S101~S104: Step S101: Preprocess the raw encrypted network traffic data to be classified, converting it into a sequence of tokens that the language model can process. This preprocessing includes organizing data packets according to network session flows and encoding the byte content of multiple data packets within each session flow into a continuous character sequence.

[0030] Step S102: Fine-tune the pre-trained language model using the preprocessed lexical sequence. During fine-tuning, a structured guidance operation is performed on the lexical sequence. The structured guidance operation guides the language model to learn the structured semantics of network traffic, including at least one of header field rearrangement and flow packet segmentation and representation. Specifically, the header field rearrangement operation involves identifying and shuffling the order of sub-sequences corresponding to network protocol header fields in the lexical sequence, and the flow packet segmentation and representation operation involves inserting separators between lexical sub-sequences corresponding to different data packets and assigning different segment embedding identifiers to lexical segments belonging to different data packets.

[0031] Step S103: Input the preprocessed traffic term sequence to be classified into the fine-tuned language model, and extract the aggregated feature vector generated by the language model for the entire input sequence as the feature representation of the traffic.

[0032] Step S104: Based on the aggregated feature vector, use the open set classification algorithm based on distance metric to calculate whether the encrypted network traffic data to be classified belongs to an unknown category; if it is determined that it does not belong to an unknown category, then further determine its known specific category.

[0033] like Figure 2 The diagram shown is a flowchart of an open set encrypted traffic classification method based on a structured fine-tuning pre-trained model.

[0034] Step S101 comprises two parts: traffic data preprocessing and lexical sequence construction. The purpose of step S101 is to transform unstructured, binary raw network traffic packets (usually stored as pcap format files) into a structured, natural language-like sequence representation, namely a lexical sequence, so that the powerful sequence modeling capabilities of pre-trained language models can be utilized for feature learning. The preprocessing mainly includes two core sub-steps: organizing data packets according to the session flow, and encoding the packet byte content.

[0035] In some embodiments, step S101 can be further refined into steps S1011 to S1015: Step S1011: Data reading and privacy anonymization processing.

[0036] First, the raw network packet capture file is read using a Python network packet processing library (such as Scapy). To protect user privacy and encourage the model to learn general traffic patterns rather than specific network endpoint information, sensitive fields in the packets need to be anonymized. Specifically, the source and destination MAC addresses in the Ethernet frame header are replaced with a uniform address "00:00:00:00:00:00"; the source and destination IP addresses in the IP header are replaced with "0.0.0.0"; and the source and destination port numbers in the transport layer (such as TCP / UDP) header are replaced with 0. This anonymization process preserves key structural information such as packet length, protocol type, and payload content while eliminating identifiers related to specific hosts or services.

[0037] Step S1012: Session flow construction and packet organization.

[0038] The essence of network communication is a bidirectional session flow. This step aggregates the data packets processed by S1011 into different network session flows based on the network 5-tuple, namely source IP address, source port, destination IP address, destination port, and transport layer protocol type. Data packets belonging to the same 5-tuple are considered part of the same session flow and are sorted according to their timestamps to reflect the time sequence of communication.

[0039] Preferably, to strike a balance between preserving critical information and controlling computational complexity, only a predetermined number (e.g., the first three) of the data packets in each session stream are extracted for subsequent processing. Research shows that data packets in the initial stage of a session (such as TCP handshakes and TLS negotiations) contain key information for identifying the application type.

[0040] Step S1013: Byte content encoding and character sequence generation.

[0041] For each data packet in the organized session stream, its payload (including the anonymized header and the encrypted application layer payload) needs to be converted into a character sequence that can be processed by the pre-trained language model.

[0042] Preferably, the entire raw byte content of each data packet is converted into two-digit hexadecimal numbers, resulting in a hexadecimal string. For example, the binary byte sequence 0x0A, 0x1B will be converted into the string "0a1b". Then, to capture local correlation patterns between bytes, the resulting hexadecimal string is split into a sequence of two adjacent characters, forming a sequence of tuples. For example, the string "0a1b2c3d" will be split into the sequence ["0a", "1b", "2c", "3d"]. These tuples constitute the basic "vocabulary" describing the content of the data packets. Finally, the sequences of tuples corresponding to all (or the first N) data packets in the same session stream are concatenated in chronological order into a long, continuous character sequence to represent the entire network session stream.

[0043] Step S1014: Vocabulary construction and lexicalization.

[0044] In order to convert the continuous character sequence obtained in the previous step into the discrete word sequence required for the language model input, a lexicalization operation is required.

[0045] Preferably, based on a large amount of unlabeled network traffic data, a dedicated word segmenter is trained using a sub-word segmentation algorithm (such as WordPiece, Byte-Pair Encoding, or SentencePiece). This segmenter learns to merge more semantically meaningful and longer new symbols (sub-words) from frequently co-occurring tuples such as "0a", "1b", and "c3", thereby constructing a customized vocabulary optimized for network traffic data. In addition to the learned traffic sub-words, this vocabulary also includes necessary special markers, such as [PAD] for padding, [SEP] for separating different sentence segments (or data packets), [CLS] for representing the classification of the entire sequence, [UNK] for unknown symbols, and [MASK] for masked language modeling tasks. After training, the word segmenter is used to segment the character sequence of each session stream generated in step S1013 into a sequence of lexical units composed of corresponding IDs from the vocabulary. For example, the sequence ["0a", "1b", "2c"] might be mapped by the tokenizer to the word ID sequence [103, 207, 458].

[0046] Step S1015: Data formatting and storage.

[0047] The word sequences obtained from the above processing are associated with their corresponding traffic category labels (for supervised data), and divided into training, validation, and test sets (e.g., in an 8:1:1 ratio). For efficient access, the data can be stored in various formats, such as easy-to-read TSV files, structured JSON files, and memory-mapped file formats suitable for efficient reading and writing of large-scale numerical data.

[0048] Through the above steps S1011 to S1015, step S101 completely transforms the original encrypted network traffic data into a series of normalized lexical sequences.

[0049] Step S101 creates a traffic language that the language model can understand and process through anonymization, hexadecimal conversion, and word segmentation. Simultaneously, by organizing and concatenating data packets according to the conversation flow, a preliminary serialization context for the traffic data is constructed at the input level, laying the foundation for the subsequent model to deeply understand the temporal and structural characteristics of the traffic. The preprocessed word sequence serves as the direct input for fine-tuning the pre-trained language model in step S102.

[0050] In step S102, the model is fine-tuned based on structured guided operations. The purpose of step S102 is to use the word sequence with task annotations (for the training data) generated in step S101 to perform targeted training on the pre-trained language model that already possesses general sequence understanding capabilities, i.e., fine-tuning. The core innovation of fine-tuning lies in the introduction of structured guided operations, which aim to force the model, during the learning process, not only to understand the semantics of traffic content, but also to deeply focus on and model the inherent hierarchical structural information of network traffic data, namely the independent semantics of protocol header fields and the sequence relationships between data packets, thereby extracting more discriminative and structure-aware feature representations for open set classification tasks.

[0051] In some embodiments, step S102 can be further refined into steps S1021~S1023: Step S1021: Selection of pre-trained model and preparation of input.

[0052] Preferably, this invention uses a generative pre-trained Transformer model with powerful sequence modeling and feature extraction capabilities as the base model. Such models (e.g., NetGPT and GPT-2 models specifically designed for network traffic understanding) have been pre-trained on massive text corpora through self-supervised learning (as in the next word prediction task), effectively capturing long-range dependencies and contextual semantics. The word sequence output in step S101 is used as input for the fine-tuning stage. Before inputting into the model, a special marker [CLS] is added to the front of the sequence to aggregate the information of the entire sequence; its corresponding final hidden state will serve as the feature representation of the entire traffic sequence.

[0053] Step S1022: Implementation of structured guided operation.

[0054] In each training iteration, before or during the input of the word sequence into the model, perform at least one of the following structured guidance operations to inject structural prior knowledge into the data: (1) Header field rearrangement operation: The purpose of header field reordering is to break the model's dependence on the fixed position of protocol header fields in the byte stream, and force it to learn the semantics of each field and its combination logic, rather than a simple local byte pattern.

[0055] Specifically, this involves identifying consecutive subsequences corresponding to protocol header fields within a token sequence based on network protocol specifications (such as IP and TCP / UDP header formats). These subsequences represent structured information such as source / destination IP addresses, ports, and flags. Then, the token order within one or more identified header field subsequences is randomly shuffled byte-by-byte (i.e., maintaining the internal integrity of each tuple token). For example, a subsequence representing a TCP header ["45", "00", "00", "3c", ...] might be rearranged to ["00", "3c", "45", "00", ...]. The token sequence in the payload remains unchanged. This operation, as a strong data augmentation, forces the model to still understand its functional semantics even when faced with a disordered header byte order.

[0056] (2) Stream inner packet segmentation and representation operations: The purpose of intra-flow packet segmentation and representation operations is to explicitly reveal the boundaries and temporal relationships of data packets in the traffic sequence to the model, guide the model to learn intra-packet context and inter-packet dependencies, and thus understand the dynamic behavior of network flows.

[0057] Specifically, this includes: inserting a special packet separator, such as [PCK], between the subsequences corresponding to different original data packets in the long character sequence (or the word sequence after segmentation) formed in step S101. Next, assigning a segment embedding identifier to each word in the sequence. Typically, all words belonging to the first data packet are assigned segment ID 0, words belonging to the second data packet are assigned segment ID 1, and so on. Through the [PCK] marker and differentiated segment embeddings, the model's self-attention mechanism can clearly perceive "which data packet the current word belongs to," thereby more effectively modeling intra-packet close associations and inter-packet temporal or response relationships.

[0058] Segment embedding is a common technique in the Transformer model used to distinguish different parts of the input sequence. This invention applies it to distinguish different data packets in a network stream to encode packet order information.

[0059] Preferably, the above header field rearrangement operation and the flow packet segmentation and representation operation are used in conjunction to guide the model to comprehensively learn the structured semantics of traffic from both protocol fields and packet sequences.

[0060] Step S1023: Model fine-tuning training.

[0061] The token sequence processed by S1022 (including [CLS] tags) is input into a pre-trained Transformer model. The model is trained based on its inherent causal language modeling or masked language modeling objective. Specifically, for the input sequence... The model needs to maximize the conditional probability likelihood function for predicting the next word based on the preceding words. Among these, Indicates the first Each word element, ; This represents the total length of the sequence. This usually refers to the [CLS] tag.

[0062] Preferably, the objective function is as shown in formula (1): (1) in, Indicates model parameters; This represents the training dataset; This represents the probability distribution of the model output; Indicates the first Each word element.

[0063] By performing backpropagation and parameter updates under this objective, the model adapts to the distribution of encrypted traffic data. As the training data is processed through structured guided operations, its internal representation (especially the hidden state marked with [CLS]) can reflect the structured semantic features of network traffic.

[0064] Step S102, through steps S1021 to S1023, completes the targeted fine-tuning of the pre-trained language model.

[0065] Step S102 effectively injects domain knowledge (protocol structure, packet boundaries) into the data-driven deep learning process by introducing structured guiding signals. This enables the fine-tuned model to no longer simply read traffic byte text, but to understand network traffic like an article with chapters and paragraphs, thereby extracting more discriminative deep feature representations that are more suitable for open-set scenarios for downstream classification tasks. This step is crucial for improving the model's ability to distinguish between known categories and identify unknown categories.

[0066] In step S103, traffic feature extraction is performed based on the fine-tuned model. The purpose of step S103 is to transform the unknown encrypted network traffic data to be classified into a fixed-dimensional, dense numerical vector, i.e., a feature representation, using the model trained in the previous steps that possesses structured semantic understanding capabilities. This feature vector highly summarizes all the key information of the original traffic and forms the basis for subsequent efficient and accurate classification decisions. This step achieves the transformation from unstructured raw data to structured, highly discriminative features.

[0067] In some embodiments, step S103 can be further refined into steps S1031~1033: Step S1031: Preprocessing of data to be classified and sequence construction.

[0068] For a new encrypted network traffic dataset (typically a pcap file or real-time captured network packets) that needs to be classified, it is first preprocessed according to the same procedure as step S101 to ensure that the input format is consistent with that used during model training. Specifically: Privacy anonymization: The MAC address, IP address, and port number in this traffic data are anonymized in the same way.

[0069] Session stream extraction: Extract the target session stream based on the 5-tuple, and similarly, in a preferred embodiment, only the first preset number of data packets of the stream (such as the first three) are captured.

[0070] Encoding and word segmentation: The data packet byte content is converted into a hexadecimal string, divided into tuples, and the generated character sequence is converted into a word ID sequence using the dedicated word segmenter trained and saved in step S1014.

[0071] Structured Input Preparation: Add a [CLS] marker to the front of the generated word sequence, and may add other special markers such as [SEP] depending on model requirements. Simultaneously, reproduce the structured guidance operation defined in step S1022. That is, if header field rearrangement was used during fine-tuning, the identified header field subsequences also need to be rearranged here (using the same random strategy as during training, or a deterministic strategy); if stream packet segmentation and representation were used during fine-tuning, a [PCK] delimiter needs to be inserted here, and the corresponding segment embedding identifier needs to be assigned.

[0072] Step S1032: Model forward propagation and feature extraction.

[0073] The word sequence prepared in step S1031, which fully conforms to the model input specification, is input into the language model that has been fine-tuned in step S102. The model processes the input sequence via forward propagation. Each layer of the Transformer model encodes each word in the sequence, and finally, in the output layer, each input word corresponds to a high-dimensional context-aware hidden state vector.

[0074] Step S1033: Aggregate feature vectors.

[0075] In the output hidden state vector, the hidden state vector corresponding to the first [CLS] tag of the sequence is extracted. In the design of the Transformer architecture, the [CLS] tag is specifically used to aggregate and represent the global semantic information of the entire input sequence. After fine-tuning for the encrypted traffic classification task, the vector corresponding to this [CLS] tag is no longer a general language representation, but a highly specialized aggregated feature vector that deeply integrates specific traffic content, protocol structure semantics (learned through header reordering), and packet-level temporal information (learned through packet segmentation and representation).

[0076] Through the above steps S1031 to S1033, step S103 completes the transformation from the raw traffic to be classified into a high-quality feature vector.

[0077] Step S103 achieves information condensation and dimensionality reduction, compressing the original traffic into a fixed-length numerical vector, greatly simplifying the complexity of subsequent processing. Simultaneously, the structured guided fine-tuning of the model eliminates a large amount of noise and redundancy in the original data and amplifies the structured semantic patterns crucial for distinguishing different application categories (including known and unknown). The final output feature vector is a standard machine learning feature format that can be directly integrated into various downstream classifiers or clustering algorithms, providing a unified and high-quality data foundation for the open-set classification decision in step S104.

[0078] In step S104, open set and closed set classification decisions are made based on distance metrics. The purpose of step S104 is to perform a two-stage classification decision using the highly discriminative aggregated feature vector extracted in step S103: first, it determines whether the traffic belongs to an unknown category not seen during model training; if not, it is further classified into a known specific application category. This step is the final execution stage of this invention for dealing with real dynamic network environments and achieving open set identification, and its effectiveness highly depends on the high-quality feature space generated by the preceding steps.

[0079] In some embodiments, step S104 can be further refined into the following steps S1041~1044: Step S1041: Open set classifier preparation and threshold setting.

[0080] The present invention preferably employs an open set identification algorithm based on distance metric. Its core idea is that in a good feature space, samples of known categories will cluster near their respective category centers (prototypes), while samples of unknown categories will be relatively far away from these clustering areas.

[0081] In some embodiments, the K-nearest neighbor distance algorithm is employed. During the model training phase, the entire training set needs to be pre-processed using a fine-tuned model to extract aggregated feature vectors for all known class samples. Based on the aggregated feature vectors of known class samples, a class prototype for each known class is calculated (typically the mean of the feature vectors of all samples in that class). Simultaneously, on an independent validation set, the open set recognition performance (e.g., accuracy, F1 score) under different distance thresholds is evaluated to determine an optimal distance threshold, which is then used as the decision boundary for distinguishing between known and unknown information.

[0082] Step S1042: Distance calculation and open set discrimination.

[0083] The feature vector obtained by extracting the traffic flow to be classified through S103 can be denoted as: It calculates the distance score between the known world class and the known world class. The K-LND algorithm provides the following optional and specific distance calculation methods: Method 1 (Distance to the nearest prototype): Calculate the feature vector The distance score is obtained by taking the Euclidean or cosine distance to the nearest prototype among all known category prototypes.

[0084] Method 2 (average distance to the K nearest samples): Calculate the feature vector The distance score is obtained by averaging the distances to the feature vectors of the K nearest samples among all known class training samples.

[0085] Method 3 (Distance Ratio): Calculate the feature vector The distance score is calculated by taking the distance to the nearest class prototype and the distance to the second nearest class prototype.

[0086] The obtained distance score is compared with a preset threshold: If the distance score is greater than a preset threshold, the distance is considered too large, indicating that the sample to be tested is far from all known category clusters. Therefore, the encrypted network traffic data is determined to belong to an unknown category. The process ends here, and an unknown or specified background category label is output.

[0087] If the distance score is less than or equal to the preset threshold, the sample is determined to be close enough to the known category and is judged not to belong to the unknown category, that is, to belong to the closed set, and then proceeds to the next stage of specific category subdivision.

[0088] Step S1043: Closed set classifier decision.

[0089] For samples identified as closed sets by S1042, their specific known application categories need to be determined. To this end, a closed-set classifier is trained. This classifier takes the aggregated feature vector of samples from known categories as input and the category label as output.

[0090] Preferably, the closed-set classifier uses a Softmax classifier, which is a single-layer fully connected neural network. The feature vector of the sample to be classified is... The input is fed into a trained Softmax classifier, which outputs a probability distribution vector. ,in, Indicates the number of known categories. This indicates that the sample belongs to the first... Estimated probabilities of known categories, Ultimately, the category with the highest probability will be selected. The known specific category to which the encrypted network traffic data belongs has been determined.

[0091] Step S1044: Output the result.

[0092] Integrate the decision results from S1042 and S1043 to output the final classification label: either an unknown category or a specific known category. .

[0093] Through the above steps S1041 to S1044, step S104 completes the final classification decision.

[0094] Step S104 effectively distinguishes new applications or malicious traffic not covered in the training by simple distance comparison, greatly improving the model's practicality and robustness in real network environments. Simultaneously, based on an intuitive distance metric, open set discrimination requires no complex generative models or additional calibration networks, resulting in low computational overhead and stability. Closed set classification is performed on high-quality feature vectors, and in conjunction with mature classifiers such as Softmax, it can achieve high-precision discrimination between known categories. Furthermore, the open set discrimination module and the closed set classification module are relatively independent, making the algorithm easy to understand, debug, and extend (e.g., different distance metrics or closed set classifiers can be replaced).

[0095] like Figure 3 The diagram shown is a flowchart illustrating the open set encrypted traffic classification method based on a structured fine-tuning pre-trained model, as described in the specific embodiments above.

[0096] Corresponding to the above method, the present invention also provides an open set encrypted traffic classification system based on a structured fine-tuning pre-trained model, including a processor, a memory, and a computer program / instructions stored in the memory. The processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of the method described above.

[0097] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned method. The computer-readable storage medium may be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art.

[0098] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0099] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0100] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0101] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An open set encrypted traffic classification method based on structured fine-tuning pre-trained model, characterized in that, The method includes: The raw encrypted network traffic data to be classified is preprocessed to convert the raw encrypted network traffic data into a lexical sequence that can be processed by a language model; wherein, the preprocessing includes organizing data packets according to network session streams and encoding the byte content of multiple data packets in each session stream into a continuous character sequence; The pre-trained language model is fine-tuned using the pre-processed lexical sequence. During the fine-tuning process, a structured guidance operation is performed on the lexical sequence. The structured guidance operation is used to guide the language model to learn the structured semantics of network traffic, including at least one of header field rearrangement operation and flow packet segmentation and representation operation. The header field rearrangement operation includes identifying and shuffling the order of subsequences corresponding to network protocol header fields in the lexical sequence. The flow packet segmentation and representation operation includes inserting separators between lexical subsequences corresponding to different data packets and assigning different segment embedding identifiers to lexicals belonging to different data packets. The preprocessed traffic word sequence to be classified is input into the fine-tuned language model, and the aggregated feature vector generated by the language model for the entire input sequence is extracted as the feature representation of the traffic. Based on the aggregated feature vector, an open set classification algorithm based on distance metric is used to calculate whether the encrypted network traffic data to be classified belongs to an unknown category; if it is determined that it does not belong to an unknown category, its known specific category is further determined.

2. The open set encrypted traffic classification method based on structured finetuning pre-trained model according to claim 1, characterized in that, Encode the byte content of multiple packets in each session stream into a continuous sequence of characters, including: Convert the raw byte content of each data packet into a hexadecimal string; The hexadecimal string is divided into two tuples of adjacent characters as a basic unit to form the continuous character sequence.

3. The open set encrypted traffic classification method based on structured finetuning pre-trained model according to claim 1 or 2, characterized in that, The preprocessing also includes: A vocabulary is constructed for the continuous character sequence, and the character sequence is divided into a word sequence using a sub-word-based segmentation algorithm.

4. The open set encrypted traffic classification method based on structured finetuning pre-trained model according to claim 1, characterized in that, The pre-trained language model is a generative pre-trained Transformer model.

5. The open set encrypted traffic classification method based on structured finetuning pre-trained model according to claim 1, characterized in that, The structured bootstrapping operation includes the header field rearrangement operation and the stream packet segmentation and representation operation.

6. The open set encrypted traffic classification method based on structured hyperparameter-tuning pre-trained model according to claim 1, characterized in that, The open set classification algorithm based on distance metric is the K-nearest neighbor distance algorithm. The determination of whether it belongs to an unknown category includes: calculating the distance between the aggregated feature vector and the known category prototype or the known category sample, and comparing the distance with a preset threshold.

7. The open set encrypted traffic classification method based on structured hyperparameter-tuning pre-trained model according to claim 1, characterized in that, Further determine its specific known category, including: The aggregated feature vector is input into a closed-set classifier to obtain the probability that the encrypted network traffic data to be classified belongs to each known category, and the category with the highest probability is taken as the classification result.

8. The open set encrypted traffic classification method based on a structured fine-tuning pre-trained model according to claim 1, characterized in that, When organizing data packets according to network session streams, only the first preset number of data packets in each session stream are extracted for the encoding operation, where the preset number is an integer greater than or equal to 1.

9. An open-set encrypted traffic classification system based on a structured fine-tuning pre-trained model, comprising a processor, a memory, and a computer program / instructions stored in the memory, characterized in that, The processor is configured to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1 to 8.

Images