An encrypted traffic classification method based on a large language model

Abstract

The present disclosure provides an encrypted traffic classification method based on a large language model. The method is composed of three modules: an encrypted traffic data preprocessing and two-dimensional table representation construction module, a two-dimensional table representation alignment module, and an end-to-end classification framework module. The two-dimensional table representation of encrypted traffic is constructed by the encrypted traffic data preprocessing and two-dimensional table representation construction module, and the structured metadata and unstructured payload information are uniformly mapped to the semantic space of the large language model. The self-supervised alignment method of the two-dimensional table representation alignment module strengthens the understanding ability of the model for traffic representation. Finally, the end-to-end classification framework module is used to realize the end-to-end process from the original encrypted traffic to the classification result. This solves the problems of insufficient labeled data and diversified classification tasks in the existing encrypted traffic classification task.

Description

Technical Field

[0001] This disclosure relates to the field of computer science, and more specifically, to a method for classifying encrypted traffic based on a large language model. Background Technology

[0002] Network traffic classification technology is a crucial means of ensuring data flow security. By identifying sensitive traffic behaviors and malicious traffic characteristics, traffic classification can promptly detect unauthorized outflows and signs of authorized channel hijacking, providing a basis for decision-making regarding response measures such as disconnection and security accountability. To ensure secure data flow, traffic can be classified according to dimensions such as user behavior, transmitted content, and communication patterns. Combining this with specific concerns about sensitive traffic behaviors in different traffic scenarios can identify high-risk operations, thereby preventing unauthorized outflows. Furthermore, classifying abnormal, attack, and malicious traffic, and distinguishing it from benign traffic, plays a vital role in targeted identification and prevention of authorized channel hijacking. Traditional traffic classification methods typically include matching common ports of common applications and matching keywords in the plaintext payload and header—this method is often called DPI (Deep Packet Inspection), primarily targeting unencrypted traffic data. However, with the widespread adoption of traffic encryption technologies, traditional traffic classification methods are no longer applicable, posing challenges to traffic classification and increasing the difficulty of monitoring and management. For encrypted traffic, existing research mainly focuses on developing classification methods based on machine learning techniques. These methods can be categorized into traditional machine learning methods, deep learning methods, and methods based on pre-trained models. Common encrypted traffic features used in these methods include packet features, packet sequence features, and statistical features. Information about the raw traffic is also typically used in deep neural network structures. For example, CN202511550043.1 discloses a WireGuard multi-behavior traffic classification method and system based on sequence similarity segmentation. This technique constructs a three-dimensional traffic path containing payload length sequences, time sequences, and interval time sequences based on the segmented sub-sequences. It further extracts the bidirectional packet length sequences and their cumulative sequences between the client and server to obtain the multi-dimensional traffic path. Then, a sliding window method is used to extract path signature features from the multi-dimensional traffic path. Finally, the path signature features are input into an LSTM model for time series modeling, and user behavior is classified through a fully connected layer. CN202511405859.5 discloses a network encrypted traffic classification method based on convolutional self-attention. This technique inserts the interval time sequence of the encrypted traffic sequence into the original packet sequence to form a processed sequence with interleaved packets and interval time. This processed sequence is then converted into a hexadecimal string sequence and input into a network consisting of convolutional layers and several sub-attention coding modules for feature extraction. Finally, the extracted network encrypted traffic features are input into a fully connected layer and a Softmax layer to achieve classification. Furthermore, CN202310356710.7 discloses an encrypted traffic classification method and apparatus based on the SwinT-CNN model. This technique converts the encrypted traffic data to be classified into a two-dimensional matrix, generates a grayscale image, inputs the grayscale image into an improved CNN module and an improved attention coding module, and finally uses a fully connected layer activated by Softmax to make predictions and generate classification results.

[0003] Existing technologies all rely on researchers' experience to select specific features from raw traffic as input. The classification performance of these methods is limited by the degree of matching between the selected features and the task scenario, and their generalization ability is often insufficient when facing diverse classification tasks. In fact, the scarcity of labeled data and the diversity of classification tasks are two major technical challenges faced by existing encrypted traffic classification methods. Unlike the three works mentioned above and other mainstream works in the field of encrypted traffic classification, this invention discloses an encrypted traffic classification technology based on a large language model. Based on task-independent encrypted traffic metadata and payload information, it generally bridges the modal differences between encrypted traffic modalities and LLM pre-training corpora by constructing a two-dimensional representation of encrypted traffic and aligning this representation with the LLM semantic space. In end-to-end classification tasks, it effectively utilizes the strong generalization ability of LLM and its adaptability to data-constrained scenarios, thus effectively addressing the problems of scarce labeled data and diverse classification tasks.

[0004] The scarcity of labeled data is primarily due to the difficulty of labeling and the scarcity of effective data collection scenarios. Specifically, traffic captured on the public internet often lacks real labels, making it difficult to accurately define its type; traffic data generated by corporate activities is more valuable for application, but it often cannot be publicly shared due to commercial privacy or confidential information; data collected in laboratory settings is easy to label, but its quantity is often limited by equipment scale, making it difficult to simulate large-scale real-world traffic environments. This situation makes it difficult for many existing methods that rely on large-scale labeled data to be trained sufficiently and effectively.

[0005] Furthermore, the tasks of categorizing encrypted traffic for ensuring secure data flow exhibit considerable diversity. For instance, the categorization dimensions for preventing unauthorized outflows include transmission protocols, service types, user behavior categories, specific applications, specific web pages, and types of transmitted content. The categorization dimensions for preventing authorized channel hijacking include abnormal traffic detection (benign / malicious binary classification), botnets, malware, network attacks, and phishing. However, some existing methods employ statistical or temporal features that are highly relevant to the task scenario and heavily reliant on expert experience. These features lack universal characterization capabilities, resulting in insufficient generalization ability when facing diverse task scenarios, making it difficult to meet the needs of ensuring secure data flow.

[0006] Currently, the mainstream methods in the field of encrypted traffic classification, including methods based on traditional learning, methods based on deep learning, and methods based on pre-trained models, cannot effectively overcome the two challenges mentioned above.

[0007] Traditional machine learning methods typically take specific statistical features and time-series features selected based on the task scenario as input and are trained using traditional machine learning paradigms. These methods offer good interpretability, but their construction often relies on expert experience, and their performance is limited by the degree to which the selected input features match the task scenario. When the task scenario changes, the original features often become invalid, resulting in insufficient generalization ability and high development costs when dealing with diverse task scenarios.

[0008] In deep learning-based methods, statistical features are used relatively infrequently; temporal features of data packets and raw traffic data are the mainstream input formats. These methods can automatically extract complex representational patterns from data, offering advantages over traditional machine learning methods in terms of cross-task generalization and development costs. However, for work based on packet temporal features, its performance and generalization are still limited by the selection of task-related temporal feature types, exhibiting limitations similar to traditional machine learning. Furthermore, deep learning models typically rely on large-scale labeled data for training to achieve better classification performance.

[0009] Pre-trained models refer to neural networks pre-trained on large-scale unlabeled datasets. They can learn general semantic information and can be fine-tuned to adapt to specific downstream tasks. Compared to traditional machine learning and deep learning methods, methods based on pre-trained models can learn the general semantics of encrypted traffic using unlabeled data, significantly reducing the dependence on labeled data and demonstrating relatively good cross-task generalization. These methods mainly use raw traffic data in the form of byte streams or grayscale images as input, and some works still use the temporal features of data packets. Their methodologies mainly draw on the fields of natural language processing (NLP) and computer vision (CV), widely adopting Transformer, Convolutional Neural Network (CNN), and Autoencoder (AE) as backbone networks, and transferring and using pre-training strategies such as Masked Language Model (MLM) and Masked Autoencoder (MAE). However, despite the significant advantages of pre-trained models, the byte stream or grayscale representation of encrypted traffic has significant modal differences from natural language text and real images, hindering the effective mining and understanding of traffic semantics by the model, and to some extent limiting the performance of these methods on encrypted traffic classification tasks. Summary of the Invention

[0010] The purpose of this disclosure is to provide a method for classifying encrypted traffic based on a large language model, which aims to solve the problems of scarce labeled data and diverse classification tasks faced by existing methods in encrypted traffic classification tasks.

[0011] In general, a method for classifying encrypted traffic based on a large language model is provided, which consists of three modules: encrypted traffic data preprocessing and two-dimensional table representation construction module, two-dimensional table representation alignment module, and end-to-end classification framework. By constructing a two-dimensional table representation of encrypted traffic through encrypted traffic data preprocessing and a two-dimensional table representation construction module, structured metadata and unstructured payload information are uniformly mapped to the semantic space of the large language model. The self-supervised alignment method of the two-dimensional table representation alignment module enhances the model's understanding of traffic representation. Finally, the end-to-end classification framework module is used to realize the end-to-end process from raw encrypted traffic to classification results.

[0012] The specific implementation method of the encrypted traffic data preprocessing and two-dimensional table representation construction module is as follows: First, the input encrypted traffic data is standardized and preprocessed, and a two-dimensional table representation of traffic metadata and a compressed representation of encrypted payload are constructed to facilitate understanding by the large language model. Finally, the structured metadata and unstructured payload information are uniformly mapped to the semantic space of the large language model through the joint representation fusion module of metadata and payload.

[0013] The specific method for standardizing and preprocessing the encrypted traffic data is as follows: The input PCAP file is parsed using a traffic session segmentation tool. Based on the five-tuple characteristics consisting of source address, destination address, source port, destination port, and transport protocol, the continuous data packet sequence is divided into TCP or UDP data stream sample units. Ethernet layer metadata information in the samples is filtered and removed, and anonymization mapping processing is performed on the network layer and transport layer address information in the samples. Within the same data stream sample, the real source / destination IP address and port number are replaced with a consistent random pseudo-identifier.

[0014] The method for constructing the two-dimensional table representation of the traffic metadata is as follows: a data stream contains Each data packet contains If a metadata field is provided, then the metadata of the stream is represented as a shape. The table contains non-header cells that store metadata values ​​and header cells that store metadata keys. Special tags "<" and ">" are added before and after the content of each cell. For the above table structure, the configuration of the two-dimensional position encoding mechanism includes: Token two-dimensional position encoding: The first dimension of the encoding corresponds to the row index, where the index of the header row is set to 0; the second dimension of the encoding corresponds to the column index, and the value of each token is the sum of the starting position encoding of the column and the token index in the cell. Column start position code: The start position code of the first column is set to 0, and the start position code of each subsequent column is the sum of the start position code of the previous column and the maximum number of tokens in all cells of that column; Sequence sparsity strategy: Only tokens with actual semantic information are retained in the input sequence, and placeholders are not included to fill empty spaces; The generation process of the metadata token sequence and two-dimensional location code includes the following steps: initializing the string table, traversing the metadata fields to fill the table header, traversing the metadata fields one by one to fill the metadata value, tokenizing the cell content, iteratively calculating the starting position code of each column, and traversing the table column by column and row by row to fill in the metadata token sequence and location code sequence.

[0015] The specific method for compressing and characterizing the encrypted payload is as follows: A BERT-like bidirectional encoder is used to compress and represent the encrypted payload. The encoder’s ability to extract general semantics is enhanced by contrastive learning pre-training. The specific design includes: contrastive learning pre-training, encoder output representation, dimension alignment projection, and encoder selection. Contrastive learning pre-training: The encoder undergoes contrastive learning based on the SimSiam architecture, using only positive examples. In the contrastive training task based on the SimSiam architecture, given two inputs... and Define Encoder as Predictor is Stop gradient operation as In the comparative training, the two intermediate representations are respectively represented as: as well as The cosine similarity is calculated as follows: The weighting coefficients are defined based on the strength levels of positive sample pairs: The overall loss function is defined as: Encoder output representation: A special token [CLS] is added to the beginning of the input sequence, and the embedding vector corresponding to the first [CLS] of the output tensor is selected as the output representation of the Encoder; Dimension-aligned projection: When integrating the Encoder with LLM, a linear projection layer is introduced to map the Encoder output dimension to the hidden dimension of the LLM embedding vector; Encoder selection: The LongFormer structure was adopted as the specific implementation of the Encoder.

[0016] The specific method for the joint representation fusion of metadata and payload is as follows: For stream-based classification tasks, each stream contains metadata and an optional encrypted payload. During the metadata table construction phase, specific column positions are reserved for the encrypted payload, and the compressed representation of the payload is embedded into the corresponding position to form a complete two-dimensional table representation of the encrypted traffic. If a packet has no payload, the position is left blank, and it neither participates in the Encoder compression nor is included in the LLM input sequence.

[0017] The specific implementation method of the two-dimensional table representation alignment module is as follows: the two-dimensional table representation of encrypted traffic is effectively aligned with the semantic space of the large language model using unlabeled data. This is achieved through two sub-modules: a metadata two-dimensional table representation alignment sub-module for metadata two-dimensional table representation and a payload-containing encrypted traffic two-dimensional table representation alignment sub-module for complete encrypted traffic two-dimensional table representation containing metadata and payload.

[0018] The implementation method of the alignment submodule represented by the metadata two-dimensional table is as follows: fine-tune the alignment task through corresponding instructions and construct a self-supervised training target; Specifically, by using unlabeled data to generate metadata table representations and combining them with self-supervised tasks such as mask prediction and row-column relationship reasoning, LLM can effectively understand the two-dimensional table structure of metadata and comprehend the collaborative information of the row and column dimensions. Based on the table structure design of metadata, the training task includes six tasks: (1) Table delimiter task, the input is the text Prompt and the table representation of the stream, and the target output is the value of the first and last cells of the table without cell delimiters; (2) Cell retrieval task, the input is the text Prompt, the table representation of the stream and the value of a random cell, the target output is the row number and column number of all cells with that value; (3) Cell reverse retrieval task, the input is the text Prompt, the table representation of the stream and the row number and column number of a random cell, and the target output is the value of the cell; (4) Column retrieval task, the input is the text Prompt, the table representation of the stream and the column number of a random column, and the target output is the column name of the column; (5) Row retrieval task, the input is the text Prompt, the table representation of the stream and the row number of a random row, and the target output is the value of all cells in the row; (6) Table size detection task; the input is the text Prompt and the table representation of the stream, and the target output is the number of rows and columns of this table; The training phase of the alignment process employs a LoRA fine-tuning strategy, freezing the LLM backbone parameters and allowing only the added LoRA low-rank matrix parameters to be learned.

[0019] The specific implementation of the alignment of the encrypted traffic two-dimensional table representation with payload is as follows: A traffic table representation with payload is generated using unlabeled data. A self-supervised task is designed to enable LLM to understand the semantic relationship between the compressed payload embedding vector and metadata, achieving a deep understanding of the overall traffic representation. The training tasks include the following four types: (1) Basic information restoration task, the input is the text Prompt and the compressed representation of a single encrypted payload, the target output is the transport layer protocol type (TCP or UDP) of the stream to which the payload belongs, the length of the payload and the first few hexadecimal characters; (2) Metadata-payload matching task. The input is a text Prompt and several data packet metadata tables with no specific association. A column of non-empty payloads corresponding to these metadata in random order is appended to the end of the table. The target output is the metadata row number corresponding to each payload row. (3) Load sequence sorting task: The input is a text Prompt and a single table consisting of a non-empty encrypted load sequence in the same burst in shuffled order. The target output is the position number of each load in the original burst. (4) Packet sorting task: The input is a text Prompt and the metadata and encrypted payload of the data packets in the same burst in shuffled order. The metadata and payload in each line come from the same data packet. The target output is the sequence number of each line in the original burst. During the alignment process, the training phase freezes the weights of the Encoder and the backbone weights of the LLM, allowing only the weights of the linear projection layer between them to be learned.

[0020] The specific implementation of the end-to-end classification framework module is as follows: Based on the encrypted traffic two-dimensional table representation and the aligned large language model, an encrypted traffic end-to-end classification framework based on the large language model is constructed. The structured metadata and unstructured payload information are uniformly mapped to the semantic space of the large language model, and fine-tuned to adapt it to downstream classification tasks.

[0021] The specific process of the end-to-end classification framework is as follows: For a given encrypted traffic stream defined by a 5-tuple , represented as a data packet sequence ,in The number of packets contained in the stream, each packet Includes metadata information and encrypted payload information ,Right now Decompose the stream into metadata sequences and encrypted payload sequence Process them separately; First, regarding the metadata sequence A two-dimensional table representation method is used for processing, and the metadata is converted into a key-value table in natural language form through text parsing. Its shape is ,in To specify the number of metadata fields containing the encrypted payload, the first row is the table header. The column containing the encrypted payload is temporarily filled with tokens of length 1, using a tokenizer. Convert the table content into an input identifier sequence The sequence length is : Assign a two-dimensional location code to each token , of which The position code of each token is represented as a row index. and column indexes tuples: in, This indicates the row number of the token in the table. This represents the starting code of the column containing the token, plus its relative sequence number within the cell. The token sequence is input through the LLM embedding layer. Mapped as a token embedding sequence : in It is the hidden layer dimension of LLM; Secondly, regarding the encrypted payload sequence A contrastive learning-based compressed representation method is adopted. The payload data is processed by a pre-trained encrypted payload encoder to extract features, and the first position [CLS] token of the output tensor is taken as the semantic representation of the payload. Subsequently, the aligned linear projection layer is used to... Hidden layer dimensions mapped to LLM The load characterization sequence was obtained. : in, and These are the parameters of the linear projection layer; if a data packet does not contain an encrypted payload, the corresponding position is empty and does not participate in subsequent feature fusion. Subsequently, feature fusion was performed. During the metadata table construction phase, specific column positions were reserved for the encrypted payload. Reserve an index set for the corresponding load positions in the input sequence; characterize the load sequence. Embedded into The corresponding index position is used to replace the original placeholder embedding, thus forming the fused input representation. : To guide the model in performing a classification task, a one-dimensional task instruction prompt text is constructed. Using a word segmenter Convert the prompt text into an input identifier sequence. The sequence length is The input identifier sequence is obtained through the embedding layer of the LLM. Mapped as a cue embedding sequence : Embed the prompt in the sequence With the fused input representation The sequences are concatenated to form the basic context embedding sequence. : in, Indicates a splicing operation; During the model fine-tuning phase, the model input sequence contains the content of the true labels; let the true label sequence be... Its embedding representation is The embedding sequence length is Constructing fine-tuned input embedding sequences : in, This represents the true label embedding, where the input embedding sequence is processed by the backbone LLM. of The Transformer decoder transforms layer by layer to obtain the fine-tuned hidden layer state sequence. : in Indicates the first The hidden state at the nth time step is then mapped to the vocabulary space through a vocabulary projection layer to obtain the nth time step. The logits vector at each time step : in, For the projection matrix, For bias terms, For a vocabulary list; Calculate the first using the Softmax function The output and the true label at each time step Vocabulary at the location Same probability: Training is performed using an autoregressive loss function, specifically defined as the negative log-likelihood loss between the predicted class token and the true class label: Through end-to-end LoRA fine-tuning, the model can jointly optimize its ability to understand and extract metadata semantics and payload semantics, minimizing the loss function. This enables high-precision encrypted traffic classification; During the inference phase, for each time step The model input sequence contains a greedy policy. The content generated previously; let the generated tag sequence be... Its embedding representation is Constructing the inference input embedding sequence : in, Indicates time The previously generated label embeddings, the input embedding sequence, are transformed layer by layer by the backbone LLM to obtain the inference hidden layer state sequence. : in Indicates the first The hidden state at the nth time step; through the vocabulary projection layer mapping, the nth time step is obtained. logits vector at each time step : The first stage of model inference In each time step, according to Vectors, also using the Softmax function to calculate the vocabulary. probability distribution on : A greedy decoding strategy is used, selecting the term with the highest probability as the prediction result: The generation process is iterative until the model generates a preset terminator or reaches the maximum generation length, resulting in a complete output word sequence. ; Then, the reverse process of the word segmenter is used to output the word sequence. Restored to natural language text string By using exact string matching or keyword extraction, from The final encrypted traffic classification result label is parsed out, completing the end-to-end classification task.

[0022] The technical effects to be achieved by the embodiments of the present invention are as follows: This invention reduces reliance on labeled data and overcomes the problem of labeled data scarcity. By constructing a self-supervised objective, it fully utilizes unlabeled data to enhance the model's semantic understanding and mining capabilities, thus reducing dependence on labeled data. Through self-supervised tasks such as mask prediction, row-column relationship retrieval, payload-metadata matching, and sequence sorting, the model can fully align the two-dimensional table representation with its semantic space based on unlabeled data, strengthening its ability to understand and mine deep traffic information. Simultaneously, a contrastive learning paradigm based on SimSiam is used to train the encrypted payload encoder, prompting it to extract general features of the encrypted payload. Through the aforementioned pre-trained self-supervised process, this invention enhances the LLM's understanding of encrypted traffic, fully leveraging its rapid adaptability in scenarios with few labeled samples in downstream tasks, effectively solving the problem of decreased model classification performance caused by the scarcity of labeled data in encrypted traffic classification scenarios.

[0023] This invention demonstrates strong generalization ability across diverse tasks in encrypted traffic classification scenarios: Unlike existing technologies that rely on expert experience to select statistical or temporal features, which suffer from feature dependency on task scenarios and easy failure when changing scenarios, this invention classifies encrypted traffic based on task-independent metadata and payload information, avoiding the dependence of feature engineering on specific task scenarios. Furthermore, this invention leverages the powerful semantic understanding and reasoning capabilities of LLM, stimulating the world knowledge learned by LLM during pre-training through natural language instructions, thereby exhibiting excellent generalization ability across various task scenarios.

[0024] Overcoming the modal differences between encrypted traffic and pre-trained data of large language models, and fully utilizing the world knowledge of large language models: Raw encrypted traffic data can be divided into structured metadata and unstructured encrypted payloads. Existing mainstream LLMs are mainly good at processing natural language text or real images. Faced with traffic data that is complex in type and encoded differently from natural language text or natural images, it is difficult to understand and process it directly and effectively, which limits the general capabilities of LLMs in classification tasks. This invention innovatively transforms metadata into a two-dimensional table representation in the form of natural language text, and combines it with two-dimensional positional encoding to reduce the understanding difficulty of LLMs and stimulate their ability to parse metadata from a two-dimensional perspective. By using compressed payload representation and contrastive learning pre-training, the complete information of the payload is preserved, the general semantics of the payload are extracted, and the token overhead is reduced. Finally, the two are integrated into a unified two-dimensional table representation, enabling LLMs to understand metadata and payload information collaboratively from a two-dimensional perspective. Through a reasonably designed alignment training process, the modal differences that still exist between the two-dimensional table representation and the LLM pre-training data are bridged. By employing a rationally designed representation and alignment process, this invention overcomes the modal differences between encrypted traffic and LLM pre-training data, enabling LLM to fully leverage the world knowledge learned during its pre-training phase in classification tasks, thereby effectively improving the classification performance of encrypted traffic. Attached Figure Description

[0025] The above and other objects and features of this disclosure will become clearer from the following description taken in conjunction with the accompanying drawings.

[0026] Figure 1 This is a schematic diagram illustrating the reasoning process of an encrypted traffic classification method based on a large language model according to an embodiment of the present disclosure; Figure 2 This is a schematic flowchart illustrating the encrypted traffic data preprocessing and two-dimensional table representation construction process according to an embodiment of the present disclosure; Figure 3 This is a schematic diagram illustrating the alignment flowchart of a two-dimensional table representation of encrypted traffic according to an embodiment of the present disclosure; Figure 4 This is a schematic flowchart illustrating an end-to-end classification framework for encrypted traffic based on a large language model according to an embodiment of the present disclosure. Detailed Implementation

[0027] The following detailed embodiments are provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but may be changed as will become clear upon understanding this disclosure, except for operations that must occur in a specific order. Furthermore, for clarity and conciseness, descriptions of features known in the art may be omitted.

[0028] The features described herein may be implemented in different forms and should not be construed as limited to the examples described herein. Rather, the examples described herein are provided only to illustrate some of the many feasible ways of implementing the methods, apparatus, and / or systems described herein, which will become clear upon understanding the disclosure of this application.

[0029] As used herein, the term “and / or” includes any one of the associated listed items and any combination of any two or more.

[0030] Although terms such as “first,” “second,” and “third” may be used herein to describe various components, assemblies, regions, layers, or parts, these components, assemblies, regions, layers, or parts should not be limited by these terms. Rather, these terms are used only to distinguish one component, assembly, region, layer, or part from another. Thus, without departing from the teaching of the examples described herein, the first component, first assembly, first region, first layer, or first part referred to as the first component, first assembly, first region, first layer, or first part may also be referred to as the second component, second assembly, second region, second layer, or second part.

[0031] In the specification, when an element (such as a layer, region, or substrate) is described as being "on" another element, "connected to," or "bonded to" another element, the element may be directly "on" another element, directly "connected to," or "bonded to" the other element, or one or more other elements may be present in between. Conversely, when an element is described as being "directly on" another element, "directly connected to," or "directly bonded to" another element, no other elements may be present in between.

[0032] The terminology used herein is for the purpose of describing various examples only and is not intended to limit disclosure. Unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. The terms “comprising,” “including,” and “having” indicate the presence of the described features, quantities, operations, components, elements, and / or combinations thereof, but do not preclude the presence or addition of one or more other features, quantities, operations, components, elements, and / or combinations thereof.

[0033] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains upon understanding this disclosure. Unless expressly defined herein, terms (such as those defined in a general dictionary) shall be interpreted as having a meaning consistent with their meaning in the context of the relevant field and in this disclosure, and shall not be interpreted in an idealized or overly formalistic manner.

[0034] Furthermore, in the description of the examples, detailed descriptions of well-known related structures or functions will be omitted when it is believed that such detailed descriptions would lead to a vague interpretation of this disclosure.

[0035] Figure 1 This is a schematic diagram illustrating an encrypted traffic classification method based on a large language model according to an embodiment of the present disclosure.

[0036] To achieve the aforementioned objectives, the present invention employs the following technical framework: Figure 1 As shown.

[0037] The encrypted traffic classification method proposed in this invention, based on a large language model, consists of three modules: encrypted traffic data preprocessing and two-dimensional table representation construction, two-dimensional table representation alignment, and an end-to-end classification framework. By constructing a two-dimensional table representation of encrypted traffic, structured metadata and unstructured payload information are uniformly mapped to the semantic space of the large language model. Furthermore, a self-supervised alignment method is used to enhance the model's understanding of the traffic representation, ultimately achieving an end-to-end process from raw encrypted traffic to classification results.

[0038] Encrypted traffic data preprocessing and two-dimensional table representation construction module: This module is responsible for standardizing and preprocessing the input encrypted traffic data and constructing a two-dimensional table representation that is easy for large language models to understand. This module includes four sub-modules: encrypted traffic preprocessing, two-dimensional table representation of traffic metadata, compressed representation of encrypted payload, and joint representation fusion of metadata and payload.

[0039] Encrypted traffic preprocessing This module is responsible for preprocessing the raw network traffic data. Specifically, the system uses a traffic session segmentation tool (such as SplitCap) to parse the input PCAP file. Based on the five-tuple characteristics consisting of source address, destination address, source port, destination port, and transport protocol, it divides continuous data packet sequences into TCP or UDP data stream sample units. To reduce potential interference with model feature learning, the system filters and removes Ethernet layer metadata information from the samples and performs anonymization mapping on the network layer and transport layer address information in the samples: within the same data stream sample, the real source / destination IP address and port number are replaced with consistent random pseudo-identifiers. This processing mechanism hides the real network topology information while preserving the communication correlation and distinguishable features between hosts within the sample, ensuring that the model can learn traffic behavior patterns rather than specific network addresses.

[0040] Two-dimensional tabular representation of traffic metadata Suppose a data stream contains Each data packet contains If a metadata field is provided, then the metadata of the stream can be represented as a shape. The table (including header) contains metadata values ​​in non-header cells and metadata keys in header cells. To clearly define cell boundaries, special tags "<" and ">" are added before and after the content of each cell.

[0041] Given that encrypted traffic metadata has a highly structured nature and the information arrangement follows specific protocol specifications, while containing information in two dimensions: data packets and feature fields, this invention adopts a tabular structure to organize the metadata and configures a corresponding two-dimensional positional encoding mechanism to support LLM in parsing metadata information from a two-dimensional perspective.

[0042] In practical implementation, for any target data stream containing N data packets, and each data packet associated with M metadata fields, the metadata of this data stream can be constructed as a table structure (including header rows) with dimensions (N+1)×M. The header row cells store the metadata field names, while the non-header row cells store the specific metadata values. To further distinguish cell boundaries and prevent the model from confusing cell content, specific identifiers "<" and ">" are appended to the beginning and end of the content of each cell.

[0043] For the above table structure, the two-dimensional position encoding mechanism configured in this embodiment includes: Token Two-Dimensional Position Encoding: The first dimension of the encoding corresponds to the row index, where the index of the header row is set to 0; the second dimension of the encoding corresponds to the column index, and the value of each token is the sum of the starting position encoding of the column and the token index within the cell.

[0044] Column start position code: The start position code of the first column is set to 0, and the start position code of each subsequent column is the sum of the start position code of the previous column and the maximum number of tokens in all cells of that column.

[0045] Sequence sparsity strategy: To reduce token consumption, the input sequence retains only tokens with actual semantic information and does not include placeholders used to fill empty spaces.

[0046] Table 1: The generation process of the metadata token sequence and two-dimensional location code follows the description of Algorithm 1 as shown in Table 1, and specifically includes the following steps: initializing the string table, traversing the metadata fields to fill the table header, traversing the metadata fields one by one to fill the metadata value, tokenizing the cell content, iteratively calculating the starting position code of each column, and traversing the table column by column and row by row to fill in the metadata token sequence and location code sequence.

[0047] Compression characterization of encrypted payload To address the issue that the encrypted payload contains a large amount of information and that direct input would lead to excessive token overhead, this invention employs a BERT-like bidirectional encoder to compress and represent the encrypted payload, and enhances the encoder's ability to extract general semantics through contrastive learning pre-training.

[0048] The specific design includes: Contrastive Learning Pre-training: The Encoder undergoes contrastive learning based on the SimSiam architecture, focusing on positive examples only. Encrypted payloads within the same burst (continuous packet sequences in the same direction) are considered strong positive samples, while encrypted payloads in the same stream but different bursts are considered weak positive samples. Weighting coefficients are introduced into the loss calculation to differentiate the strength of sample similarity. This multi-level positive sample design aims to enable the encoder to effectively extract general features at the application, session, and action levels.

[0049] Encoder output representation: A special token [CLS] is added to the beginning of the input sequence, and the embedding vector corresponding to the first [CLS] of the output tensor is selected as the output representation of the Encoder.

[0050] Dimension Alignment Projection: A linear projection layer is introduced when integrating the Encoder with LLM to map the Encoder output dimension to the hidden dimension of the LLM embedding vector.

[0051] Encoder selection: The LongFormer structure was adopted as the specific implementation of the Encoder.

[0052] In a contrastive training task based on the SimSiam architecture, given two inputs... and Define Encoder as Predictor is Stop gradient operation as The two intermediate representations in the comparative training are respectively represented as follows: as well as Cosine similarity is calculated as follows: The weighting coefficients are defined based on the strength levels of positive sample pairs: The overall loss function is defined as: Joint representation fusion of metadata and payload For stream-based classification tasks, each stream contains metadata and an optional encrypted payload. During the metadata table construction phase, specific column positions (such as the tcp.payload or udp.payload column) are reserved for the encrypted payload, and the compressed representation of the payload is embedded into the corresponding position to form a complete two-dimensional table representation of the encrypted traffic. If a packet has no payload, that position is left blank, and it neither participates in the encoder compression nor is included in the LLM input sequence.

[0053] Encrypted traffic 2D table representation alignment module: This module constructs a self-supervised instruction fine-tuning alignment training process. It effectively aligns the two-dimensional table representation of encrypted traffic with the semantic space of a large language model using unlabeled data. This bridges the modal differences that still exist between the encrypted traffic two-dimensional table representation and the pre-training corpus of the large language model, enhancing the model's understanding of these differences and improving its convergence efficiency and classification performance. This module includes two sub-modules: metadata two-dimensional table representation alignment and payload-containing encrypted traffic two-dimensional table representation alignment.

[0054] Metadata Two-Dimensional Table Representation Alignment This module addresses the two-dimensional table representation of metadata by fine-tuning alignment tasks through corresponding instructions to construct a self-supervised training objective. Specifically, it generates a metadata table representation using unlabeled data and combines it with self-supervised tasks such as mask prediction and row-column relationship reasoning to enable LLM to effectively understand the two-dimensional table structure of metadata and comprehend the collaborative information of row (per-pack) and column (per-feature) dimensions.

[0055] Based on the table structure design of the metadata, the training task mainly includes six tasks: Table delimiter task. The input is a text Prompt, a stream of table representations, and the target output is the values ​​of the first and last cells of the table without cell delimiters.

[0056] Cell retrieval task. The input consists of the text "Prompt", a tabular representation of the stream, and the value of a random cell. The target output is the row and column numbers of all cells that have the given value.

[0057] Cell reverse lookup task. The input consists of the text Prompt, a table representation of the stream, and the row and column numbers of a random cell. The target output is the value of that cell.

[0058] Column retrieval task. The input consists of a text prompt, a table representation of the stream, and the column number of a random column. The target output is the column name of that column.

[0059] Row retrieval task. The input consists of the text Prompt, a tabular representation of the stream, and the row number of a random row. The target output is the values ​​of all cells in that row.

[0060] Table size detection task. The input is a text prompt and a table representation of the stream. The target output is the number of rows and columns of this table.

[0061] The training phase of this alignment process employs a LoRA fine-tuning strategy, freezing the LLM backbone parameters and allowing only the parameters of the added LoRA low-rank matrix to be learned; since the input does not contain the cryptographic payload, it does not involve the relevant structure of the cryptographic payload.

[0062] Alignment of encrypted traffic in two-dimensional tables with payload This module designs corresponding instruction fine-tuning alignment tasks for a complete encrypted two-dimensional table representation of traffic, including metadata and payloads, and constructs a self-supervised training objective. Specifically, it generates a traffic table representation with payloads using unlabeled data, and designs self-supervised tasks such as payload-metadata association prediction and payload semantic inference to enable LLM to understand the semantic relationship between compressed payload embedding vectors and metadata, thereby achieving a deep understanding of the overall traffic representation.

[0063] There are four types of training tasks: Basic information restoration task. The input is a text prompt and a compressed representation of a single encrypted payload. The target output is the transport layer protocol type (TCP or UDP) of the stream to which the payload belongs, the length of the payload, and the first few hexadecimal characters.

[0064] Metadata-Payload Matching Task. The input consists of a text prompt and several data packet metadata tables with no specific association. At the end of the tables, a column is appended with non-empty payloads corresponding to these metadata in shuffled order. The target output is the metadata row number corresponding to each payload row.

[0065] Payload sequence sorting task. The input consists of a text Prompt and a single table of shuffled, non-empty encrypted payload sequences from the same burst. The target output is the position number of each payload in the original burst.

[0066] Packet sorting task. The input is a text prompt and shuffled metadata and encrypted payloads of packets within the same burst. The metadata and payload of each line come from the same packet. The target output is the sequence number of each line in the original burst.

[0067] During the training phase of this alignment process, the weights of the Encoder and the backbone weights of the LLM are frozen, allowing only the weights of the linear projection layer between them to be learned.

[0068] End-to-end classification framework module: Based on the aforementioned two-dimensional table representation of encrypted traffic and the aligned large language model, this module constructs an end-to-end classification framework for encrypted traffic based on the large language model. It uniformly maps structured metadata and unstructured payload information to the semantic space of the large language model, and fine-tunes it to adapt to downstream classification tasks, thereby demonstrating effective classification performance in inference scenarios.

[0069] The inference and fine-tuning process of the end-to-end classification framework is described as follows: For a given encrypted traffic stream defined by a 5-tuple It can be represented as a sequence of data packets. ,in This represents the number of packets contained in the stream. Each packet... Includes metadata information and encrypted payload information ,Right now Therefore, we decompose the stream into metadata sequences. and encrypted payload sequence Process them separately.

[0070] First, regarding the metadata sequence The two-dimensional table representation method designed in section 3.1 is used for processing. Metadata is converted into a key-value table in natural language form through text parsing. Its shape is ,in This specifies the number of metadata fields containing the encrypted payload. The first row is the table header, and the column containing the encrypted payload is temporarily filled with tokens of length 1. A tokenizer is used. Convert the table content into an input identifier sequence The sequence length is : To enable the model to understand the table structure from a two-dimensional perspective, a two-dimensional positional code is assigned to each token. , of which The position code of each token is represented as a row index. and column indexes tuples: in, This indicates the row number of the token in the table (the header row is 0). This represents the starting code of the column containing the token, plus its relative sequence number within the cell. The input token sequence is obtained through the LLM's embedding layer. Mapped as a token embedding sequence : in It is the hidden layer dimension of LLM.

[0071] Secondly, regarding the encrypted payload sequence The compression representation method based on contrastive learning, designed in Section 3.2, is adopted. Features are extracted from the payload data using a pre-trained encrypted payload encoder, and the first position ([CLS]Token) of the output tensor is taken as the semantic representation of the payload. Subsequently, the aligned linear projection layer is used to... Hidden layer dimensions mapped to LLM The load characterization sequence was obtained. : in, and These are the parameters for the linear projection layer. If a data packet does not contain an encrypted payload, the corresponding position is set to empty and does not participate in subsequent feature fusion.

[0072] Subsequently, feature fusion is performed. During the metadata table construction phase, specific column positions (such as the tcp.payload column or the udp.payload column) have been reserved for the encrypted payload. Let... This is a set of indices reserved for the corresponding load positions in the input sequence. We will define the load characterization sequence. Embedded into The corresponding index position is used to replace the original placeholder embedding, thus forming the fused input representation. : This step enables the fusion of deep semantic information of the payload and structured information of metadata at the model input layer.

[0073] In addition, to guide the model in performing classification tasks, a one-dimensional task instruction prompt text is constructed. Using a word segmenter Convert the prompt text into an input identifier sequence. The sequence length is The input identifier sequence is obtained through the embedding layer of the LLM. Mapped as a cue embedding sequence : Embed the prompt in the sequence With the fused input representation The sequences are concatenated to form the basic context embedding sequence. : in, This indicates a splicing operation.

[0074] During the model fine-tuning phase, the model input sequence contains the content of the true labels. Let the true label sequence be... Its embedding representation is The embedding sequence length is Constructing a fine-tuned input embedding sequence : in, This represents the true label embedding. The input embedding sequence is processed through a backbone LLM. of The Transformer decoder transforms layer by layer to obtain the sequence of fine-tuned hidden layer states. : in Indicates the first The hidden state at the nth time step. Then, the hidden state is mapped to the vocabulary space through an unembedding layer to obtain the nth time step. logits vector at each time step : in, For the projection matrix, For bias terms, This is a vocabulary list.

[0075] The model calculates the first... The output and the true label at each time step Vocabulary at the location Same probability: The system is trained using an autoregressive loss function, specifically defined as the negative log-likelihood loss (cross-entropy loss) between the predicted class token and the true class label: Through end-to-end LoRA fine-tuning, the model can jointly optimize its ability to understand and extract metadata semantics and payload semantics, minimizing the loss function. This enables high-precision encrypted traffic classification.

[0076] During the inference phase, for each time step The model input sequence contains a greedy policy. The previously generated content. Let the generated tag sequence be... Its embedding representation is Constructing the inference input embedding sequence : in, Indicates time The previously generated label embeddings are used. The input embedding sequence undergoes a backbone LLM layer-by-layer transformation to obtain the inference hidden layer state sequence. : in Indicates the first The hidden state at time step n. Similarly, through a vocabulary projection layer, the hidden state at time step n is obtained. logits vector at each time step : The first stage of model inference In each time step, the model is based on Vectors, also using the Softmax function to calculate the vocabulary. probability distribution on : A greedy decoding strategy is used, selecting the term with the highest probability as the prediction result: This generation process is repeated until the model generates a preset end-of-strike token (EOS token) or reaches the maximum generation length, resulting in a complete output word sequence. .

[0077] Subsequently, the output word sequence is processed using the inverse process of the tokenizer (Detokenizer). Restored to natural language text string Since the model has mapped specific traffic category labels (such as "Video_Streaming", "Web_Browsing", etc.) to specific text sequences during the fine-tuning phase, it is possible to extract the relevant text sequences through string matching or keyword extraction. The final encrypted traffic classification result label is parsed out, completing the end-to-end classification task.

[0078] While some embodiments of this disclosure have been shown and described, those skilled in the art will understand that modifications may be made to these embodiments without departing from the principles and spirit of this disclosure, which are defined by the claims and their equivalents.

Claims

1. A method for classifying encrypted traffic based on a large language model, characterized in that, It consists of three modules: encrypted traffic data preprocessing and two-dimensional table representation construction module, two-dimensional table representation alignment module, and end-to-end classification framework. By constructing a two-dimensional table representation of encrypted traffic through encrypted traffic data preprocessing and a two-dimensional table representation construction module, structured metadata and unstructured payload information are uniformly mapped to the semantic space of the large language model. The self-supervised alignment method of the two-dimensional table representation alignment module enhances the model's understanding of traffic representation. Finally, the end-to-end classification framework module is used to realize the end-to-end process from raw encrypted traffic to classification results.

2. The encrypted traffic classification method based on a large language model as described in claim 1, characterized in that, The specific implementation method of the encrypted traffic data preprocessing and two-dimensional table representation construction module is as follows: First, the input encrypted traffic data is standardized and preprocessed, and a two-dimensional table representation of traffic metadata and a compressed representation of encrypted payload are constructed to facilitate understanding by the large language model. Finally, the structured metadata and unstructured payload information are uniformly mapped to the semantic space of the large language model through the joint representation fusion module of metadata and payload.

3. The encrypted traffic classification method based on a large language model as described in claim 2, characterized in that, The method for constructing the two-dimensional table representation of the traffic metadata is as follows: a data stream contains Each data packet contains If a metadata field is provided, then the metadata of the stream is represented as a shape. The table contains non-header cells that store metadata values ​​and header cells that store metadata keys. Special tags "<" and ">" are added before and after the content of each cell. For the above table structure, the configuration of the two-dimensional position encoding mechanism includes: Token two-dimensional position encoding: The first dimension of the encoding corresponds to the row index, where the index of the header row is set to 0; the second dimension of the encoding corresponds to the column index, and the value of each token is the sum of the starting position encoding of the column and the token index in the cell. Column start position code: The start position code of the first column is set to 0, and the start position code of each subsequent column is the sum of the start position code of the previous column and the maximum number of tokens in all cells of that column; Sequence sparsity strategy: Only tokens with actual semantic information are retained in the input sequence, and placeholders are not included to fill empty spaces; The generation process of the metadata token sequence and two-dimensional location code includes the following steps: initializing the string table, traversing the metadata fields to fill the table header, traversing the metadata fields one by one to fill the metadata value, tokenizing the cell content, iteratively calculating the starting position code of each column, and traversing the table column by column and row by row to fill in the metadata token sequence and location code sequence.

4. The encrypted traffic classification method based on a large language model as described in claim 2, characterized in that, The specific method for compressing and characterizing the encrypted payload is as follows: A BERT-like bidirectional encoder is used to compress and represent the encrypted payload. The encoder’s ability to extract general semantics is enhanced by contrastive learning pre-training. The specific design includes: contrastive learning pre-training, encoder output representation, dimension alignment projection, and encoder selection. Contrastive learning pre-training: The encoder undergoes contrastive learning based on the SimSiam architecture, using only positive examples. In the contrastive training task based on the SimSiam architecture, given two inputs... and Define Encoder as Predictor is Stop gradient operation as In the comparative training, the two intermediate representations are respectively represented as: as well as The cosine similarity is calculated as follows: The weighting coefficients are defined based on the strength levels of positive sample pairs: The overall loss function is defined as: Encoder output representation: A special token [CLS] is added to the beginning of the input sequence, and the embedding vector corresponding to the first [CLS] of the output tensor is selected as the output representation of the Encoder; Dimension-aligned projection: When integrating the Encoder with LLM, a linear projection layer is introduced to map the Encoder output dimension to the hidden dimension of the LLM embedding vector; Encoder selection: The LongFormer structure was adopted as the specific implementation of the Encoder.

5. The encrypted traffic classification method based on a large language model as described in claim 2, characterized in that, The specific method for the joint representation fusion of metadata and payload is as follows: For stream-based classification tasks, each stream contains metadata and an optional encrypted payload. During the metadata table construction phase, specific column positions are reserved for the encrypted payload, and the compressed representation of the payload is embedded into the corresponding position to form a holistic two-dimensional table representation of the encrypted traffic. If a packet has no payload, that position is left blank, neither participating in the Encoder compression nor being included in the LLM input sequence.

6. The encrypted traffic classification method based on a large language model as described in claim 1, characterized in that, The specific implementation method of the two-dimensional table representation alignment module is as follows: the two-dimensional table representation of encrypted traffic is effectively aligned with the semantic space of the large language model using unlabeled data. This is achieved through two sub-modules: a metadata two-dimensional table representation alignment sub-module for metadata two-dimensional table representation and a payload-containing encrypted traffic two-dimensional table representation alignment sub-module for complete encrypted traffic two-dimensional table representation containing metadata and payload.

7. The encrypted traffic classification method based on a large language model as described in claim 6, characterized in that, The implementation method of the alignment submodule represented by the metadata two-dimensional table is as follows: fine-tune the alignment task through corresponding instructions and construct a self-supervised training target; Specifically, by using unlabeled data to generate metadata table representations and combining them with self-supervised tasks such as mask prediction and row-column relationship reasoning, LLM can effectively understand the two-dimensional table structure of metadata and comprehend the collaborative information of the row and column dimensions. Based on the table structure design of metadata, the training task includes six tasks: (1) Table delimiter task, the input is the text Prompt and the table representation of the stream, and the target output is the value of the first and last cells of the table without cell delimiters; (2) Cell retrieval task, the input is the text Prompt, the table representation of the stream and the value of a random cell, the target output is the row number and column number of all cells with that value; (3) Cell reverse retrieval task, the input is the text Prompt, the table representation of the stream and the row number and column number of a random cell, and the target output is the value of the cell; (4) Column retrieval task, the input is the text Prompt, the table representation of the stream and the column number of a random column, and the target output is the column name of the column; (5) Row retrieval task, the input is the text Prompt, the table representation of the stream and the row number of a random row, and the target output is the value of all cells in the row; (6) Table size detection task; the input is the text Prompt and the table representation of the stream, and the target output is the number of rows and columns of this table; The training phase of the alignment process employs a LoRA fine-tuning strategy, freezing the LLM backbone parameters and allowing only the added LoRA low-rank matrix parameters to be learned.

8. The encrypted traffic classification method based on a large language model as described in claim 6, characterized in that, The specific implementation of the alignment of the encrypted traffic two-dimensional table representation with payload is as follows: A traffic table representation with payload is generated using unlabeled data. A self-supervised task is designed to enable LLM to understand the semantic relationship between the compressed payload embedding vector and metadata, achieving a deep understanding of the overall traffic representation. The training tasks include the following four types: (1) Basic information restoration task, the input is the text Prompt and the compressed representation of a single encrypted payload, the target output is the transport layer protocol type (TCP or UDP) of the stream to which the payload belongs, the length of the payload and the first few hexadecimal characters; (2) Metadata-payload matching task. The input is a text Prompt and several data packet metadata tables with no specific association. A column of non-empty payloads corresponding to these metadata in random order is appended to the end of the table. The target output is the metadata row number corresponding to each payload row. (3) Load sequence sorting task: The input is a text Prompt and a single table consisting of a non-empty encrypted load sequence in the same burst in shuffled order. The target output is the position number of each load in the original burst. (4) Packet sorting task: The input is a text Prompt and the metadata and encrypted payload of the data packets in the same burst in shuffled order. The metadata and payload in each line come from the same data packet. The target output is the sequence number of each line in the original burst. During the alignment process, the training phase freezes the weights of the Encoder and the backbone weights of the LLM, allowing only the weights of the linear projection layer between them to be learned.

9. The encrypted traffic classification method based on a large language model as described in claim 1, characterized in that, The specific implementation of the end-to-end classification framework module is as follows: Based on the encrypted traffic two-dimensional table representation and the aligned large language model, an encrypted traffic end-to-end classification framework based on the large language model is constructed. The structured metadata and unstructured payload information are uniformly mapped to the semantic space of the large language model, and fine-tuned to adapt it to downstream classification tasks.

10. The encrypted traffic classification method based on a large language model as described in claim 9, characterized in that, The specific process of the end-to-end classification framework is as follows: For a given encrypted traffic stream defined by a 5-tuple , represented as a data packet sequence ,in The number of packets contained in the stream, each packet Includes metadata information and encrypted payload information ,Right now Decompose the stream into metadata sequences and encrypted payload sequence Process them separately; First, regarding the metadata sequence A two-dimensional table representation method is used for processing, and the metadata is converted into a key-value table in natural language form through text parsing. Its shape is ,in To specify the number of metadata fields containing the encrypted payload, the first row is the table header. The column containing the encrypted payload is temporarily filled with tokens of length 1, using a tokenizer. Convert the table content into an input identifier sequence The sequence length is : Assign a two-dimensional location code to each token , of which The position code of each token is represented as a row index. and column indexes tuples: in, This indicates the row number of the token in the table. This represents the starting code of the column containing the token, plus its relative sequence number within the cell. The token sequence is input through the LLM embedding layer. Mapped as a token embedding sequence : in It is the hidden layer dimension of LLM; Secondly, regarding the encrypted payload sequence A contrastive learning-based compressed representation method is adopted. The payload data is processed by a pre-trained encrypted payload encoder to extract features, and the first position [CLS] token of the output tensor is taken as the semantic representation of the payload. Subsequently, the aligned linear projection layer is used to... Hidden layer dimensions mapped to LLM The load characterization sequence is obtained. : in, and These are the parameters of the linear projection layer; if a data packet does not contain an encrypted payload, the corresponding position is empty and does not participate in subsequent feature fusion. Subsequently, feature fusion was performed. During the metadata table construction phase, specific column positions were reserved for the encrypted payload. Reserve an index set for the corresponding load positions in the input sequence; characterize the load sequence. Embedded into The corresponding index position is used to replace the original placeholder embedding, thus forming the fused input representation. : To guide the model in performing a classification task, a one-dimensional task instruction prompt text is constructed. Using a word segmenter Convert the prompt text into an input identifier sequence. The sequence length is The input identifier sequence is obtained through the embedding layer of the LLM. Mapped as a cue embedding sequence : Embed the prompt in the sequence With the fused input representation The sequences are concatenated to form the basic context embedding sequence. : in, Indicates a splicing operation; During the model fine-tuning phase, the model input sequence contains the content of the true labels; let the true label sequence be... Its embedding representation is The embedding sequence length is Constructing fine-tuned input embedding sequences : in, This represents the true label embedding, where the input embedding sequence is processed by the backbone LLM. of The Transformer decoder transforms layer by layer to obtain the fine-tuned hidden layer state sequence. : in Indicates the first The hidden state at the nth time step is then mapped to the vocabulary space through a vocabulary projection layer to obtain the nth time step. The logits vector at each time step : in, Let be the projection matrix. For bias terms, For a vocabulary list; Calculate the first using the Softmax function The output and the true label at each time step Vocabulary at the location Same probability: Training is performed using an autoregressive loss function, specifically defined as the negative log-likelihood loss between the predicted class token and the true class label: Through end-to-end LoRA fine-tuning, the model can jointly optimize its ability to understand and extract metadata semantics and payload semantics, minimizing the loss function. This enables high-precision encrypted traffic classification; During the inference phase, for each time step The model input sequence contains a greedy policy. The content generated previously; let the generated tag sequence be... Its embedding representation is Constructing the inference input embedding sequence : in, Indicates time The previously generated label embeddings, the input embedding sequence, are transformed layer by layer by the backbone LLM to obtain the inference hidden layer state sequence. : in Indicates the first The hidden state at the nth time step; through the vocabulary projection layer mapping, the nth time step is obtained. logits vector at each time step : The first stage of model inference In each time step, according to Vectors, also using the Softmax function to calculate the vocabulary. probability distribution on : A greedy decoding strategy is used, selecting the term with the highest probability as the prediction result: The generation process is iterative until the model generates a preset terminator or reaches the maximum generation length, resulting in a complete output word sequence. ; Then, the reverse process of the word segmenter is used to output the word sequence. Restored to natural language text string By using exact string matching or keyword extraction, from The final encrypted traffic classification result label is parsed out, completing the end-to-end classification task.

Images