Encrypted signaling feature representation method based on hybrid data enhancement and comparative learning

Abstract

The invention belongs to the technical field of information security. The invention provides an encrypted signaling feature representation method based on hybrid data enhancement and comparative learning. According to the embodiment of the invention, the universal basic model which can be reused by multiple tasks is generated through self-supervised training, and a flexible, efficient and sustainable evolution solution can be provided for different business scenes, so that the business scenes can quickly meet universal security requirements. The bottleneck of data annotation is fundamentally solved, and the whole life cycle cost of the model is greatly reduced. Characteristic representation has strong immunity to non-essential disturbance of the flow, and the nature of the flow behavior can be accurately captured. The model can effectively resist escape means such as confusion, encryption and the like commonly used by attackers, and has stronger generalization detection capability on unknown attacks with different modes but the same behavior essence. The decoupling of'retraining 'and'light application' is realized, and the deployment threshold and the operation overhead are greatly reduced. And the method has future adaptability to emerging encryption protocols and privacy protection technologies.

Description

Technical Field

[0001] This disclosure relates to the field of information security technology, and in particular to a method for representing encrypted signaling features based on hybrid data augmentation and contrastive learning. Background Technology

[0002] With the acceleration of global digitalization, network communication has become the cornerstone of social operation. To protect the confidentiality and integrity of data transmission, encryption technologies, especially Transport Layer Security (TLS), have been widely deployed across the internet. Statistics show that encrypted traffic currently accounts for over 90% of internet traffic. While this widespread encryption trend effectively defends against attacks such as eavesdropping and tampering, it also presents unprecedented challenges to network security supervision. Traditional network security devices, such as Intrusion Detection Systems (IDS) and Next-Generation Firewalls (NGFWs), rely heavily on Deep Packet Inspection (DPI) technology for their core detection capabilities—identifying malicious activity by examining the content of data packet payloads. However, in the face of encrypted traffic, the data packet payload becomes ciphertext that cannot be directly deciphered, rendering DPI technology almost completely ineffective and creating a huge "security blind spot."

[0003] To address this challenge, academia and industry have developed various encryption traffic analysis techniques that do not rely on decryption. The core idea behind these techniques is to infer the actual application type or behavioral pattern by analyzing the external behavioral characteristics exhibited by encryption traffic during transmission. Currently, the implementation schemes most similar to this application can be mainly divided into two categories: analysis methods based on metadata and handshake information, and behavioral fingerprinting methods based on deep learning.

[0004] With the development of machine learning, especially deep learning, researchers have found that relying solely on metadata from the handshake phase is insufficient for fine-grained behavioral analysis. Therefore, techniques based on encrypted traffic behavior fingerprinting have emerged. The core assumption of these methods is that even when content is encrypted, different applications or malicious behaviors will leave unique, identifiable statistical patterns in the network traffic sequence.

[0005] Although the aforementioned existing technical solutions can solve some problems in specific scenarios, they all have inherent and insurmountable drawbacks, which are particularly prominent when facing an increasingly complex and dynamic network threat environment.

[0006] Disadvantages of existing metadata analysis technologies: 1. Visibility is decreasing and it is becoming ineffective.

[0007] 2. The analysis is shallow and cannot identify in-application behavior.

[0008] The drawbacks of existing deep learning-based behavioral fingerprinting methods: While these techniques offer improvements in accuracy, their complex model architecture and end-to-end training paradigm introduce four problems that are difficult to solve in practice: 1. Training costs are high and reproduction is difficult.

[0009] 2. The model and application are highly coupled, resulting in poor flexibility.

[0010] 3. It has low sensitivity to unknown traffic and limited generalization ability.

[0011] 4. High dependence on high-quality labeled data.

[0012] Therefore, it is necessary to improve one or more of the problems existing in the above-mentioned related technical solutions.

[0013] It should be noted that this section is intended to provide background or context for the technical solutions of this disclosure as set forth in the claims. The description herein does not constitute an admission that it is prior art simply because it is included in this section. Summary of the Invention

[0014] The purpose of this disclosure is to provide a method for representing encrypted signaling features based on hybrid data augmentation and contrastive learning, thereby overcoming, to at least some extent, one or more problems caused by the limitations and defects of related technologies.

[0015] According to embodiments of this disclosure, a method for representing encrypted signaling features based on hybrid data augmentation and contrastive learning is provided, including: Obtain raw traffic data packets and convert them into a unified format of word sequences; A hybrid positive sample pair construction module is used to perform application-layer semantic enhancement and flow feature layer structural enhancement on the word sequence to generate positive sample pairs; wherein, the hybrid positive sample pair construction module includes a transformation enhancement module, a homology sampling module, and a positive sample pair generation module; Based on the InfoNCE loss function, the CoSTE basic encoder is trained using positive sample pairs and the remaining samples in the same batch to obtain the trained CoSTE basic encoder. The unknown encrypted data packet is input into the trained CoSTE basic encoder, converted into a high-quality feature vector, and stored in the feature repository. Feature vectors are read from the feature repository, unknown threats are discovered using unsupervised paths, and known threats are identified using supervised paths.

[0016] Furthermore, the step of acquiring the raw traffic data packets and converting them into a unified format lexical sequence includes: Obtain the raw traffic data packets and decompose them into protocol header features and encrypted payload; The vocabulary of the Transformer base model is reused and expanded based on head feature words and payload words; By concatenating the individual lexical units, a unified input lexical sequence is generated, which includes the [CLS] tag, the head lexical sequence, the [SEP] tag, and the payload lexical sequence.

[0017] Furthermore, the step of using the transformation enhancement module to perform application-layer semantic enhancement and flow feature layer structural enhancement on the lexical sequence to generate positive sample pairs includes: The transformation enhancement module uses known traffic semantic metadata to make semantically equivalent modifications to the original traffic data packets after unification and to simulate network transmission disturbances. The homology sampling module randomly samples several word sequences from the same conversation stream based on the homology principle, which serve as the positive sample set of the target word sequence; Based on the anchor samples in the positive sample set, the positive sample pair generation module uses a transform-transform strategy, a transform-homogeneous strategy, or a homogeneous-homogeneous strategy to generate positive sample pairs.

[0018] Furthermore, the transformation-transformation strategy includes: for anchor samples Perform two transformation enhancements independently to generate two different views. and view .

[0019] Transformation-homogeneous strategies include: for anchor samples Implement a transformation to enhance the generated view At the same time, it randomly samples another data packet from its session stream as a view. .

[0020] Homologous-homogeneous strategies include: from anchor samples Within the session stream, two data packets that are different from the current data packet are randomly sampled to form a pair of views. .

[0021] Furthermore, the step of training the CoSTE basic encoder based on the InfoNCE loss function using positive sample pairs and the remaining samples in the same batch to obtain the trained CoSTE basic encoder includes: Using all samples other than positive sample pairs in the batch as negative samples, and combining them with positive sample pairs, we train a CoSTE basic encoder based on the Transformer architecture, and optimize the InfoNCE loss function to obtain the trained CoSTE basic encoder.

[0022] Furthermore, the expression for the InfoNCE loss function is:

[0023] in, For positive sample pairs via encoder The generated feature vector, The set of feature vectors generated for all samples in this batch. Used to calculate the cosine similarity between two vectors. This refers to temperature hyperparameters.

[0024] Furthermore, the step of inputting the unknown encrypted data packet into the trained CoSTE basic encoder, converting it into a high-quality feature vector, and storing it in the feature repository includes: The unknown encrypted data packet is input into the trained CoSTE basic encoder to extract its corresponding feature vector; Store the feature vectors in the feature repository.

[0025] Furthermore, the steps of reading feature vectors from the feature repository, discovering unknown threats using unsupervised paths, and identifying known threats using supervised paths include: Read feature vectors from the feature repository; Feature vectors are input into an unsupervised path constructed by a clustering algorithm to discover unknown threats; Feature vectors are fed into a supervised path consisting of lightweight classifiers to identify known threats.

[0026] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: In the embodiments of this disclosure, the encrypted signaling feature representation method based on hybrid data augmentation and contrastive learning, on the one hand, produces a general basic model that can be reused for multiple tasks through self-supervised training, providing a flexible, efficient, and sustainably evolving solution for different business scenarios, enabling it to quickly respond to general security needs. This fundamentally solves the data annotation bottleneck and greatly reduces the model's total lifecycle cost. Furthermore, the feature representation generated by this method is highly immune to non-essential traffic disturbances (such as network jitter and payload stuffing), and can more accurately capture the essence of traffic behavior. Therefore, the model can more effectively resist common attacker escape techniques such as obfuscation and encryption, and has stronger generalization detection capabilities against unknown attacks (zero-day attacks) with different patterns but the same underlying behavior. On the other hand, it achieves decoupling between "heavy training" and "lightweight application," significantly reducing deployment thresholds and operational overhead. The application framework completely separates the "heavyweight" feature learning process from the "lightweight" business decision-making process. The CoSTE basic model produced by a single training iteration can be repeatedly and efficiently invoked by various low-resource-consumption traditional algorithms (such as SVM and DBSCAN) in different business scenarios. This significantly reduces the hardware and software costs and technical complexity of deploying advanced traffic analysis capabilities in edge computing devices or high-throughput network environments. Furthermore, by modeling the behavioral patterns of complete encrypted data packets (including their encrypted payloads), it enables deep analysis of specific in-application behaviors. More importantly, this method does not rely on any specific protocol plaintext fields (such as the SNI field in the TLS protocol), therefore its effectiveness is unaffected by the development of emerging privacy-preserving technologies such as ECH (Encrypted Client Greeting) in TLS 1.3, exhibiting greater technological foresight and longer technological viability. Attached Figure Description

[0027] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0028] Figure 1 The diagram illustrates the steps of an encrypted signaling feature representation method based on hybrid data augmentation and contrastive learning in an exemplary embodiment of this disclosure. Figure 2 This diagram illustrates the overall architecture of the CoSTE system in an exemplary embodiment of this disclosure. Figure 3 This diagram illustrates a contrastive learning and hybrid data augmentation process in an exemplary embodiment of this disclosure. Figure 4A flowchart illustrating the downstream application path in an exemplary embodiment of this disclosure is shown. Detailed Implementation

[0029] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0030] Furthermore, the accompanying drawings are merely illustrative diagrams of embodiments of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0031] Disadvantages of existing metadata analysis technologies: 1. Decreasing visibility, approaching ineffectiveness: The effectiveness of this scheme rests entirely on the premise of obtaining plaintext metadata. However, for further protection of user privacy, the industry is vigorously promoting technologies such as Encrypted Client Greeting (ECH) in TLS 1.3. Once ECH is widely deployed, key plaintext fields, including SNI, will be encrypted, causing this type of method to lose its most important source of information, and its identification capability will be severely degraded.

[0032] 2. Shallow analysis depth, unable to identify in-application behavior: Even in the current environment, metadata analysis can only provide connection-level information (e.g., which website the user connected to), but cannot distinguish the specific behaviors that occur within the encrypted tunnel after the connection is established (e.g., whether uploading files or browsing web pages). This shallow analysis cannot meet the needs of fine-grained behavior control and threat detection.

[0033] The drawbacks of existing deep learning-based behavioral fingerprinting methods: While these techniques offer improvements in accuracy, their complex model architecture and end-to-end training paradigm introduce four problems that are difficult to solve in practice: 1. High training costs and difficulty in reproduction: These methods typically employ deep models (especially the Transformer architecture) containing hundreds of millions or even more parameters. Their training requires massive computational resources (such as multiple high-end GPUs running in parallel for weeks) and large-scale datasets. Furthermore, to achieve ideal performance, complex self-supervised pre-training and supervised fine-tuning are often necessary. The entire process has a high technical barrier, making it difficult for ordinary enterprises or institutions to afford the high training costs and reproducibility. Moreover, the high cost makes model updates and iterations extremely difficult, hindering rapid response to emerging network threats and applications, leading to models becoming obsolete easily.

[0034] 2. The model is highly coupled with the application, resulting in poor flexibility: These methods are typically end-to-end, meaning a single, large model is directly responsible for the entire process from raw traffic input to final classification label output. The model's design and training objectives are highly coupled to a specific classification task (e.g., identifying 10 types of malware). This high coupling leads to extremely poor flexibility and scalability. If a new identification category needs to be added, or the model's capabilities need to be applied to a new unsupervised task (e.g., unknown threat detection), the entire model often needs to be redesigned and retrained on a large scale, lacking the ability to train once and reuse it across multiple scenarios.

[0035] 3. Low sensitivity to unknown traffic, limited generalization ability: Because its training paradigm is essentially supervised learning, the model learns to "remember" specific traffic patterns seen in the training set, rather than "understanding" the essence of traffic behavior. Its generalization ability is strictly limited to the known distribution of the training data. When faced with an unknown threat employing a completely new communication protocol or behavioral pattern (zero-day attack), because its characteristics differ from samples of any category in the training set, the model is highly likely to misclassify it as benign, leading to severe false negatives. Similarly, attackers can bypass detection through escape techniques that significantly alter traffic statistics without changing the nature of the attack (such as changing heartbeat intervals or adding random padding).

[0036] 4. High dependence on high-quality labeled data: While some advanced methods have introduced self-supervised pre-training to reduce the need for labeled data, their ability to ultimately solve specific tasks still heavily relies on high-quality labeled datasets during the supervised fine-tuning phase. In the field of cybersecurity, acquiring and maintaining large-scale, diverse, and accurately labeled datasets is an extremely costly process that always lags behind attackers. This dependence on labeled data is the fundamental bottleneck restricting the rapid and economical deployment and maintenance of such technologies in real-world environments.

[0037] This example implementation provides a method for representing encrypted signaling features based on hybrid data augmentation and contrastive learning. (Reference) Figure 1 As shown, this method for representing encrypted signaling features based on hybrid data augmentation and contrastive learning may include: Step S101: Obtain the raw traffic data packets and convert them into a unified format word sequence; Step S102: Use the hybrid positive sample pair construction module to perform application layer semantic enhancement and flow feature layer structural enhancement on the word sequence to generate positive sample pairs; wherein, the hybrid positive sample pair construction module includes a transformation enhancement module, a homology sampling module and a positive sample pair generation module; Step S103: Based on the InfoNCE loss function, train the CoSTE basic encoder using positive sample pairs and the remaining samples in the same batch to obtain the trained CoSTE basic encoder. Step S104: Input the unknown encrypted data packet into the trained CoSTE base encoder, convert it into a high-quality feature vector and store it in the feature repository; Step S105: Read feature vectors from the feature repository, use unsupervised paths to discover unknown threats, and use supervised paths to identify known threats.

[0038] The aforementioned method for representing encrypted signaling features based on hybrid data augmentation and contrastive learning achieves several advantages. Firstly, it generates a general-purpose foundational model that can be reused across multiple tasks through self-supervised training. This provides a flexible, efficient, and sustainably evolving solution for different business scenarios, enabling rapid responses to common security needs. It fundamentally solves the data annotation bottleneck, significantly reducing the model's lifecycle cost. Furthermore, the feature representation generated by this method is highly immune to non-essential traffic disturbances (such as network jitter and payload stuffing), accurately capturing the essence of traffic behavior. Therefore, the model can more effectively resist common attacker escape techniques such as obfuscation and encryption, and exhibits stronger generalization detection capabilities against unknown attacks (zero-day attacks) with different patterns but identical underlying behaviors. Secondly, it decouples "heavy training" from "lightweight application," significantly reducing deployment barriers and operational overhead. The application framework completely separates the "heavyweight" feature learning process from the "lightweight" business decision-making process. The CoSTE foundational model generated from a single training iteration can be repeatedly and efficiently invoked by various low-resource-consumption traditional algorithms (such as SVM and DBSCAN) in different business scenarios. This significantly reduces the hardware and software costs and technical complexity of deploying advanced traffic analysis capabilities in edge computing devices or high-throughput network environments. Furthermore, by modeling the behavioral patterns of complete encrypted data packets (including their encrypted payloads), it enables deep analysis of specific in-application behaviors. More importantly, this method does not rely on any specific protocol plaintext fields (such as the SNI field in the TLS protocol), therefore its effectiveness is unaffected by the development of emerging privacy-preserving technologies such as ECH (Encrypted Client Greeting) in TLS 1.3, exhibiting greater technological foresight and longer technological viability.

[0039] Below, we will refer to Figures 1 to 4 The steps of the above-described method for representing encrypted signaling features based on hybrid data augmentation and contrastive learning in this example embodiment will be described in more detail.

[0040] In one embodiment, this application discloses a CoSTE (Contrastive Signaling Traffic Encoder) encrypted traffic analysis method and application framework. This framework decouples the complex model training process from the lightweight business application process, dividing it into two main stages: a core training path and a downstream application path. The core training path focuses on building a general-purpose CoSTE basic encoder. This process takes unlabeled traffic generated by a self-developed traffic generation device as input. First, it converts the raw traffic into a sequence format that the model can process through a specific lexicalization step. Then, it uses a self-supervised contrastive learning paradigm that integrates transform enhancement and homology sampling for model training. This process can be executed periodically, serving as the basis for subsequent processing. The downstream application path demonstrates the application mode of the trained encoder in actual business, showcasing how the encoder acts as an efficient feature extractor. Its output high-quality feature vectors are then used through clustering algorithms or lightweight classifiers to achieve fast and accurate traffic analysis and threat identification, forming a feature repository. Figure 2 This is the overall architecture diagram of the CoSTE system in this application, which describes the decoupled architecture concept of "separation of model training and business inference" in this application and shows the whole process from model building to actual deployment and application.

[0041] The figure illustrates the two core stages of the CoSTE method in this application and their interrelationship: the "core training path" on the left and the "downstream application path" on the right.

[0042] The "Core Training Path" on the left is a periodically executed offline model building process. This process takes "unlabeled traffic" (which can originate from the self-developed traffic generation device described in this application) as initial input and proceeds through three core steps: Step S101: Input construction and lexicalization, converting the raw traffic data into a sequence format that the model can process.

[0043] Step S102: Core Module - Hybrid Positive Sample Pair Construction, which generates high-quality training samples for contrastive learning through data augmentation and sampling strategies unique to this application.

[0044] Step S103: Encoder training based on contrastive learning. The core model is trained using a self-supervised learning paradigm. The final output of this path is the "CoSTE Basic Encoder (New Version Model)," a deep learning model that has mastered the ability to represent features universally and robustly in encrypted traffic.

[0045] The "Deployment / Application" arrow in the middle represents a connection process. It indicates that the results of the training path on the left—the "CoSTE Basic Encoder"—are deployed to the actual business environment after training, becoming the core processing unit of the application path on the right.

[0046] The "Downstream Application Path" on the right is a real-time online traffic analysis process. This process takes "unknown encrypted data packets" as real-time input and performs the following operations in sequence: Step S104: The data packet is efficiently converted into a high-quality feature vector z by the "CoSTE base encoder (deployed training results)".

[0047] The feature vectors output by the encoder are written into a persistent "feature repository".

[0048] Step S105: When analysis is required, read the feature vector z from the "feature repository". The read feature vector z is then sent to a unified analysis module - "clustering algorithm / lightweight classifier" (e.g., DBSCAN or SVM).

[0049] The analysis module makes decisions based on the position of the feature vector z in the feature space, ultimately achieving the goal of "discovering known threats / unknown threats".

[0050] The core training path and downstream application path will be explained in detail below.

[0051] Phase 1: Core Training Path First, we will explain the basis for the technical approach and encoder initialization selection in this application.

[0052] In adapting general pre-trained language models to the field of encrypted traffic analysis, this application innovatively chooses the self-supervised contrastive learning approach instead of the masked language model (MLM) approach. This choice is based on the following technical considerations: Learning Objectives and Feature Quality: The core objective of contrastive learning is to bring semantically similar samples (positive sample pairs) closer together in the feature space, while semantically dissimilar samples (negative sample pairs) further apart. This directly optimizes the discriminative and invariant properties of the features, making the generated features more suitable for downstream tasks such as classification and clustering. Compared to the optimization and reconstruction capabilities of MLM, it is more targeted.

[0053] Robustness and Generalization Ability: Contrastive learning can use data augmentation strategies to force the model to ignore non-essential disturbances in traffic (such as latency jitter, redundant padding bytes, etc.) and instead focus on its actual behavioral patterns. Therefore, the model can achieve greater robustness and generalization ability against real-world network noise and attacker evasion techniques.

[0054] Data efficiency: Through data augmentation mechanisms, contrastive learning can "expand" a limited number of samples into a large number of training signals (positive and negative sample pairs), providing more effective supervision information for the model with the same amount of data.

[0055] Regarding the encoder initialization and selection criteria, this application selects a pre-trained language model based on the Transformer architecture as the CoSTE encoder. The base model is used, leveraging its sequence modeling and transfer learning capabilities. In the specific implementation, the bert-base-uncased model is chosen as the foundation. The selection criteria are: this model has 12 Transformer layers, 768 hidden unit dimensions, and approximately 110 million parameters, achieving a good balance between performance and computational overhead. Its scale is sufficient to capture complex patterns in encrypted traffic, while the cost of training and inference on standard GPU hardware is manageable. The hidden unit dimension of this model determines the CoSTE encoder in this application. Output feature vector It is a 768-dimensional vector. This application loads the pre-trained weights of the model into the CoSTE encoder as the initial state of the parameters.

[0056] The core training path is described in detail below.

[0057] Step S101: Input Construction and Tokenization This step aims to convert heterogeneous raw network packets into a unified sequence of terms that the Transformer model can process.

[0058] Feature separation and preprocessing: Decompose a single encrypted data packet into two parts: structured protocol header features and unstructured encrypted payload.

[0059] Lexical mapping: This application adopts a strategy of reusing and expanding the original vocabulary of the base model.

[0060] Retain and reuse the common vocabulary: retain the original natural language vocabulary of the base model to handle readable ASCII strings that may appear in the traffic.

[0061] Expanding Domain-Specific Terminology: Defining new special terms for specific elements in the encrypted traffic domain and adding these new terms to the existing vocabulary. The new terms include: Header feature tokens: Map key protocol header fields (e.g., the SYN flag in the TCP protocol) to a new, unique token (e.g., [T_SYN]).

[0062] Payload tokens: A unique new token is assigned to each possible bigram (65,536 combinations) in the encrypted payload.

[0063] Input layer adaptation: After loading the pre-trained model, the dimension of its word embedding layer is expanded accordingly to accommodate all newly added words. The embedding vectors corresponding to the newly added words are randomly initialized.

[0064] Sequence concatenation: The various word components are concatenated to form a unified input word sequence X, with the standard format: [CLS] [Head word sequence][SEP] [Load word sequence][SEP]. [CLS] and [SEP] are standard words with special functions in the Transformer model.

[0065] Step S102: Core Module - Hybrid Positive Sample Pair Construction Strategy The effectiveness of this multi-layered data augmentation method relies heavily on precise control over traffic semantics, which is achieved through the aforementioned self-developed traffic generation device. This device can simulate various routine network activities (such as web browsing, file transfer, and database access) to generate large-scale, diverse, and unlabeled traffic with known "behavioral labels" and "session affiliations." While generating traffic, it preserves its complete application-layer semantic metadata, ensuring that all augmentation operations maintain the core behavioral semantics unchanged.

[0066] The Transformation Augmentation Module simulates the morphological changes of network traffic in a real-world environment by applying transformations to data packets without altering their core semantics. Specifically, it includes: Application-layer semantic augmentation (L7 Semantic Augmentation): Utilizes known traffic semantic metadata to make semantically equivalent modifications to application-layer data. For example, for an HTTP GET request, randomly adding or removing a query parameter that does not affect core functionality.

[0067] Flow feature layer structural augmentation (L3 / L4): simulates network transmission disturbances, such as adding jitter to packet arrival times.

[0068] The Same-Origin Sampling Module selects positive samples based on the "same-origin" principle. In this application, "same-origin" is defined as "belonging to the same network session flow," that is, packets that have the same five-tuple (source IP, destination IP, source port, destination port, protocol) as the anchor packet. Since the traffic is generated by controllable, self-developed equipment, the session flow information to which each packet belongs is deterministic and known, providing an accurate and reliable data foundation for the Same-Origin Sampling Module.

[0069] For a single anchor sample in a batch, the system randomly selects one of the following strategies to generate positive sample pairs:

[0070] Strategy A (Transformation-Transformation): For anchor samples Perform two transformation enhancements independently to generate two different views. and .

[0071] Strategy B (Transformation-Homologous): For anchor samples Implement a transformation to enhance the generated view At the same time, it randomly samples another data packet from its session stream as a view. This strategy enables the model to learn invariance to both content transformations and in-stream context.

[0072] Strategy C (Same Origin-Same Origin): From anchor samples Within the session stream, two data packets that are different from the current data packet are randomly sampled to form a pair of views. .

[0073] Figure 3 This diagram illustrates the contrastive learning and hybrid data augmentation process of this application, showing the processing flow of a single anchor sample during contrastive learning training.

[0074] The samples first enter the "Hybrid Positive Sample Pair Construction Module," which generates a positive sample pair using the aforementioned strategy (strategy B in the figure as an example). This pair consists of a transformed view and a view from its same source session stream. This pair of views is considered a positive sample pair, while all other samples in the same batch are considered negative samples. All samples are then fed into the CoSTE encoder to extract feature vectors. Finally, optimization is performed using the InfoNCE loss function, which in the feature space drives the feature vectors of the positive sample pairs closer together, while simultaneously driving them further apart from the feature vectors of all negative samples.

[0075] Step S103: Construct the CoSTE basic encoder through contrastive learning This step is the core innovation of this application. Through an innovative self-supervised learning paradigm, a CoSTE basic encoder with general feature extraction capabilities is trained. Its training data comes from the positive sample pairs in step two. and all other samples in the same batch (negative samples) Ultimately, this enables self-supervised comparative learning. This application employs a contrastive learning framework for the encoder. Training is performed. For a batch of size B, where each anchor sample... and the positive samples generated by it All other samples within the batch are considered negative samples. The model parameters are updated by optimizing the InfoNCE loss function: Formula: InfoNCE Loss Function

[0076] in, and It is composed of positive samples For encoder The generated feature vector, It is the set of feature vectors generated from all samples within this batch. Used to calculate the cosine similarity between two vectors. It is a temperature hyperparameter, and the final generated feature vector is 768-dimensional.

[0077] Algorithm: CoSTE Basic Encoder Training Pseudocode # Input: #f_theta: Transformer encoder (initialized based on pre-trained BERT) #D_unlabeled: Massive unlabeled traffic dataset, provided in batch form #CreatePositivePair(): Hybrid positive pair generation function of this application #Tokenize(): Function to wrap the data into a sequence of terms #NUM_EPOCHS: Total number of training rounds #calculate_batch_contrastive_loss(): Function to calculate the InfoNCE loss in batches deftrain_coste_base_model(f_theta, D_unlabeled, CreatePositivePair,Tokenize, NUM_EPOCHS, calculate_batch_contrastive_loss): """This algorithm describes the core process of training the CoSTE base encoder. Through self-supervised contrastive learning, the model can learn robust feature representations of encrypted traffic.""" # Initialize the AdamW optimizer to update model parameters f_thetaoptimizer = AdamW(f_theta.parameters()) # Perform multiple training iterations on the entire dataset forepoch in range(NUM_EPOCHS): # Iterate through each batch in the dataset forbatch_P in D_unlabeled: # batch_P is a list containing B data packets # Step A: Preprocessing - Obtain the homologous context for each data packet batch_flow_contexts = find_flow_contexts(batch_P) # Initialize a list to store the feature vectors and positive pair relationships for all views in this batch all_views_vectors = [] positive_pairs_indices = [] # Step B: Positive pair generation and feature extraction for i, P_i in enumerate(batch_P): # 1. Generate a positive pair for anchor point P_i (v_i, v_j) v_i, v_j = CreatePositivePair(P_i, batch_flow_contexts[i])# 2. Lexicalize the pair of views and perform forward propagation through encoder f_theta z_i = f_theta(Tokenize(v_i)) z_j = f_theta(Tokenize(v_j))# Store the generated feature vectors into list all_views_vectors.append(z_i) all_views_vectors.append(z_j)# Record the index relationship of positive sample pairs for subsequent loss calculation positive_pairs_indices.append((len(all_views_vectors)- 2, len(all_views_vectors)- 1))# Step C: Calculate the contrastive loss total_loss = calculate_batch_contrastive_loss(positive_pairs_indices, all_views_vectors)# Step D: Backpropagation and parameter optimization optimizer.zero_grad() total_loss.backward() optimizer.step() return f_theta # Return the trained CoSTE base encoder. Phase Two: Downstream Application Path This path is the standard application model of this application in actual business deployment, and its core technical characteristics are high efficiency, low cost, and strong interpretability. Figure 4 This is a flowchart of the downstream application path of this application, which shows the parsing process of an unknown data packet in the business system.

[0078] Steps S104 and S105: The data packet is first mapped into a fixed-dimensional (768-dimensional) high-quality feature vector z by the pre-trained "CoSTE basic encoder". The generated feature vector z is then written to a persistent "feature repository", which serves as the central hub for feature vectors, decoupling the feature extraction process from subsequent analysis and decision-making. Analysis tasks can retrieve feature vectors from this repository for analysis as needed. The feature vectors retrieved from the feature repository are distributed to two parallel processing pipelines: one is an unsupervised path, feeding the feature vectors into models such as the "DBSCAN clustering algorithm" to discover unknown or novel anomalous behaviors; the other is a supervised path, feeding the feature vectors into a pre-trained shallow machine learning classifier such as an "SVM classifier" to quickly identify known threat types. Unsupervised Paths: Discovering Unknown Threats Using Clustering Algorithms Implementation plan: Each traffic data packet captured over a period of time is processed one by one using a pre-trained CoSTE basic encoder to extract its corresponding feature vector, forming a feature vector set. This set is then input into a clustering algorithm such as DBSCAN (Density-Based Spatial Clustering of Applications with Noise).

[0079] Technical Results: The DBSCAN equal-density clustering algorithm can automatically divide densely clustered samples (representing mainstream, normal network behavior) in the feature space into different "clusters," while identifying sparsely distributed "outliers." These outliers deviate significantly from the mainstream in their behavioral patterns, and therefore are highly likely to correspond to novel, rare anomalous activities or zero-day attacks.

[0080] Supervision path: Identifying known threats using shallow classifiers Implementation plan: Use a small-scale labeled sample set containing known threat types. First, extract feature vectors from all samples in the sample set using the CoSTE basic encoder. Then, use these "feature vector-label" pairs to train a lightweight shallow classifier with low computational cost and fast inference speed, such as a Support Vector Machine (SVM).

[0081] Technical Results: Due to the highly linearly separable feature vectors provided by the CoSTE base encoder, shallow machine learning classifiers such as SVM can achieve high classification accuracy with extremely low training costs and very few training samples. During the inference phase, its computational latency is significantly lower than that of directly using end-to-end deep learning models for classification, making it more suitable for high-throughput real-time detection scenarios.

[0082] In a specific embodiment, this application focuses on its end-to-end technological innovation, from input representation, training paradigm, data augmentation to application framework. This application employs different technical means at each stage, thereby systematically addressing the fundamental shortcomings of existing methods.

[0083] 1. Lexicalization Strategies: Hybrid Lexicalization with Reuse and Expansion vs. Simple Mapping or Complete Replacement The implementation scheme of this application: This application adopts a hybrid lexicalization strategy of "reuse and expansion". This scheme retains the original vocabulary of the base model, which is trained on massive natural language corpora, to inherit its general sequence understanding capabilities; at the same time, by adding tens of thousands of new lexical units (including protocol field-specific lexical units and byte tuple lexical units) specifically defined for the encrypted traffic domain to the vocabulary, the model is expanded, thereby accurately injecting the structured semantics of network protocols into the model.

[0084] Existing implementation methods: When processing encrypted traffic, existing technologies typically employ strategies of simple mapping or complete replacement. For example, some schemes map the byte stream of traffic indiscriminately into a sequence of binary tuples, failing to explicitly express the semantics of structured features such as protocol headers; other schemes completely discard the vocabulary of pre-trained models, building a completely new vocabulary from scratch for the traffic data, but this loses the transfer learning advantages brought by the pre-trained models.

[0085] The fundamental difference between the two lies in their technical approaches: the inheritance of pre-trained knowledge and the injection of domain-specific knowledge. Existing technologies either fail to fully inherit pre-trained knowledge or fail to effectively inject domain semantics. This application, through a strategy of reuse and extension, is the first to implement a collaborative mechanism that enables the model to simultaneously understand the underlying byte patterns of the traffic payload and the top-level behavioral semantics of the protocol header within a unified framework. The richness and effectiveness of its input representation far surpasses existing technologies.

[0086] 2. Learning Paradigm: Contrastive learning that optimizes feature discriminativeness vs. Masked model that optimizes context reconstruction The implementation scheme of this application: This application selects self-supervised contrastive learning as the core training paradigm for domain adaptation. The learning objective of this paradigm is "like attracts like, unlike repulses," and the InfoNCE loss function is used to directly optimize the relative distance of samples in the feature space to achieve a feature representation that better meets the requirements.

[0087] Existing implementation methods: Current preprocessing language models primarily employ Masked Language Models (MLM) as their training paradigm in the self-supervised stage. Their learning objective is to predict masked lexical units based on context, essentially learning the statistical patterns of sequence context and optimizing the model's reconstruction capabilities.

[0088] Differences and Evaluation: The most fundamental difference between the two lies in their core learning objectives. The contrastive learning paradigm of this application directly optimizes the most crucial feature attribute for downstream classification and clustering tasks—separability. In contrast, the discriminative property of features in existing MLM paradigms is merely an indirect byproduct of the model's learning and reconstruction tasks, not its direct optimization objective. Therefore, the technical approach of this application is more direct and efficient in generating high-quality discriminative features.

[0089] 3. Data Augmentation Strategies: Systematic Hybrid Augmentation vs. Missing or Single Augmentation Methods The implementation scheme of this application: This application proposes and applies a systematic hybrid positive sample pair construction strategy designed specifically for encrypted traffic. This strategy is the core technical barrier of this application. It innovatively integrates two-dimensional enhancement methods: (1) transformation enhancement, which applies semantically preserved perturbations to data packets at the application layer and the flow feature layer; (2) homology sampling, which uses different data packets within the same session flow to construct positive sample pairs, taking advantage of the inherent contextual association of traffic.

[0090] Existing implementation methods for encrypted traffic self-supervised analysis generally lack systematic, domain-specific data augmentation strategies. They either do not use data augmentation or only employ single, simple augmentation methods such as random masking, failing to provide high-quality, diverse, and semantically consistent positive sample pairs for contrastive learning.

[0091] Differences and Evaluation: The key difference between this application and existing technologies lies in the fact that this application is the first to systematically introduce data augmentation methods into self-supervised learning of encrypted traffic. Through a hybrid strategy of "transformation + homology," this application defines what constitutes "semantically consistent but morphologically different" samples in the encrypted traffic domain, thereby solving the core prerequisite problem for the application of contrastive learning in this field. This is something that existing technologies completely lack and is the fundamental reason why this application can learn robust features.

[0092] 4. Application Framework: Decoupled "Base Model + Lightweight Downstream" vs. Highly Coupled End-to-End Model The implementation scheme of this application: This application adopts an application framework that decouples training and application. In the first stage, a general basic encoder is produced through self-supervised training; in the second stage, in practical applications, this encoder is used only as an efficient feature extractor, and its output feature vector is fed into a shallow machine learning algorithm with extremely low computational overhead (such as SVM, DBSCAN) for fast decision-making.

[0093] Existing implementation methods: Most existing deep learning-based methods adopt an integrated, highly coupled end-to-end approach. That is, a large and complex deep learning model is directly responsible for the entire process from the original input to the final classification result, which means that a resource-intensive deep model must also be deployed and run at the real-time inference end.

[0094] Differences and Evaluation: The fundamental difference in technical architecture lies in whether the "feature extraction" and "task decision" stages are decoupled. Existing integrated solutions result in high costs and low flexibility at the inference end. This application achieves an ideal deployment model that separates the computationally intensive model training process from the lightweight business inference process through architectural decoupling, offering significant advantages over existing coupled architecture technologies in terms of inference efficiency, deployment cost, and application flexibility.

[0095] Furthermore, the "traffic generation device" mentioned in the embodiments of this application is a system capable of generating traffic data with specific application-layer semantics and network behavior patterns. In this application, this device is primarily used to generate diverse, semantically consistent data in a controlled environment to support the multi-layered data augmentation described in the core training path. This device is an advantageous condition for the efficient implementation of the method of this application; however, the method itself is not limited to using a specific traffic generation device and can also be applied to traffic data captured from real network environments and semantically labeled or associated in other ways.

[0096] Regarding the pre-trained model: This application mentions using bert-base-uncased as the initial backbone network for the encoder in the embodiments. This choice is based on its proven superior performance on general sequence modeling tasks. However, the core innovations of this application (hybrid data augmentation, contrastive learning framework, and decoupled application mode) are not limited to the bert-base-uncased architecture, but can also be applied to other Transformer-based pre-trained language models (such as RoBERTa, ALBERT, etc.), which are technical replacements that can be foreseen and implemented by those skilled in the art.

[0097] Regarding lightweight downstream algorithms: In the embodiments of this application, SVM and DBSCAN are used as representatives of lightweight downstream algorithms. This choice is based on their classic status and efficiency in their respective fields. However, the application framework of this application is not limited to these two algorithms. In the supervised path, other shallow classification algorithms such as Logistic Regression, Decision Tree, and Gradient Boosting Machines can be substituted. In the unsupervised path, other clustering or anomaly detection algorithms such as K-Means, Isolation Forest, and Gaussian Mixture Model can be substituted. These are all equivalent substitutions within the scope of protection of this application.

[0098] In one specific embodiment, the technical means adopted in this application can produce the following beneficial effects: 1. It fundamentally solves the bottleneck of data annotation and greatly reduces the cost of the entire model lifecycle.

[0099] Advantages: The core training process of this application is entirely based on self-supervised learning, using unlabeled traffic data from a self-developed traffic generation device. This frees the initial model building and periodic updates from the dependence on expensive, scarce, and slow-updating manual annotation resources, enabling a lower cost and faster response to and adaptation to the continuous evolution of network threats.

[0100] Technical means: This advantage is achieved by employing a hybrid positive sample pair construction strategy specifically designed for network traffic and a self-supervised contrastive learning framework designed in this application. Unlike existing technologies that still rely on a large amount of labeled data for supervised training or fine-tuning to adapt to downstream tasks, this application directly produces a high-quality general feature encoder through a superior self-supervised objective, fundamentally reducing the need for labeled data.

[0101] 2. Significantly improved the model's robustness and generalization ability to unknown threats.

[0102] Advantages: The feature representations generated in this application are highly immune to non-essential traffic disturbances (such as network jitter, payload stuffing, etc.), and can more accurately capture the essence of traffic behavior. Therefore, the model can more effectively resist escape techniques commonly used by attackers, such as obfuscation and encryption, and has a stronger generalization detection capability for unknown attacks (zero-day attacks) that have different patterns but the same essential behavior.

[0103] Technical approach: During the contrastive learning training phase, the hybrid positive sample pair construction strategy of this application introduces diverse perturbations into both the application layer and the flow feature layer to construct positive samples. This process essentially forces the model to learn a feature representation that is invariant to these perturbations, thereby capturing the behavioral essence of traffic rather than its appearance.

[0104] 3. It decouples "heavy training" from "light application", significantly reducing deployment threshold and operating overhead.

[0105] Advantages: The application framework of this application completely separates the "heavyweight" feature learning process from the "lightweight" business decision-making process. The CoSTE basic model generated from a single training iteration can be repeatedly and efficiently invoked by various low-resource-consumption traditional algorithms (such as SVM, DBSCAN) in different business scenarios. This significantly reduces the hardware and software costs and technical complexity of deploying advanced traffic analysis capabilities on edge computing devices or in high-throughput network environments.

[0106] Technical approach: By directly optimizing the topological structure of the feature space through contrastive learning, the resulting feature vectors naturally possess excellent linear separability (i.e., "cohesiveness among similar features and discreteness among dissimilar features"). This high-quality feature representation provides a solid foundation for the shallow learning algorithm in the downstream application framework designed in this application, enabling it to solve complex classification and clustering problems with performance far exceeding that of traditional manual features.

[0107] 4. It possesses future adaptability to emerging encryption protocols and privacy protection technologies.

[0108] Advantages: This application enables in-depth analysis of specific behaviors within an application by modeling the behavioral patterns of complete encrypted data packets (including their encrypted payloads). More importantly, this method does not rely on any specific protocol plaintext fields (such as the SNI field in the TLS protocol), therefore its effectiveness is unaffected by the development of emerging privacy protection technologies such as ECH (Encrypted Client Greeting) in TLS 1.3, exhibiting stronger technological foresight and longer technological viability.

[0109] Technical means: This advantage comes from the lexicalization method in the core training path. This method incorporates the complete protocol header and payload information into the analysis scope of the model, realizing end-to-end, deep behavioral modeling of traffic, rather than relying on a few plaintext fields that are easily encrypted or removed.

[0110] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0111] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A method for encrypted signaling feature representation based on hybrid data augmentation and contrastive learning, characterized in that, The method comprises the following steps: Obtaining original traffic data packets and converting them into token sequences in a unified format; Applying a mixed positive sample pair construction module to the token sequences for application layer semantic enhancement and flow feature layer structure enhancement to generate positive sample pairs; the mixed positive sample pair construction module comprises a transformation enhancement module, a homology sampling module and a positive sample pair generation module; Training a CoSTE base encoder based on an InfoNCE loss function using the positive sample pairs and the remaining samples in the same batch to obtain a trained CoSTE base encoder; Inputting unknown encrypted data packets into the trained CoSTE base encoder to convert them into high-quality feature vectors and store them in a feature storage library; Reading the feature vectors from the feature storage library, discovering unknown threats using an unsupervised path, and identifying known threats using a supervised path.

2. The method of claim 1, wherein the method is based on a hybrid data augmentation and contrastive learning based encrypted signaling feature representation. In the step of obtaining original traffic data packets and converting them into token sequences in a unified format, the following steps are included: Obtaining original traffic data packets and decomposing them into protocol header features and encrypted payloads; Multiplexing the vocabulary of the Transformer base model and expanding the vocabulary according to the header token and payload token; Splicing the token of each part to generate a unified input token sequence containing a [CLS] token, a header token sequence, a [SEP] token, and a payload token sequence.

3. The method of claim 2, wherein the method is based on a hybrid data augmentation and contrastive learning. In the step of using the transformation enhancement module to perform application layer semantic enhancement and flow feature layer structure enhancement on the token sequences to generate positive sample pairs, the following steps are included: The transformation enhancement module uses known traffic semantic metadata to make semantic equivalent changes to the original traffic data packets in a unified format and simulates network transmission disturbances; The homology sampling module randomly samples several token sequences from the same session stream as the positive sample set of the target token sequence according to the homology principle; Based on the anchor samples in the positive sample set, the positive sample pair generation module uses a transformation-transformation strategy, a transformation-homology strategy, or a homology-homology strategy to generate positive sample pairs for them.

4. The method of claim 3, wherein the method is based on a hybrid data augmentation and contrastive learning. Transform- transform strategy includes: to anchor samples Independent implementation of two transform enhancement, generating two different views And view . Transform-homogeneous strategy includes: transforming the anchor sample Implementing a transform-enhanced generation of views while randomly sampling another packet from the conversation flow it belongs to as a view . The homology-homology strategy includes: randomly sampling two data packets different from the anchor sample from the session flow in which the anchor sample is located, to form a pair of views .

5. The method of claim 4, wherein the method is based on a hybrid data augmentation and contrastive learning. In the step of training a CoSTE base encoder based on an InfoNCE loss function using positive sample pairs and the remaining samples in the same batch to obtain a trained CoSTE base encoder, the following steps are included: Using the other samples in the batch except the positive sample pairs as negative samples, training the CoSTE base encoder based on the Transformer architecture in combination with the positive sample pairs, optimizing the InfoNCE loss function, and obtaining the trained CoSTE base encoder.

6. The method of claim 5, wherein the method is based on a hybrid data augmentation and contrastive learning. The expression of the InfoNCE loss function is: wherein, is a positive sample pair encoded by the encoder generated feature vector, is a set of feature vectors generated for all samples in the batch, is used to calculate the cosine similarity between two vectors, is a temperature hyperparameter.

7. The method of claim 6, wherein the method is based on a hybrid data augmentation and contrastive learning based encrypted signaling feature representation method. In the step of inputting unknown encrypted data packets into the trained CoSTE base encoder to convert them into high-quality feature vectors and storing them in a feature storage library, the following steps are included: Inputting unknown encrypted data packets into the trained CoSTE base encoder to extract their corresponding feature vectors; Storing the feature vectors in the feature storage library.

8. The method of claim 7, wherein the method is based on a hybrid data augmentation and contrastive learning based encrypted signaling feature representation method. In the step of reading the feature vectors from the feature storage library, discovering unknown threats using an unsupervised path, and identifying known threats using a supervised path, the following steps are included: Reading the feature vectors from the feature storage library; Inputting the feature vectors into an unsupervised path composed of a clustering algorithm to discover unknown threats; The feature vectors are input into a supervised path consisting of lightweight classifiers to identify known threats. The feature vectors are input into a supervised path consisting of lightweight classifiers to identify known threats.

Images