Internet service provider-oriented encrypted multimedia traffic classification method and device

Abstract

The invention relates to the technical field of Internet information, in particular to an Internet service provider-oriented encrypted multimedia traffic classification method and device.The method comprises the steps of capturing statistical characteristics, time sequence characteristics and original byte domains of target encrypted traffic to construct a multi-domain characteristic data set; simulating transmission control protocol behaviors in real network transmission according to the multi-domain feature data set to generate an enhanced data set containing transmission control protocol semantics; inputting the enhanced data set containing the transmission control protocol semantics into a pre-constructed hybrid fusion model so as to respectively extract a feature vector of each domain; and integrating the weight of the feature vector of each domain by using maximum likelihood estimation to determine the category of the target encrypted traffic. Therefore, the problems that an existing Internet traffic classification method mostly depends on single-dimensional features for classification, but the use of the single-dimensional features causes that an existing classification model cannot comprehensively capture traffic features, and the improvement of classification precision is limited are solved.

Description

Technical Field

[0001] The present invention relates to the field of Internet information technology, and in particular to an encrypted multimedia traffic classification method and device for Internet service providers. Background Art

[0002] In the modern Internet ecosystem, Internet Service Providers (ISPs) need to efficiently and accurately detect and classify traffic passing through the Internet to ensure the provider's service load balancing, priority assurance, and other goals. However, achieving these goals faces numerous challenges in the complex real-world network environment where bandwidth is rapidly growing and network behavior is constantly changing.

[0003] With the deepening of digital transformation, bandwidth requirements for data centers and edge networks are experiencing unprecedented explosive growth. To achieve business flexibility, accessibility, and cost optimization, enterprises and organizations continue to migrate core data and applications to the cloud, driving exponential growth in cloud computing infrastructure traffic. Furthermore, the widespread adoption of 5G, the Internet of Things (IoT), and high-definition video codecs is further exacerbating network bandwidth constraints. Video streaming is projected to account for over four-fifths of global consumer internet traffic in 2022 alone. The rise of highly immersive applications like AR / VR is increasing bandwidth requirements per unit of traffic by tens of times compared to traditional text-based services. This trend poses a dual challenge to multimedia traffic classification systems: High-speed links require classification algorithms with extremely low processing latency to avoid becoming a bottleneck for traffic scheduling. Furthermore, dynamically changing bandwidth usage patterns (such as the bursty traffic of 4K live streaming and the periodic refresh of CDN (Content Delivery Network) caches) make static classification models based on historical traffic characteristics difficult to adapt to real-time demands. When network bandwidth utilization approaches its peak, even a slight delay in the classification system may lead to deviations in QoS policy execution, which in turn may cause service degradation for critical businesses. This is particularly serious for edge computing scenarios that rely on real-time decision-making.

[0004] As a foundational protocol for multimedia transmission, the Transmission Control Protocol (TCP) supports real-time services such as online audio and video with its reliable transmission characteristics. (For example, YouTube achieves low-latency streaming through the Real-Time Messaging Protocol (RTMP) over TCP.) However, the widespread adoption of Transport Layer Security (TLS) encryption technology (as of 2023, over 90% of internet traffic is transmitted using TLS encryption) is fundamentally undermining the foundations of traditional deep packet inspection (DPI) technology. Transport Layer Security (TLS) provides transport-level security directly on top of TCP, protecting communications between clients and servers by encrypting the communication payload. TLS is widely used in various traffic communications. TLS communication consists of two main phases: the TLS handshake and application data transmission. TLS traffic primarily consists of partially unencrypted handshake messages and encrypted application data messages.

[0005] However, traffic encryption is a double-edged sword. While it enhances data privacy, traditional classification methods rely on parsing packet payloads and identifying specific protocol features (such as Hypertext Transfer Protocol (HTTP) header fields and Real Time Streaming Protocol (RTSP) signaling formats) through rule matching. However, encrypted traffic payloads are deeply obfuscated, rendering payload-based detection ineffective. Even when machine learning models are constructed using traffic statistics (such as packet size distribution and connection duration), the homogeneity of encrypted traffic can lead to reduced feature differentiation. For example, encrypted Netflix streaming and encrypted web access have highly similar transport layer features, making effective classification difficult using only the five-tuple (source / destination IP, port, protocol, and timestamp). Furthermore, some multimedia applications utilize dynamic encrypted tunnels (such as WireGuard) or end-to-end encryption (such as the Signal protocol), further blurring the boundaries of traffic attributes. This forces classification techniques to shift from relying on content analysis to deep feature mining based on traffic behavior, which places higher demands on feature engineering and model generalization capabilities.

[0006] At the same time, the imbalance in the distribution of class samples in network traffic is becoming a significant factor hindering the performance of classification models. This imbalance manifests itself in two forms: "relatively too few" and "absolutely too few." In the relatively imbalanced scenario, the dominant class (such as Hangouts traffic, which accounts for over 50% in the ISCX-2016 VPN dataset) influences model optimization through gradient dominance, weakening the features of the minority class (such as the AIM protocol, which accounts for less than 1%). Even with a sample-weighted loss function, the high variance of the dominant class's data can still bias gradient updates toward its distribution space. For example, in traffic classification models, simply concatenating class labels and traffic features can lead to the model generating patterns typical of the dominant class, significantly reducing the classification accuracy of the minority class. Absolute imbalance is even more severe. When the sample size for certain applications (such as Vimeo and Spotify in the ISCX-2016 dataset, where the number of streams is less than 100) falls below the minimum data threshold required for model convergence, severe overfitting or underfitting can occur. Taking the NetShare training results as an example, as the sample size decreases, the KL divergence between the generated data and the real data increases significantly (as shown in the table below), indicating that the model struggles to capture the subtle distribution characteristics of minority classes. This data imbalance is particularly prominent in real-world network environments. Due to the sampling period and bursty traffic, the initial labeled sample size for emerging multimedia applications (such as real-time interactive live streaming platforms) is often insufficient, leading to "cold start" recognition errors in the classification model during early deployment. The high cost of manually annotated data further exacerbates the difficulty of obtaining samples.

[0007]

[0008] While current machine learning-based internet traffic classification schemes have demonstrated certain advantages in the field of encrypted traffic identification, they still face significant technical bottlenecks. Existing mainstream methods have significant shortcomings in utilizing traffic features. They often rely on single-dimensional features for classification, such as statistical features, time series features (such as packet length sequences), or raw byte features, failing to fully exploit the multi-dimensional information contained in network traffic logs and behavioral patterns. However, network traffic itself possesses rich multi-dimensional features that can provide an important basis for multimedia traffic identification. The use of single-dimensional features prevents existing classification models from fully capturing traffic characteristics, limiting improvements in classification accuracy.

[0009] At the model-building level, existing solutions lack the ability to perceive and model the semantics of the TCP protocol. Due to the effects of TCP protocol mechanisms (such as retransmission and congestion control), TLS flows will produce changes in packet sequences (such as subsequence offsets, repetitions, and variable-length subsequences) in different network environments. However, existing models have difficulty identifying such changes and are unable to understand the semantic information behind TCP traffic. This lack of understanding of TCP protocol semantics makes classification models less robust in complex and changing network environments, making it difficult to accurately complete multimedia traffic classification tasks, affecting the actual effectiveness of network service quality assurance and resource optimization. Summary of the Invention

[0010] The present invention provides an encrypted multimedia traffic classification method and device for Internet service providers to solve the problem that existing Internet traffic classification methods mostly rely on single-dimensional features for classification. However, the use of single-dimensional features will cause the existing classification model to be unable to fully capture traffic characteristics, thereby limiting the improvement of classification accuracy.

[0011] A first aspect of the present invention provides an encrypted multimedia traffic classification method for Internet service providers, comprising the following steps: capturing statistical features, time series features, and raw byte domains of target encrypted traffic to construct a multi-domain feature dataset; simulating transmission control protocol behavior in real network transmission based on the multi-domain feature dataset to generate an enhanced dataset containing transmission control protocol semantics; inputting the enhanced dataset containing transmission control protocol semantics into a pre-built hybrid fusion model to extract feature vectors for each domain respectively; and integrating the weights of the feature vectors for each domain using maximum likelihood estimation to determine the category to which the target encrypted traffic belongs.

[0012] Optionally, capturing statistical features, time series features, and raw byte domains of target encrypted traffic to construct a multi-domain feature dataset includes:

[0013] The method utilizes network nodes deployed at an Internet service provider to obtain data packet metadata, data packet arrival timing, and traffic original bytes of the target encrypted traffic; parses the data packet metadata to extract flow-level and packet-level statistical features; performs frequency domain transformation on the data packet arrival timing to obtain the time series features; extracts TLS handshake phase plaintext information and preceding data packet byte sequence of the traffic original bytes, and characterizes the original byte domain based on the TLS handshake phase plaintext information and the preceding data packet byte sequence; and constructs the multi-domain feature dataset based on the statistical features, the time series features, and the original byte domain.

[0014] Optionally, simulating transmission control protocol behavior in real network transmission based on the multi-domain feature dataset to generate an enhanced dataset containing transmission control protocol semantics includes:

[0015] A data enhancement mechanism based on transmission control protocol awareness is used to simulate the transmission control protocol behavior in real network transmission according to the multi-domain feature dataset, so that protocol-level dynamic disturbances are injected into the multi-domain feature dataset to generate the enhanced dataset containing transmission control protocol semantics, wherein the transmission control protocol behavior includes packet sequence offset, packet sequence duplication and packet length offset.

[0016] Optionally, inputting the enhanced dataset containing the transmission control protocol semantics into a pre-built hybrid fusion model to extract feature vectors of each domain separately includes:

[0017] The enhanced data set containing transmission control protocol semantics is input into the pre-built hybrid fusion model respectively, wherein the pre-built hybrid fusion model includes a tree model, a sequence neural network and a convolutional neural network; quantifiable statistical features are extracted from the enhanced data set containing transmission control protocol semantics by the tree model, and the quantifiable statistical features are used as the first feature vector of the current domain; the depth of the traffic sequence features is extracted from the enhanced data set containing transmission control protocol semantics by the sequence neural network, and the depth of the traffic sequence features is used as the second feature vector of the current domain; the original byte features of the network traffic payload are automatically extracted from the enhanced data set containing transmission control protocol semantics by the convolutional neural network, and the original byte features of the network traffic payload are used as the third feature vector of the current domain.

[0018] Optionally, the sequential neural network includes a first embedding layer, a bidirectional multi-layer LSTM network, a first dense layer and a first classification layer, wherein,

[0019] The first embedding layer maps the packet length sequence in the enhanced data set containing the transmission control protocol semantics into a high-dimensional dense vector; the bidirectional multi-layer LSTM network captures the feature vector containing the full sequence context information in the high-dimensional dense vector; the first dense layer uses a multi-layer perceptron to perform information fusion and dimensionality reduction on the feature vector containing the full sequence context information to form a representation vector of the time series feature; the first classification layer introduces a soft maximization classifier to calculate the depth of the traffic sequence feature based on the representation vector of the time series feature.

[0020] Optionally, the convolutional neural network includes a second embedding layer, a two-dimensional CNN combined with a residual network, a second dense layer and a second classification layer, wherein,

[0021] The second embedding layer maps the leading byte sequence of the upstream and downstream loads in the enhanced data set containing the transmission control protocol semantics to a Hilbert space geometric representation; the two-dimensional CNN combined with the residual network captures the local spatial correlation and long-distance structural relationship in the Hilbert space geometric representation; the second dense layer uses a multi-layer perceptron to fuse information and reduce dimensions of the local spatial correlation and the long-distance structural relationship to form a representation vector of the original byte domain; the second classification layer uses a two-layer multi-layer perceptron to map the representation vector of the original byte domain to the target label space to extract the original byte features of the network traffic load.

[0022] A second embodiment of the present invention provides an encrypted multimedia traffic classification device for an Internet service provider, comprising:

[0023] A capture module is used to capture the statistical features, time series features and raw byte domains of the target encrypted traffic to construct a multi-domain feature dataset; an enhancement module is used to simulate the transmission control protocol behavior in real network transmission based on the multi-domain feature dataset to generate an enhanced dataset containing transmission control protocol semantics; an extraction module is used to input the enhanced dataset containing transmission control protocol semantics into a pre-built hybrid fusion model to extract the feature vectors of each domain respectively; a classification module is used to integrate the weights of the feature vectors of each domain using maximum likelihood estimation to determine the category to which the target encrypted traffic belongs.

[0024] A third aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the encrypted multimedia traffic classification method for Internet service providers as described in the above embodiment.

[0025] A fourth aspect of the present invention provides a computer program product. When the computer program / instructions are executed by a processor, the computer program / instructions implement the above-mentioned encrypted multimedia traffic classification method for Internet service providers.

[0026] A fifth aspect of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned encrypted multimedia traffic classification method for Internet service providers.

[0027] The encrypted multimedia traffic classification method and device for Internet service providers proposed in the embodiments of the present invention address the difficult problem of encrypted multimedia traffic classification. By integrating statistical, temporal and raw byte multi-dimensional features, combining dedicated domain models (StatRF, SeqNet, RBNet) and TCP semantic enhancement strategies, they overcome the limitations of single feature domain analysis bias and insufficient understanding of TCP protocol semantics, and achieve robust classification performance in complex network scenarios. The TCP semantic enhancement strategy simulates the dynamic behavior of the protocol in real network transmission (such as retransmission and congestion control), thereby improving the model's generalization ability for encrypted traffic under dynamic conditions, reducing the need for frequent model updates, and providing technical support for Internet service providers to build an efficient QoS-driven network traffic management system.

[0028] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0030] Figure 1 A flowchart of a method for classifying encrypted multimedia traffic for Internet service providers provided by an embodiment of the present invention;

[0031] Figure 2 This is a diagram illustrating an implementation framework of an encrypted multimedia traffic classification method for Internet service providers provided by an embodiment of the present invention;

[0032] Figure 3 A schematic diagram of three key factors that cause data packet length sequence fluctuations in different network environments, provided by an embodiment of the present invention, wherein (a) is packet sequence offset, (b) is packet sequence duplication, and (c) is packet length offset;

[0033] Figure 4 A diagram of the execution framework of a sequential neural network provided by an embodiment of the present invention, wherein (a) is the first embedding layer, (b) is a bidirectional multi-layer LSTM network, (c) is the first dense layer, and (d) is the first classification layer;

[0034] Figure 5 A diagram of the execution framework of a convolutional neural network provided by an embodiment of the present invention, wherein (a) is the second embedding layer, (b) is a two-dimensional CNN combined with a residual network, (c) is the second dense layer, and (d) is the second classification layer;

[0035] Figure 6A block diagram of an encrypted multimedia traffic classification device for Internet service providers provided by an embodiment of the present invention;

[0036] Figure 7 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.

[0038] The following describes an encrypted multimedia traffic classification method and apparatus for Internet service providers according to an embodiment of the present invention with reference to the accompanying drawings.

[0039] Figure 1 A flowchart of an encrypted multimedia traffic classification method for Internet service providers provided by an embodiment of the present invention.

[0040] like Figure 1 As shown, the method for classifying encrypted multimedia traffic to an Internet service provider includes the following steps:

[0041] In step S101, the statistical features, time series features, and raw byte domains of the target encrypted traffic are captured to construct a multi-domain feature dataset.

[0042] In some embodiments, statistical features, time series features, and raw byte fields of target encrypted traffic are captured to construct a multi-field feature dataset, including:

[0043] Utilize network nodes deployed at Internet service providers to obtain packet metadata, packet arrival timing, and raw bytes of target encrypted traffic;

[0044] Parse data packet metadata to extract flow-level and packet-level statistical features;

[0045] Perform frequency domain transformation on the arrival time sequence of data packets to obtain time series features;

[0046] Extract the TLS handshake phase plaintext information and the preceding data packet byte sequence of the original bytes of the traffic, and characterize the original byte field based on the TLS handshake phase plaintext information and the preceding data packet byte sequence;

[0047] A multi-domain feature dataset is constructed based on statistical features, time series features and raw byte domains.

[0048] In the actual implementation process, the embodiment of the present invention considers the characteristics of multiple dimensions simultaneously in the network traffic classification technical solution, and provides a new approach to the problem of complex traffic identification by integrating the complementary advantages of statistical features, time series features and raw byte information. At present, with the surge in encrypted traffic and the diversification of application scenarios, the limitations of most traditional methods that rely on a single feature domain (such as only the statistical value of the statistical flow or the raw byte information) are becoming increasingly apparent. Although statistical features can capture the differences in the macro distribution of traffic (such as the upstream and downstream byte flow ratios of video streams and real-time communications), it is difficult to characterize the dynamic transmission process; time series features can use technologies such as fast Fourier transform (FFT) to analyze the frequency domain laws of packet arrival intervals and size sequences, and can mine the pseudo-random patterns of encrypted traffic payloads; raw byte features use plaintext interactive information such as server name indication (SNI) during the TLS handshake phase, or through the protocol structure of the preceding packet (such as the FTP command field) to achieve earlier identification. The fusion of three types of features can comprehensively characterize traffic. Statistical features can provide basic classification information for distinguishing major traffic categories. Timing analysis reveals the dynamic interaction characteristics of the transmission process. Raw bytes supplement protocol-level semantic information, showing the potential to surpass the performance of a single feature model in traffic classification.

[0049] Specifically, if Figure 2 As shown, the embodiment of the present invention uses the core network nodes deployed at the Internet Service Provider (ISP) to synchronously capture the flow-level and packet-level statistical features, time series features and original byte domains in the target encrypted traffic to form a multi-dimensional traffic characterization space. Specifically, the data packet metadata is parsed to extract flow-level and packet-level statistics (such as packet size distribution, byte flow ratio, connection duration); the arrival sequence of data packets is recorded based on high-precision timestamps to generate packet interval sequences and ordered packet size sequences, and the packet interval sequences and ordered packet size sequences are transformed in the frequency domain (such as fast Fourier transform) to capture dynamic transmission patterns and obtain time series features; the original bytes of the traffic are processed to characterize the traffic features by extracting the plaintext information in the TLS handshake phase (such as the server name indication SNI, certificate field) and the preceding data packet byte sequence to obtain a multi-domain feature data set.

[0050] In step S102, the transmission control protocol behavior in real network transmission is simulated according to the multi-domain feature dataset to generate an enhanced dataset containing transmission control protocol semantics.

[0051] In some embodiments, simulating transmission control protocol behavior in real network transmission based on the multi-domain feature dataset to generate an enhanced dataset containing transmission control protocol semantics includes:

[0052] Based on the data enhancement mechanism of Transmission Control Protocol awareness, the Transmission Control Protocol behavior in real network transmission is simulated according to the multi-domain feature dataset, so that protocol-level dynamic perturbations are injected into the multi-domain feature dataset to generate an enhanced dataset containing Transmission Control Protocol semantics, where the Transmission Control Protocol behavior includes packet sequence offset, packet sequence duplication and packet length offset.

[0053] In actual implementation, when network conditions change, TCP's retransmission timeout (RTO) can cause subsequence offsets, rapid retransmission of data packets can cause packet sequence duplication, and Nagle algorithm optimization can cause packet size adjustments. These phenomena often cause traditional models to misjudge the true business attributes of encrypted traffic. For example, when the packet size of video key frames fluctuates due to MTU differences, if the model lacks understanding of TCP retransmission logic, it can easily confuse it with abnormal traffic.

[0054] Therefore, if Figure 3 As shown, the embodiment of the present invention addresses the problem that existing traffic classification technologies lack modeling of TCP protocol-level semantics in dynamic network environments. It proposes a TCP protocol-aware data enhancement mechanism, specifically targeting delay-sensitive multimedia traffic such as video streaming and real-time communication. It reproduces these dynamic characteristics while retaining the original traffic characteristics. By simulating the behavior of the real TCP protocol stack to achieve semantic feature enhancement, it can learn the essential laws of the dynamic change characteristics of traffic rather than just the surface patterns. The mechanism first identifies three key phenomena caused by the TCP transmission mechanism: subsequence offset caused by retransmission timeout (RTO), subsequence duplication caused by fast retransmission, and packet size changes caused by Nagle algorithm optimization, which seriously interfere with the feature consistency of TLS traffic classification.

[0055] To preserve the original traffic distribution characteristics and inject protocol-level dynamic semantics, the enhanced framework simulates TCP retransmission behavior by generating sequence variants that conform to the real RTO and fast retransmit mode, while maintaining the correctness of the packet header fields to preserve key protocol signatures such as the TLS handshake. It also dynamically adjusts the packet size based on the simulated maximum segment size (MSS) negotiation results to reproduce the fragmentation scenarios caused by MTU differences in video streaming platforms. By accurately modeling network changes guided by TCP transmission semantics, this enhancement mechanism enables the classification model to learn invariant features that are strongly correlated with protocol behavior, effectively improving its robustness in classifying encrypted multimedia traffic under complex network conditions and effectively addressing the lack of generalization capabilities of traditional methods due to their neglect of TCP dynamic behavior.

[0056] Specifically, if Figure 2As shown, the embodiment of the present invention delivers three types of features in the multi-domain feature dataset to the transmission control protocol-aware data enhancement mechanism to simulate the TCP protocol behavior in real network transmission (such as subsequence offset caused by retransmission timeout, packet sequence duplication caused by fast retransmission, and packet size adjustment generated by Nagle algorithm optimization), embeds protocol-level dynamic features (such as subsequence offset and packet size adjustment mode) into the traffic representation space, and generates an enhanced dataset containing TCP semantics, which effectively enhances the robustness and adaptability of the classification model to dynamic conditions such as network congestion and delay jitter, reduces dependence on specific network environments, and avoids detection performance degradation due to changes in protocol behavior.

[0057] In step S103, the enhanced dataset containing the transmission control protocol semantics is input into the pre-built hybrid fusion model to extract the feature vector of each domain respectively.

[0058] In some embodiments, an enhanced dataset containing transmission control protocol semantics is input into a pre-built hybrid fusion model to extract feature vectors for each domain separately, including:

[0059] The enhanced dataset containing transmission control protocol semantics is input into a pre-built hybrid fusion model respectively, wherein the pre-built hybrid fusion model includes a tree model, a sequential neural network and a convolutional neural network;

[0060] Extracting quantifiable statistical features from an enhanced dataset containing transmission control protocol semantics through a tree model, and using the quantifiable statistical features as the first feature vector of the current domain;

[0061] The depth of traffic sequence features is extracted from an enhanced dataset containing transmission control protocol semantics through a sequence neural network, and the depth of traffic sequence features is used as the second feature vector of the current domain;

[0062] Through convolutional neural networks, the original byte features of network traffic payload are automatically extracted from an enhanced dataset containing transmission control protocol semantics, and the original byte features of network traffic payload are used as the third feature vector of the current domain.

[0063] In the actual implementation process, Figure 2 As shown in the figure, the enhanced dataset containing the semantics of the transmission control protocol is input into a hybrid fusion model that integrates dedicated processing units in the statistical domain, timing domain, and raw byte domain, and the deep features of each domain (i.e., the feature vector of each domain) are extracted through the tree model StatRF, the sequence neural network SeqNet, and the convolutional neural network RBNet respectively.

[0064] Furthermore, the tree-based model StatRF is a classification module built based on the random forest algorithm. It aims to extract a quantifiable statistical feature system from traffic data to support classification decisions. This tree-based model StatRF integrates three statistical indicators: flow metadata, distribution characteristics, and TLS metadata. These include flow duration, packet size percentile, upstream and downstream byte distribution ratio, bidirectional port connection status, and features such as certificate validity period and cipher suite type during the TLS handshake phase. By constructing a decision tree ensemble model and aggregating the classification results of multiple trees using a voting mechanism, the tree-based model StatRF effectively captures the complex nonlinear relationships in traffic data. It also utilizes bootstrap resampling technology (bagging) to balance model bias, suppress overfitting, and ensure stable operation in high-dimensional, dynamically changing multimedia traffic scenarios. The statistical feature vector output by the tree-based model StatRF (i.e., the first feature vector of the current domain, such as packet size distribution and byte flow ratio) can complement features from other domains (time series domain and raw byte domain), providing a foundational decision basis for hybrid domain fusion classification.

[0065] Furthermore, if Figure 4 As shown, the sequence neural network SeqNet includes a first embedding layer, a bidirectional multi-layer LSTM network, a first dense layer and a first classification layer, wherein the first embedding layer maps the packet length sequence in the enhanced dataset containing the transmission control protocol semantics into a high-dimensional dense vector; the bidirectional multi-layer LSTM network captures the feature vector containing the full sequence context information in the high-dimensional dense vector; the first dense layer uses a multi-layer perceptron to fuse information and reduce the dimension of the feature vector containing the full sequence context information to form a representation vector of the time series feature; the first classification layer introduces a soft maximization classifier to calculate the depth of the traffic sequence feature according to the representation vector of the time series feature.

[0066] Specifically, the SeqNet neural network employs a four-layer hierarchical architecture to achieve deep extraction and classification of traffic sequence features. Its core lies in the fusion of supervised learning and contrastive learning mechanisms to capture fine-grained temporal dynamic patterns in network flow packet sequences. SeqNet first maps packet length sequences into high-dimensional dense vectors through the first embedding layer. It then calculates the Fourier-transformed Pearson correlation coefficient (PCC) and KL divergence of adjacent packet pairs to quantify the pseudo-random nature of encrypted traffic payloads. This is then used as input along with temporal data such as packet length and inter-arrival intervals.

[0067] The middle layer uses a bidirectional multi-layer LSTM network (Bi-LSTM) to process embedding vectors. This captures upstream and downstream temporal dependencies in communication patterns through bidirectional encoding of forward and backward hidden states. The shallow layer focuses on local sequence patterns, while the deep layer integrates global features to enhance the ability to model long-range dependencies, ultimately generating a feature vector that contains contextual information for the entire sequence. The features processed by the bidirectional LSTM enter the first dense layer, where a multilayer perceptron (MLP) is used to achieve information fusion and dimensionality reduction, forming the core representation vector in the time series domain.

[0068] The first classification layer introduces a soft maximization classifier, combining focal loss and triplet loss to construct a joint training objective. The former addresses the class imbalance problem by adjusting the focal parameter γ, while the latter optimizes contrastive learning using positive and negative sample pairs, guiding the model to learn the intrinsic similarities of data structures and avoiding over-reliance on label information. The use of a hybrid loss function enables the SeqNet neural network to effectively handle sequence variations in encrypted traffic caused by the dynamic behavior of the TCP protocol (such as retransmissions and congestion control). While maintaining computational efficiency, it also retains the ability to distinguish the temporal characteristics of delay-sensitive traffic such as real-time communications and video streaming. The resulting second feature vector in the current domain (such as the packet arrival interval sequence and the frequency domain transformation pattern) complements the statistical domain and raw byte domain modules to jointly support end-to-end classification decisions for equipment.

[0069] Furthermore, if Figure 5 As shown in the figure, the convolutional neural network RBNet includes a second embedding layer, a two-dimensional CNN combined with a residual network, a second dense layer and a second classification layer, wherein the second embedding layer maps the leading byte sequence of the upstream and downstream loads in the enhanced data set containing the semantics of the transmission control protocol to the Hilbert space geometric representation; the two-dimensional CNN combined with the residual network captures the local spatial correlation and long-distance structural relationship in the Hilbert space geometric representation; the second dense layer uses a multi-layer perceptron to fuse information and reduce the dimension of the local spatial correlation and long-distance structural relationship to form a representation vector of the original byte domain; the second classification layer uses two layers of multi-layer perceptrons to map the representation vector of the original byte domain to the target label space to extract the original byte features of the network traffic load.

[0070] Specifically, the convolutional neural network (RBNet) is based on a convolutional neural network (CNN) combined with representation learning technology. It can automatically extract the raw byte features of network traffic payloads and capture high-order statistical properties and local structural patterns that are difficult to characterize with traditional methods. The RBNet first processes the leading byte sequences of the upstream and downstream payloads (such as the first 2500 bytes of each upstream and downstream) through the second embedding layer of the reproducing kernel Hilbert space (RKHS). With the help of the mean embedding method of RKHS, the byte distribution is mapped to the geometric representation of the Hilbert space. While avoiding complex probability density estimation, it accurately depicts the nonlinear characteristics in edge probabilities and conditional probabilities - such as the block structure of burst video stream transmission or the differentiated distribution patterns of elephant and mouse streams.

[0071] The mid-level architecture utilizes a two-dimensional CNN combined with a residual network, using convolutional kernels to capture local spatial correlations and long-range structural relationships within byte sequences. The initial convolution-pooling layer extracts local features after embedding. Subsequent residual blocks capture global sequence patterns while mitigating the vanishing gradient problem in deeper networks, effectively preserving protocol-level fingerprints implicit in the raw bytes (e.g., TLS handshake certificate fields and FTP command structures). The convolutional upstream and downstream features are flattened and fed into the second dense layer of a multilayer perceptron with a Selu activation function, achieving feature fusion and dimensionality reduction to generate the final representation vector for the raw byte domain.

[0072] The second classification layer is mapped to the target label space through two layers of MLP, enabling the convolutional neural network RBNet to extract fine-grained byte features from the early handshake packets or the first few data packet payloads of the encrypted traffic. It is particularly sensitive to plaintext information such as the server name indication (SNI) and protocol version number in the TLS handshake packets, as well as the command field structure unique to protocols such as FTP / SMTP (i.e., the third eigenvector of the current domain). It forms a three-dimensional feature complement with the statistical domain and timing domain modules, jointly supporting the equipment's accurate classification of encrypted multimedia traffic, while maintaining efficient processing, and significantly improving the feature resolution capability in complex protocol environments.

[0073] In step S104 , the weights of the feature vectors of each domain are integrated using maximum likelihood estimation to determine the category to which the target encrypted traffic belongs.

[0074] In the actual implementation process, when classifying network flows, the embodiment of the present invention adopts a weighted soft voting strategy to fuse the classification outputs of the three single-domain models: the statistical domain model StatRF, the time series domain model SeqNet, and the raw byte domain model RBNet. Unlike the existing hard voting that only considers the contribution of a certain type of model, this strategy uses the probability distribution predicted by each model as the collective voting score. In view of the differences in the performance of different feature domain models, different weighting coefficients will be assigned to each model according to its historical accuracy. The final decision is made by linearly combining these weighted probability vectors. This method can effectively reduce the impact of uncertain predictions of a single model while highlighting the contribution of high-performance models. In the feature domain collection In the example, weight ω is defined for each feature domain f f .

[0075] For each sample in the training set, the probability vector output by the feature field f constitutes P f , the sample label data is uniquely encoded to form Z. The task of determining the optimal weight of the model can be transformed into a constrained optimization problem, namely:

[0076]

[0077] The following constraints must also be met:

[0078]

[0079] The maximum likelihood estimation (MLE) is used to calculate the weights of the tree model StatRF, the sequential neural network SeqNet, and the convolutional neural network RBNet. The weights of the feature vectors of each domain are integrated, and the classification decision module finally outputs the application type of encrypted traffic (such as video streaming, real-time communication, file transfer, etc.).

[0080] Experiments on a real-world encrypted multimedia traffic dataset have demonstrated the superior performance of this invention: achieving an F1 score of up to 97.75% in multi-class classification tasks, a 3.51%-8.09% improvement over the leading baseline approach while maintaining low computational complexity. Experimental results demonstrate that the present invention significantly improves classification accuracy for latency-sensitive traffic, such as encrypted video streams and real-time communications, compared to traditional methods.

[0081] In summary, the encrypted multimedia traffic classification method for Internet service providers proposed in the embodiment of the present invention has the following beneficial effects:

[0082] (1) It integrates multi-dimensional feature collection and TCP semantic enhancement technology. The former improves the richness of representation through cross-domain information integration, and the latter enhances the robustness of the model through TCP protocol-level modeling. Together, they provide a feasible solution for encrypted traffic classification, dynamic network environment adaptation, and QoS assurance for delay-sensitive services, thereby alleviating the problems of traditional solutions in encrypted traffic identification, such as one-sided feature analysis, lack of protocol semantic understanding, and poor model adaptability.

[0083] (2) The TCP semantic enhancement unit simulates real protocol behavior, which helps to enhance the model's adaptability to dynamic network changes;

[0084] (3) The fusion model uses a dedicated sub-model and a weighted soft voting strategy to break through the limitations of a single feature domain;

[0085] (3) Through the coordinated multi-domain feature collection, model hybrid fusion and TCP semantic enhancement, multi-faceted analysis and classification of traffic can be achieved; multi-domain integration of statistical, time series and raw byte features can comprehensively capture traffic behavior;

[0086] (5) It provides an efficient and reliable network security protection and traffic management solution for ISPs to implement QoS-driven traffic classification and management, effectively improving the efficiency and accuracy of encrypted traffic classification, and can cope with the challenges of complex network environments. It is of great significance to ensure service quality, improve user experience, and maintain network security under the new network architecture.

[0087] Next, an encrypted multimedia traffic classification device for Internet service providers proposed in an embodiment of the present invention will be described with reference to the accompanying drawings.

[0088] Figure 6 A block diagram of an encrypted multimedia traffic classification device for Internet service providers provided by an embodiment of the present invention.

[0089] like Figure 6 As shown, the encrypted multimedia traffic classification device 60 for Internet service providers includes: a capture module 601, an enhancement module 602, an extraction module 603 and a classification module 604.

[0090] The capture module 601 is used to capture the statistical characteristics, time series characteristics, and raw byte domains of the target encrypted traffic to construct a multi-domain feature dataset. The enhancement module 602 is used to simulate the transmission control protocol behavior in real network transmission based on the multi-domain feature dataset to generate an enhanced dataset containing transmission control protocol semantics. The extraction module 603 is used to input the enhanced dataset containing transmission control protocol semantics into a pre-built hybrid fusion model to extract feature vectors for each domain. The classification module 604 is used to integrate the weights of the feature vectors for each domain using maximum likelihood estimation to determine the category to which the target encrypted traffic belongs.

[0091] In some embodiments, the capture module 601 includes:

[0092] an acquisition unit, configured to acquire data packet metadata, data packet arrival timing, and raw bytes of the target encrypted traffic using a network node deployed at an Internet service provider;

[0093] A parsing unit, used to parse data packet metadata to extract flow-level and packet-level statistical features;

[0094] A transformation unit, configured to perform frequency domain transformation on the arrival timing of data packets to obtain time series features;

[0095] A characterization unit is used to extract the TLS handshake phase plaintext information and the preceding data packet byte sequence of the original bytes of the traffic, and characterize the original byte field based on the TLS handshake phase plaintext information and the preceding data packet byte sequence;

[0096] A construction unit is used to construct a multi-domain feature dataset based on statistical features, time series features, and raw byte fields.

[0097] In some embodiments, the enhancement module 602 includes:

[0098] Based on the data enhancement mechanism of Transmission Control Protocol awareness, the Transmission Control Protocol behavior in real network transmission is simulated according to the multi-domain feature dataset, so that protocol-level dynamic perturbations are injected into the multi-domain feature dataset to generate an enhanced dataset containing Transmission Control Protocol semantics, where the Transmission Control Protocol behavior includes packet sequence offset, packet sequence duplication and packet length offset.

[0099] In some embodiments, the extraction module 603 includes:

[0100] An input unit, configured to input the enhanced data set containing transmission control protocol semantics into a pre-built hybrid fusion model, wherein the pre-built hybrid fusion model includes a tree model, a sequential neural network, and a convolutional neural network;

[0101] a first extraction unit, configured to extract quantifiable statistical features from an enhanced data set containing transmission control protocol semantics through a tree model, and use the quantifiable statistical features as a first feature vector of a current domain;

[0102] a second extraction unit, configured to extract the depth of the traffic sequence feature from the enhanced data set containing the transmission control protocol semantics through a sequence neural network, and use the depth of the traffic sequence feature as a second feature vector of the current domain;

[0103] The third extraction unit is used to automatically extract the original byte features of the network traffic payload in the enhanced data set containing the transmission control protocol semantics through a convolutional neural network, and use the original byte features of the network traffic payload as the third feature vector of the current domain.

[0104] In some embodiments, the sequential neural network includes a first embedding layer, a bidirectional multi-layer LSTM network, a first dense layer, and a first classification layer, wherein:

[0105] The first embedding layer maps the packet length sequence in the augmented dataset containing the transmission control protocol semantics into a high-dimensional dense vector;

[0106] The bidirectional multi-layer LSTM network captures the feature vector containing the full sequence context information in the high-dimensional dense vector;

[0107] The first dense layer uses a multi-layer perceptron to fuse information and reduce the dimension of the feature vector containing the full sequence context information to form a representation vector of the time series features;

[0108] The first classification layer introduces a soft maximization classifier to calculate the depth of traffic sequence features based on the representation vector of time series features.

[0109] In some embodiments, the convolutional neural network includes a second embedding layer, a two-dimensional CNN combined with a residual network, a second dense layer, and a second classification layer, wherein

[0110] The second embedding layer maps the preamble byte sequence of the upstream and downstream payloads in the enhanced dataset containing the transmission control protocol semantics to a Hilbert space geometric representation;

[0111] Two-dimensional CNN combined with residual network captures local spatial correlation and long-range structural relationship in Hilbert space geometric representation;

[0112] The second dense layer uses a multi-layer perceptron to fuse information and reduce dimensions of local spatial correlation and long-distance structural relationships to form a representation vector of the original byte domain;

[0113] The second classification layer maps the representation vector of the original byte domain to the target label space through two layers of multi-layer perceptrons to extract the original byte features of the network traffic payload.

[0114] It should be noted that the above explanation of the embodiment of the encrypted multimedia traffic classification method for Internet service providers is also applicable to the encrypted multimedia traffic classification device for Internet service providers in this embodiment, and will not be repeated here.

[0115] The encrypted multimedia traffic classification device for Internet service providers proposed in accordance with an embodiment of the present invention has the following beneficial effects:

[0116] (1) It integrates multi-dimensional feature collection and TCP semantic enhancement technology. The former improves the richness of representation through cross-domain information integration, and the latter enhances the robustness of the model through TCP protocol-level modeling. Together, they provide a feasible solution for encrypted traffic classification, dynamic network environment adaptation, and QoS assurance for delay-sensitive services, thereby alleviating the problems of traditional solutions in encrypted traffic identification, such as one-sided feature analysis, lack of protocol semantic understanding, and poor model adaptability.

[0117] (2) The TCP semantic enhancement unit simulates real protocol behavior, which helps to enhance the model's adaptability to dynamic network changes;

[0118] (3) The fusion model uses a dedicated sub-model and a weighted soft voting strategy to break through the limitations of a single feature domain;

[0119] (3) Through the coordinated multi-domain feature collection, model hybrid fusion and TCP semantic enhancement, multi-faceted analysis and classification of traffic can be achieved; multi-domain integration of statistical, time series and raw byte features can comprehensively capture traffic behavior;

[0120] (5) It provides an efficient and reliable network security protection and traffic management solution for ISPs to implement QoS-driven traffic classification and management, effectively improving the efficiency and accuracy of encrypted traffic classification, and can cope with the challenges of complex network environments. It is of great significance to ensure service quality, improve user experience, and maintain network security under the new network architecture.

[0121] Figure 7 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device may include:

[0122] Memory 701 , processor 702 , and computer programs stored in the memory 701 and executable on the processor 702 .

[0123] When the processor 702 executes the program, the encrypted multimedia traffic classification method for Internet service providers provided in the above embodiment is implemented.

[0124] Furthermore, the electronic device further includes:

[0125] The communication interface 703 is used for communication between the memory 701 and the processor 702 .

[0126] The memory 701 is used to store computer programs that can be run on the processor 702 .

[0127] The memory 701 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0128] If the memory 701, processor 702, and communication interface 703 are implemented independently, the communication interface 703, memory 701, and processor 702 can be interconnected via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0129] Optionally, in a specific implementation, if the memory 701, the processor 702 and the communication interface 703 are integrated on a chip, the memory 701, the processor 702 and the communication interface 703 can communicate with each other through an internal interface.

[0130] The processor 702 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0131] An embodiment of the present invention further provides a computer program product. When the computer program / instructions are executed by a processor, the computer program / instructions implement the above-mentioned encrypted multimedia traffic classification method for Internet service providers.

[0132] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the program implements the above-mentioned encrypted multimedia traffic classification method for Internet service providers.

[0133] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.

[0134] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "N" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0135] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or N executable instructions for implementing a custom logical function or step of a process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0136] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or N wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program can be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing it in other suitable ways as necessary, and then storing it in a computer memory.

[0137] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiment, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0138] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0139] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

[0140] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and are not to be construed as limiting the present invention. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for classifying encrypted multimedia traffic for Internet service providers, characterized in that: The following steps are involved: Capture the statistical characteristics, time series characteristics, and raw byte domains of target encrypted traffic to build a multi-domain feature dataset; Simulating transmission control protocol behavior in real network transmission according to the multi-domain feature dataset to generate an enhanced dataset containing transmission control protocol semantics; Inputting the enhanced dataset containing transmission control protocol semantics into a pre-built hybrid fusion model to extract feature vectors of each domain respectively; The weights of the feature vectors of each domain are integrated using maximum likelihood estimation to determine the category to which the target encrypted traffic belongs.

2. The method for classifying encrypted multimedia traffic for Internet service providers according to claim 1, wherein: The statistical features, time series features, and raw byte domains of the target encrypted traffic are captured to construct a multi-domain feature dataset, including: Using a network node deployed at an Internet service provider, obtaining data packet metadata, data packet arrival timing, and raw bytes of the target encrypted traffic; parsing the data packet metadata to extract flow-level and packet-level statistical features; Performing frequency domain transformation on the arrival time sequence of the data packets to obtain the time series characteristics; Extracting the TLS handshake phase plaintext information and the preceding data packet byte sequence of the original bytes of the traffic, and characterizing the original byte field according to the TLS handshake phase plaintext information and the preceding data packet byte sequence; The multi-domain feature dataset is constructed according to the statistical features, the time series features and the original byte domain.

3. The method for classifying encrypted multimedia traffic for Internet service providers according to claim 1, wherein: The simulating the transmission control protocol behavior in real network transmission according to the multi-domain feature dataset to generate an enhanced dataset containing transmission control protocol semantics includes: A data enhancement mechanism based on transmission control protocol awareness is used to simulate the transmission control protocol behavior in real network transmission according to the multi-domain feature dataset, so that protocol-level dynamic disturbances are injected into the multi-domain feature dataset to generate the enhanced dataset containing transmission control protocol semantics, wherein the transmission control protocol behavior includes packet sequence offset, packet sequence duplication and packet length offset.

4. The method for classifying encrypted multimedia traffic for Internet service providers according to claim 1, wherein: The step of inputting the enhanced dataset containing the transmission control protocol semantics into a pre-built hybrid fusion model to extract feature vectors of each domain separately includes: Inputting the enhanced data sets containing transmission control protocol semantics into the pre-built hybrid fusion model respectively, wherein the pre-built hybrid fusion model includes a tree model, a sequential neural network and a convolutional neural network; Extracting quantifiable statistical features from the enhanced data set containing transmission control protocol semantics by using the tree model, and using the quantifiable statistical features as the first feature vector of the current domain; Extracting the depth of traffic sequence features from the enhanced data set containing transmission control protocol semantics through the sequence neural network, and using the depth of the traffic sequence features as a second feature vector of the current domain; The convolutional neural network is used to automatically extract the original byte features of the network traffic payload from the enhanced data set containing the transmission control protocol semantics, and the original byte features of the network traffic payload are used as the third feature vector of the current domain.

5. The method for classifying encrypted multimedia traffic for Internet service providers according to claim 4, characterized in that: The sequential neural network includes a first embedding layer, a bidirectional multi-layer LSTM network, a first dense layer and a first classification layer, wherein, The first embedding layer maps the packet length sequence in the enhanced dataset containing transmission control protocol semantics into a high-dimensional dense vector; The bidirectional multi-layer LSTM network captures the feature vector containing the full sequence context information in the high-dimensional dense vector; The first dense layer performs information fusion and dimension reduction on the feature vector containing the full sequence context information through a multi-layer perceptron to form a representation vector of the time series feature; The first classification layer introduces a soft maximization classifier to calculate the depth of the traffic sequence feature according to the representation vector of the time series feature.

6. The method for classifying encrypted multimedia traffic for Internet service providers according to claim 4, characterized in that: The convolutional neural network includes a second embedding layer, a two-dimensional CNN combined with a residual network, a second dense layer and a second classification layer, wherein, The second embedding layer maps the leading byte sequences of the upstream and downstream payloads in the enhanced data set containing the transmission control protocol semantics to a Hilbert space geometric representation; The two-dimensional CNN is combined with the residual network to capture the local spatial correlation and long-range structural relationship in the Hilbert space geometric representation; The second dense layer performs information fusion and dimensionality reduction on the local spatial correlation and the long-distance structural relationship through a multi-layer perceptron to form a representation vector of the original byte domain; The second classification layer maps the representation vector of the original byte domain to the target label space through a two-layer multi-layer perceptron to extract the original byte features of the network traffic load.

7. An encrypted multimedia traffic classification device for Internet service providers, characterized in that: include: The capture module is used to capture the statistical characteristics, time series characteristics, and raw byte fields of the target encrypted traffic to construct a multi-domain feature dataset; an enhancement module, configured to simulate a transmission control protocol behavior in real network transmission based on the multi-domain feature dataset to generate an enhanced dataset containing transmission control protocol semantics; an extraction module, configured to input the enhanced dataset containing transmission control protocol semantics into a pre-built hybrid fusion model to extract feature vectors of each domain respectively; The classification module is used to integrate the weights of the feature vectors of each domain by using maximum likelihood estimation to determine the category to which the target encrypted traffic belongs.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the encrypted multimedia traffic classification method for Internet service providers according to any one of claims 1 to 6.

9. A computer program product, characterized in that When the computer program / instruction is executed by a processor, the method for classifying encrypted multimedia traffic for Internet service providers according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the encrypted multimedia traffic classification method for Internet service providers according to any one of claims 1 to 6.

Images