Apparatus and process for automatically classifying network flows of ISP-level network traffic in real-time

Abstract

A computer-implemented process for processing packets of ISP-level network traffic to automatically classify in real-time network flows of the network traffic, and including the steps of: receiving, at respective packet arrival times, packets of network flows of general network traffic of a plurality of network users of the ISP; processing the received packets to generate, for each network flow of the received packets, a corresponding packet-level time-series representation of each packet of the network flow, and a corresponding slot-level time-series representation of aggregated packets of the network flow received at respective packet arrival times within each of a plurality of successive time slots; and for each of the network flows: (i) processing the packet-level time-series representation with a corresponding trained packet-level classifier to generate a packet-level classification of the network flow as one of a plurality of known flow types and a corresponding confidence score; and (ii) processing the slot-level time-series representation with a corresponding trained slot-level classifier to generate a slot-level classification of the network flow as one of the plurality of known flow types and a corresponding confidence score; and (iii) processing the packet-level and slot-level classifications of the network flow and the respective confidence scores with a time-series flow classifier including long short term memory (LSTM) cells trained by reinforcement learning to determine a confident classification of the network flow as one of the plurality of known flow types or as an unknown flow type.

Description

[0001] APPARATUS AND PROCESS FOR AUTOMATICALLY CLASSIFYING NETWORK FLOWS OF ISP-LEVEL NETWORK TRAFFIC IN REAL-TIME

[0002] TECHNICAL FIELD

[0003] The present invention relates to network traffic management, and in particular to an apparatus and process for processing data packets of network traffic in large (ISP-level) networks to automatically classify in real-time network flows of the network traffic.

[0004] BACKGROUND

[0005] In order to effectively manage their networks, network operators that offer Internet access services to broadband and / or mobile clients, or manage network infrastructure for large enterprises and / or university campuses, require visibility into the bandwidth demands of different applications, such as video streaming, online gaming, and conferencing, and their service providers (e.g., YouTube and Netflix for video streaming), and the associated experiences of their end-users. In particular, having visibility into the user experience associated with each application session facilitates proactive network troubleshooting to maintain high network performance for users. Moreover, providing contextual labels for network flows through a network can serve as signals for dynamic performance optimization techniques, such as prioritizing network flows of latency-sensitive applications such as video conferencing via low-latency 5G slices, and using network APIs for video streaming flows to guarantee sufficient bandwidth allocations, for example.

[0006] Currently, internet service providers (ISPs) deploy classifiers that categorize network flows into corresponding application types for usage accounting and user experience measurements. However, for practical deployment in large networks that can serve hundreds of thousands of users using diversified applications supported by millions of concurrent network flows, the classification of flows needs to not only provide accurate flow labels that are useful to network operators, but also to produce classification results for each flow at the earliest possible time, ideally from only the initial packets of each flow, and the least possible number of those initial packets. This would reduce computing resources, classify more queued flows, and quickly enable subsequent monitoring tasks (e.g., inference on application user experience) that require flow classification results as a prerequisite. Although existing flow classification methods have gone some way to meet these goals, their performance remains limited. For example, flow classifiers should also be robust to packet sequence disorders (drops and retransmissions) in candidate flows,and capable of detecting unknown flow types that are not within the existing classification scope. Neither of these goals is well achieved by existing methods.

[0007] It is desired to address or alleviate one or more disadvantages or limitations of the prior art, or to at least provide a useful alternative.

[0008] SUMMARY

[0009] In accordance with some embodiments of the present invention, there is provided a computer-implemented process for processing packets of ISP-level network traffic to automatically classify in real-time network flows of the network traffic, and including the steps of:

[0010] receiving, at respective packet arrival times, packets of network flows of general network traffic of a plurality of network users of the ISP;

[0011] processing the received packets to generate, for each network flow of the received packets, a corresponding packet-level time-series representation of each packet of the network flow, and a corresponding slot-level time-series representation of aggregated packets of the network flow received at respective packet arrival times within each of a plurality of successive time slots; and

[0012] for each of the network flows:

[0013] (i) processing the packet-level time-series representation with a corresponding trained packet-level classifier to generate a packetlevel classification of the network flow as one of a plurality of known flow types and a corresponding confidence score; and (ii) processing the slot-level time-series representation with a corresponding trained slot-level classifier to generate a slot-level classification of the network flow as one of the plurality of known flow types and a corresponding confidence score; and

[0014] (iii) processing the packet-level and slot-level classifications of the network flow and the respective confidence scores with a timeseries flow classifier including long short term memory (LSTM) cells trained by reinforcement learning to determine a confident classification of the network flow as one of the plurality of known flow types or as an unknown flow type.In some embodiments, the process further includes a step of, responsive to the flow classification, changing one or more network settings to effect one or more of the following network reconfigurations: provisioning bandwidth and network capability in accordance with bandwidth demands of the flow type, prioritising traffic flows, and mapping traffic to network slices.

[0015] In some embodiments, the step of processing the packet-level and slot-level classifications of the network flow and the corresponding confidence scores includes:

[0016] comparing each of the confidence scores with a corresponding confidence threshold; responsive to determining that each of the confidence scores is less than the corresponding confidence threshold, waiting to receive a further corresponding packet-level or slot-level classification of the network flow and corresponding confidence score; and responsive to determining that at least one of the confidence scores is equal to or greater than the corresponding confidence threshold, selecting the greater of the confidence scores and the corresponding packet-level or slot-level classification of the network flow as the confident classification of the network flow as one of the plurality of known flow types.

[0017] In some embodiments, the step of processing the packet-level and slot-level classifications of the network flow and the respective confidence scores includes, responsive to determining that a number of processed packets of the network flow has reached or exceeded a threshold value, determining the confident classification of the network flow as the unknown flow type.

[0018] In some embodiments, the time-series flow classifier dynamically determines a smallest number of packets required to accurately classify the network flow.

[0019] In accordance with some embodiments of the present invention, there is provided a network flow classification apparatus for processing data packets of ISP level network traffic to automatically identify in real-time user platforms and content providers of video streams of the network traffic, the apparatus including:

[0020] random access memory; and

[0021] at least one processor configured to execute any one of the above processes.In accordance with some embodiments of the present invention, there is provided a computer-readable storage medium having stored thereon executable instructions that, when executed by at least one processor, cause the at least one processor to execute any one of the above processes.

[0022] In accordance with some embodiments of the present invention, there is provided a network flow classification apparatus for processing data packets of ISP-level network traffic to automatically classify in real-time network flows of the network traffic, the apparatus including:

[0023] random access memory;

[0024] at least one processor;

[0025] at least one network interface to receive, at respective packet arrival times, packets of network flows of general network traffic of a plurality of network users of a network service provider;

[0026] a packet-level time-series generator to generate, for each network flow of the received packets, a corresponding time-series representation of each packet of the network flow;

[0027] a slot-level time-series generator to generate, for each network flow of the received packets, a corresponding time-series representation of aggregated packets received at respective packet arrival times within each of a plurality of successive time slots;

[0028] a trained packet-level time series classifier to process each time-series representation of each packet of the network flow to generate a corresponding packetlevel classification of the network flow as one of a plurality of known flow types and a corresponding confidence score;

[0029] a slot-level time series classifier including long short term memory (LSTM) cells trained by reinforcement learning to process each slot-level time-series representation to generate a corresponding slot-level classification of the network flow as one of the plurality of known flow types and a corresponding confidence score; and a time-series flow classifier to process the packet-level and slot-level classifications of the network flow and the corresponding confidence scores to determinea confident classification of the flow as one of the plurality of known flow types or as an unknown flow type.

[0030] In some embodiments, the apparatus further includes a network API component to effect one or more of the following network reconfigurations: provisioning bandwidth and network capability in accordance with bandwidth demands of the confidently classified flow type, prioritising traffic flows, and mapping traffic to network slices.

[0031] In some embodiments, the time-series flow classifier compares each of the confidence scores with a corresponding confidence threshold, and, responsive to determining that the confidence score is less than the corresponding confidence threshold, waits to receive a further corresponding provisional classification of the network flow and corresponding confidence score.

[0032] In some embodiments, the slot-level time series classifier is further configured to determine, responsive to determining that a number of processed packets of the network flow has reached or exceeded a threshold value, the confident classification of the network flow as the unknown flow type.

[0033] The time-series flow classifier may dynamically determine a smallest number of packets required to accurately classify the network flow.

[0034] Also described herein is a computer-implemented process for processing packets of ISP-level network traffic to automatically classify in real-time network flows of the network traffic, and including the steps of:

[0035] receiving packets of network flows of general network traffic of a plurality of network users of the ISP;

[0036] processing the received packets to generate, for each network flow of the received packets, a corresponding time-series representation of each packet of the network flow, and a corresponding time-series representation of aggregated packets received within each of a plurality of time slots of the network flow; and

[0037] for each of the network flows:(i) processing the time-series representation of each packet of the network flow with a corresponding trained packet-level classifier to generate a first provisional classification of the network flow as one of a plurality of know flow types and a corresponding confidence score; and

[0038] (ii) processing the time-series representation of each slot of the network flow with a corresponding trained slot-level classifier to generate a second provisional classification of the network flow as one of the plurality of know flow types and a corresponding confidence score; and

[0039] (iii) processing the first and second provisional classifications of the network flow and the corresponding confidence scores with a timeseries flow classifier to determine a confident classification of the flow as one of the plurality of know flow types or as an unknown flow type.

[0040] The time-series flow classifier may include long short term memory (LSTM) cells trained by reinforcement learning. The time-series flow classifier may dynamically determine a smallest number of packets required to accurately classify the network flow.

[0041] The process may further include a step of, responsive to the flow classification, changing one or more network settings to effect one or more of the following network reconfigurations: provisioning bandwidth and network capability in accordance with bandwidth demands of the flow type, prioritising traffic flows, and mapping traffic to network slices.

[0042] The step of processing the first and second provisional classifications of the network flow and the corresponding confidence scores may include comparing each of the confidence scores with a corresponding confidence threshold, and, responsive to determining that the confidence score is less than the corresponding confidence threshold, waiting to receive a further corresponding provisional classification of the network flow and corresponding confidence score.Also described herein is a network flow classification apparatus for processing data packets of ISP level network traffic to automatically identify in real-time user platforms and content providers of video streams of the network traffic, the apparatus including:

[0043] random access memory; and

[0044] at least one processor configured to execute any one of the above processes.

[0045] Also described herein is a network flow classification apparatus for processing data packets of ISP-level network traffic to automatically classify in real-time network flows of the network traffic, the apparatus including:

[0046] random access memory;

[0047] at least one processor;

[0048] at least one network interface to receive data packets of general network traffic of a plurality of network users of the network service provider;

[0049] a packet-level time-series generator to generate, for each network flow of the received packets, a corresponding time-series representation of each packet of the network flow;

[0050] a slot-level time-series generator to generate, for each network flow of the received packets, a corresponding time-series representation of aggregated packets received within each of a plurality of time slots of the network flow;

[0051] a trained packet-level time series classifier to process each time-series representation of each packet of the network flow and generate a first provisional classification of the network flow as one of a plurality of know flow types and a corresponding confidence score;

[0052] a trained slot-level time series classifier to process each time-series representation of aggregated packets received within each of a plurality of time slots of the network flow and generate a second provisional classification of the network flow as one of the plurality of know flow types and a corresponding confidence score;

[0053] a time-series flow classifier to process the first and second provisional classifications of the network flow and the corresponding confidence scores to determine a confident classification of the flow as one of the plurality of know flow types or as an unknown flow type.The process may further include a network API component to change one or more network settings to effect one or more of the following network reconfigurations: provisioning bandwidth and network capability in accordance with bandwidth demands of the confidently classified flow type, prioritising traffic flows, and mapping traffic to network slices.

[0054] The time-series flow classifier may include long-short-term memory (LSTM) cells trained by reinforcement learning.

[0055] BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Some embodiments of the present invention are hereinafter described, by way of example only, with reference to the accompanying drawings, in which:

[0057] Figure 1 is a schematic diagram of a network flow classification apparatus arranged to receive mirrored packets flowing between an ISP's network and the Internet;

[0058] Figure 2 is a block diagram of the network flow classification apparatus in accordance with an embodiment of the present invention;

[0059] Figure 3 is a schematic diagram of a network flow classification process (also referred to herein as FastFlow) executed by the network flow classification apparatus and in accordance with an embodiment of the present invention;

[0060] Figure 4 is a schematic diagram illustrating how a candidate flow is represented as per packet and per slot time-series data sequences;

[0061] Figure 5 is a schematic diagram showing the architecture of a time-series flow classifier of the network flow classification apparatus and process;

[0062] Figures 6 to 8 are respective schematic graphs illustrating how: (Figure 6) a classifier trained with only known classes can naively split the entire feature space according to the known classes, resulting in (Figure 7) unknown instances being misclassified as belonging to one of the existing classes, and how (Figure 8) augmentation can be used in classifier training to generate closely outlying synthetic unknown flows that help bind the class decision borders;

[0063] Figures 9 to 12 are respective graphs illustrating the performance of classifiers when using time-series statistics from a fixed number of packets or time-interval slots onthree public datasets, where the bounded range of each data point shows the variation of accuracy / Macro-F1 across flow types in each dataset; Figure 9: accuracy when using fixed numbers of packets, Figure 10: Macro-F1 when using a fixed number of packets, Figure 11: accuracy when using fixed time slots, and Figure 12: Marco-F1 when using fixed time slots;

[0064] Figures 13 to 15 are respective cumulative density function (CDF) plots showing the number of packets required by the FastFlow process to classify each flow type of the three public datasets UTMobileNet (Figure 13), UNIBS (Figure 14) and VNAT (Figure 15);

[0065] Figures 16 to 18 are respective cumulative density function (CDF) plots showing the time required by the FastFlow process to classify each flow type of the three public datasets UTMobileNet (Figure 13), UNIBS (Figure 14) and VNAT (Figure 15);

[0066] Figures 19 to 21 are respective cumulative density function (CDF) plots showing the number of packets required by the packet-level classifier of the FastFlow process to classify each flow type of the three public datasets UTMobileNet (Figure 19), UNIBS (Figure 20) and VNAT (Figure 21); and

[0067] Figures 22 to 24 are respective cumulative density function (CDF) plots showing the time required by the slot-level classifier of the FastFlow process to classify each flow type of the three public datasets UTMobileNet (Figure 22), UNIBS (Figure 23) and VNAT (Figure 24).

[0068] DETAILED DESCRIPTION

[0069] In order to address limitations of the prior art such as those described above, the inventors have performed extensive research to investigate possible flow classification processes with improved performance. As a result of this research, the inventors have developed an apparatus and process that enable network operators to accurately classify network flows in general network traffic as one of a plurality of known flow types or as an unknown flow type, with what the inventors believe to be the smallest possible number of packets for each flow, and where that number is dynamically determined at runtime.

[0070] This information can then be used to perform selective processing on selected flow types, and / or to improve the ISPs' network configuration based on the network demands required by specific network flows. Such network reconfigurations can be achieved (dynamically or otherwise) manually by network operators, and / or automatically, using network APIs. In either case, the network configuration can be improved, by, for example, dynamicallyreconfiguring routing paths, provisioning bandwidth, and / or mapping one or more selected flow types to network slices with high bandwidth and low latency.

[0071] In addition, capacity planning is a key ongoing activity for ISPs as they strive to meet the ever-increasing demands for quality service assurance from their customers. A key aspect to this is bandwidth management. Given the significant load imposed by some flow types (e.g., video streaming), the real-time classification of network flows gives ISPs valuable information about bandwidth demands. This information can be used to optimize capacity planning decisions such as right-sizing regions to reduce over- or under-provisioning, create custom plans that allow prioritization of video streams on specific user platforms, and improve caching mechanisms to reduce associated transit costs, for example.

[0072] The flow classification process described herein receives and processes packets of general network traffic to automatically classify its network flows, and in real-time. In particular, the process is able to classify the network flows in extremely high volume network traffic such as those of ISP networks. The packets are processed to generate, for each network flow, corresponding packet-level and slot-level time-series representations of the network flow. The packet-level time-series representations is generated from individual packets, whereas the slot-level time-series representation is generated from aggregated packets that were received at arrival times falling within each of a plurality of successive time slots.

[0073] The two time-series representations are processed by respective classifiers to generate respective packet-level and slot-level classifications of the network flow as being one of a plurality of known flow types, together with respective confidence scores representing the confidence levels of the two classifications. Finally, the packet-level and slot-level classifications and their confidence scores are processed by a time-series flow classifier to determine a confident classification of the network flow as either one of the known flow types, or as an unknown flow type. The confident classification of the network flow may be displayed and / or stored and / or used to reconfigured network components and / or used to process selected network flows as described herein.

[0074] The time-series flow classifier includes long short term memory (LSTM) cells trained by reinforcement learning. The time-series flow classifier determines the confident classification only after at least one of the confidence scores from the packet-level and slot-level classifiers reaches a threshold value, or when a maximum number of timesteps / packets has been reached, thereby dynamically determining the confidentclassification of each network flow from the minimum number of packets of that flow. The use of slot-level classification enables the flow classification process to be robust with respect to packet sequence disorders resulting from packet drops and retransmissions, which render packet-level classification ineffective.

[0075] Accordingly, and as described further below, the flow classification process effectively converts packet streams to precise packet statistics at both per-packet and per-slot granularities. These packet statistics are then input to a sequential decision-based classification model that leverages a long-short-term memory (" LSTM") architecture trained with reinforcement learning. The model dynamically determines the smallest number of time-series data points per flow that provides a confident classification of the flow as one of a plurality of known flow types or as an unknown flow type.

[0076] The flow classification process has been evaluated using public datasets, demonstrating its superior performance for early and accurate flow classification. Deployment insights on the classification of over 22.9 million flows across seven application types and 33 content providers in a university campus network over one week are discussed below, showing that the process requires an average of only 8.37 packets and 0.5 seconds to classify the application type of a flow with over 91% accuracy, and to classify the content provider of the flow with over 96% accuracy.

[0077] In work leading up to the invention, the inventors recognised that existing network traffic classifiers have started shifting away from rule-based mechanisms that match flow metadata, such as port numbers and server name indication (SNI) fields in TLS handshake payloads, because they have become less effective due to the increasing use of encrypted application traffic and randomized or non-standardized port numbers. Consequently, statistical models are increasingly being used to classify network flows based on their timeseries volumetric profiles, which are determined by the actual content carried per application and are agnostic to metadata obfuscation and encryption. However, for a large network provider such as an ISP that can have millions of concurrent flows to be quickly classified for downstream tasks such as usage accounting and performance measurement in real-time, legacy metadata-based approaches can classify a candidate flow by its first or certain signalling packet(s), whereas time-series statistical models often require a lengthy time-series input (e.g., packets) before concluding a confident flow classification result. Consequently, existing statistical models are impractical for use with ISP-level network traffic.In view of the above, to enable the practical deployment of time-series statistical flow classifiers in largescale networks, some models have recently been developed to use a fixed number of initial packets in a flow that carry static signatures in initialization requests pertinent to a certain application (e.g., video streaming or conferencing) or content provider. However, the inventors recognised that, given the diversified flow types in an ISP network, each with its static initial content carried by a different number of initial packets, having this number fixed for every flow cannot always produce a reliable classification. Specifically, the inventors recognised that, with a fixed number, unpredictable packets carrying dynamic user content may be included for flows that inherently require a smaller number of initial packets for classification, and initial packets carrying static flow content might not be sufficiently captured for flows that require a larger number. Moreover, packet drops and retransmissions, which are common in network communications, further render ineffective the use of a fixed number of initial packets for classifying diversified flows in realistic network environments.

[0078] Having identified these limitations of prior art network flow classification processes for large-scale networks, the inventors determined that a dynamic, run-time optimization is required to determine the smallest number of packets for a confident classification result of a given flow. For example, an insufficient number of time-series inputs can cause inaccurate results, whereas a number greater than the smallest number causes needless delays in classification. This recognition motivated the inventors to develop a time-series network flow classifier with sequential decision-making capability trained by reinforcement learning, which dynamically determines the smallest number of time-series data points at run-time to confidently classify each candidate flow.

[0079] As described below, the flow classification process described herein (also referred to for convenience as " FastFlow") represents raw packet streams of a flow as time-series data sequences with respective granularities of packets and slots (i.e., time intervals), and uses time-series classifiers leveraging a long-short-term memory (LSTM) architecture tuned by reinforcement learning techniques to make real-time decisions on the smallest number of time-series inputs required to accurately classify each flow. The process is inherently capable of detecting unknown flow types as outliers instead of mislabelling them as one of the known types, and is robust to packet sequence disorders within each flow due to packet drops and retransmissions, which are common occurrences in large networks.Key Requirements for Flow Classification in Large Networks

[0080] As described above, the optimisation of flow classification performance in large networks with massive throughputs scaling to millions of concurrent flows presents three major requirements / challenges: (i) early classification from the smallest possible number of time-series packet statistics, (ii) detection of unknown flow types, and (iii) robustness to packet sequence disorders. Table 1 below provides a comparison of the FastFlow process described herein and state-of-the-art methods and with respect to these three requirements.

[0081] The state of the art methods are described in the following publications:

[0082] (i) Julien Piet, Dubem Nwoji, and Vern Paxson (2023) GGFAST: Automating Generation of Flexible Network Traffic Classifiers, in Proc. ACM SIGCOMM. New York, USA, 850–866 (" Piet"),

[0083] (ii) Lixuan Yang, Alessandro Finamore, Feng Jun, and Dario Rossi (2021) Deep Learning and Zero-Day Traffic Classification: Lessons Learned From a Commercial-Grade Dataset, IEEE Transactions on Network and Service Management 18, 4 (Sep 2021), 4103–4118 (" Yang"),

[0084] (iii) Jordan Holland, Paul Schmitt, Nick Feamster, and Prateek Mittal (2021) New Directions in Automated Traffic Analysis, in Proc. ACM SIGSAC Conference on Computerand Communications Security. Virtual Event, Republic of Korea, 3366–3383 (" Holland"),

[0085] (iv) Jun Zhang, Xiao Chen, Yang Xiang, Wanlei Zhou, and Jie Wu (2015) Robust Network Traffic Classification, IEEE / ACM Trans. Netw. 23, 4 (Aug 2015), 1257–1270 (" Zhang 2015"),

[0086] (v) Steven Jorgensen, John Holodnak, Jensen Dempsey, Karla de Souza, Ananditha Raghunath, Vernon Rivet, Noah DeMoes, Andres Alejos, and Allan Wollaber (2024) Extensible Machine Learning for Encrypted Network Traffic Application Labeling via Uncertainty Quantification, IEEE Transactions on Artificial Intelligence 5, 1 (Jan 2024), 420–433 (" Jorgensen"),

[0087] (vi) Hassan Alizadeh, Harald Vranken, Andre Zuquete, and Ali Miri (2020) Timely Classification and Verification dNetwork Traffic Using Gaussian Mixture Models, IEEE Access 8 (May 2020), 91287–91302 (" Alizadeh"),(vii) Nen-Fu Huang, Gin-Yuan Jai, Han-Chieh Chao, Yih-Jou Tzang, and Hong-Yi Chang (2013) Application Traffic Classification at the Early Stage by Characterizing Application Rounds, Information Sciences 232 (May 2013), 130–142 (" Huang"),

[0088] (viii) Lizhi Peng, Bo Yang, and Yuehui Chen (2015) Effective Packet Number for Early Stage Internet Traffic Identification, in Neurocomputing 156 (May 2015), 252–267 (" Peng"), and

[0089] (ix) Jun Zhang, Chao Chen, Yang Xiang, Wanlei Zhou, and Athanasios V. Vasilakos (2013) An Effective Network Traffic Classification Method with Unknown Flow Detection, IEEE Trans. Netw. Serv. Manag. 10, 2 (Jun 2013), 133–147 (" Zhang 2013").

[0090] Table 1. Qualitative comparison between the FastFlow process and state-of-the-art methods in terms of the three key requirements of flow classification for deployment in large networks.

[0091] Method Number of timeAble to detect Robust to packet series data points unknown flow sequence types? disorder? Piet fixed input size y^ X

[0092] Yang a unified input size y^ X per TCP / UDP

[0093] Holland fixed input size X X Zhang 2015 a unified size per y^ X network

[0094] Jorgensen fixed input size y^ y^ Alizadeh a unified size per X X network

[0095] Huang a unified size per X X application type

[0096] Peng a unified size per X X network

[0097] Zhang 2013 a unified size per y^ y^

[0098] network

[0099] FastFlow optimal size per y^ y^

[0100]

[0101] candidate flowRegarding the first requirement, network operators seek to minimise the computational resources required to process packets for classification, because these resources increase in magnitude with the number of concurrent flows to be classified. In particular, flow classifiers deployed in large networks may be required to classify millions of concurrent flows. Network operators also require each flow to be classified rapidly for precise postclassification telemetry. Thus there is a need for early classification that aims to provide a prediction for each flow as early as possible while maintaining classification accuracy. This requires determining the smallest number of time-series data points (e.g., packets or slot statistics) for classifying each candidate flow because an unnecessarily large input size would delay classification, whereas a too small number would lead to inaccurate classification. The inventors determined that all state-of-the-art methods use either a fixed input size, such as first 10 packets, or a unified input size for each deployment network or per application type, regardless of the possible variations and complexities in flow profiles during practical deployment.

[0102] Regarding the second requirement, for a flow classifier trained on a labelled dataset of known flow types (e.g., corresponding to applications such as video streaming or conferencing, and content providers such as YouTube or Zoom), processing flows that do not belong to any of the known types is inevitable in practical deployments. Although classifying flows into a finite coarse-grained scope can bypass unknown flow types, such as annotating flows by their network protocols and port numbers, the value of such classification results diminishes for large network operators that require visibility into trending applications and popular providers (e.g., for network optimization). Therefore, flow classifiers should be able to produce fine-grained application and provider-level classifications while also being capable of detecting unknown flow types, rather than mislabelling them as one of the known types. This is known as out-of-distribution classification in the machine learning community for scenarios in which it is infeasible to classify all possible flow types. However, as shown in the third column of Table 1, many state-of-the-art classifiers lack this capability due to their architectures.

[0103] Regarding the third requirement, packet sequence disorders in a flow caused by packet drops and retransmissions are commonly observed in large networks, where network conditions are generally not ideal for every flow, such as client devices served by lossy wireless connections, network congestion or bottlenecks on the routing path, and / or overwhelmed servers dropping packets. Therefore, flow classifiers should be reasonably robust to deviated time-series flow patterns caused by packet sequence disorders. Asshown in the last column of Table 1 and experimentally evaluated below, prior art flow classifiers that precisely match a fixed number of packets for flow classification are not inherently robust to packet sequence disorders. Some prior art methods are arguably robust, but only because they use aggregated statistics over a relatively lengthy interval (e.g., per 40.96 seconds or the entire duration of a flow); however, this inevitably sacrifices the speed of flow classification.

[0104] FastFlow - Flow Classification

[0105] The flow classification process 'FastFlow') described herein addresses all three of the above requirements for deployment in large networks, using a combination of a finegrained packet sequence and a coarse-grained slot sequence to represent time-series statistics per candidate flow. This approach also uses time-series classifiers trained with reinforcement learning to enable dynamic determinations of the smallest number of data points for making a confident classification per flow.

[0106] Figure 1 is a simplified schematic diagram of an embodiment of the flow classification apparatus 102 deployed by an ISP at a gateway 104 between the ISP's network 106 and the Internet 108. Figure 2 is a simplified block diagram of the flow classification apparatus 102 of the described embodiment, and Figure 3 is a high-level schematic diagram of the flow classification process executed by the flow classification apparatus 102 to classify network flows.

[0107] As shown in Figure 1, an ISP provides its customers 110 with access to the Internet 108. In a typical scenario, each ISP customer 110 has a wireless and / or hardwired (copper wire or optical fibre) connection to the ISP's network 106, from which the broader internet 108 can be accessed via a gateway 104 of the ISP. Thus all network traffic to and from those customers passes through the ISP's gateway 104 and network 106. In a typical example, at any given time the ISP may serve hundreds of thousands of users 110 using diverse applications supported by millions of concurrent network flows. The challenge therefore is to accurately classify those millions of concurrent network flows as rapidly as possible -and in real-time, to enable subsequent dedicated real-time processing of selected flow classes, and / or dynamic optimisation of the ISP's network 106 to provide high quality network services with low latency and high throughput.In the described embodiments, the flow classification apparatus 102 is a blade server configured with an 8-core Intel Xeon E5-2620 CPU 202, as shown in Figure 2, and the flow classification processes executed by the apparatus 102 are implemented as executable instructions of one or more software modules 204, as shown in Figure 3, stored on nonvolatile (e.g., hard disk or solid-state drive) storage 206 associated with the blade server. However, it will be apparent to those skilled in the art that in other embodiments at least parts of the flow classification processes described herein can alternatively be implemented in one or more other forms, such as configuration data of one or more field programmable gate arrays (FPGAs), or as one or more dedicated hardware components, such as application-specific integrated circuits (ASICs), or as any practical combination of these various hardware and software forms.

[0108] In the described embodiments, the flow classification apparatus 102 includes 64GB of DDR4 random access memory (RAM) 208, a bus 210, and two 10 Gbps network interface connectors (NIC) 212, which respectively receive upstream and downstream network traffic between the ISP's customers 110 and the Internet 108. In the described embodiments, the flow classification apparatus 102 also includes a number of standard software modules 214 to 218, including an operating system 214 such as Linux or Microsoft Windows, and structured query language (SQL) support 216 such as PostgreSQL, which allows data to be stored in and retrieved from an SQL database 218.

[0109] Finally, the apparatus 102 includes Network APIs 220 to allow the apparatus 102 to change network settings to improve streaming network performance of selected flow types as required, based on the flow classifications. In particular, the ISP can configure the flow classification processes 204 to use the Network APIs 220 to automatically reconfigure the ISP's network 106 to satisfy the different network demands required by some flow types, by mapping those flow types to network slices, provisioning sufficient bandwidth, and / or configuring high-priority routing paths for flow types with high network demands.

[0110] The ISP monitors the network traffic of its users 110 by mirroring network packets as they pass through the ISP's gateway 104 to the internet 108. Mirrored packets are sent to the flow classification apparatus 102 for processing by the flow classification process. Packets received by the flow classification apparatus 102 are first assigned to corresponding network flows using any suitable method known to those skilled in the art. For example, in the described embodiments, the flow classification process uses the method described in Y. Wang, M. Lyu, and V. Sivaraman, Characterizing User Platforms for Video Streaming in Broadband Networks, in Proceedings of the 2024 ACM on Internet MeasurementConference or M. Lyu, R. D. Tripathi, and V. Sivaraman, MetaVRadar: Measuring Metaverse Virtual Reality Network Activity. Proc. ACM Meas. Anal. Comput. Syst. However, it will be apparent to those skilled in the art that other methods may be used in other embodiments.

[0111] As shown in Figure 3, the packets 302 of each resulting candidate ( / .e., unclassified) flow 304 are processed to generate a corresponding time-series sequence 306 containing (packet level) statistics of the individual packets 302, and a corresponding time-series sequence 308 of (time interval or slot level) statistics of aggregated packets per (fixed duration) time slot. As described below, the use of both packet-level 306 and slot-level 308 statistics combines the advantages of the precise flow statistics provided by packet sequence under ideal network conditions, and the robustness to packet sequence disorders provided by interval-based aggregation.

[0112] For each candidate flow 304, its time-series sequence of packet-level statistics 306 is input to a time-series classifier 310 for packet-level statistics, and its time-series sequence of slot-level statistics 308 is input to a time-series classifier 312 for slot-level statistics. Each of these classifiers 310, 312 generates a corresponding classification output 314, 316, including a corresponding flow type of the candidate flow 304 and a corresponding confidence score of that classification.

[0113] In contrast to prior art methods, each of the classifiers 308, 310 is trained with reinforcement learning techniques instead of supervised learning so that they are capable of making confident classifications from the smallest number of time-series inputs 310, 312 for each candidate flow 304, and also identifying flows that do not belong to the known flow types.

[0114] The respective classifiers 310, 312 for packet and time interval real-time sequence data 310, 312 can produce outputs 314, 316 that are unsynchronized. For example, a flow with packet drops might not provide a confident classification result 314 until a relatively large number of packets is received. In contrast, the data sequence representing multiple packets aggregated over a time interval 308 can more rapidly provide prediction results 316 with relatively high confidence. Accordingly, if the outputs 314, 316 of the classifiers 310, 312 include at least one high-confidence classification score, then the corresponding classification is selected 318 as a robust classification result 320 per flow in the least possible time.Time-Series Flow Data Sequence at Packet Granularity

[0115] Under good network conditions, network flows are expected to have no packet loss or disorder in their packet sequences caused by packet drops and retransmissions. Certain types of network flow, such as those of video streaming or online gaming, and certain specific applications of these types, or different user device operating systems (e.g., iOS and Android), often exhibit unique sequences in their packet directions, sizes, and interpacket arrival times. As previously revealed by the inventors (see, for example, M. Lyu, R. D. Tripathi, and V. Sivaraman, MetaVRadar: Measuring Metaverse Virtual Reality Network Activity, Proceedings of the ACM on Measurement and Analysis of Computing Systems 7, 3 (Sep 2023), 1-29, and S. Madanapalli, H. Gharakheili, and V. Sivaraman, Know Thy Lag: In-Network Game Detection and Latency Measurement, Proc. International Conference on Passive and Active Network Measurement, Virtual Event, 395-410), this is particularly true for their initial packets that carry static content (e.g., requested services or application metadata) in contrast to subsequent packets that vary in dependence on the user's actions.

[0116] Therefore, to precisely preserve the packet-level statistics, the fine-grained time-series packet-level representation 306 of a candidate flow is defined as:

[0117] P= [P1. P2,

[0118]

[0119] (1)

[0120] where -> represents statistics of the nth packet in this flow, including packet direction dirn), Pn

[0121] packet size (sn), and inter-arrival time (Δtn), expressed as:

[0122] p

[0123]

[0124] n ~ n $n

[0125] Figure 4 is a schematic illustration of the conversion of an on-going candidate flow 304 (top row of Figure 4) into corresponding per-packet (middle row) 306 and per-slot (bottom row) 308 time-series representations. The flow 304 consists of both upstream (T) and downstream (l) packets, as indicated by the vertical arrows, the arrival time between successive packets pnand pn+1being given by Atn+1.

[0126] Statistics for every packet of the candidate flow 304 that has arrived until the current timestamp are included in a corresponding runtime array 306. This fine-grained timeseries flow representation 306 with packet attributes in a matrix format is compatible with known time-series classification models such as Long-Short Term Memory Cell (LSTM) withoutrequiring any additional overhead for pre-processing. Specifically, the packet direction dirnis represented as 1 for upstream or 0 for downstream, the packet size snis calculated as the packet payload size excluding the network and transport layer headers in the unit of MTU (i.e., sn= payloadn / MTU), and the packet inter-arrival time is included as its log-scale conversion ( / .e., log(Atn+1)).

[0127] Time-Series Flow Data Sequence at Slot Granularity

[0128] The per packet sequence p defined in Equation 1 works well for time-series flow classification when the packet-level profiles precisely match their expected patterns learned during the training process. However, this nearly ideal assumption is not realistic when the expected packet sequence of a candidate flow is disordered due to packet drops and retransmissions caused by practical factors such as network, client, and server conditions. To address such occurrences, the flow classification process also generates the coarse-grained time-series representation 308 for each candidate flow with each data point representing aggregated packets arriving within a corresponding time slot of fixed duration 6. This representation better handles packet sequence disorders caused by non-ideal network conditions, and preserves both volumetric and temporal information, at the expense of losing visibility into each packet. The inventors have found that temporal information of a flow, such as the burstiness of the arrived packets, is also an effective metric for early classification.

[0129] With a first packet of a flow arriving at a timestamp 0, and a time-interval slot of fixed duration 6 for aggregation, the slot time-series representation 308 of a flow is defined as:

[0130] V — [cJo-tJb ^<5— 2<sb • • • »

[0131]

[0132] — - - where \n-\}8-nb represents the statistics aggregated from the packets that arrived between the timestamps of (n - 1)5 and n6.

[0133] Aggregating packet statistics per time-interval slot

[0134] In the described embodiments, each aggregation data point of the time-series is generated as follows. For simplicity of description, this data point is written as v in the following, omitting its index.The aim is to explicitly preserve important contextual characteristics of a network flow that are inherently impacted by its functions and types, including packet directions, sizes of light and heavy packets, and dominant data transmission direction. Also, the representation should be computationally lightweight to allow generation in real time, e.g., using online algorithms. To meet these criteria, each aggregated data point v is calculated as the following matrix:

[0135] T

[0136]

[0137] (4)

[0138] $(••?> s / i and

[0139] where

[0140]

[0141] ’: 1respectively denote the average packet payload sizes for upstream heavy packets, downstream heavy packets, upstream light packets, and downstream light packets, and the ratio between total upstream and downstream packet sizes, over the corresponding time-interval slot of duration 6 (with a default value of 100 ms in the described embodiments). The average function is used for packet payload sizes because it is statistically relevant and can be computed using online algorithms in real-time. The light and heavy packet payload sizes are defined by a threshold value that can be set empirically by the network operator for each deployment network. In the described embodiments, a threshold value of 1200 bytes is used to group packet payloads into light and heavy categories.

[0142] Selecting Flow Classification Results at Real-Time

[0143] As shown in Figure 3, the time-series data sequences 306, 308 of a candidate flow 304 at respective granularities of packet and time slot are input to their respective classifiers 310, 312, each of which generates a corresponding classification result 314, 316 of flow type and a corresponding classification confidence score. As described in detail below, the classifiers 310, 312 process their respective packet and slot time-series data sequences 306, 308 for each candidate flow asynchronously. For example, a flow without packet sequence disorders can be confidently classified by the packet sequence classifier 310 when the fifth packet arrives at a 0.5-second timestamp. In contrast, the slot classifier 312 might not produce its results until a 3-second timestamp. Alternatively, a flow with packet sequence disorders might never be confidently classified by its packet sequence classifier 310, whereas the slot classifier 312 might generate a confident result within several seconds. In addition, both classifiers 310, 312 can produce respective confidentresults without noticeable time differences, i.e., synchronously. Therefore, to select the most accurate classification label for a candidate flow at the earliest possible timestamp, a real-time output selector 318 executes a real-time selection process to select the first available confident classification result generated by the packet and slot classifiers 310, 312, asynchronously or synchronously, for a candidate flow 302. Pseudo-code representing the real-time selection process is provided below.

[0144] The real-time output selector 318 receives flow classification results 314, 316 and their confidence scores 318, 320 that are generated in real-time from both packet and slot sequence classifiers 310, 312. As described below, the packet sequence classifier 310 generates a packet-level classification 314 whenever a new packet arrives, whereas the slot classifier 312 only generates a slot-level classification 316 per time interval (encompassing multiple packet arrivals). The real-time output selector 318 continuously checks the packet-level classification 314 and, if available, the slot-level classification 316 against respective preset confidence thresholds TPand Ttto select a flow type classification with high confidence. In the described embodiments, this is achieved using a 90th-percentile of all confidence values obtained during the training process. A practical selection time window ^select (with a default value of 100 ms in the described embodiments) is also used to recognize synchronously and asynchronously generated results from both classifiers 310, 312.

[0145] For each asynchronous event, i.e., a flow classification result is generated by one of the classifiers 310, 312, and no result is generated by the other classifier within the selection time window ^select, as specified by blocks (1) and (2) of Algorithm 1 below, the classification confidence score is compared to the corresponding threshold to decide whether to accept the predicted flow type, or to wait for more classification results to be generated. For each synchronous event processed by block (3) of Algorithm 1, the classification results 314, 316 from both classifiers 310, 312 are first compared to each other before being compared to their respective confidence thresholds for a decision. In the described embodiments, the final confidence score is increased by 20% whenever both classifiers 310, 312 predict the same flow type.Algorithm 1: Selecting flow classification results produced by packet and slot classifiers nearly synchronously or asynchronously.

[0146] Input, (classp, conf, <— classification results with confidence scores on packet flow data sequence, (classt, conf <— results on slot flow data sequence, (Tp, Ti) <— confidence thresholds for packet and slot flow data sequence, Sselect <— user-defined selection time window;

[0147] Output, (classs, confs) <— the selected flow classification result and its confidence score; Real-time flow classification result selection process:

[0148] / / asynchronous event: a new result from the packet sequence classifier (t) if an arrived (cdasSp, confi, and no (class,, Confi) arrive within Asc ecf then

[0149] if confp > Tp then

[0150] j return {classs, confi) (classp, confi,) / / confident flow type selected and exi t else

[0151] I wait for another data point to arrive

[0152] end

[0153] end

[0154] / / asynchronous event: a new result from the slot sequence classifier

[0155] (s) if an arrived (class,, confi) and no (clasSp, COtlfp) arrive within Asclect then

[0156] if con ff> Ttthen

[0157] | return {classs, confi ) ♦— (class,, confi ) / / confident flow type selected and exit

[0158] else

[0159] wait for another result to arrive

[0160] end

[0161] end

[0162] / / synchronous event: new results from both classifiers

[0163] {») if a (classp, confi, ) and a (class / , confi ) arrive within Aselect then

[0164] if confp > confi and con fi, > Tp then

[0165] | return (classs, confi) ^— (classp, confi,) flow type by pocket sequence selected and exit else

[0166] | wait for another result to arrive

[0167] end

[0168] if conft> confpand conft> Ttthen

[0169] | return (classs, confs) ← (classt, conft) / / flow type by slot sequence selected and exit else

[0170] wait for another result to arrive

[0171] end

[0172] endTime-Series Flow Classifier with the Shortest Data Sequence

[0173] In order to dynamically determine the shortest data sequences required for accurate classification, the packet-level and slot-level classifiers 310, 312 are time-series (LSTM-based) classifiers. The LSTM classifiers 310, 312 trained by a reinforcement learning technique on augmented labelled training datasets.

[0174] Time-Series Classifier Architecture for Early Flow Classification

[0175] The classifiers 310, 312 are expected to process a large number of concurrent flows, timely and accurately. The classifiers 310, 312 provide early predictions using the smallest possible number of time-series inputs (i.e., packets or slots) for each flow. However, as there is no single optimal number of time-series inputs for all flows, a one-fits-all approach would not provide optimal results in practice.

[0176] Prior art flow classification methods typically train their models with a standard supervised approach that requires a fixed number of timesteps that is considered to be a model hyperparameter optimized during training. Therefore, such methods process the same amount of input for every flow, and cannot adapt to the requirements of each candidate flow. However, in work leading up to the invention, the inventors determined that the optimal time-series input length required for each flow varies according to both the classification objective and minor variations in the data sequence caused by network conditions. Moreover, traditional classifiers make a "closed-world" assumption that, after deployment, they will only observe the class labels present in the training set. However, this is an unrealistic assumption given the number and diversity of flows in operational networks. Consequently, these classifiers are constrained to misclassify unknown flow types, unseen in the training data, as one of the known flow types.

[0177] Although some existing approaches can address this open-world assumption, they have additional requirements such as additional data, extra computing power, or constraints on data distribution. In contrast, the FastFlow process described herein addresses both requirements with a uniquely elegant solution: the real-time output selector 318 that takes one of the following two actions after receiving a new time-series data point: output a prediction, or wait for the next data point, based on the classification confidence up to the current data point. This approach has two major benefits: (i) a prediction is issued as soon as it has gained enough confidence, and (ii) if no prediction is issued after a configurablenumber of data points (with a default value of 20 in the described embodiments), the classifier outputs the class label "unknown".

[0178] Figure 5 is a schematic diagram showing the architecture of each classifier model. The part 502 of the model trained with reinforcement learning includes an LSTM cell 504 that provides a feature set 506 at each time step. This feature set 506 is fed into a fully connected network 508 that outputs one of the known classes or an "unknown" label.

[0179] As described in A. Graves and A. Graves, Long short-term memory, in Supervised sequence labelling with recurrent neural networks (Feb 2012), 37-45, LSTM classifier architectures are able to process sequential data. Traditional LSTM-based classifiers extract features from a fixed number of timesteps, and feed these features to a softmax layer to obtain scores for each known class label. As shown in Figure 5, the classifier architecture of the FastFlow process and apparatus differs from the prior art in two important ways. First, instead of waiting for a fixed number of timesteps, features are extracted from the LSTM every time a new data point arrives, and are used by an inference linear layer 508 to generate the classification confidence scores for the candidate classes. Second, the inference linear layer 508 is also able to classify a flow as an 'unknown' flow type in addition to the known flow types. Consequently, given a new flow, the model can output a temporary unknown label for the initial timesteps, and then wait for more data points of the same flow until either a high confidence classification can be made, or a configurable maximum number of timesteps (with a default value of 20 in the described embodiments) has been reached. The run-time decision at each time step is made by a dynamic inference process 510 of the output selector 318, as represented by the pseudocode of Algorithm 2 below.Algorithm 2: Dynamic inference on time-series flow data sequence.

[0180] input: Confidence scores of flow types c̃tfrom the classifier, after processing the tthdata point

[0181] in the flow data sequence;

[0182] confidence threshold Tunk, and the maximum time steps to make prediction Cunk.

[0183] output: Predicted flow type I and the prediction confidence p.

[0184] c̃t← softmax(s̃t)

[0185] i

[0186]

[0187] f t — then

[0188] / / End flow classification as the maximum number of time step

[0189]

[0190] has been reached. i I «— ‘unknown’;

[0191] i P <■■■■ ['unknown'] i return I, p end if c, [ ’unfcnowre’] > I',,,,;- or argmax(ct ) — unknown' then j / / ysrr sjcf hicc-: crccsih tc c 'iwt ijcicccwc’ tweikn®, wait fww i I « — ‘unknown’

[0192]

[0193] i p «■■■■ cf[‘unknown’] i continue for the nest time step f + 1.

[0194] else

[0195] I / / Confident enough to make a ‘not unknown’ predict ion

[0196] l ← argmax(c̃t)

[0197] |

[0198]

[0199] i n

[0200] | return I, p

[0201] end

[0202] For each incoming time-series data point, the dynamic inference process 510 processes the confidence scores of the k known classes and the unknown class. If the confidence score of the unknown class is greater than the corresponding threshold value Tunk, or if it is the most confident class, a temporary unknown classification is made, and classification continues (512) for the next time series data point. Otherwise, the most confident class among the k known classes is output (514). The process 510 executes until an output is issued (514) or the number of processed time-series data points reaches a configurable maximum number Cunk. Both thresholds, Tunkand Cunk, are hyperparameters. In the training process, Cunk is tuned to provide a balance between prediction accuracy and speed.Training FastFlow Classifiers with Reinforcement Learning

[0203] In the described embodiments, the network flow classification task is treated as a sequential decision problem, making traditional supervised learning methods unsuitable for training the packet-level and slot-level classifier models. Traditional supervised learning defines a loss function that is computed for every training case. Therefore, these methods assume that every case should immediately lead to some loss after classification. In contrast, the dynamic inference process introduces a postponing decision that does not immediately lead to a loss of value.

[0204] Reinforcement learning (RL), as described in A. G. Barto, R. S. Sutton, and CJCH Watkins, Learning and Sequential Decision Making, 1989 Technical Report, University of Massachusetts, Amherst, MA, 1989, is better suited to training the models. RL involves an agent (a sequential decision-making classifier) that observes the current environment state and chooses to take an action from a predefined action space. Based on the selected action, the environment updates its current state. Each time the agent / classifier selects an action, it gets a reward based on its fit to the current state. This process repeats until the environment arrives at a terminal state, where the agent cannot change the environment's state further. RL learning aims to maximize the total reward granted to the agent from the start state to the terminal state.

[0205] Although any one of the many popular RL algorithms may be used, for the described embodiments the inventors chose the Q-learning algorithm described in C. J. C. H. Watkins and P. Dayan, Q-learning, in Machine Learning 8, 3 (Jul 1992), 279-292 (" Watkins"), since the flow classification leads to a discrete action space. Q-learning is also known to be sample efficient, which is crucial when collecting labelled data is expensive. Finally, Q-learning has had several enhancements over the initial version described in Watkins, including its model architecture structure, loss function, and training process, which have improved its performance. Q-learning trains the agent to choose the best action from the action space given a state. Given a current state s as input, the agent outputs a number (Q value) for each action potential a e A in the action space A. The Q-value Q (s, a) represents the expected reward for taking action a in state s. Therefore, after training, the agent always picks the action with the maximum Q-value given an input state.In the FastFlow classification process, the state is represented by the first p timesteps of the flow representation (in both the packet and the slot representations). The action space for a fc-way classification problem has k + 1 actions, one for each flow type, and the k + 1th action for the unknown class. The terminal state occurs when the agent chooses to make a prediction, or when it has processed the maximum number of time-series data points, Cunk. The reward given to the agent is decided by a reward function that takes the current state s, the action a, and the actual class label I of the current flow, according to:

[0206] wait_penalty if a == k + 1

[0207] positive_reward else if a == l negative_reward otherwise

[0208]

[0209] (5)

[0210] In the training process described herein, the reward function outputs a positive reward (+1) if the classifier outputs a correct prediction, and a negative reward (-1) for an incorrect prediction. However, if the classifier decides to output 'unknown' and continues to the next timestep, a negative reward 'wait_penalty', a hyperparameter that determines the speed of classification, is awarded. The value of 'wait_penalty' is set as '-.03' in the described embodiments. Lowering its value places more importance on classification speed than accuracy, and vice versa. The model is then trained using double Q learning loss with sampling priority (as described in T. Schaul, J. Quan, I. Antonoglou, and D. Silver, Prioritized Experience Replay, in Proc. International Conference on Learning Representations, San Juan, Puerto Rico, and in H. van Hasselt, A. Guez, and D. Silver, Deep Reinforcement Learning with Double Q-learning, in Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, USA, 2094–2100) that inputs tuples of (state, action, next_state, reward).

[0211] Training Flow Data Augmentation

[0212] A final component of the FastFlow process involves the generation of flows with unknown labels. In many real scenarios, these unknown flows are readily available, consisting of all flows that do not belong to the known classes. However, in practice, collecting these flows is challenging. For example, these flows might need to be manually labeled to guarantee that they do not fall into one of the existing classes. Another practical challenge involves training models using benchmark datasets, as most of these datasets do not include a general unknown class.A solution to these problems is to generate synthetic unknown flows using augmentation techniques. Augmentation is a widely used approach in deep learning to solve data deficiency, as several deep models have a large number of parameters that require extensive training sets. To this end, a strong augmentation methodology was used to generate synthetic unknown flows away from the high-density areas that characterize each known flow type.

[0213] Figures 6 and 7 are schematic graphs illustrating classification by a classifier trained on known classes, showing classification of inputs belonging to those classes (Figure 6), and misclassification of additional inputs belonging to unknown classes. Figure 8 shows classification of the same inputs, but where the class borders have been refined. This simplified example illustrates the use of strongly augmented flows to restrain the decision border around each class, so that the trained classifier is capable of detecting flow instances belonging to the unknown types (i.e., not statistically belonging any known class) with the refined orders of each flow type.

[0214] Specifically, this synthetic unknown flow augmentation is achieved by distorting the attributes of packets before converting them into packet or slot flow data representation. This process starts by sampling αattr∈ [0,1] fraction of packets in each flow. The payload size, pst, direction, din, and timestamp, tt for each sampled ith packet are then distorted as follows:

[0215] psiaug= psi* αps+ U(0, MTU) * (1 - αps) (6)

[0216] tiaug= ti+ ((t

[0217]

[0218] i+1- ti) * U(0, 1) - (ti- ti-1) * U(0, 1)) * αts(7)

[0219] where αps∈ [0, 1] and αts∈ [0, 1] are the strength of payload and timestamp distortion, respectively. U(a, b) denotes the uniform distribution over the range [a, b]. To apply augmentation for the attribute din, the direction of a flow is randomly reversed with probability adir. In practice, aps, ats and adir are set to '0.2' after experimental selections. aattr is sampled from U (0.6, 0.9) to generate highly augmented pseudo unknown flows. This augmentation over attributes using Fattr, is formally defined as follows:a

[0220]

[0221] ugmente<l_traiiiing_flow ~ (flow__knaivn, a;tte, a / ?s, a / 5, adir) (8)

[0222] Notably, the pseudo unknown flows used in the classifier training process are labelled 'unknown'. Therefore, during training, the classifier always receives a negative reward when it misclassifies these flows into one of the known types. This enhances the unknown flow detection capability of the trained classifier. A weaker form of augmentation is also provided by setting αattrfrom U (0, 0.2) to increase the size of the training data and mitigate the class imbalance. Both approaches were found to be effective using public datasets and deployment in the inventors' university campus network.

[0223] EXAMPLES

[0224] Evaluation, Benchmarking and Deployment Insights

[0225] The inventors evaluated the performance of FastFlow for achieving two primary objectives: i) classifying network flows accurately and rapidly through the use of the smallest number of packets per flow, and ii) maintaining classification robustness to packet sequence disorders and unknown flow types commonly found in practical deployments.

[0226] To achieve these objectives, the inventors chose three popular public datasets containing labeled packet captures (PCAPs) of network flows for various network types, namely the VNAT dataset from MIT Lincoln lab for VPN networks, the UTMobileNet dataset for mobile networks, and the UNIBS dataset collected from the campus network edge of the University of Brescia. Table 2 below lists the flow types in each dataset.

[0227] A common limitation of these public datasets is that they were collected under nearly ideal network conditions, with no packet drops or retransmissions in each labelled flow. Also, they do not contain unknown (unlabelled) flows. Consequently, these datasets were modified by introducing packet sequence disorders and some flow types were randomly excluded to mimic unknown flows in the evaluation. To simulate packet sequence disorder, a percentage αdrop∈ [0, 100] of packets in the flow were randomly dropped. For TCP flows, these packets were reinserted after a variable retransmission delay. In practice, the retransmission delay was set as the RTT calculated by a three-way handshake. The drop probability adrop of each packet was randomly selected in accordance with a normal distribution with a mean value of 5% and a standard deviation of 3.5%, as suggested bythe network operation community to model a scenario with more realistic network conditions.

[0228] Table 2: Specification of the three public network flow datasets used in the evaluation.

[0229] UTMobileNet UNIBS

[0230] Flow type #flows Flow type #flows

[0231] Google-Maps 2584 Browsers 18820 Netflix 1680 P2P 14175 Reddit 1295 Mail 4432

[0232] Youtube 1031 Other 2368

[0233] Facebook 1028 Skype 499

[0234] Instagram 994

[0235] Pinterest 994 VNAT

[0236] Google-Drive 709 Streaming 1628

[0237] Spotify 699 Chat 811

[0238] Twitter 682 Control 611

[0239] Gmail 400 File Transfer 456

[0240] Hangout 351

[0241] Messenger 329

[0242]

[0243] To assess the detection of unknown flow types, around 20-25% of the flows were randomly excluded from the training set for each training and evaluation iteration. More precisely, three flow types were removed from the UTMobileNet dataset, and one flow type from each of the UNIBS and VNAT datasets. Ten evaluation iterations were performed for each dataset, with each flow type excluded from the training set at least once. During each evaluation iteration, 70% of the data was randomly selected and used for model training, the remaining 30% of the data being used for testing.

[0244] These augmentations were chosen to give existing datasets more realistic data characteristics. To ensure that the FastFlow process was not favored by the augmentations, the packet drops were only introduced in the test data. Thus, neither FastFlow nor its competitors had information about the packet drop rate during training.For completeness, results of the non-augmented dataset are also presented below to provide a complete picture of FastFlow performance with all combinations of presence and absence of packet drops and unknown flows.

[0245] Different performance metrics were used in the assessments. Accuracy and the "macro Fl" metric are reported for identifying known flow types. Accuracy is a prevalent performance measure, but it is difficult to interpret in the presence of several flow types and when some are uncommon. The " Macro Fl" metric is the unweighted average of the Fl predictive performance measure for each flow. This measure favours classifiers that perform well for all flow types, independently of the number of flows. Both measures are shown as percentages, with higher values indicating better classification performance.

[0246] Also reported are the false positive rate (FPR) and the true positive rate (TPR) to assess the recognition of unknown flows. The FPR measures the percentage of unknown flows incorrectly assigned to one of the existing flow types. The TPR measures the percentage of unknown flows correctly recognized as such. Finally, the number of packets and inference time in seconds are also reported as measures of inference efficiency.

[0247] The hyper-parameters for the FastFlow' LSTM cells were set using standard tuning processes, including a hidden size of 128, an Adam optimizer (as described in Diederik P. Kingma and Jimmy Ba, Adam: A Method for Stochastic Optimization, in ArXiv) with a learning rate of 3E- 4, and a capped training epoch of 200. The static parameters of the FastFlow classifier architecture are empirically tuned for a balanced classification and speed per deployment network (or dataset). For example, Cunk is set to 20, 30, and 15 for UNIBS, UTMobilenet and VNAT dataset, respectively. In the campus deployment, this value is set to 25 during the training process. Tunk and wait_penalty were consistently set to 0.8 and -0.03, respectively.

[0248] Evaluation and Comparison to the State-of-the-art

[0249] The performance of the FastFlow process was evaluated and compared to two of the state-of-the-art network flow classification methods recently developed by the research community: GGFast and Grad-BP (as described in Piet and Yang, respectively), as well as some baselines. Both GGFast and Grad-BP have been practically deployed in large networks as discussed in in Piet and Yang, respectively. GGFast classify flows by theirpacket length sequences, and Grad-BP uses deep learning models to classify flow types by their packet lengths and directions in time-series sequences.

[0250] Table 3 below summarizes the evaluation results. The FastFlow process outperforms GGFast and Grad-BP in terms of classification performance for known flows, and inference efficiency. GGFast performs better than FastFlow for identifying unknown flows. However, GGFast's small decrease in FPR and increase in TPR for unknown flows comes with a significant decrease in performance for the classification of known flows. For example, for the VNAT dataset, GGFast macro Fl is only 70.45% where FastFlow achieved 96.24%. Similar results can be observed for the other datasets.C la s sification performance on known flow type s Unknown types Dataset Method

[0251] Macro Fl (%) Accuracy (%) Packets (#) Time (s) FPR (%) TPR (%) FastFlow 85.42 86.32 12.92 ± 9.48 0.57 ± 1.61 4.56 86.94 UTMobileNet GGFast

[0042] 77.82 83.90 50 2.56 ± 0.49 1.04 90.42

[0252] Grad-BP

[0064] 80.33 83.85 100(TCP) 10(UDP) 4.86 ± 1.98 4.92 61.35 Pkt.-S 65.28 74.20 5 0.28 ± 1.07

[0253] Pkt.-45 80.02 81.17 45 2.40 ± 1.24

[0254] Time-int.-S 73.39 75.96 4.39 ± 1.96 0.25

[0255] Time-int.-45 85.29 87.05 42.02 ± 9.23 2.25

[0256] FastFlow 96.24 98.03 4.16 ± 1.34 0.033 ± 1.43 0 97.32 VNAT GGFast

[0042] 70.45 86.68 50 1.08 ± 0.42 0 99.47 Grad-BP

[0064] 95.64 95.91 100(TCP) 10(UDP) 1.91 ± 0.56 4.98 93.07 Pkt.- 5 63.53 71.48 5.035 ± 0.81

[0257] Pkt.’4S 80.41 85.66 45 0.96 ± 1.59

[0258]

[0259] Time-int-5 96.48 97.59 14.22 ± 5.42 0.25

[0260] Time-int.-45 95.20 97.45 100.47 ± 23.44 2.25

[0261] FastFlow 92.30 95.21 9.92 ± 3.92 0.36 ± 0.82 2.54 98.16 UNIBS GGFast

[0042] 87.93 91.95 50 1.52 ± 1.87 0.87 99.37 Grad-BP

[0064] 90.15 93.46 100(TCP) 10(UDP) 2.95 ± 1.97 4.93 68.45 Pkt.- 5 81.49 87.02 5 0.19 ± 1.18

[0262] Pkt.-45 90.53 94.45 45 1.36 ± 1.46

[0263] Time-int.-5 81.91 82.84 6.83 ± 2.80 0.25

[0264] Time-int.-45 92.36 95.55 75 ± 17.15 2.25

[0265]

[0266] Table 3: Classification performance comparison between FastFlow, state-of-the-art and baseline

[0267] methods under conditions of packet sequence disorder and unknown flow types.Table 3 also shows some baselines that represent families of methods in the literature. For example, several prior works classify network flows using a fixed number of timeseries data points of either packets or time-interval slots. Accordingly, Table 3 includes the performance of standard LSTM time-series classifiers that take a fixed number of initial flow data points. These classifiers use the first five packets (labelled as 'Pkt. -5' in Table 3), the first forty-five packets ('Pkt.45'), the first five slots ( 'Time-int.-5'), and the first forty-five slots ( 'Time-int.-45'). FastFlow outperforms most of these baselines. Pkt. -5 clearly does not have enough information to make precise classifications. Pkt. -45 shows that increasing the number of packets does improve classification performance but not enough to achieve FastFlow's performance. The performance of packet classifiers with more than 45 packets was not investigated since the inference times for Pkt. -45 are already much larger than FastFlow. Slot classifiers perform better than packet classifiers. Overall, five slots are not enough to guarantee good performance, but with 45 intervals, the classifiers can reach similar (or even slightly better) classification performance to FastFlow.

[0268] However, this comes at the expense of a longer inference time. Also, not all baselines can handle unknown flows, so their performance in this task cannot be measured. In addition to the two fixed numbers (i.e., 5 and 45) of packets or slots for flow classification, other fixed numbers from 5 to 45 with a step of 5 were also tested. The results, including accuracy and macro Fl score, are averaged for each dataset and are shown in Figures 9 to 12. It is apparent that FastFlow outperforms all settings with fixed number of inputs in both accuracy and macro Fl score.

[0269] It can also be observed that using more initial packets (Figures 9 and 10) or time-interval slots (Figures 11 and 12) to classify a flow does not necessarily improve accuracy, validating the merit of the FastFlow methodology that dynamically determines the smallest number of packets or slots required to accurately classify each candidate flow.

[0270] Ablation Study with Only Packet or Slot Flow Data Sequence

[0271] Described below are additional analyses that provide a better understanding of FastFlow performance, starting with an ablation study that characterizes FastFlow's performance with either packet or slot representation.As described above, FastFlow collectively uses two data representations: packet and slot data sequence. The importance of using both representations (instead of only one) was also assessed by excluding one of the time-series classifiers 310, 312 trained over the respective one of the flow data representations 306, 308. Table 4 below summarizes the results, where Packet seq. stands for a FastFlow version using only a packet representation and Time-int. seq. using only a slot representation.

[0272] This assessment shows that the slot representation is the best representation to classify known flows, and the packet representation performs best for unknown flows. In terms of inference time efficiency, the packet representation is the best. By design, FastFlow inherits the best of each representation. Its known flow classification performance is inferior but close to the slot representation. Conversely, its unknown flow identification is close to the packet representation, which performs best. Regarding inference efficiency, FastFlow is faster than the time representation but slower than the packet representation.Classification performance on known flow ty pes | Unknown types Dataset Method Macro Fl (%) Accuracy (%) Packets (#) Time (s) | FPR (%) TPR (%) FastFlow 85.42 86.32 12.92 ± 9.48 0.57 + 1.61 4.56 86.94 UTMobileNet Packet seq. 79.08 79.81 8.71 ± 4.05 0.51 ± 0.43 3.63 88.24

[0273] Time-int. seq. 87.54 88.61 26.30 ± 9.10 1.63 ± 0.11 4.68 83.21 FastFlow' 96.24 98.03 4.16 ± 1.34 0.033 ± 1.43 0 97.32 VNAT

[0274] Packet seq. 94.29 96.49 3.97 ± 1.73 0.006 ± 1.18 0 99.24 Time-int. seq. 99.40 99.61 7.93 ± 5.06 0.093 ± 1.32 0 98.78 FastFlow 92.30 95.21 9.92 ± 3.92 0.36 ± 0.82 2.54 98.16 UNIBS

[0275] Packet seq. 90.51 92.37 5.44 ± 3.29 0.27 ± 1.20 0 97.92

[0276]

[0277] Time-int. seq. 92.60 96.07 18.02 ± 9.83 0.48 ± 0.42 3.18 94.4

[0278]

[0279] The results in Tables 3 and 4 indicate that FastFlow is efficient, but only report the mean number of packets and inference time for all classes. However, Figures 13 to 18 provide details on the inference efficiency of the FastFlow process for each flow type. Specifically, Figures 13 to 15 are respective graphs showing the fraction of packets per flow type classified by FastFlow for a given number of packets of the UTMobileNet, UNIBS, and VNAT datasets, respectively. Similarly, Figures 16 to 18 show the fraction of packets per flow classified for a given time threshold in seconds, for the UTMobileNet, UNIBS, and VNAT datasets, respectively.

[0280] These Figures show a high amount of variability across flow types and datasets. Regarding the datasets, UTMobileNet required the largest number of packets (30 packets, Figure 13) to classify all packets, whereas UNIBS required half of this amount (15 packets, Figure 14), and VNAT required only 10 packets (Figure 15). The inventors believe that the number of packets may be correlated with the number of flow types in each dataset, because a larger number of flow types makes the recognition of individual flows more challenging.

[0281] Regarding the recognition of individual flow types in each dataset, it is apparent that some types can be rapidly classified, whereas others require more packets. This contributes to the standard deviation numbers observed in Tables 3 and 4. In both cases, Figures 13 to 18 demonstrate the adaptability of FastFlow to recognize which flow types are more straightforward to classify.

[0282] FastFlow Deployment in a Live Network

[0283] The classification performance and inference efficiency of FastFlow in a realistic network deployment with a massive volume of data and a large number of unknown flows was also assessed. During a one-week deployment from 1st to 7th November 2024, a copy of the network traffic exchanged between the inventors' university campus border router and the Internet was streamed to the flow classification apparatus 102 via two 10 Gbps network interfaces for inbound and outbound traffic, respectively. Ethical clearance approval was obtained, allowing the analysis of campus traffic for network application classifications without collecting information that could identify users, such as university IDs and names. The data comprised 22.9 million flows that belong to seven application types served by 32 content providers as labelled by a commercial network traffic classification system deployed in parallel with the flow classification apparatus 102. Table 5 below summarizesthe data collected, including the number of flows used for training FastFlow one week before the deployment date.

[0284] The performance of FastFlow was assessed for application types and their content providers. Table 6 below shows the results for each application type. On average, FastFlow achieves an accuracy of 91.45%, and a Macro-Fl score of 90.23%. The average number of packets is 8.37, and the average time to classify a flow is 0.5 seconds. Among the seven application types, FastFlow achieves very high classification accuracy for conferencing (over 99%), mail (over 95%), and software update (over 93%), all of which have quite deterministic initial packet patterns compared to other application types. For mail and software, such patterns become confidently clear within the first eight packets with slight variations within 3 or 4 packets. In comparison, conferencing flows require about 3 to 18 packets for confident classification by FastFlow due to the diversified flow functionalities such as for video, audio, chats, and screen sharing, each of which can have a unique initial packet sequence for function-specific requests.

[0285] The other four application types have decent classification performance compared to reports in the literature, with fewer than 10 packets and 1 to 2 seconds for over 85% accuracy or Macro-Fl score. The one exception is social media flows, which are only classified with about 82% accuracy using up to 20 packets in 4 seconds. As shown in the provider list of the social media application type in Table 5 below, the flows labelled as social media by the commercial system belong to providers offering a mixed set of services such as short videos (e.g., Tiktok), picture sharing (e.g., Instagram), online social platforms (e.g., Facebook) and forums (e.g., Reddit), which are inherently different in the content delivered via network flows compared to each other.Table 5: Summary of the deployment data, including the number of flows per application type and content provider.

[0286] Application Provider #flow #train. flow MS Stream 1835585 136358 Youtube 1835508 124088 Video Streaming QQ 592163 12424

[0287] WeChat 387700 31024 Fastly 96750 15339 Adobe 1814389 101212 Windows 310918 34429 Software Update

[0288] Apple 820493 31090 Ubuntu 10921 1763 Discord 337876 38945 WhatsApp 30959 2589 Googl eMeet 30034 1415 Conferencing MS Teams 154832 2010

[0289] Facetime 9866 815

[0290] Zoom 5424 361 TikTok 1492556 115313 Instagram 1344128 73665 Facebook 759719 42566 Social Media

[0291] Linkedln 185968 16363 Reddit 154804 10566 Twitter 126376 6679 Apple iCloud 908163 13808 MS Sharepoint 679058 25600 File Storage Dropbox 112415 11778

[0292] Google Drive 253460 8327 OneDrive 59111 3485 AmazonAWS 1212689 35159 GoogleServices 882736 37417 Download

[0293] Google 755756 21928 MS DotNET 10880 13701 Microsoft 738633 26950 Mail

[0294] Google 87650 4216

[0295] - —

[0296]

[0297] Unknown – 4075187 –Table 6: Performance metrics of flow classification by FastFlow by application type. Application i Macro FT (%) Accuracy ('<) Packets (ff) T ime (s) Unknown FPR (%) Video Streaming 89.90 88.73 8.31 ± 4.38 0.21 ± 1.32 0.00 Software Update 92.50 93.18 7.22 + 4.50 0.10 ± 1.06 0.00 Conferencing 98.63 99.82 10.57 ± 7.27 1.34 ± 3.89 4.34 Social Media 81.91 82.42 11.89 ± 5.65 1.59 + 2.38 5.17 File Storage 89.13 84.75 8.83 ± 4.11 0.23 + 2.17 0.00 Download 83.60 87.97 10.10 ± 6.22 0.20 ± 1.21 1.43 Mail 93.66 95.20 6.12 ± 3.33 1.68 + 9.58 0.00 Unknown 91.94 20 ± 0.00 0.77 ± 2.69

[0298]

[0299] Average | 90.23 91.45 8.37 0.5

[0300] With a maximum time step threshold Cunk value tuned to a value of 25 packets, FastFlow can accurately detect 91.94% unknown flows that are labelled by the commercial system. The remaining unknown flows are classified as mostly 'social media' and 'conferencing', followed by 'download', as reported in the 'Unknown FPR' column of Table 6. However, those false positives might not truly be misclassifications because the labels provided by the commercial system were used as estimations of ground truths, which will not always be correct, particularly for the applications (e.g., social media and software update) that have providers not included in the ground-truth labels.

[0301] The inventors also developed and deployed flow classifiers to determine the content provider of each flow with its identified application type. Given that classifying content providers within each application type reduces the classification scope and possible variations within the same flow type, this improved performance for both accuracy and speed in all subsequent content provider classifiers, compared to its preliminary application type classifiers. Table 7 below shows the classification performance of these content provider classifiers for two representative application types, including video streaming and software update.

[0302] For video streaming content providers, the classifiers achieve superior performance in the three content providers primarily providing streaming services, including Microsoft Stream (used by the inventors' university for organizational video content such as lecture recording), YouTube, and Fastly. Over 97% accuracy was achieved with about 3 to 4 initial packets (less than 0.15 seconds) per flow. The video flows supported by QQ and WeChat, mainly known as social media applications with video streaming as their side features, require an average of 5 to 8 packets (less than 0.2 seconds) for classification accuracy over 89%. This shows the complexity of flow patterns for content providers that offerservices across application types. Notably, all flows labelled by the commercial system as video flows are confidently classified by their provider types instead of being classified as the 'unknown' type shown in the last column of Table 7.Application Provider Macro Fl (%) Accuracy (%) Packet (#) Time (s) Unknown FPR (%)

[0303] Microsoft 98.70 99.29 3.25 ± 2.32 0.05 ± 0.44 0.00 yabeeoace oo casscato basto Tl 7: Prfrmnf flwlifiin FFlw YouTube 97.98 97.14 4.25 ± 3.98 0.15 ± 1.51 0.00

[0304] Video Streaming 89.77 86.32 8.32 ± 4.68 0.19 ± 0.36 0.00 ppp poeesetate acatoodes fr rrnivliinrvir.

[0305] QQ

[0306] WeChat 91.01 91.56 5.78 ± 5.43 0.11 ± 0.40 0.00

[0307] Fastly 99.05 99.33 4.76 ± 1.56 0.01 ± 0.02 0.00

[0308] Adobe 98.09 98.24 4.75 ± 2.44 0.02 ± 0.09 0.48

[0309] Windows 93.84 94.20 6.40 ± 2.97 0.08 ± 0.51 0.38

[0310] Software Update

[0311] Apple 95.20 93.12 6.61 ± 4.16 0.06 ± 0.46 0.00

[0312] Ubuntu 98.77 99.38 6.63 ± 1.55 0.13 ± 0.15 0.00

[0313] Discord 98.74 99.70 1.20 ± 1.59 0.04 ± 0.03 0.00

[0314] Whatsapp 99.22 99.38 2.99 ± 2.30 0.39 ± 2.45 0.00

[0315] Conferencing

[0316] GoogleMeet 98.41 96.87 4.00 ± 4.09 0.13 ± 0.05 0.03

[0317] MS Teams 98.68 99.20 2.51 ± 1.45 0.57 ± 2.51 0.00

[0318] FaceTime 97.77 95.65 3.65 ± 3.60 0.38 ± 1.80 0.00

[0319] Zoom 97.41 98.62 4.12 ± 4.45 0.99 ± 3.62 0.07

[0320] TikTok 83.51 86.36 6.31 ± 3.52 0.13 ± 0.18 0.00

[0321] Instagram 85.17 88.97 11.16 ± 5.57 0.29 ± 1.39 0.00

[0322] Social Media

[0323] Facebook 82.35 82.12 9.90 ± 5.23 0.06 ± 0.28 0.05

[0324] LinkedIn 81.16 89.09 10.52 ± 5.26 0.27 ± 1.62 0.00

[0325] Reddit 84.61 88.85 9.09 ± 4.62 0.67 ± 4.10 0.00

[0326] Twitter 84.06 88.14 3.85 ± 4.11 0.02 ± 0.10 0.00

[0327] Apple iCloud 96.44 95.00 11.89 ± 4.62 0.05 ± 0.10 4.76 MS Sharepoint 91.94 95.65 7.35 ± 4.40 0.05 ± 0.27 2.42 File Storage

[0328] Dropbox 96.42 97.29 8.12 ± 2.63 0.08 ± 0.12 0.00

[0329] Google Drive 97.77 96.24 3.85 ± 4.11 0.02 ± 0.10 1.48

[0330]

[0331] OneDrive 88.37 84.73 9.63 ± 3.95 0.03 ± 0.07 0.00Software update has all its popular content providers accurately (over 93%) predicted with about six initial packets (less than 0.1 seconds) per flow. Windows and Apple, which have a diversified firmware catalog potentially supported by different product teams, are classified with lower accuracies (94.20% and 93.12%) compared to Adobe and Ubuntu (98.24% and 99.38%), which are with unified firmware / software portals for updates. From the last column of Table 7, a minority (less than 0.48%) of Adobe and Windows flows labelled by the commercial system are misclassified as 'unknown' by the FastFlow classifier, suggesting that further improvements on the fine-grained labelled dataset are needed to train a more accurate classifier on those application / provider types.

[0332] Additional Lab Evaluation Results of FastFlow Classification Performance:

[0333] I. Under ideal Conditions without Packet Sequence Disorder and Unknown Flow Type Table 8 below shows the classification results with no changes to all three public datasets ( / .e., without unknown flows or induced packet sequence disorder). As expected, the Macro-Fl and accuracy scores for all methods are higher compared to when the evaluations are conducted with packet sequence disorder on the test set. FastFlow performs slightly better than state of the art methods 89.44% vs 89.04% for UTMobilenet, 95% vs 93% for VNAT, and 95% vs 94% for UNIBS. However, FastFlow uses less than half the number packets on average to classify in case of UTMobilenet, less than 1 / 10 the number of packets for VNAT, and similarly for UNIBS. It can also be observed that the results from the packet representation only classifier 310, represented as 'Packet seq.' is faster and more accurate than both fast flow and the slot representation only classifier 312, represented as Time-int.seq'. This empirically shows that, in the absence of packet sequence disorder, packet level representation is better than slot representation.

[0334] II. With Packet Sequence Disorders but no Unknown Flow Types

[0335] Table 9 below shows classification results when packet sequence disorder is introduced in the test dataset, but without unknown flow detection. Compared to Table 8, where no packet sequence disorder is introduced, the Macro-Fl and Accuracy scores are reduced, however the scores for packet representation based models such as 'Packet seq.' and 'Pkts' are lowered more than time representation based models such as Time-int seq.' and Time-int. -5'.For example, for the UTMobilenet dataset, the Macro-Fl drop for 'Packet seq.' method is 10% whereas for Time-int. seq.' method it is less than 2% percent. Because FastFlow uses both packet and slot representations, its scores are not as heavily impacted as methods using only packet representation. This empirically highlights the importance of using both representations.

[0336] III. With Unknown Flow Types but no Packet Sequence Disorder

[0337] Table 10 below shows results when no packet sequence disorder is induced on the test dataset, but unknown flow detection is enabled by randomly sampling classes from each dataset and treating flows from these classes as unknown. It can be observed when comparing to Table 3, where packet sequence disorder is induced in the test dataset, that packet sequence disorder has only a small impact on unknown flow detection. The MacroFl and Accuracy scores follow a similar trend as discussed in the previous section.

[0338] Speed of FastFlow Classifiers:

[0339] I. When using Only Packet Flow Data Sequences

[0340] Figures 19 to 21 are cumulative distribution function (CDF) plots for the packet representation only classifier when tested on packet sequence dropped dataset without unknown flow detection, for the UTMobileNet (Figure 19), UNIBS (Figure 20) and VNAT (Figure 21) public datasets.

[0341] It can be observed that the UTMobilenet dataset (Figure 19) takes the most time / packets to classify, where the majority of flows require fewer than 8 packets for classification. In case of the VNAT dataset (Figure 21), most chat flows are classified using only the first packet, as more than 99 percent of chat flows in VNAT dataset start with a 52 byte payload from client to server, which does not happen in case of other classes. When comparing to the CDF plots for FastFlow in Figures 13 to 15, we can see that the packet-level classifier is slightly faster; however, this speed gain is not enough to justify the loss in Accuracy and Macro-Fl scores.k ()kk () ()hd () Ti T Pt T M Fl M Dt A#%tt"meaensacesaenacroeoaaseccuracy;■

[0342] Table 8: Classification performance of FastFlow for an ideal network flow dataset bl UTMiNtoee without packet sequence disorder or unknown flow types.

[0343] 0 <» xq <> 0 ’’F © •'j; co 0 0 ■«< «n CM d so I / , 41 41 41 41 41 41 41 41 4j 4: 4| 4| 4| 4| 4| 41 41 41 41 41 41 41 41 «

[0344] CM

[0345] =© 0 «« q q «h N N “-J PM

[0346] 0 0 6 4 « d

[0347] VNAT

[0348] y

[0349] ,!□ o K ” irt ™ »7~j.

[0350] wi Jk ri j>-5 ’•-; d o 2 - °o E s’ o Arj f S3 0 A ”•

[0351] 41 -H 4i 0 _Xd® 41 414141 41 g X ® 2 41 -+I

[0352] sor, ■ CM 41 *1 _< CNI t-5C4 4| 41

[0353] o JR 4 >n uinin « ULrt£ o

[0354] Q -S d F 4i ZE 3;.s 2 fc 5 s

[0355] 'S7-' F«H|

[0356] o g

[0357] 522914 ±..

[0358] 522194 ±..

[0359] W2866 ±6..

[0360] UNIBS

[0361] 50 r-i 0 r’. CM CO £M

[0362] CQ O -v $5 0

[0363] XQ >0 k*) 0 un c^5 cq OQ ()()0QTCP n)UDP 1 ON. CX CX <&:?3 5> CO

[0364] 5000 ±.

[0365] 54000 ±.

[0366] 484237 ±..

[0367] 6609754 ±..

[0368] 0 71 \© 0 ^0 r* 0 O t M -M tn t> flO 1F< ts

[0369] iri id cd cd id r< 0 0 sd <?\ r r r- 0^ O 00 oq

[0370] m s> r—!. CP in

[0371] -L. MO m $ «i >0 in > Jj, j4t----JL ',

[0372] o “•■= s & d s *"< -w- •<-* r / i 41 r«4 t! ffl m I I

[0373] £ TS S tS '. T O W

[0374] mc; M *" ’*•' S E ~ 4 S 0 S t? tS E E « « y y -a sS >5 ^ ^ p^ o o cM pM y y PM SM H O O PH CM H H (L 4 >" C U PH H H

[0375]

[0376] k ()kk () () ()hd i Ti T P T A M F M Dt%l%ttt#meaensacesaenccuracyacroeoaase

[0377] 1283674 ±.,

[0378] 929004 ±,.

[0379] Table 9: Classification performance ofFastFlow with realistic packet 1821996 ±..

[0380] seque 50nce disorders in each candidate flow but no unknown flow types.

[0381] ()()blTCPUDP UTMiN 10010toee

[0382] 5000 ±.

[0383] 54000 ±.

[0384] i>- c? <1 m cQ 739228 ±..

[0385] cn e*j ’d’ r-4 n> ©s 't r-l PM- <,■ ' / O!s$ H x© oc o 5440983 ±.. rr m CM S' O' O' Gh CM ©v cn ©% CO d d © d *1 d tn d c4 d T-5 d d m +1 -H -l-i +1 +1 +1 41 S 488679 + ±..1 4-1 41 41 41 41 41 CM « 41 41 41 41 41 41 41 CM ^3 cn t'x ch « in rl © 90 t'- 'Cf rM © ■CJ' O CM M CM r-i ' CM Ch nn © CM en M3 5 C 30329 ±M.. CM " V © <-j sn tn w t* m « o o <4 d in d OJ A <5 H rl d 4 d d d tn d <4

[0386] 19280...

[0387] 50

[0388] ()A VNTTCP(UDP) lOO IC

[0389] 5000 ±.

[0390] 45000 ±.

[0391] 1223602 ±..

[0392] 6 8771702 ±..

[0393] 7 69941 ±..

[0394] 559238 ±..

[0395] 10910020 ±..

[0396] 50

[0397] *-i krs ® er> in tn vh CM CM w O CO «*5 Tf O O' *•< oO 'ti m eo oo in,. © '^ CM © f i w rs M q CO £>. X 4 w M N op CM eq CM tn ()() UNIBSTCPUDP ■ 10010Mj so • oo o; oo S sMa 0 c5 C t-O> T Kj:< T-5 c «4 c r4, O MS M5rd -4 d d CM «-i nq

[0398] 0‘s o. 0". S O' O' 00 O' OC i 0s6- ^ d £h oi? gbi w

[0399] 5 o S « « 000 ±O.wi>- Ch 45000 ±.

[0400] 212242 ±..

[0401] 7 639679 ±.. C-3 O 0-3 O co o CM -TM Cl i-H O MB O* ’sf <5 in *0 0^ kf) i> CSj CM 0** cn T? X- CM %o Ch r* CM O O on 171 tn E> 03 in d d ri CM 4 o vi **4 £h 4 tfi $d i» ri a' d CN d d □> d *-4 fid t-s MB CO O W Ch SQ Ch © £Q oo QQ Q0 fiQ (h ^i ^ b Q\ W W 'O C\

[0402] H in.. K M 9 tn tn

[0403] « tn1 / 1- <$ tn £ 2; Z & S ■'; A > J j, ’.

[0404] Q V3 r-1. r-9rj

[0405] ” 1 tn. S -S tn.5 ± ■£ PQ tn.5.is td z »: m > t th <• css 1. tn A Js & US J U «5 V +- 4- H E H Z us u rq *s ts« 4j '.h. fe s J ni z t£ >. 1. y y « « ± % -± -id -S.5 cs «,xs!— ' SK -S -5 «.2 0 S S. S. S LM ^ H C O (± I± H H Hi fc H O O PL Ph H H

[0406]

[0407] Classification Performance Unk. flow detection Dataset Method

[0408] Macro Fl (%) Accuracy (%) Packets (#) Time (s) FPR (%) TPR (%)

[0409] FastFlow 89.21 90.2 12.74 ± 5.99 0.59 ± 1.07 4.16 88.21 Packet seq. 90.07 90.7 8.51 ± 3.63 0.41 ± 0.23 2.81 90.01 Time-int. seq. 89.33 29.63 ± 10.29 1.68 ± 3.79 4.59 GGFast 90.35 91.49 50 2.92 ±.96 4.97 92.81 UTMobileNet Grad-BP 91.21 91.6S 100(TCP) 10(UDP) 5.73 ± 1.49 4.88 69.87

[0410] Pkt.-5 72.95 77.78 5 0.15 ± 1.99 Pkt. -45 88.96 90.07 45 2.61 ± 0.79 Time-int.-5 69.51 74.93 6.49 ± 2.94 0.25 Time-in t. -45 88.85 89.49 39 ± 6.04 2.25 FastFlow 99.33 99.46 3.89 ± 1.61 .06 ±0.74 0 99.39 Packet seq. 99.33 99.46 3.89 ± 1.61 .06 ±0.07 0 99.20 Time-int. seq. 99.70 99.80 7.85 ± 4.35.088 ± 1.64 0 98.93 GGFast 84.28 90.72 50 0.95 ±.62 2.79 99.42 VNAT Grad-BP 99.77 99.79 100(TCP) 10(UDP) 1.97 ± 0.84 4.98 99.30

[0411] Pkt.-5 92.01 96.50 5 0.03 + 0.83 Pkt. 45 94.72 98.67 45 0.84 ± 0.59 Time-in t. -5 93.42 98.96 15.79 ± 3.07 0.25 Time-int.-45 90.43 93.50 93.52 ± 11.96 2.25 FastFlow 94.56 97.80 6.32 ± 2.68 0.28 ± 1.30 1.33 97.32 Packet seq. 95.10 97.79 5.19 -± 3.02 0.23 ± 1.16 0 98.29 Time-int. seq. 92.63 96.18 11.41 ± 7.31 6.39 ± 0.83 448 98.40 GGFast 93.12 97.45 50 1.35 ± 1.31 3.43 99.03 UNIBS Grad-BP 89.09 96.11 100(TCP) 10(UDP) 2.53 ± 1.63 4.90 73.90

[0412] Pkt. -5 85.08 89.81 5 0.23 ± 0.93 Pkt.-45 93.56 97.54 45 1.22 + 1.84

[0413]

[0414] Time-int.-5 82.45 87.01 5.62 ± 1.92 0.25

[0415]

[0416] Time-in t. -45 92.73 95.63 86.01 ± 21.02 2.25II. Using Only Slot Flow Data Sequences

[0417] Figures 22 to 24 are CDF plots showing the fraction of flows classified as a function of elapsed time for the slot representation only classifier 312, and for the UTMobileNet, UNIBS and VNAT public datasets, respectively. It can be observed that the classifier 312 adjusts according to the complexity of the classification task. For example, Figure 22 shows that most flows of the UTMobileNet dataset are classified within 1.5 seconds, whereas Figure 24 shows that almost all flows of the VNAT dataset are classified within 0.4 seconds. When compared to the corresponding plots for FastFlow in Figures 16 to 18, it is apparent that FastFlow performs significantly faster than the slot representation only classifier 312, where most of the flows from the UTMobilenet dataset are classified within 0.5 seconds.

[0418] As described herein, the flow classification apparatus 102 and process (also referred to herein as FastFlow) can be practically deployed in large networks such as ISPs to classify network flows rapidly and accurately, in real-time. By providing dual-grained timeseries flow representations and a corresponding time-series flow classifier architecture trained with reinforcement learning techniques, FastFlow is the first of its kind that addresses the three key challenges for deployment in large networks, including accurate classification with the smallest number of initial packets in each candidate flow, robustness to packet sequence disorders, and the ability to detect unknown flow types.

[0419] The classification performance of FastFlow has been extensively validated using public datasets, and its performance compared to ablation alternatives and state-of-the-art methods. FastFlow was deployed in a large campus network to classify application types and content providers of network flows. The deployment insights described above demonstrate that FastFlow can accurately classify flows by application and content provider with only about 10 initial packets of each flow, and in less than one or two seconds, and are able to detect flows that do not belong to any of the known types.

[0420] Many modifications will be apparent to those skilled in the art without departing from the scope of the present invention.

Claims

CLAIMS1. A computer-implemented process for processing packets of ISP-level network traffic to automatically classify in real-time network flows of the network traffic, and including the steps of:receiving, at respective packet arrival times, packets of network flows of general network traffic of a plurality of network users of the ISP;processing the received packets to generate, for each network flow of the received packets, a corresponding packet-level time-series representation of each packet of the network flow, and a corresponding slot-level time-series representation of aggregated packets of the network flow received at respective packet arrival times within each of a plurality of successive time slots; andfor each of the network flows:(i) processing the packet-level time-series representation with a corresponding trained packet-level classifier to generate a packetlevel classification of the network flow as one of a plurality of known flow types and a corresponding confidence score; and (ii) processing the slot-level time-series representation with a corresponding trained slot-level classifier to generate a slot-level classification of the network flow as one of the plurality of known flow types and a corresponding confidence score; and(iii) processing the packet-level and slot-level classifications of the network flow and the respective confidence scores with a timeseries flow classifier including long short term memory (LSTM) cells trained by reinforcement learning to determine a confident classification of the network flow as one of the plurality of known flow types or as an unknown flow type.

2. The process of claim 1, further including a step of, responsive to the confident classification of the network flow, changing one or more network settings to effect one or more of the following network reconfigurations: provisioning bandwidth and network capability in accordance with bandwidth demands of the flow type, prioritising traffic flows, and mapping traffic to network slices.

3. The process of claim 1 or 2, wherein the step of processing the packet-level and slot-level classifications of the network flow and the corresponding confidence scores includes:comparing each of the confidence scores with a corresponding confidence threshold;responsive to determining that each of the confidence scores is less than the corresponding confidence threshold, waiting to receive a further corresponding packetlevel or slot-level classification of the network flow and corresponding confidence score; andresponsive to determining that at least one of the confidence scores is equal to or greater than the corresponding confidence threshold, selecting the greater of the confidence scores and the corresponding packet-level or slot-level classification of the network flow as the confident classification of the network flow as one of the plurality of known flow types.

4. The process of claim 3, wherein the step of processing the packet-level and slot-level classifications of the network flow and the respective confidence scores includes, responsive to determining that a number of processed packets of the network flow has reached or exceeded a threshold value, determining the confident classification of the network flow as the unknown flow type.

5. The process of any one of claims 1 to 4, wherein the time-series flow classifier dynamically determines a smallest number of packets required to accurately classify the network flow.

6. A network flow classification apparatus for processing data packets of ISP level network traffic to automatically identify in real-time user platforms and content providers of video streams of the network traffic, the apparatus including:random access memory; andat least one processor configured to execute the process of any one of claims 1 to 5.

7. A computer-readable storage medium having stored thereon executable instructions that, when executed by at least one processor, cause the at least one processor to execute the steps of any one of claims 1 to 5.

8. A network flow classification apparatus for processing data packets of ISP-level network traffic to automatically classify in real-time network flows of the network traffic, the apparatus including:random access memory;at least one processor;at least one network interface to receive, at respective packet arrival times, packets of network flows of general network traffic of a plurality of network users of a network service provider;a packet-level time-series generator configured to generate, for each network flow of the received packets, a corresponding time-series representation of each packet of the network flow;a slot-level time-series generator configured to generate, for each network flow of the received packets, a corresponding time-series representation of aggregated packets received at respective packet arrival times within each of a plurality of successive time slots;a trained packet-level time series classifier configured to process each time-series representation of each packet of the network flow to generate a corresponding packet-level classification of the network flow as one of a plurality of known flow types and a corresponding confidence score;a slot-level time series classifier including long short term memory (LSTM) cells trained by reinforcement learning and configured to process each slot-level time-series representation to generate a corresponding slot-level classification of the network flow as one of the plurality of known flow types and a corresponding confidence score; anda time-series flow classifier to process the packet-level and slot-level classifications of the network flow and the corresponding confidence scores to determine a confident classification of the network flow as one of the plurality of known flow types or as an unknown flow type.

9. The apparatus of claim 8, further including a network API component configured to change, responsive to the confident classification of the network flow, one or more network settings to effect one or more of the following network reconfigurations: provisioning bandwidth and network capability in accordance with bandwidth demands of the flow type, prioritising traffic flows, and mapping traffic to network slices.

10. The apparatus of claim 8 or 9, wherein the time-series flow classifier is configured to compare each of the confidence scores with a corresponding confidence threshold, and, responsive to determining that the confidence score is less than the corresponding confidence threshold, waits to receive a further corresponding provisional classification of the network flow and corresponding confidence score.

11. The apparatus of any one of claims 8 to 10, wherein the slot-level time series classifier is further configured to, responsive to determining that a number of processed packets of the network flow has reached or exceeded a threshold value, determine the confident classification of the network flow as the unknown flow type.

12. The apparatus of any one of claims 8 to 11, wherein the time-series flow classifier dynamically determines a smallest number of packets required to accurately classify the network flow.

Images