A traffic management method, system and device based on dynamic classification

Abstract

The present invention relates to a traffic management method, system and device based on dynamic classification. The data preparation module collects network traffic screening feature data and labels the data according to the application type; the online training module constructs a network traffic classification model through the XGBoost algorithm; based on the model, the traffic classification module classifies the network traffic; and the traffic type trend analysis module statistically analyzes the changing trend state of each type of network traffic within a period of time after classification and divides each type of network traffic into trend state types; the PHB mapping module maps different types of network traffic to different PHB types; and the traffic distribution module is used to forward the network traffic in real time according to the forwarding strategies of different PHB types. The present invention combines the traffic management method of network traffic type and the dynamic and rapid classification of each type of traffic change trend to meet the real-time requirements and improve user experience.

Description

Technical Field

[0001] The present invention relates to the field of Internet, and in particular to a method, system and device for network traffic management based on dynamic classification. Background Art

[0002] With the continuous development of Internet technology, the number of various applications has skyrocketed. Especially with the rise of short videos, cloud disks, cloud games and other applications, users' demand for network traffic has entered a period of explosive growth, and higher requirements have been placed on network quality.

[0003] According to the 2020 China Network Quality Annual Report released by the Internet Data Center IDC, 45% of Chinese netizens said that the network experience was "poor", among which "network lag" ranked first.

[0004] Operators facing this problem first consider that many network applications have their own network characteristics, and their real-time requirements are also different. The traditional "equal treatment" forwarding method cannot meet the high real-time requirements of some applications and seems a bit "wasteful" for other applications. Therefore, under the premise that the overall bandwidth does not change much, taking measures to classify and manage network traffic in a differentiated manner and giving priority to applications with high real-time requirements is a feasible response method. Specifically, based on the classification of network traffic, a suitable transmission environment is customized for different applications (traffic types), thereby improving network application access perception and customer satisfaction.

[0005] Traditional traffic classification methods include TCP port identification-based and deep packet inspection (DPI)-based. Their essence is to parse network traffic data packets to obtain the valid data contained therein, and then achieve classification by matching certain feature fields.

[0006] The network traffic classification technology based on TCP port identification is to classify traffic according to ports based on the TCP protocol. Although the algorithm of this method is simple, with the emergence of port hopping and port masquerading technology, its accuracy rate continues to decrease, and its scope of application is also greatly reduced.

[0007] Network traffic classification technology based on deep packet inspection (DPI) is a technology that classifies network traffic by analyzing the valid data of network traffic packets and matching them with known programs or protocols. However, this technology is affected by data encryption and some privacy issues, and the classification results cannot meet commercial conditions.

[0008] In addition, whether based on TCP port identification technology or deep packet inspection (DPI) technology, both classifications classify network traffic according to static rules set by humans, which cannot meet the requirements of real-time intelligence.

[0009] For example, the network traffic determination method disclosed in CN112187653A includes obtaining a flow, determining its corresponding business model, calling a traffic classification model based on the business model, and determining the traffic type of the flow based on the traffic classification model. This method is a static classification and does not establish a dynamic classification and identification model for complex network scenarios, and is not universal in the industry.

[0010] For another example, the network traffic determination method disclosed in CN112187652A collects multiple sample traffic within a period of time, clusters the multiple sample traffic to obtain multiple sample traffic types, and establishes initial feature extraction rules and initial feature determination rules based on the multiple sample traffic types. This method establishes traffic determination rules based on clustered samples within a specific time, without considering the multi-dimensional time series characteristics of traffic, which can easily lead to large errors in classification results.

[0011] Therefore, there is an urgent need for a traffic management method that combines network traffic types and the dynamic and rapid classification trends of each type of traffic to meet real-time requirements and improve user experience. Summary of the invention

[0012] This Summary is provided to introduce some concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter; nor is it intended to be used to determine or limit the scope of the claimed subject matter.

[0013] The present invention proposes a dynamic network traffic classification method, system and device based on machine learning. It collects network traffic feature data, establishes an effective feature database for traffic classification, and then uses the extreme gradient boosting XGBoost algorithm to build a network traffic classification model online, outputs the network traffic classification result according to the network traffic classification model, and then maps each type of traffic to different hop-by-hop behavior PHB types in combination with the real-time requirements and change trends, matches differentiated traffic management strategies, and forwards network traffic in real time through the forwarding strategies of different PHB types. This ensures that its subsequent services can meet their respective real-time requirements and improves customer experience. Its classification effect and accuracy are higher than the existing TCP classification technology and DPI classification technology.

[0014] The traffic management system based on dynamic classification of the present invention comprises: a data preparation module for collecting network traffic screening feature data and marking type labels according to application types, an online training module for building a network traffic classification model based on the network traffic marked with type labels through an XGBoost algorithm, a traffic classification module for classifying network traffic using the network traffic classification model, a traffic type trend analysis module for statistically analyzing the changing trend state of each type of network traffic after classification within a period of time and dividing each type of network traffic into trend state types, a PHB mapping module for mapping different types of network traffic to different PHB types in combination with network traffic types and changing trend state types, and a traffic distribution module for forwarding network traffic in real time according to forwarding strategies provided by different PHB types.

[0015] The traffic management method based on dynamic classification of the present invention comprises: collecting traffic data and extracting feature data, and marking type labels according to application types; using the XGBoost algorithm to perform online training using the traffic data collected and marked with type labels within a time period to obtain a traffic classification model; using the traffic classification model to classify each collected traffic; constructing feature engineering to identify the change trend state of each type of traffic; mapping each type of traffic to different PHB types; and forwarding according to the strategy corresponding to the PHB type of the real-time traffic.

[0016] These and other features and advantages will become apparent by reading the following detailed description and by reference to the associated drawings.It is to be understood that the foregoing general description and the following detailed description are illustrative only and are not restrictive of the aspects of what is claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The present invention will be described in more detail below with reference to specific embodiments shown in the accompanying drawings.

[0018] Figure 1 is a block diagram of a dynamic network traffic management system of the present invention;

[0019] Figure 2 It is an overall flow chart of the dynamic network traffic management method of the present invention;

[0020] Figure 3 yes Figure 2 A detailed flow chart of step S5 in the method;

[0021] Figure 4 It is a flow chart of Chat and Stream traffic classification and forwarding according to a specific embodiment of the dynamic network traffic management method of the present invention.

[0022] The flowcharts and block diagrams in the accompanying drawings show the system architecture, functions and operations that may be implemented by the system and method according to the embodiments of the present application. In this regard, each box in the flowchart or block diagram may represent a module, a program segment or a part of a code, and the module, program segment or a part of a code contains one or more executable instructions for implementing a specified logical function. DETAILED DESCRIPTION

[0023] The present invention will be described in more detail below by referring to the specific embodiments shown in the accompanying drawings. By reading the detailed description of the specific embodiments below, various advantages and benefits of the present invention will become clear to those of ordinary skill in the art. However, it should be understood that the present invention can be implemented in various forms and should not be limited by the various embodiments set forth herein. The following embodiments are provided in order to be able to more thoroughly understand the present invention. Unless otherwise stated, the technical terms or scientific terms used in this application should be the common meanings understood by those skilled in the art to which this application belongs.

[0024] The present invention provides a traffic management method and system based on dynamic classification. By counting the characteristic data of the network traffic passing through the recent network nodes (the number of data packets, the number of bytes of the data packet length, the value of the fragmentation flag Flags field, etc.), the network traffic is divided into 9 types according to the application type, and each type of traffic is divided into shrinking type, stable type and growing type by counting the recent change trend state of the traffic. In combination with the requirements of the traffic type for network real-time performance and the change trend of the traffic type, different types of network traffic are mapped to the PHB type in the QoS service quality, and a mapping strategy is generated. After the real-time network traffic is classified, it is forwarded according to the mapping strategy, which not only ensures the requirements of different types of traffic for network real-time performance but also ensures that high-frequency traffic can be forwarded with a higher priority.

[0025] like Figure 1 As shown, the traffic management system based on dynamic classification of the present invention includes: a data preparation module, an online training module, a traffic classification module, a traffic type trend analysis module, a PHB mapping module and a traffic distribution module. The details are as follows:

[0026] Data preparation module: collects network traffic at a certain observation point (e.g., router or layer 3 switch) in the network within a period of time (e.g., one month, three months, six months, etc.), selects different traffic feature data that play a positive role in traffic classification for classification, performs statistical analysis and labels the data according to the application type, and obtains a data set with the type label; in addition, the data preparation module is also responsible for real-time collection of network traffic data;

[0027] Online training module: Train the network traffic data set with type labels obtained from the network traffic feature data collected over a period of time analyzed by the data preparation module, and build a network traffic classification model through the XGBoost algorithm;

[0028] Traffic classification module: Based on the different characteristic data that play an active role in classifying network traffic obtained by screening in the data preparation module, the network traffic classification model trained by the online training module is used to classify the network traffic to obtain classified network traffic;

[0029] Traffic type trend analysis module: By statistically analyzing the changing trend of each type of network traffic over a period of time, each type of network traffic is divided into status types, for example, including shrinking traffic, stable traffic and growing traffic;

[0030] PHB mapping module: By combining the network traffic type obtained by the traffic classification module and the change trend status of each type obtained by the traffic type trend analysis module, different traffic types are mapped to different PHB types;

[0031] Traffic forwarding module: forwards network traffic in real time according to the forwarding strategies provided by different PHB types.

[0032] like Figure 2 As shown, the traffic management method based on dynamic classification of the present invention includes:

[0033] In step S1, network traffic is collected for a period of time at a certain observation point (router or three-layer switch) in the network, and characteristic data is extracted, including the number of data packets, data packet length; the source IP, destination IP, service type Type of Service field value, and fragmentation flag Flags field value of the IP header; the source port, destination port, header length Data Offset field value, and emergency flag URG field value of the TCP header; the source port, destination port and duration of the entire network flow of the UDP header.

[0034] The debugging personnel classify these network traffic feature data according to the application type and label them with type. The specific classification can be divided into the following nine types: Chat (instant messaging application type), DNS (domain name system service application type), Email (email application type), File transfer (file transfer application type), Game (multiplayer online game application type), HTTP (hypertext transfer protocol application type), P2P (P2P file sharing application type), Stream (streaming media application type) and VoIP (IP phone application type). Each type of traffic has different requirements for real-time performance and will be given different weights in subsequent steps.

[0035] In step S2, the network traffic feature data collected in step S1 is split into a training set and a test set, and the XGBoost algorithm is selected to complete the training and testing work; the network traffic feature data in the data set is used as a node for tree splitting, and different types of network traffic generate trees with different scores, thereby obtaining a traffic classification model.

[0036] In step S3, each network flow is classified according to the real-time characteristic data of each network flow obtained in step S1 and the classification model trained in step S2 to obtain the type of each network flow. After t decision trees are trained, the classification prediction score of each network flow is:

[0037]

[0038] in, represents the output network traffic classification prediction score, t represents the number of decision trees, and f k Represents a specific tree, x i Represents the input network traffic and predicts the score through classification The current traffic type is determined by the size of

[0039] In step S4: construct feature engineering for the network data collected in step S1: count the characteristics of the total amount of each type of traffic, such as the mean, maximum, median, minimum and other statistical characteristics of the time granularity of the past 7 days, 1 month, 6 months, etc.; use the time series decomposition algorithm to decompose the trend items of each indicator, and judge the changing trend of the indicator; according to the output of the feature engineering, use the SVM classification algorithm to construct a traffic type trend state classification model, so as to identify whether the trend state of each traffic type is shrinking, stable or growing.

[0040] In step S5, different types of traffic classes are mapped to different PHB types in combination with the different types of traffic obtained in step S3 and the changing trends of the corresponding types of traffic obtained in step S4. The PHB types include DF (default forwarding) type, four levels of AF (assured forwarding) types, including AF1, AF2, AF3 and AF4, and EF (expedited forwarding) type.

[0041] In step S6, the real-time network traffic is forwarded according to the type of the real-time network traffic obtained in step S3 and the PHB type corresponding to each traffic type obtained in step S5, thereby improving the network service quality.

[0042] The following combination Figure 3 discuss Figure 2 The specific steps of mapping different types of traffic classes to different PHB types in step S5 are as follows:

[0043] Step S5-1: adding initial weight values ​​W(i) to different types of network traffic in step S1 according to the real-time requirements. For example, the initial weight values ​​of HTTP type, P2P type, Email type, File transfer type, DNS type, Chat type, Game type, Stream type and VoIP type are set to 1, 1, 2, 2, 3, 4, 4, 5 and 6 respectively;

[0044] Step S5-2: adding additional weight values ​​A(i) to the traffic types of different change trend states obtained in step S4, for example, setting the additional weight values ​​of the shrinking, stable and growing traffic types to 1, 2 and 3 respectively;

[0045] Step S5-3: Adding the initial weight value W(i) in step S5-1 and the additional weight value A(i) in step S5-2 is the final weight value of different types of traffic, for example, the possible results are 2, 3, 4, 5, 6, 7, 8 and 9;

[0046] Step S5-4: According to the final weight value obtained in step S5-3, different types of traffic classes are marked as EF, AF4, AF3, AF2, AF1 and DF types in the PHB, among which the traffic types with final weight values ​​of 2 and 3 are marked as DF type default forwarding, the traffic types with final weight values ​​of 4, 5, 6 and 7 are marked as AF1, AF2, AF3 and AF4 types for guaranteed forwarding respectively, and the traffic types with weight values ​​of 8 and 9 are marked as EF type for rapid forwarding.

[0047] Figure 4 This is a flow chart of traffic classification and forwarding marked with Chat and Stream class labels in step S1 according to an embodiment of the present invention. The specific steps are as follows:

[0048] Step S100: by analyzing the network real-time requirements of different types of traffic, initial weight values ​​are added to them, W_(Chat)=4, W_(Stream)=5;

[0049] Step S200: Add additional weight values ​​to the two types of traffic according to their changing trends. Assuming that Chat traffic is growing and Stream traffic is shrinking, add additional weight values ​​to them, A_(Chat)=3, A_(Stream)=1;

[0050] Step S300: Combine the initial weight value and the additional weight value to obtain a final weight value of 7 for Chat traffic and a final weight value of 6 for Stream traffic, corresponding to PHB types AF4 and AF3 respectively, and generate a PHB mapping policy;

[0051] Step S400: collect real-time network traffic and analyze its type. If it is Chat type traffic, forward it according to AF4; if it is Stream type traffic, forward it according to AF3. AF4 has a higher priority than AF3.

[0052] The traffic management method and system based on dynamic classification of the present invention establish a set of characteristic data models under complex network scenarios, fully combine the recent trend change characteristics of traffic, assign different weights, generate differentiated management strategies, and meet the diverse needs of identifying general traffic. It is suitable for traffic identification and traffic scheduling in large and medium-sized Internet environments (such as operator networks, large IDC networks, etc.), and has wide applicability. The method and system of the present invention make full use of Internet bandwidth resources, improve network reuse rate, can effectively improve the response speed of large and medium-sized Internet, and effectively solve the classification and scheduling problems of large and medium-sized networks.

[0053] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features may be replaced by equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application, and they should all be included in the scope of the claims and specification of the present application.

Claims

1. A traffic management method based on dynamic classification, include: Collect traffic data and extract feature data, and label them according to the application type; Use the XGBoost algorithm to perform online training using traffic data collected within a period of time and labeled with type labels to obtain a traffic classification model; Using the traffic classification model to classify each collected traffic; Construct feature engineering to identify the changing trend of each type of traffic; Map each type of traffic to a different PHB type; Mapping each type of traffic to a different PHB type further includes: adding an initial weight value to each type of traffic according to real-time requirements; adding an additional weight value to each type of traffic according to different change trend states; adding the initial weight value and the additional weight value to obtain a final weight value; and mapping each type of traffic to a different PHB type according to the final weight value; And forward according to the policy corresponding to the PHB type of real-time traffic.

2. The method according to claim 1, It is characterized in that The characteristic data includes one or more of the following: the number of data packets, the length of data packets; the source IP, destination IP, service type Type of Service field value, and fragmentation flag Flags field value of the IP header; the source port, destination port, header length Data Offset field value, and emergency flag URG field value of the TCP header; the source port, destination port and duration of the entire network flow of the UDP header.

3. The method according to claim 1, It is characterized in that The type labels according to the application types include: instant messaging application type, domain name system service application type, email application type, file transfer application type, multiplayer online game application type, hypertext transfer protocol application type, P2P file sharing application type, streaming media application type and IP phone application type.

4. The method according to claim 1, It is characterized in that The step of obtaining a traffic classification model includes: splitting the data obtained in the previous step into a training set and a test set, selecting the XGBoost algorithm for training and testing, using the feature data as nodes for tree splitting, and obtaining different scores after generating trees for different types of network traffic, thereby obtaining a traffic classification model.

5. The method according to claim 4, It is characterized in that The classification prediction score of each real-time network traffic is: in, represents the output network traffic classification prediction score, 𝑡 represents the number of trees, 𝑓 𝑘 represents a specific tree, 𝑥 𝑖 Represents the input network traffic and predicts the score through classification The current traffic type is determined by the size of 6. The method according to claim 1, It is characterized in that Identify the changing trend status of each type of traffic including: Statistics on the mean, maximum, median, and minimum characteristics of the total amount of each type of traffic at the time granularity; Use the time series decomposition algorithm to decompose the trend items of characteristic indicators and determine the changing trend of the indicators; Using the SVM classification algorithm to build a traffic type trend status classification model; and Identify for each traffic type whether the trend status is shrinking, stable, or growing.

7. The method according to claim 1, It is characterized in that The PHB types include DF default forwarding, EF fast forwarding and AF assured forwarding, wherein the AF further includes AF1, AF2, AF3 and AF4 with different weights.

8. A traffic management system based on dynamic classification, include: The data preparation module is used to collect network traffic, filter feature data, and label it according to the application type; Online training module, which is used to build a network traffic classification model based on the network traffic with type labels through the XGBoost algorithm; A traffic classification module, used to classify network traffic using the network traffic classification model; The traffic type trend analysis module is used to statistically analyze the changing trend of each type of network traffic within a period of time after classification and to divide each type of network traffic into trend status types; The PHB mapping module is used to map different types of network traffic to different PHB types based on the network traffic type and the change trend status type; Mapping each type of traffic to a different PHB type further includes: adding an initial weight value to each type of traffic according to the real-time requirement; Add additional weight values ​​to each type of traffic according to different change trend states; Adding the initial weight value and the additional weight value to obtain a final weight value; and Mapping each type of traffic to a different PHB type according to the final weight value; and The traffic distribution module is used to forward network traffic in real time according to the forwarding strategies provided by different PHB types.

9. The system of claim 8, Features: The type labels marked according to the application type include: instant messaging application type, domain name system service application type, email application type, file transfer application type, multiplayer online game application type, hypertext transfer protocol application type, P2P file sharing application type, streaming media application type and IP phone application type; The trend status types include shrinking, stable, and growing; and The PHB types include DF default forwarding, EF fast forwarding and AF assured forwarding.

Images