Data stream identification method and device, computer storage medium

Abstract

The application discloses a data stream identification method and device, and a computer storage medium, and belongs to the technical field of networks. A network device or a server inputs a first data packet of a to-be-identified data stream into a deep learning model; receives target feature data output by the deep learning model according to the first data packet; and determines the type of the to-be-identified data stream based on the target feature data, wherein the type includes a known application or an unknown application; wherein the deep learning model is trained by a plurality of sample data streams labeled with application tags. The application automatically learns the distinguishing features of the known application data stream and the unknown application data stream through the deep learning model, the reliability of the feature data output by the deep learning model is relatively high, and therefore the accuracy of data stream identification can be improved.

Description

Technical Field

[0001] This application relates to the field of network technology, and in particular to a data stream identification method and apparatus, and a computer storage medium. Background Technology

[0002] The purpose of data flow identification is to determine the category of a data flow, that is, to determine whether the data flow belongs to an unknown application or a known application. Data flow identification is the prerequisite and foundation for data flow control, network security management, improving network service quality, and optimizing network performance.

[0003] Most current data stream recognition methods are based on supervised machine learning techniques. These methods extract features from data streams of known categories, train a classifier using these features, and then use the trained classifier to classify the data stream to be recognized. However, because the classifier's performance is highly dependent on the design of the data stream features, which are often determined based on human experience and have low reliability, the accuracy of data stream recognition is low. Summary of the Invention

[0004] This application provides a data stream identification method and apparatus, and a computer storage medium, which can solve the problem of low accuracy in current data stream identification.

[0005] Firstly, a data stream identification method is provided. A network device or server inputs the first data packet of the data stream to be identified into a deep learning model; receives target feature data output by the deep learning model based on the first data packet; and determines the type of the data stream to be identified based on the target feature data, which includes known applications and unknown applications; wherein, the deep learning model is trained from multiple sample data streams labeled with application tags.

[0006] The data stream identification method provided in this application determines the type of the data stream by inputting the first data packet of the data stream to be identified into a deep learning model, and then determining the target feature data output by the deep learning model based on the first data packet. Since the deep learning model can automatically learn the distinguishing features between known and unknown application data streams, the feature data output by the deep learning model has higher reliability compared to feature data determined based on human experience, thereby improving the accuracy of data stream identification.

[0007] Optionally, the deep learning model includes a convolutional neural network and an optimization network. The convolutional neural network includes an input layer, a hidden layer, and an output layer connected in sequence, and the optimization network is connected to the output layer. The input layer sends the first data packet of the data stream to be identified to the hidden layer; the hidden layer performs feature extraction and feature combination on the first data packet of the data stream to be identified to obtain feature data; the output layer sends the feature data to the optimization network; and the optimization network performs feature reorganization on the feature data to obtain the target feature data.

[0008] The hidden layer extracts and combines features from the input data through multidimensional convolutional filtering and nonlinear mapping. By employing an optimized network to reconstruct the features output from the convolutional neural network, feature data with low coupling and strong orthogonality can be obtained.

[0009] Optionally, the hidden layer includes nine sequentially connected data processing layers, each of which includes a convolutional layer, a residual layer, and a pooling layer connected in sequence. The pooling layer can be a max pooling layer or an average pooling layer.

[0010] In this application, the process of determining the type of a data stream to be identified based on target feature data includes: obtaining a clustered sample set, which includes multiple sample feature data, including sample feature data labeled with known application tags and sample feature data labeled with unknown application tags; performing clustering processing on the target feature data and the multiple sample feature data; and determining the type of the data stream to be identified based on the sample feature data in the target cluster where the target feature data is located.

[0011] Optionally, the process of determining the type of the data stream to be identified based on the sample feature data in the target cluster where the target feature data is located includes: determining the label of the target cluster based on the sample feature data in the target cluster; and determining the type of the data stream to be identified based on the label of the target cluster.

[0012] In one possible implementation, the label of the target cluster is determined based on the sample feature data in the target cluster, including: calculating the posterior probability of each type of sample feature data in the target cluster, wherein the application label of sample feature data of the same type is the same; and determining the application label of the sample feature data of the type with the highest posterior probability in the target cluster as the label of the target cluster.

[0013] In another possible implementation, the label of the target cluster is determined based on the sample feature data in the target cluster, including: when there is no sample feature data in the target cluster labeled with a known application label, the label of the target cluster is determined to be an unknown application.

[0014] Optionally, the two implementation methods described above can be combined. After obtaining the target cluster where the target feature data is located, it is detected whether there is sample feature data labeled with known application tags in the target cluster. When there is sample feature data labeled with known application tags in the target cluster, the posterior probability of each type of sample feature data in the target cluster is calculated. The application tag of the sample feature data with the highest posterior probability in the target cluster is determined as the tag of the target cluster.

[0015] Optionally, clustering processing is performed on the target feature data and multiple sample feature data, including: using n clusterers to cluster the target feature data and multiple sample feature data respectively, to obtain n target clusters where the target feature data belongs, where n is an integer greater than 1, and the number of clusters and / or the initial cluster centers of different clusterers are different. Then, based on the labels of the target clusters, the type of the data stream to be identified is determined, including: determining the type of the data stream to be identified based on the labels of the n target clusters.

[0016] Optionally, the process of determining the type of the data stream to be identified based on the labels of n target clusters includes: using a weighted integration function to determine the type of the data stream to be identified, wherein the weighted integration function is:

[0017]

[0018] in, Lable x The label represents the category of the data stream to be identified, and the label represents the set of possible application categories, which includes known applications and unknown applications. ω i J represents the maximum posterior probability corresponding to the i-th target cluster out of n target clusters. i (x) represents the label of the i-th target cluster.

[0019] Multiple clusterers are used to cluster the target feature data and multiple sample feature data respectively, resulting in multiple target clusters where the target feature data belongs. After determining the labels and corresponding posterior probabilities of these multiple target clusters, a weighted ensemble method is used to determine the category of the data stream to be identified where the target feature data belongs. Compared with using a single clusterer, this method can reduce the influence of noise and independent points, and enhance the stability and robustness of the clustering results.

[0020] Optionally, after determining the type of the data stream to be identified based on the sample feature data within the target cluster where the target feature data resides, the method further includes:

[0021] When the data stream to be identified belongs to an unknown application, the target feature data is labeled with an unknown application label; the target feature data labeled with the unknown application label is added to the cluster sample set to obtain the updated cluster sample set.

[0022] Optionally, if the multiple sample data streams include sample data streams labeled with known application tags and sample data streams labeled with unknown application tags, then the process of obtaining the clustered sample set includes: inputting the first data packet of each of the multiple sample data streams into the deep learning model, and using the feature data output by the deep learning model as the sample feature data.

[0023] Secondly, a data stream identification device is provided, which includes functional modules for implementing the method and its various embodiments described in the first aspect. These functional modules may be hardware modules, software modules, or a combination of hardware and software modules.

[0024] Thirdly, a data stream identification device is provided, comprising a processor and a memory. The memory stores a computer program, the computer program including program instructions; the processor invokes the computer program to implement the data stream identification method as described in the first aspect and any implementation thereof.

[0025] Fourthly, a computer storage medium is provided, wherein instructions are stored on the computer storage medium, and when the instructions are executed by a processor, the data stream identification method as described in the first aspect and any implementation thereof is implemented.

[0026] Fifthly, a chip is provided, the chip including programmable logic circuits and / or program instructions, which, when the chip is running, implement the data stream identification method as described in the first aspect and any implementation thereof.

[0027] The beneficial effects of the technical solution provided in this application include at least the following:

[0028] By inputting the first data packet of the data stream to be identified into a deep learning model, and based on the target feature data output by the deep learning model according to the first data packet, the type of the data stream to be identified is determined. Since the deep learning model can automatically learn the distinguishing features between known and unknown application data streams, the feature data output by the deep learning model is more reliable than feature data determined based on human experience, thus improving the accuracy of data stream identification. Furthermore, multiple clusterers are used to cluster the target feature data and multiple sample feature data respectively, resulting in multiple target clusters to which the target feature data belongs. After determining the labels and corresponding posterior probabilities of these multiple target clusters, a weighted ensemble method is used to determine the category of the data stream to which the target feature data belongs. This reduces the influence of noise and independent points, enhances the stability and robustness of the clustering results, and further improves the accuracy of data stream identification. Attached Figure Description

[0029] Figure 1 This is a flowchart illustrating a data stream identification method provided in an embodiment of this application;

[0030] Figure 2 This is a flowchart of a method for determining the category of a data stream to be identified, provided in an embodiment of this application;

[0031] Figure 3 This is a schematic diagram of the structure of a deep learning model provided in an embodiment of this application;

[0032] Figure 4 This is a schematic diagram of the structure of a convolutional neural network provided in an embodiment of this application;

[0033] Figure 5 This is a schematic diagram of an optimized network structure provided in an embodiment of this application;

[0034] Figure 6 This is a schematic diagram of the structure of a data stream identification device provided in an embodiment of this application;

[0035] Figure 7 This is a schematic diagram of the structure of a determining module provided in an embodiment of this application;

[0036] Figure 8 This is a schematic diagram of another data stream identification device provided in an embodiment of this application;

[0037] Figure 9 This is a block diagram of a data stream identification device provided in an embodiment of this application. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0039] Currently, data stream identification typically relies on human experience to determine the data stream features to be extracted. However, the reliability of data stream features determined by human experience is low, leading to low accuracy in data stream identification. This application provides a data stream identification method that trains a deep learning model using multiple sample data streams labeled with application tags, and then performs data stream identification based on the feature data output by the deep learning model. Because the deep learning model can automatically learn the distinguishing features between known and unknown application data streams, the reliability of the feature data output by the deep learning model is high, thereby improving the accuracy of data stream identification.

[0040] The data flow identification method provided in this application can be applied to network devices or servers in a communication network. Optionally, the communication network can be a data center network (DCN), a metropolitan area network, a wide area network, or a campus network, etc., and this application does not limit the application scenario of the data flow identification method. The network device can be a router or a switch, etc.

[0041] Figure 1 This is a flowchart illustrating a data stream identification method provided in an embodiment of this application. Figure 1 As shown, the method includes:

[0042] Step 101: Input the first data packet of the data stream to be identified into the deep learning model.

[0043] Optionally, after acquiring the data stream to be identified, the first 512 bytes of the first data packet of the data stream are obtained and input as a 512-dimensional data source into the deep learning model. The first 512 bytes of the first data packet in the data stream include the packet header and payload data. Since the packet header and payload data of the first data packet in the data stream carry the natural attributes of the application to which the data stream belongs, as well as key information about the application's communication with other applications, such as quintuple information, the reliability of the feature data extracted based on the first 512 bytes of the first data packet in the data stream is relatively high. When the number of bytes in the first data packet of the acquired data stream to be identified is less than 512, it can be padded with zeros.

[0044] Step 102: Receive the target feature data output by the deep learning model based on the first data packet.

[0045] Step 103: Determine the type of the data stream to be identified based on the target feature data.

[0046] This type includes known applications and unknown applications. Determining the type of the data stream to be identified means determining whether the data stream to be identified belongs to a known application or an unknown application.

[0047] Optionally, the implementation process of step 103 can be as follows: Figure 2 As shown, it includes:

[0048] Step 1031: Obtain a cluster sample set, which includes multiple sample feature data.

[0049] The sample feature data includes sample feature data labeled with known application tags and sample feature data labeled with unknown application tags. For example, suppose the clustered sample set includes sample feature data 1, sample feature data 2, and sample feature data 3. If sample feature data 1 belongs to known application 1, then the application tag labeled with sample feature data 1 is known application 1; if sample feature data 2 belongs to known application 2, then the application tag labeled with sample feature data 2 is known application 2; if sample feature data 3 belongs to an unknown application, then the application tag labeled with sample feature data 3 is unknown application. In this embodiment, all sample feature data belonging to unknown applications have the same application tag (all application tags are unknown application).

[0050] Optionally, if the multiple sample data streams used to train the deep learning model include sample data streams labeled with known application tags and sample data streams labeled with unknown application tags, then the process of obtaining the clustered sample set includes: inputting the first data packet of each of the multiple sample data streams into the deep learning model, and using the feature data output by the deep learning model as sample feature data to obtain the clustered sample set.

[0051] Step 1032: Perform clustering processing on the target feature data and multiple sample feature data.

[0052] Optionally, the k-means algorithm can be used to cluster the target feature data and multiple sample feature data.

[0053] In this embodiment, n clusterers can be used to cluster the target feature data and multiple sample feature data respectively, resulting in n target clusters where the target feature data belongs, where n is an integer greater than 1. The number of clusters and / or the initial cluster centers of the different clusterers among the n clusterers are different. Optionally, each clusterer randomly selects the initial cluster centers.

[0054] For example, five clusterers are used to cluster the target feature data and multiple sample feature data respectively. The number of clusters produced by these five clusterers are 5, 6, 7, 8, and 9, respectively. That is, after clustering the target feature data and multiple sample feature data, these five clusterers can obtain 5 clusters, 6 clusters, 7 clusters, 8 clusters, and 9 clusters, respectively. In this embodiment, the cluster containing the target feature data is called the target cluster. Each clusterer can obtain one target cluster after clustering the target feature data and multiple sample feature data.

[0055] Step 1033: Determine the type of the data stream to be identified based on the sample feature data in the target cluster where the target feature data is located.

[0056] Optionally, the implementation process of step 1033 includes: determining the label of the target cluster based on the sample feature data in the target cluster; and determining the type of the data stream to be identified based on the label of the target cluster.

[0057] In an optional embodiment of this application, the posterior probability of each type of sample feature data in the target cluster is calculated, and the application labels of sample feature data of the same type are the same; the application label of the sample feature data with the highest posterior probability in the target cluster is determined as the label of the target cluster.

[0058] Optionally, the target cluster includes feature data of q class samples. The label L of the target cluster can be determined using a label determination function, which is:

[0059] L = arg max P = (Y = y j ), j = 1, 2, ..., q, where q is a positive integer.

[0060] Where, P = (Y = y j ) represents the probability that the label of the target cluster is the application label of the feature data of the j-th class sample. argmax is a function that evaluates the parameters of a function, which means finding the parameter with the maximum score.

[0061] For example, suppose the target cluster includes 100 sample feature data points. These 100 sample feature data points include 10 sample feature data points labeled as known application 1, 7 sample feature data points labeled as known application 2, 8 sample feature data points labeled as known application 3, and 75 sample feature data points labeled as unknown applications. The posterior probability of the sample feature data belonging to known application 1 is 0.1, the posterior probability of belonging to known application 2 is 0.07, the posterior probability of belonging to known application 3 is 0.08, and the posterior probability of belonging to unknown applications is 0.75. Since the posterior probability of belonging to unknown applications is the highest, the application label for this target cluster is determined to be "unknown application." Similarly, if the posterior probability of the sample feature data belonging to known application 2 is the highest in the target cluster, then the application label for this target cluster is determined to be "known application 2."

[0062] In another optional embodiment of this application, when there is no sample feature data labeled with a known application tag in the target cluster, the tag of the target cluster is determined to be an unknown application.

[0063] Optionally, after obtaining the target cluster where the target feature data is located, it is detected whether there is sample feature data labeled with known application tags in the target cluster. When there is sample feature data labeled with known application tags in the target cluster, the posterior probability of each type of sample feature data in the target cluster is calculated. The application tag of the sample feature data with the highest posterior probability in the target cluster is determined as the tag of the target cluster.

[0064] By detecting whether there are sample feature data labeled with known application tags in the target cluster, when there are no sample feature data labeled with known application tags in the target cluster, the tag of the target cluster can be directly determined to be an unknown application, without having to calculate the posterior probability of various sample feature data, which can reduce the amount of computation.

[0065] Optionally, when in step 1032, n clusterers are used to perform clustering processing on the target feature data and multiple sample feature data respectively to obtain n target clusters where the target feature data is located, the implementation of step 1033 above includes: determining the type of the data stream to be identified based on the labels of the n target clusters.

[0066] Optionally, a weighted integration function can be used to determine the type of the data stream to be identified. This weighted integration function is:

[0067]

[0068] in, Lable xThe label represents the category of the data stream to be identified, and the label represents the set of optional application categories, which includes known applications and unknown applications. ω i J represents the maximum posterior probability corresponding to the i-th target cluster out of n target clusters. i (x) represents the label of the i-th target cluster.

[0069] For example, five clusterers are used to cluster the target feature data and multiple sample feature data respectively, resulting in five target clusters. The target cluster corresponding to clusterer 1 is labeled "Unknown Application" with a posterior probability of 100%; the target cluster corresponding to clusterer 2 is labeled "Known Application 1" with a posterior probability of 60%; the target cluster corresponding to clusterer 3 is labeled "Unknown Application" with a posterior probability of 80%; the target cluster corresponding to clusterer 4 is labeled "Known Application 2" with a posterior probability of 50%; and the target cluster corresponding to clusterer 5 is labeled "Unknown Application" with a posterior probability of 75%. Based on the weighted ensemble function, the probability that the data stream to be identified belongs to an unknown application is calculated as: 1×100%+0+1×80%+0+1×75%=2.55, and the probability that the data stream to be identified belongs to a known application is: 0+1×60%+0+1×50%+0=1.1. Therefore, the data stream to be identified belongs to an unknown application.

[0070] Multiple clusterers are used to cluster the target feature data and multiple sample feature data respectively, resulting in multiple target clusters where the target feature data belongs. After determining the labels and corresponding posterior probabilities of these multiple target clusters, a weighted ensemble method is used to determine the category of the data stream to be identified where the target feature data belongs. Compared with using a single clusterer, this method can reduce the influence of noise and independent points, and enhance the stability and robustness of the clustering results.

[0071] Optionally, after performing step 1033, after determining that the data stream to be identified belongs to an unknown application, the target feature data is labeled with an unknown application label; and the target feature data labeled with the unknown application label is added to the cluster sample set to obtain an updated cluster sample set.

[0072] Since unknown applications constitute a large and diverse set, after determining that the data stream to be identified belongs to an unknown application, adding the target feature data labeled with the unknown application tag to the clustering sample set can increase the richness of the clustering sample set and improve the accuracy of subsequent classification. Furthermore, in this embodiment, after obtaining new feature data belonging to an unknown application, there is no need to manually label new unknown sample instances, thus reducing labor costs.

[0073] Optionally, in the embodiments of this application, the implementation process of step 103 may further include: inputting the target feature data into a pre-trained classifier, and determining the type of the data stream to be identified based on the output of the classifier.

[0074] The deep learning model provided in this application embodiment is trained from multiple sample data streams labeled with application tags. These multiple sample data streams include sample data streams labeled with known application tags and sample data streams labeled with unknown application tags. Optionally, a network device or server collects multiple data streams in a data center network, and the collected data streams are labeled with application tags to obtain sample data streams.

[0075] Optionally, Figure 3 This is a schematic diagram of the structure of a deep learning model provided in an embodiment of this application. Figure 3 As shown, this deep learning model includes convolutional neural networks (CNNs) and an optimization network. The CNNs consist of an input layer, hidden layers, and an output layer connected in sequence. The optimization network is connected to the output layer of the CNNs. Optionally, the optimization network can be a multilayer perceptron (MLP).

[0076] The input layer sends the first data packet of the data stream to be identified to the hidden layer; the hidden layer extracts and combines features from the first data packet of the data stream to be identified to obtain feature data; the output layer sends the feature data to the optimization network; the optimization network reassembles the feature data to obtain the target feature data.

[0077] Optionally, the hidden layer includes nine sequentially connected data processing layers, each including a sequentially connected convolutional layer, residual layer, and pooling layer. Optionally, the pooling layer can be a max pooling layer or an average pooling layer; this embodiment does not limit this. The hidden layer may also include eight, ten, or other numbers of data processing layers; this embodiment does not limit the number of data processing layers included in the hidden layer. The hidden layer performs feature extraction and feature combination on the data input to the input layer through multidimensional convolutional filtering and nonlinear mapping.

[0078] For example, Figure 4 This is a schematic diagram of the structure of a convolutional neural network provided in an embodiment of this application. For example... Figure 4 As shown, the input to this convolutional neural network is the first N bytes of the first data packet in the data stream, and the output is a k-dimensional feature vector, where N can be 512. The hidden layers consist of nine sequentially connected data processing layers, each of which includes a sequentially connected convolutional layer, residual layer, and pooling layer. Please continue reading... Figure 4The hidden layer also includes a feature output layer and a fully connected layer connected in sequence. The feature output layer is used to convert the data output by the data processing layer into feature vectors, and the fully connected layer is used to integrate local features with class discrimination in the convolutional layer, residual layer and / or pooling layer.

[0079] During the training of a convolutional neural network (CNN), the output layer of the CNN is connected to a supervision module, which can be a classifier. This supervision module receives sample feature data from the output layer, identifies the sample data stream based on this feature data, outputs the identification result, and then calculates the loss value of the loss function in the current supervision module based on the identification result and the application labels of the sample data stream. It then transmits the gradient information of the loss function back to the input layer. This gradient information is used to adjust the network parameters in the CNN. The sample feature data is the feature data output by the output layer after the first data packet of the sample data stream is input into the deep learning model.

[0080] For example, Figure 5 This is a schematic diagram of an optimized network structure provided in an embodiment of this application. For example... Figure 5 As shown, the input to this optimization network is a k-dimensional feature vector [C1, C2, ..., C]. k The output is another k-dimensional feature vector [Z1, Z2, ..., Z]. k The optimization network is used to optimize the input k-dimensional feature vector [C1, C2, ..., C]. k Feature recombination is performed. This optimized network can use features other than the i-th feature from the input k-dimensional feature vector to predict the i-th feature, and use the predicted value of the i-th feature as the output i-th feature. The objective function of this optimized network can be defined as follows:

[0081]

[0082] In this context, a smaller value of L indicates lower coupling and stronger orthogonality among the k-dimensional feature vectors output by the optimized network. During the training of the optimized network, the network parameters are adjusted by minimizing the value of L to achieve optimal performance. Therefore, using the optimized network to reconstruct features from the output of a convolutional neural network can yield feature data with low coupling and strong orthogonality.

[0083] Optionally, the deep learning model provided in the embodiments of this application may also include only the following: Figure 4 The convolutional neural network shown above outputs the same feature data as the deep learning model.

[0084] The order of steps in the data stream identification method provided in this application can be adjusted appropriately, and steps can also be added or removed as needed. Any variations that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application, and therefore will not be elaborated further.

[0085] In the data stream identification method provided in this application embodiment, the first data packet of the data stream to be identified is input into a deep learning model, and the type of the data stream to be identified is determined based on the target feature data output by the deep learning model according to the first data packet. Since the deep learning model can automatically learn the distinguishing features between known and unknown application data streams, the reliability of the feature data output by the deep learning model is higher than that determined based on human experience, thus improving the accuracy of data stream identification. Furthermore, multiple clusterers are used to cluster the target feature data and multiple sample feature data respectively, obtaining multiple target clusters where the target feature data belongs. After determining the labels and corresponding posterior probabilities of these multiple target clusters, a weighted ensemble method is used to determine the category of the data stream to be identified where the target feature data belongs. This weakens the influence of noise and independent points, enhances the stability and robustness of the clustering results, and further improves the accuracy of data stream identification.

[0086] Figure 6 This is a schematic diagram of the structure of a data stream identification device provided in an embodiment of this application. Figure 6 As shown, device 60 includes:

[0087] Input module 601 is used to input the first data packet of the data stream to be identified into the deep learning model.

[0088] The receiving module 602 is used to receive the target feature data output by the deep learning model based on the first data packet.

[0089] The determination module 603 is used to determine the type of the data stream to be identified based on the target feature data, which includes known applications or unknown applications.

[0090] The deep learning model is trained from multiple sample data streams labeled with application tags.

[0091] In the data stream identification device provided in this application embodiment, the first data packet of the data stream to be identified is input into the deep learning model through the input module, and the type of the data stream to be identified is determined by the determination module based on the target feature data output by the deep learning model according to the first data packet. Since the deep learning model can automatically learn the distinguishing features between known application data streams and unknown application data streams, the feature data output by the deep learning model has higher reliability compared with feature data determined based on human experience, thereby improving the accuracy of data stream identification.

[0092] Optionally, the deep learning model includes a convolutional neural network and an optimization network. The convolutional neural network includes an input layer, a hidden layer, and an output layer connected in sequence, and the optimization network is connected to the output layer.

[0093] The input layer sends the first data packet of the data stream to be identified to the hidden layer; the hidden layer extracts and combines features from the first data packet of the data stream to be identified to obtain feature data; the output layer sends the feature data to the optimization network; the optimization network reassembles the feature data to obtain the target feature data.

[0094] Optionally, the hidden layer includes nine data processing layers connected in sequence, each of which includes a convolutional layer, a residual layer, and a pooling layer connected in sequence.

[0095] Optionally, such as Figure 7 As shown, module 603 includes:

[0096] The acquisition submodule 6031 is used to acquire a clustered sample set, which includes multiple sample feature data, including sample feature data labeled with known application tags and sample feature data labeled with unknown application tags.

[0097] Clustering submodule 6032 is used to perform clustering processing on target feature data and multiple sample feature data.

[0098] The determination submodule 6033 is used to determine the type of the data stream to be identified based on the sample feature data in the target cluster where the target feature data is located.

[0099] Optionally, a submodule is defined for:

[0100] Based on the sample feature data in the target cluster, determine the label of the target cluster; based on the label of the target cluster, determine the type of the data stream to be identified.

[0101] In one implementation, determining the submodule is also used for:

[0102] Calculate the posterior probability of feature data for each class of samples in the target cluster. Samples of the same class have the same application label. Determine the application label of the sample feature data with the highest posterior probability in the target cluster as the label of the target cluster.

[0103] In another implementation, identifying submodules is also used for:

[0104] When there is no sample feature data labeled with a known application in the target cluster, the label of the target cluster is determined to be an unknown application.

[0105] Optionally, the clustering submodule is used to: use n clusterers to perform clustering processing on the target feature data and multiple sample feature data respectively, to obtain n target clusters where the target feature data is located, where n is an integer greater than 1, and the number of clusters and / or the initial cluster centers of different clusterers are different among the n clusterers; correspondingly, the determination submodule is used to: determine the type of the data stream to be identified based on the labels of the n target clusters.

[0106] Optionally, determining the submodule is also used for:

[0107] A weighted integration function is used to determine the type of the data stream to be identified. The weighted integration function is as follows:

[0108]

[0109] in, Lable x The label represents the category of the data stream to be identified, and the label represents the set of possible application categories, which includes known applications and unknown applications. ω i J represents the maximum posterior probability corresponding to the i-th target cluster out of n target clusters. i (x) represents the label of the i-th target cluster.

[0110] Optionally, such as Figure 8 As shown, device 60 also includes:

[0111] The annotation module 604 is used to annotate the target feature data with unknown application labels when the data stream to be identified belongs to an unknown application;

[0112] The update module 605 is used to add target feature data labeled with unknown application tags to the cluster sample set, so as to obtain the updated cluster sample set.

[0113] Optionally, the multiple sample data streams include sample data streams labeled with known application tags and sample data streams labeled with unknown application tags. The acquisition submodule is used to: input the first data packet of the multiple sample data streams into the deep learning model respectively, and use the feature data output by the deep learning model as sample feature data.

[0114] In the data stream identification device provided in this application embodiment, the first data packet of the data stream to be identified is input into the deep learning model through the input module, and the type of the data stream to be identified is determined by the determination module based on the target feature data output by the deep learning model according to the first data packet. Since the deep learning model can automatically learn the distinguishing features between known application data streams and unknown application data streams, the feature data output by the deep learning model has higher reliability compared with feature data determined based on human experience, thereby improving the accuracy of data stream identification. In addition, multiple clusterers are used to cluster the target feature data and multiple sample feature data respectively to obtain multiple target clusters in which the target feature data belongs. After determining the labels and corresponding posterior probabilities of the multiple target clusters, a weighted ensemble method is used to determine the category of the data stream to be identified in which the target feature data belongs. This can reduce the influence of noise and independent points, enhance the stability and robustness of the clustering results, and further improve the accuracy of data stream identification.

[0115] Figure 9 This is a block diagram of a data stream identification device provided in an embodiment of this application. The data stream identification device can be a server or a network device. Figure 9 As shown, the data stream identification device 90 includes: a processor 901 and a memory 902;

[0116] The memory 902 is used to store a computer program, the computer program including program instructions;

[0117] The processor 901 is used to call the computer program to implement, for example... Figure 1 The data stream identification method shown.

[0118] Optionally, the network device 90 also includes a communication bus 903 and a communication interface 904.

[0119] The processor 901 includes one or more processing cores, and the processor 901 executes various functional applications and data processing by running computer programs.

[0120] The memory 902 can be used to store computer programs. Optionally, the memory can store the operating system and application program units required for at least one function. The operating system can be a real-time operating system (RTX), Linux, UNIX, Windows, or OS X.

[0121] There can be multiple communication interfaces 904, which are used to communicate with other storage devices or network devices. For example, in this embodiment, the communication interface 904 can be used to receive data streams sent by network devices in a communication network.

[0122] The memory 902 and the communication interface 904 are respectively connected to the processor 901 via the communication bus 903.

[0123] This application embodiment also provides a computer storage medium storing instructions that, when executed by a processor, implement the following: Figure 1 The data stream identification method shown.

[0124] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0125] In the embodiments of this application, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The term "at least one" refers to one or more, and the term "multiple" refers to two or more, unless otherwise expressly defined.

[0126] In this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0127] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the concept and principles of this application should be included within the protection scope of this application.

Claims

1. A data stream identification method, characterized in that, The method, applied to network devices or servers in a communication network, includes: The first data packet of the data stream to be identified is input into the deep learning model; the data stream to be identified is a data stream from the application. Receive the target feature data output by the deep learning model based on the first data packet; Obtain a clustered sample set, which includes multiple sample feature data, including sample feature data labeled with known application tags and sample feature data labeled with unknown application tags; n clusterers are used to perform clustering processing on the target feature data and the multiple sample feature data respectively, to obtain n target clusters in which the target feature data belongs, where n is an integer greater than 1, and the number of clusters and / or the initial cluster centers of different clusterers are different among the n clusterers; Based on the sample feature data of the n target clusters, determine the labels of the n target clusters; Based on the labels of the n target clusters and the posterior probabilities corresponding to the n target clusters, the type of the data stream to be identified is determined, and the type includes known applications or unknown applications; The deep learning model is trained from multiple sample data streams labeled with application tags, including sample data streams labeled with known application tags and sample data streams labeled with unknown application tags.

2. The method according to claim 1, characterized in that, The deep learning model includes a convolutional neural network and an optimization network. The convolutional neural network includes an input layer, a hidden layer, and an output layer connected in sequence. The optimization network is connected to the output layer. The input layer is used to send the first data packet of the data stream to be identified to the hidden layer; The hidden layer is used to extract and combine features from the first data packet of the data stream to be identified, so as to obtain feature data; The output layer is used to send the feature data to the optimized network; The optimized network is used to reorganize the feature data to obtain the target feature data.

3. The method according to claim 2, characterized in that, The hidden layer comprises nine data processing layers connected in sequence, each of which includes a convolutional layer, a residual layer, and a pooling layer connected in sequence.

4. The method according to claim 1, characterized in that, The step of determining the labels of the n target clusters based on the sample feature data in the n target clusters includes: Calculate the posterior probability of each class of sample feature data in any target cluster among the n target clusters, where sample feature data of the same class have the same application label; The application label of the sample feature data with the highest posterior probability in any target cluster is determined as the label of any target cluster.

5. The method according to claim 1, characterized in that, The step of determining the labels of the n target clusters based on the sample feature data in the n target clusters includes: If no sample feature data labeled with a known application tag exists in any of the n target clusters, then the tag of any target cluster is determined to be an unknown application.

6. The method according to any one of claims 1 to 5, characterized in that, The process of determining the type of the data stream to be identified based on the labels of the n target clusters and the posterior probabilities corresponding to the n target clusters includes: The type of the data stream to be identified is determined by using a weighted integration function, wherein the weighted integration function is: in, Lable x The label represents the category of the data stream to be identified, and the label represents the set of optional application categories, which includes known applications and unknown applications. ω i J represents the maximum posterior probability corresponding to the i-th target cluster among the n target clusters. i (x) represents the label of the i-th target cluster.

7. The method according to any one of claims 1 to 5, characterized in that, After determining the type of the data stream to be identified, the method further includes: When the data stream to be identified belongs to an unknown application, the target feature data is labeled with an unknown application tag; The target feature data labeled with unknown application tags is added to the clustering sample set to obtain the updated clustering sample set.

8. The method according to any one of claims 1 to 5, characterized in that, The process of obtaining the clustering sample set includes: The first data packet of each of the multiple sample data streams is input into the deep learning model, and the feature data output by the deep learning model is used as the sample feature data.

9. A data stream identification device, characterized in that, The device, used in network equipment or servers in communication networks, includes: The input module is used to input the first data packet of the data stream to be identified into the deep learning model, wherein the data stream to be identified is a data stream from the application; The receiving module is used to receive the target feature data output by the deep learning model based on the first data packet; The module to be determined includes: The acquisition submodule is used to acquire a clustered sample set, which includes multiple sample feature data, including sample feature data labeled with known application tags and sample feature data labeled with unknown application tags. The clustering submodule is used to perform clustering processing on the target feature data and the multiple sample feature data using n clusterers respectively, to obtain n target clusters where the target feature data is located, where n is an integer greater than 1, and the number of clusters and / or the initial cluster centers of different clusterers among the n clusterers are different; The determination submodule is used to determine the labels of the n target clusters based on the sample feature data in the n target clusters; and to determine the type of the data stream to be identified based on the labels of the n target clusters and the posterior probabilities corresponding to the n target clusters, wherein the type includes known applications and unknown applications. The deep learning model is trained from multiple sample data streams labeled with application tags, including sample data streams labeled with known application tags and sample data streams labeled with unknown application tags.

10. The apparatus according to claim 9, characterized in that, The deep learning model includes a convolutional neural network and an optimization network. The convolutional neural network includes an input layer, a hidden layer, and an output layer connected in sequence. The optimization network is connected to the output layer. The input layer is used to send the first data packet of the data stream to be identified to the hidden layer; The hidden layer is used to extract and combine features from the first data packet of the data stream to be identified, so as to obtain feature data; The output layer is used to send the feature data to the optimized network; The optimized network is used to reorganize the feature data to obtain the target feature data.

11. The apparatus according to claim 10, characterized in that, The hidden layer comprises nine data processing layers connected in sequence, each of which includes a convolutional layer, a residual layer, and a pooling layer connected in sequence.

12. The apparatus according to claim 9, characterized in that, The determining submodule is further configured to: Calculate the posterior probability of each class of sample feature data in any target cluster among the n target clusters, where sample feature data of the same class have the same application label; The application label of the sample feature data with the highest posterior probability in any target cluster is determined as the label of any target cluster.

13. The apparatus according to claim 9, characterized in that, The determining submodule is further configured to: If no sample feature data labeled with a known application exists in any of the n target clusters, then the label of any target cluster is determined to be an unknown application.

14. The apparatus according to any one of claims 9 to 13, characterized in that, The determining submodule is further configured to: The type of the data stream to be identified is determined by using a weighted integration function, wherein the weighted integration function is: in, Lable x The label represents the category of the data stream to be identified, and the label represents the set of optional application categories, which includes known applications and unknown applications. ω i J represents the maximum posterior probability corresponding to the i-th target cluster among the n target clusters. i (x) represents the label of the i-th target cluster.

15. The apparatus according to any one of claims 9 to 13, characterized in that, The device further includes: The annotation module is used to annotate the target feature data with unknown application tags when the data stream to be identified belongs to an unknown application; The update module is used to add the target feature data labeled with unknown application tags to the clustering sample set to obtain the updated clustering sample set.

16. The apparatus according to any one of claims 9 to 13, characterized in that, The acquisition submodule is used for: The first data packet of each of the multiple sample data streams is input into the deep learning model, and the feature data output by the deep learning model is used as the sample feature data.

17. A data stream identification device, characterized in that, include: Processor and memory; The memory is used to store computer programs, the computer programs including program instructions; The processor is used to invoke the computer program to implement the data stream identification method as described in any one of claims 1 to 8.

18. A computer storage medium, characterized in that, The computer storage medium stores instructions that, when executed by a processor, implement the data stream identification method as described in any one of claims 1 to 8.

Images