Traffic monitoring method, device, equipment, storage medium and program product

Abstract

The application provides a traffic monitoring method and device, equipment, storage medium and program product, and relates to the field of communication. The method comprises the following steps: obtaining a plurality of network traffic data samples of a plurality of time periods; the plurality of time periods correspond to the plurality of network traffic data samples one by one; extracting semantic features of the plurality of network traffic data samples by using a first model; extracting time sequence features of the plurality of network traffic data samples by using a second model; clustering the plurality of network traffic data samples according to the semantic features and the time sequence features to obtain network traffic data samples of different categories; and determining abnormal network traffic data samples from the network traffic data samples of different categories. The method is suitable for the network traffic monitoring process. The method is used for solving the problem that the deep packet detection monitoring precision is not high.

Description

Technical Field

[0001] This application relates to the field of communications, and more particularly to a traffic monitoring method, apparatus, device, storage medium, and program product. Background Technology

[0002] With the rapid development of Internet technology, network traffic has become more complex and larger than ever before. Accurate monitoring of network traffic plays a decisive role in ensuring network security, optimizing bandwidth usage, and improving service quality.

[0003] However, traditional deep packet inspection (DPI) suffers from low monitoring accuracy when dealing with large-scale and complex network traffic, especially when it comes to encrypted traffic (such as traffic related to Secure Sockets Layer (SSL) protocol, Transport Layer Security (TLS) protocol, and traffic related to other new network protocols). Summary of the Invention

[0004] This application provides a traffic monitoring method, device, equipment, storage medium, and program product, which can extract the features of network traffic data samples through a model and perform clustering to achieve accurate clustering of network traffic and effectively identify abnormal traffic.

[0005] In a first aspect, this application provides a traffic monitoring method, which includes: acquiring multiple network traffic data samples over multiple time periods; corresponding each time period to the multiple network traffic data samples; extracting semantic features of each of the multiple network traffic data samples using a first model; extracting temporal features of each of the multiple network traffic data samples using a second model; clustering the multiple network traffic data samples based on the semantic features and temporal features to obtain network traffic data samples of different categories; and identifying abnormal network traffic data samples from the network traffic data samples of different categories.

[0006] Optionally, multiple network traffic data samples are clustered based on semantic and temporal features, including: using a clustering model to cluster multiple network traffic data samples based on semantic and temporal features at an adaptive learning rate; wherein the magnitude of the adaptive learning rate is positively correlated with the severity of the network environment when acquiring multiple network traffic data samples.

[0007] Optionally, a clustering model is used to cluster multiple network traffic data samples based on semantic and temporal features at an adaptive learning rate, including:

[0008] Clustering is performed using the K-means clustering model, which satisfies the following formula:

[0009]

[0010] Among them, C i (t+1) represents the position of the i-th cluster center in the (t+1)-th iteration; α represents the adaptive learning rate; γ represents the severity of the network environment; C i (t) represents the position of the i-th cluster center in the t-th iteration; w j (t) represents the weight of the j-th network traffic data sample at the t-th iteration; x j (t) represents the feature vector of the j-th network traffic data sample at the t-th iteration; the feature vector is used to represent the semantic and temporal features of the network traffic data sample.

[0011] Optionally, γ includes any of the following: the total amount of network traffic data, the volatility of network traffic data, or the security threat level.

[0012] Optionally, the first model is trained by: obtaining a first training sample set; the first training sample set includes multiple first training samples; each first training sample includes training network traffic data and semantic feature labels corresponding to the training network traffic data; and training the first initial model based on the first training sample set to obtain the first model.

[0013] Optionally, the second model is trained by: obtaining a second training sample set; the second training sample set includes multiple second training samples; each second training sample includes training network traffic data and the time-series feature label corresponding to the training network traffic data; and training the second initial model based on the second training sample set to obtain the second model.

[0014] Optionally, the method further includes: visualizing different categories of network traffic data samples in at least one of the following ways: line charts, bar charts, and heatmaps.

[0015] The traffic monitoring method provided in this application can perform deep analysis of network traffic data samples based on a deep learning model, extract semantic and temporal features, and cluster the network traffic data samples according to the extracted semantic and temporal features to achieve accurate clustering of network flows, which can effectively identify abnormal traffic. It can also be optimized for identifying encrypted traffic, improving the ability to identify complex traffic patterns and traffic related to emerging protocols.

[0016] Furthermore, the traffic monitoring method provided in this application can cluster multiple network traffic data samples using an adaptive learning rate. The magnitude of the adaptive learning rate is positively correlated with the severity of the network environment when acquiring the multiple network traffic data samples. Thus, when the network environment is more severe, potentially introducing more noise and interference, a higher learning rate helps the clustering model adapt to this noise and interference more quickly, improving its robustness. Conversely, when the network environment is more stable when acquiring multiple network traffic data samples, the clustering model can cluster more smoothly, updating weights with smaller changes in each iteration, thereby reducing oscillations during the clustering process and improving model stability.

[0017] Secondly, this application provides a traffic monitoring device, which includes an acquisition module and a processing module.

[0018] The acquisition module is used to acquire multiple network traffic data samples across multiple time periods; each time period corresponds one-to-one with a single network traffic data sample.

[0019] The processing module is used to extract the semantic features of multiple network traffic data samples using a first model; extract the temporal features of multiple network traffic data samples using a second model; cluster the multiple network traffic data samples according to the semantic features and temporal features to obtain network traffic data samples of different categories; and identify abnormal network traffic data samples from the network traffic data samples of different categories.

[0020] Optionally, the processing module is specifically used to cluster multiple network traffic data samples based on semantic features and temporal features using a clustering model at an adaptive learning rate; wherein the magnitude of the adaptive learning rate is positively correlated with the severity of the network environment when acquiring multiple network traffic data samples.

[0021] Optionally, the processing module is specifically used to perform clustering using a K-means clustering model, which satisfies the following formula:

[0022]

[0023] Among them, C i (t+1) represents the position of the i-th cluster center in the (t+1)-th iteration; α represents the adaptive learning rate; γ represents the severity of the network environment; C i (t) represents the position of the i-th cluster center in the t-th iteration; w j (t) represents the weight of the j-th network traffic data sample at the t-th iteration; x j(t) represents the feature vector of the j-th network traffic data sample at the t-th iteration; the feature vector is used to represent the semantic and temporal features of the network traffic data sample.

[0024] Optionally, γ includes any of the following: the total amount of network traffic data, the volatility of network traffic data, or the security threat level.

[0025] Optionally, the first model is trained by: obtaining a first training sample set; the first training sample set includes multiple first training samples; each first training sample includes training network traffic data and semantic feature labels corresponding to the training network traffic data; and training the first initial model based on the first training sample set to obtain the first model.

[0026] Optionally, the second model is trained by: obtaining a second training sample set; the second training sample set includes multiple second training samples; each second training sample includes training network traffic data and the time-series feature label corresponding to the training network traffic data; and training the second initial model based on the second training sample set to obtain the second model.

[0027] Optionally, the processing module is also used to visualize different categories of network traffic data samples in at least one of the following ways: line charts, bar charts, and heatmaps.

[0028] Thirdly, this application provides a computer program product that, when run on a computer, causes the computer to perform the method described in the first aspect above.

[0029] Fourthly, this application provides an electronic device comprising: a processor and a memory; the memory storing processor-executable instructions; when the processor is configured to execute the instructions, causing the electronic device to perform the method described in the first aspect above.

[0030] Fifthly, this application provides a computer-readable storage medium comprising: computer software instructions; when the computer software instructions are executed in an electronic device, they cause the electronic device to perform the method described in the first aspect above.

[0031] The beneficial effects of the second to fifth aspects mentioned above can be referred to the first aspect, and will not be repeated here. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a schematic diagram of the composition of the traffic monitoring system provided in the embodiments of this application;

[0034] Figure 2 A flowchart illustrating the traffic monitoring method provided in this application embodiment;

[0035] Figure 3 A flowchart of traffic monitoring provided for embodiments of this application;

[0036] Figure 4 Another flowchart illustrating the traffic monitoring method provided in this application embodiment;

[0037] Figure 5 This is a schematic diagram of the composition of the traffic monitoring device provided in the embodiments of this application;

[0038] Figure 6 This is a schematic diagram illustrating the composition of an electronic device provided in an embodiment of this application. Detailed Implementation

[0039] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0040] It should be noted that in the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0041] To facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish the same or similar items with essentially the same function and effect. Those skilled in the art can understand that the terms "first" and "second" are not intended to limit the quantity or execution order.

[0042] With the rapid development of Internet technology, network traffic has become more complex and larger than ever before. Accurate monitoring of network traffic plays a decisive role in ensuring network security, optimizing bandwidth usage, and improving service quality.

[0043] However, traditional deep packet inspection (DPI) suffers from low monitoring accuracy when dealing with large-scale and complex network traffic, especially when it comes to encrypted traffic (such as traffic related to Secure Sockets Layer (SSL) protocol, Transport Layer Security (TLS) protocol, and traffic related to other new network protocols).

[0044] Based on this, embodiments of this application provide a traffic monitoring method, apparatus, device, storage medium, and program product, which can extract the features of network traffic data samples through models and perform clustering to achieve accurate clustering of network traffic and effectively identify abnormal traffic.

[0045] The following section will discuss possession in conjunction with spirit possession.

[0046] Figure 1 This is a schematic diagram illustrating the composition of the traffic monitoring system provided in an embodiment of this application. Figure 1 As shown, the system may include a data acquisition device 100 and a traffic monitoring device 200. The data acquisition device 100 and the traffic monitoring device 200 can be connected via a wired network or a wireless network.

[0047] The data acquisition device 100 may be a high-performance packet sniffing device (or may also be called a sniffer or network sniffer, etc.) deployed at critical network nodes.

[0048] The data acquisition device 100 can be used to collect network traffic data.

[0049] In some embodiments, the data acquisition device 100 can also be used to send the acquired network traffic data to the traffic monitoring device 200.

[0050] The traffic monitoring device 200 can be an electronic device such as a computer or server.

[0051] The server can be a single server or a server cluster consisting of multiple servers. In some embodiments, the server cluster can also be a distributed cluster. Optionally, the server can also be implemented on a cloud platform, such as a private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, inter-cloud, and multi-cloud, or any combination thereof. This application does not impose any limitations on this.

[0052] The traffic monitoring device 200 can be used to analyze and identify abnormal network traffic based on the network traffic data collected by the data acquisition device 100. The specific process can be referred to in the following method embodiments, and will not be repeated here.

[0053] In some embodiments, the traffic monitoring device 200 can also be used to receive network traffic data sent from the data acquisition device 100.

[0054] It should be noted that the above description uses the data acquisition device 100 and the traffic monitoring device 200 as independent devices. Optionally, the data acquisition device 100 and the traffic monitoring device 200 can also be integrated into one unit. That is, the data acquisition device 100 or its corresponding function, and the traffic monitoring device 200 or its corresponding function can be integrated into one device. For example, a data packet sniffing device with traffic monitoring function. This application embodiment does not limit this.

[0055] For simplicity, the following description will use a traffic monitoring device as the main execution unit.

[0056] Figure 2 This is a flowchart illustrating the traffic monitoring method provided in an embodiment of this application. Figure 2 As shown, the method includes the following steps:

[0057] S101. Obtain multiple network traffic data samples from multiple time periods.

[0058] In this embodiment, multiple time periods correspond one-to-one with multiple network traffic data samples. Network traffic data samples may include basic attributes of data packets, such as packet size, transmission direction (inbound or outbound), source Internet Protocol (IP) address, destination IP address, type of network protocol used (e.g., Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Internet Control Message Protocol (ICMP), SSL, or TLS, etc.), and timestamps. This disclosure does not limit the specific types of basic attributes.

[0059] For example, as mentioned above, a traffic monitoring device can receive network traffic data from a data acquisition device. In this case, the traffic monitoring device can divide the network traffic data into different network traffic data samples according to different time periods.

[0060] Optionally, after receiving network traffic data from the data acquisition device, the traffic monitoring device can also perform a series of preprocessing operations on the network traffic data.

[0061] The main purpose of preprocessing is to clean and prepare the data, making it more suitable for input to deep learning models. Preprocessing operations can include at least one of the following: denoising, filtering, feature extraction, and standardization. Denoising involves removing meaningless or erroneous data packets. Filtering involves filtering out data packets that do not provide useful information, such as empty packets or packets containing only protocol headers. Feature extraction extracts meaningful information from the data packets, such as the distribution of traffic volume within a statistical period and the arrival interval of data packets. Standardization normalizes numerical features, ensuring that each feature is compared and learned on the same scale.

[0062] Optionally, the traffic monitoring device can also dynamically adjust the number of network traffic data samples based on the network environment status when acquiring multiple network traffic data samples, ensuring that subsequent clustering can promptly reflect the changing trends of traffic characteristics. For example, the traffic monitoring device can remove noisy data and add newly incoming data points.

[0063] S102. Use the first model to extract the semantic features of each of the multiple network traffic data samples.

[0064] Optionally, the traffic monitoring device may specifically utilize a convolutional neural network (CNN) model to extract semantic features from network traffic data samples.

[0065] For example, a network traffic data sample can be viewed as a special "image". Each row or column can represent a set of traffic features within a time period. A CNN model can scan and extract features from this "image" to obtain semantic features.

[0066] For example, a convolutional neural network model may include convolutional layers, pooling layers, and fully connected layers.

[0067] Convolutional layers can learn various traffic patterns, such as brief traffic spikes and periodic traffic fluctuations. Pooling layers then perform dimensionality reduction and feature filtering, reducing data redundancy while preserving key features. Finally, after a series of convolutional and pooling layers, fully connected layers integrate the extracted local features to generate high-level semantic features with semantic information, which can then be used for subsequent classification or anomaly detection tasks.

[0068] S103. Use the second model to extract the temporal features of each of the multiple network traffic data samples.

[0069] Optionally, the traffic monitoring device can specifically utilize a long short-term memory (LSTM) network model to extract the temporal features of network traffic data samples.

[0070] Among them, the LSTM model can focus on capturing the time-series characteristics of network traffic. It can understand and remember long-term relationships between traffic, which is important for identifying and understanding network traffic data with time-series characteristics, such as traffic related to SSL and TLS protocols. Through its unique gating mechanism, LSTM can selectively retain past information when processing network traffic data over long periods, thereby better understanding the contextual information of the traffic and identifying potentially anomalous traffic.

[0071] S104. Cluster multiple network traffic data samples based on semantic features and temporal features to obtain network traffic data samples of different categories.

[0072] In some possible embodiments, the traffic monitoring device may specifically use a clustering model to cluster multiple network traffic data samples based on semantic and temporal features at an adaptive learning rate.

[0073] The adaptive learning rate is positively correlated with the severity of the network environment when acquiring multiple network traffic data samples. For example, the more severe the network environment when acquiring multiple network traffic data samples, the higher the adaptive learning rate; conversely, the more stable the network environment, the lower the adaptive learning rate.

[0074] Alternatively, the traffic monitoring device can specifically utilize the K-means clustering model for clustering.

[0075] Alternatively, the K-means clustering model can satisfy the following formula (1):

[0076]

[0077] In formula (1), C i (t+1) represents the position of the i-th cluster center in the (t+1)-th iteration. α represents the adaptive learning rate. γ represents the severity of the network environment; for example, γ can include any of the following: the total amount of network traffic data, the volatility of network traffic data (volume), or the level of security threats. α(γ,t) indicates that the adaptive learning rate varies in different iterations of clustering and is related to γ. C i (t) represents the position of the i-th cluster center in the t-th iteration; w j (t) represents the weight of the j-th network traffic data sample at the t-th iteration; x j (t) represents the feature vector of the j-th network traffic data sample at the t-th iteration. The feature vector is used to represent the semantic and temporal features of the network traffic data sample.

[0078] S105. Identify abnormal network traffic data samples from different categories of network traffic data samples.

[0079] For example, abnormal network traffic data samples may include network traffic data samples during sudden traffic surges, network traffic data samples during frequent network scanning activities, and other network traffic data samples that do not conform to normal business logic.

[0080] In one possible implementation, abnormal network traffic data samples can be manually identified by staff, who then input instructions (e.g., marking instructions) into the traffic monitoring device. The traffic monitoring device can then specifically identify abnormal network traffic data samples from different categories of network traffic data samples based on the staff's instructions.

[0081] In another possible implementation, the traffic monitoring device can pre-store reference network traffic data samples (such as normal network traffic data samples or abnormal network traffic data samples). The traffic monitoring device can determine the abnormal network traffic data samples by comparing each type of network traffic data sample with the reference network traffic data samples and based on the comparison results.

[0082] For example, a traffic monitoring device can calculate the key feature values ​​of network traffic data samples of each category and the key feature values ​​of a reference network traffic data sample, and then compare these two key feature values. If the key feature values ​​are similar (for example, the difference between the two is less than or equal to a preset threshold), the traffic monitoring device can determine that the network traffic data sample of that category and the reference network traffic data sample are network traffic data samples of the same type.

[0083] In other words, when the reference network traffic data sample is a normal network traffic data sample, and the difference between the key feature value of a certain category of network traffic data sample and the key feature value of the reference network traffic data sample is less than or equal to a preset threshold, the traffic monitoring device can determine that the network traffic data sample of that category is a normal network traffic data sample, and then gradually filter out abnormal network traffic data samples that do not belong to the normal network traffic data samples.

[0084] If the reference network traffic data sample is an abnormal network traffic data sample, and the difference between the key feature value of a certain category of network traffic data sample and the key feature value of the reference network traffic data sample is less than or equal to a preset threshold, the traffic monitoring device can determine that the network traffic data sample of that category is an abnormal network traffic data sample.

[0085] In the traffic monitoring method provided in this application embodiment, the traffic monitoring device can perform deep analysis of network traffic data samples based on a deep learning model, extract semantic and temporal features, and cluster the network traffic data samples according to the extracted semantic and temporal features to achieve accurate clustering of network flows, which can effectively identify abnormal traffic. It can also be optimized for identifying encrypted traffic, improving the ability to identify complex traffic patterns and traffic related to emerging protocols.

[0086] Furthermore, in the traffic monitoring method provided in this application embodiment, when the traffic monitoring device clusters multiple network traffic data samples, it can use a clustering model with an adaptive learning rate. The magnitude of the adaptive learning rate is positively correlated with the severity of the network environment when acquiring multiple network traffic data samples. Thus, when the network environment is more severe when acquiring multiple network traffic data samples, more noise and interference may be introduced. Using a higher learning rate helps the clustering model adapt to this noise and interference more quickly, improving the robustness of the clustering model. Conversely, when the network environment is more stable when acquiring multiple network traffic data samples, the clustering model can cluster more smoothly, updating smaller weight changes in each iteration, thereby reducing oscillations in the clustering process and improving model stability.

[0087] For example, consider an enterprise network environment containing a large volume of internal and external communication traffic. When the network suffers a distributed denial-of-service (DDoS) attack, the sudden surge in traffic will cause a significant change in traffic patterns. In this situation, the clustering model can respond quickly by increasing its adaptive learning rate to accelerate the convergence of cluster centers towards the new traffic pattern, thereby rapidly identifying this abnormal traffic and distinguishing it from other normal business traffic. Furthermore, once the attack has ended, the clustering model can quickly revert to a stable clustering state by reducing its adaptive learning rate, tracking and identifying the normal traffic patterns after recovery.

[0088] In some possible embodiments, after identifying abnormal network traffic data samples, the traffic monitoring device can also perform a series of response processes on the abnormal network traffic data samples.

[0089] For example, traffic monitoring devices can be pre-set with rule bases or algorithm bases for automatic decision-making, such as triggering alarms, adjusting network bandwidth allocation, and temporarily blocking certain IP addresses, to ensure network security and effective use of resources.

[0090] In other possible embodiments, after S104 or S105 described above, the traffic monitoring device can also perform a visual display through a monitoring interface.

[0091] For example, traffic monitoring devices can visualize different types of network traffic data samples in at least one of the following ways: line graphs, bar charts, and heatmaps.

[0092] In the traffic monitoring method provided in this application embodiment, the traffic monitoring device can display information such as the global status of network traffic, the location and severity of abnormal events in real time, facilitating maintenance personnel to quickly understand and assess the current network condition. Maintenance personnel can interact with this interface in real time, such as manually reviewing identified abnormal network traffic data samples and providing feedback based on the actual situation. The traffic monitoring device can dynamically adjust its monitoring strategy based on this feedback information, forming a closed-loop feedback mechanism to continuously optimize and improve the accuracy of the entire traffic monitoring process.

[0093] In some embodiments, prior to S102 described above, the traffic monitoring device may also acquire a first model.

[0094] In one possible implementation, the traffic monitoring device can directly obtain the first trained model from other devices.

[0095] For example, traffic monitoring devices can obtain the first trained model from other devices by downloading or transferring it to an intermediate storage medium.

[0096] In another possible implementation, the traffic monitoring device can also train the first model using the first training sample set.

[0097] Alternatively, the first model can be trained in the following way:

[0098] Step 1: Obtain the first training sample set.

[0099] The first training sample set includes multiple first training samples, each of which includes training network traffic data and corresponding semantic feature labels. The training network traffic data can be referenced from the network traffic data sample section above, and will not be repeated here. Semantic feature labels may include at least one of the following: traffic type, traffic behavior, traffic source and destination, traffic size, protocol characteristics, and time characteristics, etc.

[0100] Step 2: Based on the first training sample set, train the first initial model to obtain the first model.

[0101] For example, the initial model could be the CNN model mentioned above or other models capable of extracting semantic features.

[0102] Optionally, as described above, the first training sample set may include multiple first training samples. The traffic monitoring device may input one or more first training samples into the first initial model each time to obtain the semantic features extracted by the first initial model. Then, it may calculate the loss function based on the semantic features extracted by the first initial model and the semantic feature labels, and adjust the parameters in the first initial model until the first initial model converges.

[0103] Optionally, the conditions for the first initial model to converge (or end training) may specifically include: the number of times the first training sample is input into the first initial model reaches a threshold, and / or, the error between the semantic features extracted by the first initial model and the semantic feature labels is less than a preset error threshold.

[0104] The frequency threshold can be preset by the administrator in the traffic monitoring device. For example, the frequency threshold can be set to 10,000 times, 20,000 times, or 30,000 times, etc. This application embodiment does not limit the specific value of the frequency threshold. The error threshold can also be preset by the administrator in the traffic monitoring device; for example, the error threshold can be set to 10%, 15%, or 20%, etc. This application embodiment does not limit the specific value of the error threshold.

[0105] In other embodiments, prior to S103 described above, the traffic monitoring device may also acquire a second model.

[0106] In one possible implementation, the traffic monitoring device can directly obtain the trained second model from other devices.

[0107] For example, traffic monitoring devices can obtain a trained second model from other devices by downloading or transferring it to an intermediate storage medium.

[0108] In another possible implementation, the traffic monitoring device can also train a second model using a second training sample set.

[0109] Alternatively, the second model can be trained in the following way:

[0110] Step 1: Obtain the second training sample set.

[0111] The second training sample set includes multiple second training samples, each of which includes training network traffic data and corresponding temporal feature labels. The training network traffic data can be referenced from the network traffic data samples described above, and will not be repeated here. The temporal feature labels can, for example, be training network traffic data for one or more future time periods.

[0112] Step 2: Based on the second training sample set, train the second initial model to obtain the second model.

[0113] For example, the second initial model can be the LSTM model mentioned above or other models that can extract temporal features.

[0114] Step 2 here can be referred to as step 2 in the first model training process described above, and will not be repeated here.

[0115] Based on the understanding of the above embodiments, Figure 3 A flowchart illustrating the traffic monitoring process provided in this application embodiment. Figure 3 As shown, the main flow of the traffic monitoring method provided in this application embodiment includes, in sequence: data acquisition and preprocessing, deep learning parsing, adaptive parameter optimization clustering, abnormal traffic identification and intelligent decision-making, and visualization and feedback. Data acquisition and preprocessing can be referred to in S101 above, and will not be repeated here. Deep learning parsing can be referred to in S102 and S103 above, and will not be repeated here. Adaptive parameter optimization clustering can be referred to in S104 above, and will not be repeated here. Abnormal traffic identification and intelligent decision-making can be referred to in S105 above, and will not be repeated here. Visualization and feedback can be referred to in the above embodiments, and will not be repeated here.

[0116] Based on the understanding of the above embodiments, Figure 4 This is another flowchart illustrating the traffic monitoring method provided in an embodiment of this application. Figure 4As shown, taking the first model as a CNN model and the second model as an LSTM model as an example, the traffic monitoring device can first perform data preprocessing, preprocessing the raw network traffic data to convert it into a form suitable for deep learning model processing, then perform CNN feature extraction, using the CNN convolutional layers to scan and extract features from the network traffic data "image", then perform feature integration, after multiple convolution and pooling operations, the extracted local features are integrated through fully connected layers to form a high-level, semantically meaningful feature representation, then perform LSTM time series analysis, using the LSTM network to capture the time series features of network traffic, and finally perform model fusion and anomaly detection, combining CNN and LSTM to comprehensively analyze network traffic from both spatiotemporal dimensions.

[0117] The foregoing primarily describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the aforementioned functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0118] In an exemplary embodiment, this application also provides a traffic monitoring device. Figure 5 This is a schematic diagram illustrating the composition of the traffic monitoring device provided in an embodiment of this application. Figure 5 As shown, the device includes an acquisition module 501 and a processing module 502.

[0119] The acquisition module 501 is used to acquire multiple network traffic data samples from multiple time periods; each time period corresponds one-to-one with a single network traffic data sample.

[0120] The processing module 502 is used to extract the semantic features of multiple network traffic data samples using the first model; extract the temporal features of multiple network traffic data samples using the second model; cluster the multiple network traffic data samples according to the semantic features and temporal features to obtain network traffic data samples of different categories; and identify abnormal network traffic data samples from the network traffic data samples of different categories.

[0121] In some possible embodiments, the processing module 502 is specifically used to cluster multiple network traffic data samples according to semantic features and temporal features using a clustering model at an adaptive learning rate; wherein the magnitude of the adaptive learning rate is positively correlated with the severity of the network environment when acquiring multiple network traffic data samples.

[0122] In other possible embodiments, processing module 502 is specifically used to perform clustering using a K-means clustering model, which satisfies the following formula:

[0123]

[0124] Among them, C i (t+1) represents the position of the i-th cluster center in the (t+1)-th iteration; α represents the adaptive learning rate; γ represents the severity of the network environment; C i (t) represents the position of the i-th cluster center in the t-th iteration; w j (t) represents the weight of the j-th network traffic data sample at the t-th iteration; x j (t) represents the feature vector of the j-th network traffic data sample at the t-th iteration; the feature vector is used to represent the semantic and temporal features of the network traffic data sample.

[0125] In some other possible embodiments, γ includes any of the following: the total amount of network traffic data, the volatility of network traffic data, or the security threat level.

[0126] In some other possible embodiments, the first model is trained by: obtaining a first training sample set; the first training sample set includes multiple first training samples; each first training sample includes training network traffic data and semantic feature labels corresponding to the training network traffic data; and training the first initial model based on the first training sample set to obtain the first model.

[0127] In some other possible embodiments, the second model is trained by: obtaining a second training sample set; the second training sample set includes multiple second training samples; each second training sample includes training network traffic data and the time-series feature label corresponding to the training network traffic data; and training the second initial model based on the second training sample set to obtain the second model.

[0128] In some other possible embodiments, the processing module 502 is also configured to visualize different categories of network traffic data samples in at least one of the following ways: line charts, bar charts, and heatmaps.

[0129] In an exemplary embodiment, this application also provides an electronic device that can be applied to the above-described traffic monitoring device. Figure 6 This is a schematic diagram illustrating the composition of an electronic device provided in an embodiment of this application. For example... Figure 6 As shown, the electronic device may include a processor 601 and a memory 602; the memory 602 stores instructions executable by the processor 601; when the processor 601 is configured to execute instructions, the electronic device causes the electronic device to implement the method described in the foregoing method embodiments.

[0130] In an exemplary embodiment, this application also provides a computer program product that, when run on a computer, causes the computer to implement the methods described in the foregoing method embodiments.

[0131] In an exemplary embodiment, this application also provides a computer-readable storage medium storing computer program instructions thereon; when the computer program instructions are executed by an electronic device, the electronic device causes the electronic device to perform the method described in the foregoing embodiments. The computer-readable storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0132] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer-executable instructions. When these computer-executable instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer-executable instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer-executable instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0133] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple components. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0134] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

[0135] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method of flow monitoring, characterized by, The method comprises: obtaining a plurality of network traffic data samples of a plurality of time periods; the plurality of time periods correspond one-to-one to the plurality of network traffic data samples; extracting semantic features of the plurality of network traffic data samples respectively by using a first model; extracting timing features of the plurality of network traffic data samples respectively by using a second model; clustering the plurality of network traffic data samples according to the semantic features and the timing features by using a clustering model at an adaptive learning rate to obtain network traffic data samples of different categories; wherein the size of the adaptive learning rate is positively correlated with the severity of the network environment state when the plurality of network traffic data samples are obtained; determining abnormal network traffic data samples from the network traffic data samples of different categories.

2. The method of claim 1, wherein, The clustering of the plurality of network traffic data samples according to the semantic features and the timing features by using the clustering model at the adaptive learning rate comprises: clustering by using a K-means clustering model, wherein the K-means clustering model satisfies the following formula: ； wherein, denotes a position of a -th cluster center at a -th iteration; denotes the adaptive learning rate; denotes a degree of severity of the network environment state; denotes a position of a -th cluster center at a -th iteration; denotes a position of a -th cluster center at a -th iteration; denotes a weight of a -th network traffic data sample at a -th iteration; the feature vector is used to represent the semantic feature and the timing feature of the network traffic data sample.

3. The method of claim 2, wherein, The including any of: a total amount of network traffic data, a volatility rate of network traffic data, or a security threat level.

4. The method of claim 1, wherein, The first model is obtained by training in the following manner: obtaining a first training sample set; the first training sample set comprises a plurality of first training samples; each first training sample comprises training network traffic data and a semantic feature label corresponding to the training network traffic data; training a first initial model based on the first training sample set to obtain the first model.

5. The method of claim 1, wherein, The second model is obtained by training in the following manner: obtaining a second training sample set; the second training sample set comprises a plurality of second training samples; each second training sample comprises training network traffic data and a timing feature label corresponding to the training network traffic data; training a second initial model based on the second training sample set to obtain the second model.

6. The method according to any one of claims 1 to 5, characterized in that, The method further comprises: visually displaying the network traffic data samples of different categories in at least one of the following manners: line chart, columnar body, and heat map.

7. A flow monitoring device, characterized by The device comprises an obtaining module and a processing module; The obtaining module is configured to obtain a plurality of network traffic data samples of a plurality of time periods; the plurality of time periods correspond one-to-one to the plurality of network traffic data samples; The processing module is configured to extract semantic features of the plurality of network traffic data samples respectively by using a first model; extract timing features of the plurality of network traffic data samples respectively by using a second model; cluster the plurality of network traffic data samples according to the semantic features and the timing features by using a clustering model at an adaptive learning rate to obtain network traffic data samples of different categories; wherein the size of the adaptive learning rate is positively correlated with the severity of the network environment state when the plurality of network traffic data samples are obtained; and determine abnormal network traffic data samples from the network traffic data samples of different categories.

8. An electronic device, comprising: comprise: a processor and a memory; The memory stores instructions executable by the processor; The processor is configured to execute the instructions, so that the electronic device implements the method of any one of claims 1-6.

9. A readable storage medium, characterized by, comprise: software instructions; When the software instructions are run in an electronic device, they cause the electronic device to implement the method of any one of claims 1-6.

10. A computer program product, characterised in that, Comprising: When the computer program product is run in an electronic device, it causes the electronic device to implement the method of any one of claims 1-6.

Images