A traffic data classification method, device, equipment and storage medium

Abstract

The application relates to the technical field of computers and discloses a traffic data classification method, device and equipment and a storage medium. The method comprises the following steps: acquiring network traffic and corresponding access logs, and acquiring data dimension features according to preset data dimensions; acquiring network data according to network access information corresponding to the access logs, and acquiring service features matched with the network data based on a preset machine learning algorithm when it is detected that the network data is encrypted data; performing intelligent content analysis on the network data to acquire content features; and acquiring traffic classification corresponding to the network traffic according to the data dimension features, the service features and the content features, and a preset classification grading rule set. According to the technical scheme, network data is collected according to network access information corresponding to the access logs, and the traffic classification is determined according to data dimension features corresponding to the access logs and service features and content features corresponding to the network data, so that the classification accuracy of the traffic data can be improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus, device, and storage medium for classifying traffic data. Background Technology

[0002] Data security is crucial for safeguarding national security, protecting corporate trade secrets, and protecting the legitimate rights and interests of individuals. The classification and grading of network data is the foundation for effective data management and utilization, as well as for ensuring data security.

[0003] Currently, existing network traffic classification methods typically use pre-trained machine learning models to process collected network traffic in order to obtain corresponding traffic classification levels. However, in existing technologies, a large number of data samples need to be collected in advance to train the machine learning model, resulting in high time costs. Secondly, determining the traffic classification based solely on the currently collected network traffic itself can easily lead to low classification accuracy and misclassification. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and storage medium for classifying traffic data, which can reduce the time cost of network traffic classification, improve the accuracy of traffic data classification, and avoid misclassification.

[0005] According to one aspect of the present invention, a method for classifying traffic data is provided, comprising:

[0006] Network traffic is acquired through pre-deployed traffic acquisition probes, and access logs corresponding to the network traffic are obtained.

[0007] The access logs are divided according to a preset data dimension to obtain at least one log dataset and the data dimension features corresponding to each log dataset.

[0008] Based on the network access information corresponding to the access log, obtain the network data corresponding to the access log, and when the network data is detected to be encrypted, extract the business features of the network data based on a preset machine learning algorithm to obtain the business features that match the network data.

[0009] The network data is subjected to intelligent content parsing to obtain content features that match the network data;

[0010] Based on the data dimension features corresponding to each of the log datasets and the business and content features matching the network data corresponding to the access logs, as well as the preset classification and grading rule set, the traffic classification corresponding to the network traffic is obtained.

[0011] According to another aspect of the present invention, a traffic data classification apparatus is provided, comprising:

[0012] The access log acquisition module is used to acquire network traffic through a pre-deployed traffic collection probe and acquire the access logs corresponding to the network traffic.

[0013] The data dimension feature acquisition module is used to divide the access logs according to preset data dimensions to obtain at least one log dataset and the data dimension features corresponding to each log dataset.

[0014] The business feature acquisition module is used to acquire network data corresponding to the access log based on the network access information corresponding to the access log, and when the network data is detected to be encrypted, to extract business features from the network data based on a preset machine learning algorithm to obtain the business features matching the network data.

[0015] The content feature acquisition module is used to perform intelligent content parsing on the network data to obtain content features that match the network data.

[0016] The traffic classification acquisition module is used to acquire the traffic classification corresponding to the network traffic based on the data dimension features corresponding to each of the log datasets, the business features and content features matching the network data corresponding to the access logs, and a preset classification and grading rule set.

[0017] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0018] At least one processor; and

[0019] A memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the traffic data classification method according to any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the traffic data classification method according to any embodiment of the present invention.

[0022] The technical solution of this invention first acquires network traffic and corresponding access logs through a pre-deployed traffic acquisition probe. Then, the access logs are divided according to preset data dimensions to obtain multiple log datasets and their corresponding data dimension features. Next, network data is acquired based on the network access information in the access logs. When encrypted network data is detected, business features are extracted from the network data using a preset machine learning algorithm to obtain matching business features. Further, intelligent content parsing is performed on the network data to obtain matching content features. Finally, based on the data dimension features of each log dataset, the matching business and content features of the network data in the access logs, and a preset classification and grading rule set, the traffic classification corresponding to the network traffic is obtained. By acquiring network data based on the network access information in the access logs and determining the traffic classification based on the data dimension features and the corresponding business and content features, the time cost of network traffic classification can be reduced, the accuracy of traffic data classification can be improved, and misclassification can be avoided.

[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1A This is a flowchart of a traffic data classification method provided in Embodiment 1 of the present invention;

[0026] Figure 1B This is a schematic diagram of the process for extracting business features based on a combination of static and dynamic detection of APKs, according to Embodiment 1 of the present invention.

[0027] Figure 1C This is a schematic diagram of static detection plus dynamic detection provided in Embodiment 1 of the present invention;

[0028] Figure 1D This is a flowchart illustrating the process of extracting business features using a preset field content extraction method, as provided in Embodiment 1 of the present invention.

[0029] Figure 1EThis is a schematic diagram of an automated operation report provided according to Embodiment 1 of the present invention;

[0030] Figure 1F This is a schematic diagram of the process for identifying sensitive APP protocols according to Embodiment 1 of the present invention;

[0031] Figure 1G This is a schematic diagram of the process for identifying sensitive APP data streams based on a protocol behavior recognition model according to Embodiment 1 of the present invention;

[0032] Figure 1H This is a flowchart of another method for classifying traffic data according to Embodiment 1 of the present invention;

[0033] Figure 2 This is a schematic diagram of the structure of a traffic data classification device according to Embodiment 2 of the present invention;

[0034] Figure 3 This is a schematic diagram of the structure of an electronic device that implements the traffic data classification method of this invention. Detailed Implementation

[0035] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0036] It should be noted that the terms "first," "second," "target," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0037] Example 1

[0038] Figure 1AThis is a flowchart illustrating a traffic data classification method provided in Embodiment 1 of the present invention. This embodiment is applicable to the classification of Internet data center traffic. The method can be executed by a traffic data classification device, which can be implemented in hardware and / or software. Typically, the traffic data classification device can be configured in an electronic device, such as a computer or server. Figure 1A As shown, the method includes:

[0039] S110. Obtain network traffic through a pre-deployed traffic acquisition probe, and obtain the access logs corresponding to the network traffic.

[0040] In this embodiment, a traffic acquisition probe can be deployed at the Internet Data Center (IDC) egress to obtain IDC traffic as network traffic; then, a preset traffic parsing engine, such as a Persues engine, can be used to decode the network traffic based on the communication protocol corresponding to the network traffic to obtain its corresponding access log.

[0041] Typically, network traffic can be analyzed based on more than 100 different protocols, such as HTTP (Hypertext Transfer Protocol), POP3 (Post Office Protocol version 3), SMTP (Simple Mail Transfer Protocol), VPN (Virtual Private Network), Industrial Internet protocols, and Vehicle Networking protocols.

[0042] The access log is used to record users' network access information, which may include user identity information (user name, ID number, etc.), source IP address, source port number, destination IP address, destination port number, and Uniform Resource Locator (URL) address.

[0043] It should be noted that user authorization is required before network traffic collection and analysis. Once user authorization is successfully obtained, network traffic can be collected using pre-deployed traffic collection probes.

[0044] S120. Divide the access logs according to preset data dimensions to obtain at least one log dataset and the data dimension features corresponding to each log dataset.

[0045] The preset data dimensions may include application programming interfaces (APIs), system names, and / or company names. In this embodiment, access logs can be clustered and summarized using dimensions such as API interfaces, system names, or company names to form log datasets. Furthermore, feature values ​​corresponding to each log dataset can be extracted as data dimension features for each data dimension; for example, the API interface address and interface identifier corresponding to the log dataset.

[0046] In a specific example, access logs can be aggregated based on IP address and port number, forming a dataset of access logs with the same IP address and port number to obtain system inventory and system API inventory; at the same time, IP address and port number can be used as data dimension features for the corresponding inventory.

[0047] S130. Based on the network access information corresponding to the access log, obtain the network data corresponding to the access log, and when the network data is detected to be encrypted, extract the business features of the network data based on a preset machine learning algorithm to obtain the business features matching the network data.

[0048] In a specific example, application data or website data can be crawled based on the IP address, port number, and URL address in the network access information to obtain network data corresponding to the access logs, such as page titles, user information, and page content.

[0049] Optionally, a city's app profile can be created based on factors such as the number of registered apps nationwide, the number of app stores, and app monitoring data. This profile can then be used to identify various app assets within the network environment through proactive detection engines or AI analysis engines. Furthermore, multi-dimensional correlation analysis based on developer name, developer office location, and developer registration location can be performed, and the app's geographical location can be determined. Secondly, when crawling application data, information such as the scope of app stores to be crawled and the crawling frequency can be configured.

[0050] It should be noted that encrypting network data is becoming increasingly common to enhance data security and protect data privacy. Therefore, after acquiring network data, one can first determine whether the data is encrypted. If it is determined not to be encrypted, rule matching can be used directly to extract the corresponding business features. If it is encrypted, pre-defined machine learning algorithms, such as fingerprint algorithms or convolutional neural network algorithms, can be used to extract the matching business features.

[0051] Business characteristics may include enterprise domain name, business domain name, business profile, application name, and whether it is a sensitive application.

[0052] S140. Perform intelligent content parsing on the network data to obtain content features that match the network data.

[0053] In this embodiment, after obtaining the network data, further content feature extraction can be performed on the network data. Specifically, a pre-trained Divergence From Randomness (DFR) model or a Divergence From Independence (DFI) model can be used to intelligently parse the content such as cookies or URLs in the network data in order to extract the content features that match the network data.

[0054] Content features can include content data and related descriptions such as live streams, videos, music, pictures, novels, and products, as well as user information such as account name, browsing history, membership information, user usage time, and data usage.

[0055] S150. Based on the data dimension features corresponding to each of the log datasets and the business and content features matching the network data corresponding to the access logs, as well as the preset classification and grading rule set, obtain the traffic classification corresponding to the network traffic.

[0056] The preset classification and grading rule set can be a pre-established data classification and grading rule library, which may include at least one of the following: personal information classification rules, telecommunications industry data classification rules, industrial internet classification rules, vehicle network classification rules, instant messaging classification rules, basic telecommunications enterprise classification rules, and important data classification rules.

[0057] In this embodiment, based on the data characteristics of various industries, classification rules corresponding to each traffic category can be pre-constructed, thereby forming a classification and grading rule set. Each classification rule can be a mapping relationship between traffic categories, data dimension features, business features, and content features. In a specific example, after obtaining the data dimension features, business features, and content features corresponding to the current network traffic, the traffic category corresponding to the current network traffic can be obtained by searching the classification rules in the pre-constructed classification and grading rule set. The traffic category can be the industry to which the network traffic belongs, such as telecommunications data, industrial internet, or vehicle-to-everything (V2X) data.

[0058] Typically, personal information categories can include names, mobile phone numbers, ID card numbers, email addresses, or bank card numbers; vehicle networking categories can include license plate numbers, vehicle identification numbers (VINs), vehicle models, or location information; industrial internet categories can include industrial equipment, industrial apps, industrial drawings, and industrial documents; important data categories can include cloud services, e-commerce, finance, credit reporting, meteorology, and healthcare; and abnormal data behavior categories can include cross-border data transmission, plaintext data transmission, and unauthorized API interfaces. For example, if the length of one data item in the content characteristics is equal to the length of an ID card number and conforms to the rules for ID card numbers, then the current network traffic can be determined to belong to the personal information category.

[0059] Optionally, in this embodiment, when the network traffic is determined to be classified as abnormal data behavior, a data anomaly alarm corresponding to the network traffic can be generated and sent to a designated user. The data anomaly alarm may include network traffic and anomaly type, such as cross-border data transmission or plaintext data transmission.

[0060] The technical solution of this invention first acquires network traffic and corresponding access logs through a pre-deployed traffic acquisition probe. Then, the access logs are divided according to preset data dimensions to obtain multiple log datasets and their corresponding data dimension features. Next, network data is acquired based on the network access information in the access logs. When encrypted network data is detected, business features are extracted from the network data using a preset machine learning algorithm to obtain matching business features. Further, intelligent content parsing is performed on the network data to obtain matching content features. Finally, based on the data dimension features of each log dataset, the matching business and content features of the network data in the access logs, and a preset classification and grading rule set, the traffic classification corresponding to the network traffic is obtained. By acquiring network data based on the network access information in the access logs and determining the traffic classification based on the data dimension features and the corresponding business and content features, the time cost of network traffic classification can be reduced, the accuracy of traffic data classification can be improved, and misclassification can be avoided.

[0061] In an optional implementation of this embodiment, after obtaining the network data corresponding to the access log based on the network access information corresponding to the access log, the method may further include:

[0062] If the network data is detected to be unencrypted, then the network data is subjected to business feature extraction based on a preset rule recognition algorithm to obtain the business features that match the network data.

[0063] In a specific example, when the network data is unencrypted, business features can be extracted using methods such as pre-defined field content extraction or a combination of static and dynamic detection of the Android application package (APK) to obtain business features matching the network data. These business features can include full-dimensional features such as hash, signature, package name, class name, domain name, and server.

[0064] Specifically, the business feature extraction process based on a combination of static and dynamic APK detection can be as follows: Figure 1B As shown, the system can utilize static or dynamic engines to perform timed checks on the APK and generate automated reports based on the results. Typically, these automated reports include permission lists, third-party SDKs, potential API calls, network activity, API calls, and file operations. Alternatively, the APK can be decompiled to obtain its source code, which can then be manually inspected to determine if it illegally collects user privacy data. Finally, a final output report can be generated based on the manual inspection results and the automated report.

[0065] Among them, static detection plus dynamic detection can be like... Figure 1C As shown. In this embodiment, various features that can reflect the behavior of Android malicious programs, such as sensitive permissions, sensitive API function call sequences, filename strings, SO file names, file sizes, and file permission characteristics, can be used to determine whether an APP is a sensitive APP. Specifically, binary classification can be performed using the Random Forest algorithm and the XGBoost algorithm, and multi-class identification can be performed using convolutional neural networks and K-means clustering. Furthermore, anomaly detection can be used to discover unknown types of black samples. The results of multiple algorithms can be mutually verified. After the identification results are verified, they can be added to the sample set for iterative training to achieve online model updates.

[0066] In a specific example, the business feature extraction process using the preset field content extraction method can be as follows: Figure 1D As shown. For HTTPS data, the command `dynamic Net Content: "contacts"` can be used to search for contacts and determine whether the search results contain uploaded user privacy information; for example, software A uploaded the user's application list, while software B uploaded the user's contact information. This allows for HTTP identification.

[0067] In another optional implementation of this embodiment, after obtaining the traffic category corresponding to the network traffic, it may further include:

[0068] The network traffic is clustered according to the traffic classification to obtain at least one clustered dataset.

[0069] Based on the clustered datasets and preset report templates, generate automated operation reports.

[0070] In a specific example, network traffic belonging to the same traffic category can be added to a clustered dataset to generate multiple clustered datasets, each corresponding to a traffic category. For example, clustered datasets could include API interface clusters, system clusters, or enterprise clusters. Furthermore, data items from a pre-defined report template can be retrieved, and the corresponding data values ​​for each data item can be statistically obtained based on each clustered dataset (e.g., the number of discovered IPs, application systems, and API interfaces, as well as the number of sensitive data items, the number of involved application systems, and the number of API interfaces, etc.). Then, these data values ​​can be populated into the corresponding data items to generate an automated operations report.

[0071] For example, automated operations reports can be like Figure 1E As shown. It should be noted that the report template can be customized to meet the needs of different enterprises, users, or scenarios.

[0072] The advantage of the above settings is that they provide flexible scenario configuration, multi-dimensional cluster analysis, and personalized report template capabilities, which can provide valuable output for enterprises.

[0073] In another optional implementation of this embodiment, extracting service features from the network data based on a preset machine learning algorithm to obtain service features matching the network data may include:

[0074] The network data is subjected to content extraction of preset fields to obtain the field values ​​corresponding to the preset fields, and the fingerprint features corresponding to the network data are extracted by a preset fingerprint algorithm.

[0075] Based on the field values ​​corresponding to the preset fields and the fingerprint features, the service features matching the network data are obtained.

[0076] In this embodiment, key identifying features in the Client Hello and Server Hello fields can be extracted during the TLS handshake process using DFI feature modeling and DPI (Deep Packet Inspection) feature extraction algorithms, and used as field values. The Client Hello and Server Hello fields may include information such as SSLVersion, password, extensions, Elliptic Curves, and Elliptic Curve Point Formats. By analyzing the values ​​of the Client Hello and Server Hello fields, the server name and domain name in the data stream can be obtained, and the enterprise to which the network data belongs can be determined based on the domain name.

[0077] Secondly, fingerprint features corresponding to network data can be extracted using pre-defined fingerprint algorithms such as JA3 and JA3S. These fingerprint features can then be concatenated with the values ​​of pre-defined fields to generate business characteristics that match the network data. By combining the extracted server name with fingerprint features, the enterprise and business to which the network data belongs can be identified. Therefore, HTTP identification of encrypted network data can be achieved.

[0078] It should be noted that as sensitive app protocols continue to upgrade and evolve, in order to counter the blocking of the GFW, a variety of open-source encrypted sensitive app communication protocols have been developed. The design of these protocols will try to hide their unique identification features during the communication process and adopt a communication mode similar to HTTPS, making it difficult for DPI devices to distinguish such sensitive app protocols from normal HTTPS protocols based on message characteristics.

[0079] In this embodiment, the identification process for sensitive APP protocols can be as follows: Figure 1F As shown; specifically, by monitoring VPN protocols, important information such as the identity of sensitive APP users, time, protocol type used, and resources of sensitive APP servers on the other end are obtained. Based on the information obtained, a protocol behavior recognition model is established, and the evaluated protocol behavior recognition model is deployed to identify sensitive APP data streams.

[0080] The process of identifying sensitive APP data streams based on the protocol behavior recognition model can be as follows: Figure 1G As shown, firstly, sample features corresponding to each business sample set are extracted, such as the number of uplink and downlink bytes, the number of packets, the length sequence, and the byte distribution probability; then, a preset machine learning algorithm is used to identify the data stream based on the extracted sample features in order to determine the corresponding sensitive apps.

[0081] In another optional embodiment of this invention, the technical solution of this embodiment may further include:

[0082] Based on the data dimension features corresponding to each of the log datasets and the business and content features matching the network data corresponding to the access logs, as well as the preset classification and grading rule set, if the traffic category corresponding to the network traffic is not obtained, the network traffic will be regarded as abnormal traffic.

[0083] Obtain the manually labeled classification corresponding to the abnormal traffic, update the preset classification and grading rule set according to the manually labeled classification corresponding to the abnormal traffic, and replace the preset classification and grading rule set with the updated preset classification and grading rule set.

[0084] In a specific example, when abnormal traffic fails to be categorized based on a preset classification and grading rule set, regulatory self-learning can be performed based on this abnormal traffic to update the classification and grading rule set. Specifically, a manual annotation prompt can be sent to the user, and the manual annotation category corresponding to the abnormal traffic can be obtained. Then, the data dimension features, business features, and content features corresponding to the abnormal traffic can be added to the rules corresponding to the manual annotation category to update the preset classification and grading rule set. The updated preset classification and grading rule set can then be used as the new preset classification and grading rule set.

[0085] In one specific implementation of this embodiment, the traffic data classification process can be as follows: Figure 1H As shown. First, the collected IDC traffic is decoded to obtain access logs and sample logs; then, IP addresses and ports are aggregated from the access logs to obtain system lists and system API lists, as well as the data dimension characteristics corresponding to each list; further, based on the network access information corresponding to each access log, the content of the APP and website is crawled in batches to obtain network data; for example, user click operations can be simulated to obtain the page content of different levels of pages in the APP.

[0086] Furthermore, when the network data is unencrypted, HTTP identification and mainstream app identification can be performed based on preset rule-based identification methods; while when the network data is encrypted, sensitive app identification, HTTP identification, or niche app identification can be performed based on preset machine learning algorithms. This allows the acquisition of business characteristics corresponding to IDC traffic. Then, content characteristics corresponding to the network data can be extracted using the DFI enhanced module and deep learning-based DFR. Finally, based on data dimensional characteristics, business characteristics, content characteristics, and a preset set of classification and grading rules, the traffic classification corresponding to IDC traffic can be obtained.

[0087] For IDC traffic that has been identified and categorized, clustering can be performed based on traffic classification to obtain API interface clusters, system clusters, and enterprise clusters. Automated operation reports can then be generated based on each cluster. For IDC traffic that has not been identified and categorized, continuous monitoring and rule self-learning modules can be used to update the classification and grading rule set.

[0088] Secondly, the acquired sample logs can be restored to obtain the full data; for example, content extraction can be performed on sample logs in formats such as CAD and PDF to obtain the substantive content. Then, the full data can be identified based on a preset classification and grading rule set to obtain the traffic category corresponding to the IDC traffic. The sample logs may include files, links, and other content from network traffic.

[0089] The technical solution of this invention can significantly reduce traffic classification time and lower the difficulty of system use. It can completely extract systems containing plaintext transmission of sensitive information from logs, making it possible to analyze data across borders, non-standard de-identification of personal information, plaintext transmission of important data, batch access to sensitive data, and high-frequency access to sensitive data. Secondly, it can flexibly adapt to the requirements of data classification and hierarchical management in multiple scenarios, and more proactively respond to industry supervision. Moreover, data security ultimately serves regulatory agencies and enterprises. With clear management targets, a clear list of issues, and closed-loop handling measures, valuable output suggestions can be provided to enterprises and regulatory departments through flexible scenario configuration, multi-dimensional cluster analysis, and personalized report templates, guiding the improvement of users' data security protection capabilities.

[0090] Example 2

[0091] Figure 2 This is a schematic diagram of a traffic data classification device provided in Embodiment 2 of the present invention. Figure 2 As shown, the device includes: an access log acquisition module 210, a data dimension feature acquisition module 220, a business feature acquisition module 230, a content feature acquisition module 240, and a traffic classification acquisition module 250; wherein,

[0092] The access log acquisition module 210 is used to acquire network traffic through a pre-deployed traffic acquisition probe and acquire the access logs corresponding to the network traffic.

[0093] The data dimension feature acquisition module 220 is used to divide the access log according to a preset data dimension to obtain at least one log dataset and the data dimension features corresponding to each log dataset.

[0094] The business feature acquisition module 230 is used to acquire network data corresponding to the access log based on the network access information corresponding to the access log, and when the network data is detected to be encrypted, to extract business features from the network data based on a preset machine learning algorithm to obtain the business features matching the network data.

[0095] The content feature acquisition module 240 is used to perform intelligent content parsing on the network data to obtain the content features matched by the network data.

[0096] The traffic classification acquisition module 250 is used to acquire the traffic classification corresponding to the network traffic based on the data dimension features corresponding to each of the log datasets, the business features and content features matching the network data corresponding to the access logs, and a preset classification and grading rule set.

[0097] The technical solution of this invention first acquires network traffic and corresponding access logs through a pre-deployed traffic acquisition probe. Then, the access logs are divided according to preset data dimensions to obtain multiple log datasets and their corresponding data dimension features. Next, network data is acquired based on the network access information in the access logs. When encrypted network data is detected, business features are extracted from the network data using a preset machine learning algorithm to obtain matching business features. Further, intelligent content parsing is performed on the network data to obtain matching content features. Finally, based on the data dimension features of each log dataset, the matching business and content features of the network data in the access logs, and a preset classification and grading rule set, the traffic classification corresponding to the network traffic is obtained. By acquiring network data based on the network access information in the access logs and determining the traffic classification based on the data dimension features and the corresponding business and content features, the time cost of network traffic classification can be reduced, the accuracy of traffic data classification can be improved, and misclassification can be avoided.

[0098] Optionally, the preset data dimensions include application programming interfaces, system names, and / or company names.

[0099] Optionally, the preset classification and grading rule set includes at least one of the following: personal information classification rules, telecommunications industry data classification rules, industrial internet classification rules, vehicle network classification rules, instant messaging classification rules, basic telecommunications enterprise classification rules, and important data classification rules.

[0100] Optionally, the service feature acquisition module 230 is further configured to, if it is detected that the network data is unencrypted data, perform service feature extraction on the network data based on a preset rule recognition algorithm to obtain the service features matching the network data.

[0101] Optionally, the classification device for traffic data further includes:

[0102] A clustering data set acquisition module, configured to perform clustering processing on the network traffic according to the traffic classification to obtain at least one clustering data set;

[0103] An automated operation report generation module, configured to generate an automated operation report according to each of the clustering data sets and a preset report template.

[0104] Optionally, the service feature acquisition module 230 is specifically configured to extract the content of a preset field from the network data to obtain a field value corresponding to the preset field, and extract the fingerprint feature corresponding to the network data through a preset fingerprint algorithm;

[0105] Obtain the service features matching the network data according to the field value corresponding to the preset field and the fingerprint feature.

[0106] Optionally, the classification device for traffic data further includes:

[0107] An abnormal traffic acquisition module, configured to, according to the data dimension features corresponding to each of the log data sets, the service features and content features matching the network data corresponding to the access log, and a preset classification and grading rule set, if the traffic classification corresponding to the network traffic is not obtained, regard the network traffic as abnormal traffic;

[0108] A classification and grading rule set update module, configured to obtain the manually marked classification corresponding to the abnormal traffic, update the preset classification and grading rule set according to the manually marked classification corresponding to the abnormal traffic, and replace the preset classification and grading rule set with the updated preset classification and grading rule set.

[0109] The classification device for traffic data provided in the embodiments of the present invention can execute the traffic data classification method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method.

[0110] It should be noted that in the technical solution of this embodiment, the acquisition, storage, and application of the user's personal information involved all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0111] Embodiment III

[0112] Figure 3A schematic diagram of an electronic device 30 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0113] like Figure 3 As shown, the electronic device 30 includes at least one processor 31 and a memory, such as a read-only memory (ROM) 32 or a random access memory (RAM) 33, communicatively connected to the at least one processor 31. The memory stores computer programs executable by the at least one processor. The processor 31 can perform various appropriate actions and processes based on the computer program stored in the ROM 32 or loaded from storage unit 38 into the RAM 33. The RAM 33 can also store various programs and data required for the operation of the electronic device 30. The processor 31, ROM 32, and RAM 33 are interconnected via a bus 34. An input / output (I / O) interface 35 is also connected to the bus 34.

[0114] Multiple components in electronic device 30 are connected to I / O interface 35, including: input unit 36, such as keyboard, mouse, etc.; output unit 37, such as various types of monitors, speakers, etc.; storage unit 38, such as disk, optical disk, etc.; and communication unit 39, such as network card, modem, wireless transceiver, etc. Communication unit 39 allows electronic device 30 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0115] Processor 31 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 31 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 31 performs the various methods and processes described above, such as methods for classifying traffic data.

[0116] In some embodiments, the traffic data classification method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 38. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 30 via ROM 32 and / or communication unit 39. When the computer program is loaded into RAM 33 and executed by processor 31, one or more steps of the traffic data classification method described above may be performed. Alternatively, in other embodiments, processor 31 may be configured to perform the traffic data classification method by any other suitable means (e.g., by means of firmware).

[0117] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0118] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0119] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0120] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0121] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0122] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0123] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0124] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for classifying traffic data, characterized in that, include: Network traffic is acquired through pre-deployed traffic acquisition probes, and access logs corresponding to the network traffic are obtained. The access logs are divided according to a preset data dimension to obtain at least one log dataset and the data dimension features corresponding to each log dataset. Based on the network access information corresponding to the access log, obtain the network data corresponding to the access log, and when the network data is detected to be encrypted, extract the business features of the network data based on a preset machine learning algorithm to obtain the business features that match the network data. If the network data is detected to be unencrypted, then the network data is subjected to business feature extraction based on a preset rule recognition algorithm to obtain the business features that match the network data. The network data is subjected to intelligent content parsing to obtain content features that match the network data; Based on the data dimension features corresponding to each log dataset and the business and content features matching the network data corresponding to the access log, the traffic category corresponding to the network traffic is obtained by searching a preset classification and grading rule set.

2. The method according to claim 1, characterized in that, The preset data dimensions include application programming interfaces, system names, and / or company names.

3. The method according to claim 1, characterized in that, The preset classification and grading rule set includes at least one of the following: personal information classification rules, telecommunications industry data classification rules, industrial internet classification rules, vehicle network classification rules, instant messaging classification rules, basic telecommunications enterprise classification rules, and important data classification rules.

4. The method according to claim 1, characterized in that, After obtaining the traffic category corresponding to the network traffic, the process also includes: The network traffic is clustered according to the traffic classification to obtain at least one clustered dataset. Based on the clustered datasets and preset report templates, generate automated operation reports.

5. The method according to claim 1, characterized in that, Based on a preset machine learning algorithm, business features are extracted from the network data to obtain business features that match the network data, including: The network data is subjected to content extraction of preset fields to obtain the field values ​​corresponding to the preset fields, and the fingerprint features corresponding to the network data are extracted by a preset fingerprint algorithm. Based on the field values ​​corresponding to the preset fields and the fingerprint features, the service features matching the network data are obtained.

6. The method according to claim 1, characterized in that, Also includes: Based on the data dimension features corresponding to each of the log datasets and the business and content features matching the network data corresponding to the access logs, the network traffic is classified as abnormal traffic if the traffic category corresponding to the network traffic is not obtained by searching a preset classification and grading rule set. Obtain the manually labeled classification corresponding to the abnormal traffic, update the preset classification and grading rule set according to the manually labeled classification corresponding to the abnormal traffic, and replace the preset classification and grading rule set with the updated preset classification and grading rule set.

7. A device for classifying traffic data, characterized in that, include: The access log acquisition module is used to acquire network traffic through a pre-deployed traffic collection probe and acquire the access logs corresponding to the network traffic. The data dimension feature acquisition module is used to divide the access logs according to preset data dimensions to obtain at least one log dataset and the data dimension features corresponding to each log dataset. The business feature acquisition module is used to acquire network data corresponding to the access log based on the network access information corresponding to the access log, and when the network data is detected to be encrypted, to extract business features from the network data based on a preset machine learning algorithm to obtain the business features matching the network data. If the network data is detected to be unencrypted, then the network data is subjected to business feature extraction based on a preset rule recognition algorithm to obtain the business features that match the network data. The content feature acquisition module is used to perform intelligent content parsing on the network data to obtain content features that match the network data. The traffic classification acquisition module is used to obtain the traffic classification corresponding to the network traffic by searching a preset classification and grading rule set, based on the data dimension features corresponding to each of the log datasets and the business features and content features matching the network data corresponding to the access logs.

8. An electronic device, characterized in that, The electronic device includes: At least one processor, and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the traffic data classification method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method for classifying traffic data according to any one of claims 1-6.

Images