Business identification method and device, electronic equipment and storage medium

Abstract

The embodiment of the present application discloses a service identification method and device, electronic equipment and computer storage medium, the method comprises the following steps: obtaining the original code stream of network data; by analyzing the original code stream, the service characteristic data of the original code stream is determined; the type of the service characteristic data is determined; according to the type of the service characteristic data, the first identification method for service identification of the original code stream is determined; the service characteristic data is processed by using the first identification method, and the service identification result of the original code stream is obtained.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence (AI) technology, and specifically relates to a business identification method, device, electronic device, and computer storage medium. Background Technology

[0002] In related technologies, a common network traffic service identification scheme is to identify network traffic services through manual rule verification, that is, to determine the service type of network traffic based on experience-based judgment conditions. This scheme has the problem of low identification accuracy. Alternatively, simple and lightweight algorithm models can be used to classify and identify data services. However, related technologies only use a single model to identify data services, which is not suitable for identifying different types of data services. In scenarios where new data services need to be added, the model needs to be replaced on a large scale, resulting in low scalability. Summary of the Invention

[0003] This application provides a service identification method, apparatus, electronic device, and computer storage medium.

[0004] This application provides a service identification method, the method comprising:

[0005] Obtain the raw bitstream of network data;

[0006] By analyzing the original bitstream, the service characteristic data of the original bitstream is determined; the type of the service characteristic data is determined.

[0007] Based on the type of the business feature data, a first identification method for identifying the business in the original code stream is determined;

[0008] The service feature data is processed using the first identification method to obtain the service identification result of the original code stream.

[0009] In some embodiments, the step of analyzing the original bitstream to determine the service characteristic data of the original bitstream includes: determining the protocol type of the original bitstream; and determining the service characteristic data of the original bitstream based on the protocol type of the original bitstream.

[0010] As can be seen, the embodiments of this application can determine the service feature data of the original bitstream more accurately based on the protocol type of the original bitstream, which is conducive to determining the first identification method for service identification of the original bitstream more accurately based on the type of service feature data of the original bitstream, and thus improving the accuracy of service identification of the original bitstream.

[0011] In some embodiments, determining the service characteristic data of the original bitstream based on its protocol type includes: when the protocol type of the original bitstream is Hypertext Transfer Protocol (HTTP), determining the bitstream information and key plaintext fields of the original bitstream as the service characteristic data of the original bitstream, wherein the key plaintext fields are fields that use plaintext to characterize the service type of the original bitstream; when the protocol type of the original bitstream is HTTPS or a newly added supported protocol, determining at least the key plaintext fields of the original bitstream as the service characteristic data of the original bitstream; when the protocol type of the original bitstream is other protocols, determining the target field of the original bitstream as the service characteristic data of the original bitstream, wherein the other protocols represent network communication protocols other than HTTP, HTTPS, and the newly added supported protocols, and the target field is a predefined field.

[0012] As can be seen, the embodiments of this application can determine the service feature data in the original bitstream more accurately based on the protocol type of the original bitstream, which is conducive to determining the first identification method for service identification of the original bitstream more accurately, thereby improving the accuracy of service identification of the original bitstream.

[0013] In some embodiments, determining the business characteristic data of the original bitstream based on its protocol type further includes: if the protocol type of the original bitstream is HTTPS, determining the content in the original bitstream from which plaintext information cannot be obtained as the business characteristic data of the original bitstream.

[0014] It can be seen that for the HTTPS protocol, since the content in the original bitstream from which plaintext information cannot be obtained can be identified as the business characteristic data of the original bitstream, it is beneficial to more accurately identify the business of the original bitstream, thereby improving the accuracy of business identification of the original bitstream.

[0015] In some embodiments, determining the first identification method for business identification of the original bitstream based on the type of the business feature data includes: when the business feature data is a key plaintext field, determining that the first identification method includes a method for business identification using a Natural Language Processing (NLP) model; the key plaintext field is a field that uses plaintext to represent the business type of the original bitstream; when the business feature data is bitstream information of the original bitstream, determining that the first identification method includes a method for business identification using a Convolutional Neural Networks (CNN) model; or, when the protocol type of the original bitstream is HTTPS and the business feature data is content in the original bitstream for which plaintext information cannot be obtained, determining that the first identification method includes a method for business identification using a CNN model; when the business feature data is a target field, determining whether the target field contains the key plaintext field, and obtaining a determination result; determining the first identification method based on the determination result; the target field is a predefined field.

[0016] As can be seen, the embodiments of this application can determine the first identification method for business identification of the original bitstream more accurately according to the type of business feature data, thereby improving the accuracy of business identification of the original bitstream.

[0017] In some embodiments, determining the first identification method based on the judgment result includes: if the judgment result is that the target field contains the key plaintext field, determining that the first identification method includes a method for business identification using an NLP model; if the judgment result is that the target field does not contain the key plaintext field, determining that the first identification method includes a method for business identification using pre-defined manual rules.

[0018] As can be seen, the embodiments of this application can flexibly determine the method for business identification of the original bitstream based on the content in the target field, which is beneficial for business identification of different types of data services.

[0019] In some embodiments, processing the service feature data using the first identification method to obtain the service identification result of the original bitstream includes: when the first identification method includes a method for service identification using the NLP model, or when the first identification method includes a method for service identification using the CNN model, processing the service feature data using the first identification method to obtain a preliminary service identification result and a model prediction probability of the original bitstream, wherein the model prediction probability represents the accuracy of obtaining the preliminary service identification result using the NLP model or the CNN model; when the model prediction probability is greater than or equal to a probability threshold, using the preliminary service identification result of the original bitstream as the service identification result; when the model prediction probability is less than the probability threshold, re-determining the first identification method as a method for service identification using pre-defined manual rules, processing the service feature data using the re-determined first identification method to obtain the service identification result of the original bitstream.

[0020] As can be seen, in this embodiment, if the model predicts a probability greater than or equal to the probability threshold, the preliminary business identification result obtained by using the NLP model or CNN model can be retained; if the model predicts a probability less than the probability threshold, it indicates that the accuracy of the preliminary business identification result is low. Therefore, manual rules are used for business identification, which helps to improve the accuracy of business identification.

[0021] This application embodiment also provides a service identification device, the device comprising:

[0022] The acquisition module is used to acquire the raw bitstream of network data;

[0023] The first processing module is used to analyze the original bitstream to determine the service feature data of the original bitstream and to determine the type of the service feature data.

[0024] The second processing module is used to determine a first identification method for identifying the business in the original code stream based on the type of the business feature data.

[0025] The third processing module is used to process the service feature data using the first identification method to obtain the service identification result of the original code stream.

[0026] This application also provides an electronic device, including a processor and a memory for storing a computer program that can run on the processor; wherein the processor is used to run the computer program to perform any of the above-described service identification methods.

[0027] This application also provides a computer storage medium storing a computer program thereon, which, when executed by a processor, implements any of the above-described business identification methods.

[0028] As can be seen, the embodiments of this application can determine the type of service feature data of the original bitstream by analyzing the original bitstream. Based on the type of service feature data, the first identification method for service identification of the original bitstream can be determined more accurately, which is beneficial to improving the accuracy of service identification of the original bitstream. Furthermore, since the first identification method is determined based on the type of service feature data, the embodiments of this application adopt different service identification methods for network data of different service types. Compared with the scheme of using a single model to identify data services in related technologies, it has a wider range of applications. Moreover, for scenarios that need to expand new network data services, it is not necessary to replace the algorithm model on a large scale, but can directly determine the service identification method more accurately for new network data services, which has high scalability. Attached Figure Description

[0029] Figure 1 This is a flowchart of the service identification method according to an embodiment of this application;

[0030] Figure 2 This is a flowchart illustrating the use of an end-to-end system for service identification in this application embodiment;

[0031] Figure 3 This is a schematic diagram of the structure of the service identification device according to an embodiment of this application;

[0032] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0033] In related technologies, network traffic service identification schemes either identify network traffic services through manual rule verification or use simple and lightweight algorithm models to classify and identify data services. The following examples of network traffic service identification schemes are illustrated using schemes 1 to 4.

[0034] Solution 1 provides a traffic service identification method and related equipment, which can classify network traffic based on service fields. The main implementation steps of Solution 1 include: determining whether the network traffic is asymmetric traffic, that is, determining whether the corresponding Deep Packet Inspection (DPI) device uses HTTP. If the DPI device uses the HTTP protocol, the interactive information transmitted by the DPI device can be used for service identification.

[0035] Scheme 2 provides a method and apparatus for generating DPI rules; the main implementation steps of the scheme include: acquiring Internet data, identifying the Internet data based on DPI rules in the DPI rule base; when there is unidentified data in the Internet data, analyzing the unidentified data to obtain a first feature of the unidentified data; compiling and generating DPI rules based on the first feature; and storing the DPI rules in the DPI rule base.

[0036] Scheme 3 provides a network function orchestration method based on service identification in a mobile edge computing platform. The dataset in Scheme 3 consists of service traffic captured based on DPI (Data Point Index), which is then used in conjunction with an algorithm for service identification. The main implementation steps of Scheme 3 include: capturing user service request data streams of known service types and generating training, testing, and validation sets; constructing a low-level back propagation (BP) neural network structure and training the BP neural network; and orchestrating mobile edge computing network functions based on the service type results of the user service requests obtained during training, and implementing service delivery.

[0037] Solution 4 provides a service identification method, apparatus, terminal device, and storage medium. The main implementation steps of Solution 4 include: acquiring service data to be identified; extracting identification keywords corresponding to the identification elements in the service data to be identified based on pre-defined identification elements; performing matrix processing on the identification keywords to obtain the service data matrix to be identified; establishing a service identification model using the K-Nearest Neighbor (KNN) algorithm to determine the service to which the service data to be identified belongs; exemplarily, calculating the Manhattan distance between the service data matrix to be identified and each sample data matrix in a pre-established sample database; selecting the K sample data with the smallest Manhattan distance, where the service with the highest frequency among the K sample data is the service to which the service data to be identified belongs.

[0038] Both Scheme 1 and Scheme 2 suffer from low network resource utilization and low scalability. These two schemes use only a single model to analyze specific data fields in network traffic, resulting in excessive data being discarded and wasting resources. Furthermore, when the model needs to be extended to accommodate new data services, a large-scale replacement of the entire model is required, leading to low performance of existing interface extensions.

[0039] Schemes 3 and 4 suffer from a lack of specificity in their identification methods. Different network traffic flows exhibit significant variations in some data characteristics, making them unsuitable for using the same model to identify different types of data services. Furthermore, Schemes 3 and 4 suffer from low accuracy in service identification performance; given the sheer volume of data in the current network, the model's performance is significantly impacted.

[0040] In view of the above-mentioned technical problems, the technical solutions of the embodiments of this application are proposed.

[0041] The embodiments of this application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the embodiments provided herein are merely illustrative of the embodiments of this application and are not intended to limit the embodiments of this application. Furthermore, the embodiments provided below are some embodiments for implementing this application, and not all embodiments for implementing this application. Unless otherwise specified, the technical solutions described in the embodiments of this application can be implemented in any combination.

[0042] It should be noted that, in the embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a method or apparatus that includes a list of elements includes not only the elements expressly described, but also other elements not expressly listed, or elements inherent to implementing the method or apparatus. Without further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other related elements (e.g., steps in the method or units in the apparatus, such as portions of circuitry, processors, programs, or software, etc.) in the method or apparatus that includes that element.

[0043] For example, the service identification method provided in this application includes a series of steps, but the service identification method provided in this application is not limited to the steps described. Similarly, the service identification device provided in this application includes a series of modules, but the device provided in this application is not limited to the modules explicitly described, but may also include modules that need to be set up for obtaining relevant information or processing based on information.

[0044] This application embodiment can apply deep learning algorithms to network traffic identification services, and can analyze network data by combining semantic field information contained in each layer of the network protocol. This application embodiment can be implemented based on DPI technology. The DPI device can monitor traffic sending and receiving from the network application layer, and can realize network data analysis by analyzing the network traffic data packets involved and translating them into binary raw code streams.

[0045] In this embodiment, the data packets captured using DPI technology are stored in the following format: Data Packet Header #1, Data Packet Information #1, Data Packet Header #2, Data Packet Information #2, ..., Data Packet Header #n, Data Packet Content #n. Each data packet header contains several objective data parameters, such as protocol, host port, sender / receiver address, user information, device information, etc. The data packet information includes the data transmitted in a single session; the network records and saves possible user actions, encrypts them, and stores them in several data packet contents. The data packet header and data packet information constitute the smallest unit in network transmission. The payload data in this embodiment can adopt the above data packet storage format for subsequent analysis.

[0046] This application's embodiments can achieve service identification of network data based on a series of relatively mature and high-performance models in the AI ​​field. These models are mainly divided into CNN models and NLP models. In this application's embodiments, the CNN part can use mainstream algorithm frameworks, such as ResNet, Inception v3, and VGG-16 network structures. These algorithms are all based on a common basic structure of input layer-hidden layer-output layer. The NLP part uses mainstream frameworks such as Self-Attention, FastText, and TextCNN, which extract field words from network traffic to form sentences, analyze the semantic information between sentences, and classify the sentence categories.

[0047] Figure 1 This is a flowchart of the service identification method according to an embodiment of this application, such as... Figure 1 As shown, the process may include:

[0048] Step 101: Obtain the raw bitstream of the network data.

[0049] Here, network data can be network traffic data of any service. In practical applications, DPI devices can be used to monitor the raw bitstream of network data from the network application layer. The raw bitstream captured by the DPI device can be stored in the form of data packet storage. The data packet can include a data packet header and data packet information.

[0050] Step 102: Analyze the original bitstream to determine the service characteristic data of the original bitstream; determine the type of service characteristic data.

[0051] Step 103: Determine the first identification method for business identification of the original code stream based on the type of business feature data.

[0052] In this embodiment of the application, for two different original bitstreams, the first identification method used for the original bitstreams may be different when the types of business features are different. For example, the first identification method may be a method of business identification using an algorithmic model or a method of business identification using pre-defined manual rules; the method of business identification using an algorithmic model may be a method of business identification using a CNN model or a method of business identification using an NLP model.

[0053] Step 104: Process the business feature data using the first recognition method to obtain the business recognition result of the original code stream.

[0054] Here, the service identification result of the original bitstream is used to represent the service type corresponding to the original bitstream.

[0055] In practical applications, steps 101 to 104 can be implemented based on the processor of an electronic device. The processor can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor.

[0056] As can be seen, the embodiments of this application can determine the type of service feature data of the original bitstream by analyzing the original bitstream. Based on the type of service feature data, the first identification method for service identification of the original bitstream can be determined more accurately, which is beneficial to improving the accuracy of service identification of the original bitstream. Furthermore, since the first identification method is determined based on the type of service feature data, the embodiments of this application adopt different service identification methods for network data of different service types. Compared with the scheme of using a single model to identify data services in related technologies, it has a wider range of applications. Moreover, for scenarios that need to expand new network data services, it is not necessary to replace the algorithm model on a large scale, but can directly determine the service identification method more accurately for new network data services, which has high scalability.

[0057] In some embodiments of this application, the process of analyzing the original bitstream to determine its service characteristic data may include: determining the protocol type of the original bitstream; and determining the service characteristic data of the original bitstream based on its protocol type.

[0058] In this embodiment of the application, the protocol type of the original bitstream can be HTTP or HTTPS, or it can be a newly added supported protocol. For example, the newly added supported protocol can be a protocol such as Transmission Control Protocol (TCP) or User Datagram Protocol (UDP).

[0059] As can be seen, the embodiments of this application can determine the service feature data of the original bitstream more accurately based on the protocol type of the original bitstream, which is conducive to determining the first identification method for service identification of the original bitstream more accurately based on the type of service feature data of the original bitstream, and thus improving the accuracy of service identification of the original bitstream.

[0060] In some embodiments of this application, the process of determining the service characteristic data of the original bitstream based on its protocol type may include:

[0061] When the protocol type of the original bitstream is HTTP, the bitstream information and key plaintext fields of the original bitstream are identified as the business characteristic data of the original bitstream. The key plaintext fields are fields that use plaintext to represent the business type of the original bitstream.

[0062] If the protocol type of the original bitstream is HTTPS or a newly added supported protocol, at least the key plaintext fields of the original bitstream should be identified as the business characteristic data of the original bitstream.

[0063] When the protocol type of the original bitstream is other protocols, the target field of the original bitstream is determined as the business characteristic data of the original bitstream. Other protocols refer to network communication protocols other than HTTP, HTTPS and newly added supported protocols, and the target field is a predefined field.

[0064] In some embodiments, for supported existing protocols such as HTTP, the bitstream information and key plaintext fields of the original bitstream can be identified as the business feature data of the original bitstream. For example, the bitstream information of the original bitstream can be the feature field, where the feature field is the translated hexadecimal bitstream. The key plaintext fields can be understood as fields containing semantic information strongly associated with the current business, such as host name, source Internet Protocol (IP) information, and destination IP information.

[0065] When training a model for business identification of bitstreams, feature fields and key plaintext fields can be used as training data for the model for supported existing protocols. The feature fields can be fed as one-dimensional data into a one-dimensional convolutional network for training.

[0066] In some embodiments, for the HTTPS protocol, since the information in the original bitstream is encrypted by the Transport Layer Security (TLS) protocol, only translatable field information such as the hostname can be obtained; these translatable field information are key plaintext fields.

[0067] In some embodiments, for newly added supported protocols such as TCP and UDP, at least the key plaintext fields of the original bitstream can be identified as the business characteristic data of the original bitstream.

[0068] In some embodiments, for other protocols that are currently unidentifiable and unextendable, business identification can be performed based on target fields. Here, the method for performing business identification based on target fields may include at least one of the following: a method for performing business identification using an NLP model, or a method for performing business identification using pre-defined manual rules.

[0069] As can be seen, the embodiments of this application can determine the service feature data in the original bitstream more accurately based on the protocol type of the original bitstream, which is conducive to determining the first identification method for service identification of the original bitstream more accurately, thereby improving the accuracy of service identification of the original bitstream.

[0070] In some embodiments of this application, the process of determining the business characteristic data of the original bitstream based on the protocol type of the original bitstream further includes: when the protocol type of the original bitstream is HTTPS, determining the content in the original bitstream from which plaintext information cannot be obtained as the business characteristic data of the original bitstream.

[0071] It can be seen that for the HTTPS protocol, since the content in the original bitstream from which plaintext information cannot be obtained can be identified as the business characteristic data of the original bitstream, it is beneficial to more accurately identify the business of the original bitstream, thereby improving the accuracy of business identification of the original bitstream.

[0072] In some embodiments of this application, the process of determining a first identification method for identifying services in the original bitstream based on the type of service feature data may include:

[0073] When the business feature data is a key plaintext field, the first identification method is determined to include the method of using an NLP model for business identification.

[0074] When the business feature data is the bitstream information of the original bitstream, the first identification method is determined to include the method of using a CNN model for business identification; or, when the protocol type of the original bitstream is HTTPS protocol, and the business feature data is content in the original bitstream for which plaintext information cannot be obtained, the first identification method is determined to include the method of using a CNN model for business identification.

[0075] When the business feature data is the target field, determine whether the target field contains the key plaintext field and obtain the judgment result; determine the first identification method based on the judgment result.

[0076] In some embodiments, for supported existing protocols such as HTTP, since the business feature data includes the bitstream information of the original bitstream and key plaintext fields, the first identification method includes a method for business identification using an NLP model and a method for business identification using a CNN model.

[0077] In some embodiments, for the HTTPS protocol, translatable field information can be input into an NLP model, and the NLP model can be used to process the translatable field information to obtain the business identification result; alternatively, content in the original bitstream from which plaintext information cannot be obtained can be input into a CNN model, and the CNN model can be used to process content in the original bitstream from which plaintext information cannot be obtained to obtain the business identification result.

[0078] In some embodiments, for newly added supported protocols such as TCP and UDP, key plaintext fields can be input into the NLP model, and the NLP model can be used to process the key plaintext fields to obtain business identification results.

[0079] As can be seen, the embodiments of this application can determine the first identification method for business identification of the original bitstream more accurately according to the type of business feature data, thereby improving the accuracy of business identification of the original bitstream.

[0080] In some embodiments of this application, if the determination result is that the target field contains the key plaintext field, the first identification method is determined to include a method of business identification using an NLP model; if the determination result is that the target field does not contain the key plaintext field, the first identification method is determined to include a method of business identification using pre-defined manual rules.

[0081] Here, for other protocols that are currently unidentifiable and cannot be extended, if the target field includes a key plaintext field, an NLP model can be used to process the key plaintext field in the target field to obtain the business identification result; if the target field does not contain a key plaintext field, pre-defined manual rules can be used for business identification. For example, the target field can be judged according to a pre-defined manual rule table to obtain the business identification result.

[0082] As can be seen, the embodiments of this application can flexibly determine the method for business identification of the original bitstream based on the content in the target field, which is beneficial for business identification of different types of data services.

[0083] In some embodiments of this application, the process of processing service feature data using a first identification method to obtain the service identification result of the original code stream may include:

[0084] When the first identification method includes a method for identifying services using the NLP model, or when the first identification method includes a method for identifying services using a CNN model, the first identification method is used to process the service feature data to obtain a preliminary service identification result and a model prediction probability of the original bitstream. The model prediction probability represents the accuracy of obtaining the preliminary service identification result using the NLP model or the CNN model.

[0085] If the model predicts a probability greater than or equal to the probability threshold, the preliminary service identification result of the original bitstream is taken as the service identification result; if the model predicts a probability less than the probability threshold, the first identification method is redefined as the method of using pre-defined manual rules for service identification, and the service feature data is processed using the redefined first identification method to obtain the service identification result of the original bitstream.

[0086] Here, the probability threshold is a preset model threshold parameter, which is between 0 and 1.

[0087] As can be seen, in this embodiment, if the model predicts a probability greater than or equal to the probability threshold, the preliminary business identification result obtained by using the NLP model or CNN model can be retained; if the model predicts a probability less than the probability threshold, it indicates that the accuracy of the preliminary business identification result is low. Therefore, manual rules are used for business identification, which helps to improve the accuracy of business identification.

[0088] According to the embodiments of this application, the dataset created from the original bitstream can be divided according to the protocol based on the data packets representing the original bitstream captured by the DPI device, and the corresponding service identification method can be determined for different protocols. Different AI models for service identification can be determined for different fields of different protocols, so as to train and apply the models.

[0089] In network traffic service identification scenarios, given the complexity of protocols and service scenarios in existing network traffic, this application embodiment can determine service feature data for service identification based on judgment conditions, thereby determining the AI ​​model used for service identification. Here, the judgment conditions can be at least one of the following: the original bitstream contains key plaintext fields, the original bitstream contains bitstream information, or the original bitstream contains target fields; the AI ​​model can be an NLP model or a CNN model. This application embodiment can perform service identification on multiple types of data in the original bitstream by adding parallel algorithm middleware and using two different types of algorithm frameworks, thus broadening the data coverage and enabling deeper mining of more effective feature information in the data. Here, the parallel algorithm middleware can be multiple AI models that implement service identification in parallel, and the two different types of algorithm frameworks can be the algorithm framework of the NLP model and the algorithm framework of the CNN model.

[0090] Because network data has diverse data characteristics, wide data sources, and artificial rules prior to its use, the embodiments of this application can adopt a multi-layered and multi-dimensional judgment method to achieve business identification.

[0091] The embodiments of this application can realize service identification of the raw bitstream based on an end-to-end system. Figure 2 This is a flowchart illustrating the use of an end-to-end system for service identification in this application embodiment, referencing... Figure 2 The original bitstream is preprocessed to obtain a dataset; then, the original bitstream is classified according to the protocol field to determine the protocol type of the original bitstream.

[0092] When the protocol type of the original bitstream is an existing supported protocol such as HTTP or HTTPS, key plaintext fields can be extracted from the original bitstream; for example, when the protocol type of the original bitstream is an existing supported protocol such as HTTP or HTTPS, bitstream information can be extracted from the original bitstream.

[0093] When the protocol type of the original bitstream is a newly added supported protocol, the end-to-end system can achieve system interface extensibility for the newly added supported protocol. By providing an interface for the new field requirements, the key plaintext fields determined based on the new field requirements are identified. That is, after determining the new field requirements, the key plaintext field extraction step is performed. It can be seen that, through the extensibility of the system interface, the implementation of this application enables the operation of other substructures within the end-to-end system to be unaffected when a new supported protocol is added.

[0094] After determining the AI ​​model compatible with the original bitstream based on its protocol type, the AI ​​model can be used to process the business feature data in the original bitstream to obtain preliminary business identification results and model prediction probabilities. If the model prediction probability is greater than or equal to a probability threshold, the preliminary business identification results of the original bitstream are retained. If the model prediction probability is less than the probability threshold, business identification based on manually generated rules can be performed to obtain the business identification results. In some embodiments, the manually generated rule table is a series of rules generated manually, representing rules based on expert experience and representing human suggestions for data classification.

[0095] If the protocol type of the original bitstream is other than the protocol type, business identification can be performed based on the manual rule table to obtain the business identification result.

[0096] from Figure 2 As can be seen from the overall process shown, the business identification method of this application embodiment has strong scalability. For new protocols or other protocols, the algorithm model of the AI ​​model can be a new algorithm model.

[0097] This application's embodiments can classify the original bitstream according to set judgment conditions, with the only issue being incorrect scenario classification due to problems with the data source itself. By setting probability thresholds, for cases where the accuracy of the initial business identification results is low, switching to manual rules for business identification can improve the accuracy of business identification. The various AI models do not interfere with each other, and the processing of different types of business feature data also does not interfere with each other, enabling parallel processing of different types of business feature data, thereby simultaneously obtaining business identification results corresponding to different types of business feature data.

[0098] The training process of the aforementioned end-to-end system is illustrated below. After obtaining the raw bitstream used for training the end-to-end system, data preprocessing is first performed on the raw bitstream to obtain a dataset created from it. For example, each data entry in the dataset contains a series of fields, including label ID, feature field, hostname, protocol, package length, and Uniform Resource Locator (URL). Table 1 shows the detailed information of these fields.

[0099] Table 1

[0100] label id int Data tags, business category feature int Hexadecimal code stream host string hostname protocol string Protocol Category Package length list The length of each data packet URL string Link

[0101] When reading the dataset, the original bitstream is first categorized by protocol type. This means the original bitstream in the dataset can be categorized into several types, such as Protocol 1, Protocol 2, ..., Protocol N. Each protocol has multiple available fields, therefore the compatible AI models are also diverse. For example, the HTTP protocol includes both translatable feature fields and key plaintext fields, making both CNN and NLP models applicable. This allows for a horizontal comparison of the prediction accuracy of different models, enabling multi-dimensional determination of the business category and resulting in superior business identification performance.

[0102] In this embodiment, key plaintext fields are defined as fields in the database that possess clear semantic information and can intuitively represent the specific information of the current business category. These fields can be used as training data for language models, such as BERT and FastText algorithms.

[0103] This application embodiment can set different judgment conditions to use different contents in the original code stream as business feature data for end-to-end system training. Here, the different contents in the original code stream can be code stream information, key plaintext fields, target fields, etc.

[0104] In this embodiment, AI model adaptation is a model adaptation mechanism designed for different protocols. This maximizes the utilization of field information. Based on prior knowledge, it is known that the feature information contained in different fields is not suitable for training with a single model. However, the payload data contains significant differences in data divided by protocol. For example, the HTTP protocol contains unencrypted bitstream information and key plaintext fields; the bitstream information in HTTPS is encrypted. Therefore, the key plaintext fields are selected for training language models such as NLP.

[0105] After training the end-to-end system using the loaded model framework, the trained system can be used to perform business identification on the original bitstream. For example, after determining the AI ​​model compatible with the original bitstream based on its protocol type, the AI ​​model can be used to process the business feature data in the original bitstream to obtain preliminary business identification results and model prediction probabilities. If the model prediction probability is less than a probability threshold, it indicates that the AI ​​model's business identification accuracy is insufficient to meet the accuracy requirements. Therefore, business identification based on manual rules can be used to obtain a business identification result, which then overwrites the preliminary business identification result. It should be noted that manual rules cannot completely replace the algorithm model; they only serve as an auxiliary tool in the business identification of data.

[0106] Table 2 shows the training accuracy, testing accuracy, and model size of various AI models. As can be seen from Table 2, for feature fields, models such as Inception and ResNet can be used for training and application, while for host fields, models such as FastText and TextCNN can be used. FastText and TextCNN models are generally lightweight and have relatively simple structures, offering high inference speeds but also having the problem of limited model data.

[0107] Table 2

[0108] Inception feature 60％ 40％ 400MB ResNet feature 86％ 78％ 1.1GB FastText host, etc. 94％ 87％ 20MB TextCNN host, etc. 88％ 82％ 100MB

[0109] Taking residual networks as an example, Table 3 shows the recognition coverage and recognition accuracy of residual network models corresponding to different probability thresholds. Here, recognition coverage represents the proportion of preliminary business recognition results that are higher than the probability threshold among all preliminary business recognition results after setting the probability threshold, and recognition accuracy represents the accuracy of preliminary business recognition results that are higher than the probability threshold.

[0110] Table 3

[0111] 0.3 87 70 0.5 79 69 0.7 61 76 0.9 42 83 0.99 31 85

[0112] In summary, the embodiments of this application can determine the AI ​​model that is compatible with the original bitstream based on the protocol type of the original bitstream; for different protocol types, the final business identification result can be determined by combining the AI ​​model and manual rules; for cases where the model's predicted probability is less than the probability threshold, manual rules are switched to mitigate the risk of the output business identification result, thereby ensuring the inference accuracy of the overall framework.

[0113] Compared to related technologies that rely solely on manual rule verification for business identification, this application embodiment applies AI models to the business identification scheme, thereby improving the accuracy of business identification. For example, compared to related technologies that rely solely on manual rule verification for business identification, the business identification scheme of this application embodiment can improve the identification accuracy by 10%-20%.

[0114] Compared to related technologies that rely solely on manual rule verification for business identification, this application's embodiments organically combine manual rules and AI models, with the two complementing each other. The AI ​​model can accurately identify most of the original bitstreams. If the accuracy of the preliminary business identification result obtained by the AI ​​model is low, it indicates that the preliminary business identification result obtained by the AI ​​model is an abnormal result. For abnormal results, business identification based on manual rules can be used to obtain the business identification result.

[0115] Compared to lightweight machine learning models, the embodiments of this application, by using deep learning AI models, can more accurately predict the service type of network data and support larger data volumes. Furthermore, the embodiments of this application provide a high degree of scalability for new services and scenarios; the coupling between various substructures in the end-to-end system is low, and the model structure has high reliability.

[0116] The embodiments of this application can be applied to scenarios of data mining and deep learning using DPI devices, and can realize the identification of network data services. They can be widely used in specific data extraction and algorithm training.

[0117] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0118] Based on the service identification method proposed in the foregoing embodiments, this application also proposes a service identification device.

[0119] Figure 3 This is a schematic diagram of the structure of the service identification device according to an embodiment of this application, such as... Figure 3 As shown, the device may include an acquisition module 300, a first processing module 301, a second processing module 302, and a third processing module 303; wherein,

[0120] Module 300 is used to acquire the raw bitstream of network data.

[0121] The first processing module 301 is used to analyze the original bitstream to determine the service feature data of the original bitstream and to determine the type of the service feature data.

[0122] The second processing module 302 is used to determine a first identification method for identifying services in the original code stream based on the type of the service feature data.

[0123] The third processing module 303 is used to process the service feature data using the first identification method to obtain the service identification result of the original code stream.

[0124] In some embodiments, the first processing module 301 is configured to determine the service characteristic data of the original bitstream by analyzing the original bitstream, including: determining the protocol type of the original bitstream; and determining the service characteristic data of the original bitstream based on the protocol type of the original bitstream.

[0125] In some embodiments, the first processing module 301 is configured to determine the service characteristic data of the original bitstream based on the protocol type of the original bitstream, including:

[0126] When the protocol type of the original bitstream is HTTP, the bitstream information and key plaintext fields of the original bitstream are determined as the business characteristic data of the original bitstream, and the key plaintext fields are fields that use plaintext to represent the business type of the original bitstream;

[0127] If the protocol type of the original bitstream is HTTPS or a newly added supported protocol, at least the key plaintext fields of the original bitstream shall be identified as the business characteristic data of the original bitstream.

[0128] If the protocol type of the original bitstream is other protocols, the target field of the original bitstream is determined as the business characteristic data of the original bitstream. The other protocols refer to network communication protocols other than HTTP, HTTPS and the newly added supported protocols. The target field is a predefined field.

[0129] In some embodiments, the first processing module 301 is configured to determine the service characteristic data of the original bitstream based on the protocol type of the original bitstream, and further includes:

[0130] When the protocol type of the original bitstream is HTTPS, the content in the original bitstream from which plaintext information cannot be obtained is identified as the business characteristic data of the original bitstream.

[0131] In some embodiments, the second processing module 302 is configured to determine a first identification method for identifying services in the original bitstream based on the type of the service feature data, including:

[0132] When the business feature data is a key plaintext field, it is determined that the first identification method includes a method of business identification using an NLP model; the key plaintext field is a field that uses plaintext to characterize the business type of the original bitstream.

[0133] If the business feature data is the bitstream information of the original bitstream, the first identification method is determined to include a method of business identification using a CNN model; or, if the protocol type of the original bitstream is HTTPS and the business feature data is content in the original bitstream for which plaintext information cannot be obtained, the first identification method is determined to include a method of business identification using a CNN model.

[0134] When the business feature data is a target field, it is determined whether the target field contains the key plaintext field, and a determination result is obtained; the first identification method is determined based on the determination result; the target field is a predefined field.

[0135] In some embodiments, the second processing module 302 is configured to determine the first identification method based on the judgment result, including:

[0136] If the determination result is that the target field contains the key plaintext field, it is determined that the first identification method includes a method of using an NLP model for business identification;

[0137] If the determination result is that the target field does not contain the key plaintext field, then the first identification method is determined to include a method of business identification using pre-defined manual rules.

[0138] In some embodiments, the third processing module 303 is used to process the service feature data using the first identification method to obtain the service identification result of the original code stream, including:

[0139] When the first identification method includes a method for business identification using the NLP model, or when the first identification method includes a method for business identification using the CNN model, the first identification method is used to process the business feature data to obtain a preliminary business identification result and a model prediction probability of the original bitstream. The model prediction probability represents the accuracy of obtaining the preliminary business identification result using the NLP model or the CNN model.

[0140] If the model predicts a probability greater than or equal to a probability threshold, the preliminary service identification result of the original bitstream is taken as the service identification result; if the model predicts a probability less than the probability threshold, the first identification method is redefined as a method for service identification using pre-defined manual rules, and the redefined first identification method is used to process the service feature data to obtain the service identification result of the original bitstream.

[0141] In practical applications, the acquisition module 300, the first processing module 301, the second processing module 302, and the third processing module 303 can all be implemented based on the processor of the electronic device.

[0142] It should be noted that the description of the above device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0143] It should be noted that, in the embodiments of this application, if the above methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a terminal, server, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0144] Correspondingly, this application embodiment further provides a computer program product, the computer program product including computer executable instructions, which are used to implement any of the business identification methods provided in this application embodiment.

[0145] Accordingly, this application embodiment further provides a computer storage medium storing computer-executable instructions, which are used to implement any of the service identification methods provided in the above embodiments.

[0146] This application also provides an electronic device. Figure 4 This is a schematic diagram of the composition structure of the electronic device provided in the embodiments of this application, such as... Figure 4 As shown, the electronic device 40 may include:

[0147] Memory 401 is used to store executable instructions;

[0148] The processor 402 is used to implement any of the above-mentioned service identification methods when executing the executable instructions stored in the memory 401.

[0149] The processor 402 mentioned above can be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor.

[0150] The aforementioned computer-readable storage medium / memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.; it can also be various terminals that include one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0151] In some embodiments, the functions or modules of the apparatus provided in this application can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0152] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0153] The methods disclosed in the various method embodiments provided in this application can be arbitrarily combined to obtain new method embodiments without conflict.

[0154] The features disclosed in the various product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0155] The features disclosed in the various method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0156] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0157] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims. All of these forms are within the protection scope of this application.

Claims

1. A business identification method, characterized in that, The method includes: Obtain the raw bitstream of network data; By analyzing the original bitstream, the service characteristic data of the original bitstream is determined; the type of the service characteristic data is determined. Based on the type of the business feature data, a first identification method for identifying the business in the original code stream is determined; The first identification method is used to process the service feature data to obtain the service identification result of the original code stream; The first identification method for determining the service identification of the original code stream based on the type of the service feature data includes: When the business feature data is a key plaintext field, it is determined that the first identification method includes a method for business identification using a Natural Language Processing (NLP) model; the key plaintext field is a field that uses plaintext to represent the business type of the original bitstream. If the business feature data is the bitstream information of the original bitstream, the first identification method is determined to include a method of business identification using a convolutional neural network (CNN) model; or, if the protocol type of the original bitstream is HTTPS and the business feature data is content in the original bitstream for which plaintext information cannot be obtained, the first identification method is determined to include a method of business identification using a CNN model. When the business feature data is a target field, it is determined whether the target field contains the key plaintext field, and a determination result is obtained; the first identification method is determined based on the determination result; the target field is a predefined field.

2. The method according to claim 1, characterized in that, The step of analyzing the original bitstream to determine its service characteristic data includes: Determine the protocol type of the original bitstream; based on the protocol type of the original bitstream, determine the service characteristic data of the original bitstream.

3. The method according to claim 2, characterized in that, The step of determining the service characteristic data of the original bitstream based on its protocol type includes: When the protocol type of the original bitstream is Hypertext Transfer Protocol (HTTP), the bitstream information and key plaintext fields of the original bitstream are determined as the business characteristic data of the original bitstream, and the key plaintext fields are fields that use plaintext to characterize the business type of the original bitstream; If the protocol type of the original bitstream is HTTPS or a newly added supported protocol, at least the key plaintext fields of the original bitstream shall be identified as the business characteristic data of the original bitstream. The newly added supported protocols include: Transmission Control Protocol and User Datagram Protocol; If the protocol type of the original bitstream is other protocols, the target field of the original bitstream is determined as the business characteristic data of the original bitstream. The other protocols refer to network communication protocols other than HTTP, HTTPS and the newly added supported protocols. The target field is a predefined field.

4. The method according to claim 3, characterized in that, The step of determining the service characteristic data of the original bitstream based on its protocol type further includes: When the protocol type of the original bitstream is HTTPS, the content in the original bitstream from which plaintext information cannot be obtained is identified as the business characteristic data of the original bitstream.

5. The method according to claim 1, characterized in that, The step of determining the first identification method based on the judgment result includes: If the determination result is that the target field contains the key plaintext field, it is determined that the first identification method includes a method of using an NLP model for business identification; If the determination result is that the target field does not contain the key plaintext field, then the first identification method is determined to include a method of business identification using pre-defined manual rules.

6. The method according to claim 1, characterized in that, The step of processing the service feature data using the first identification method to obtain the service identification result of the original code stream includes: When the first identification method includes a method for business identification using the NLP model, or when the first identification method includes a method for business identification using the CNN model, the first identification method is used to process the business feature data to obtain a preliminary business identification result and a model prediction probability of the original bitstream. The model prediction probability represents the accuracy of obtaining the preliminary business identification result using the NLP model or the CNN model. If the model predicts a probability greater than or equal to a probability threshold, the preliminary service identification result of the original bitstream is taken as the service identification result; if the model predicts a probability less than the probability threshold, the first identification method is redefined as a method for service identification using pre-defined manual rules, and the redefined first identification method is used to process the service feature data to obtain the service identification result of the original bitstream.

7. A service identification device, characterized in that, The device includes: The acquisition module is used to acquire the raw bitstream of network data; The first processing module is used to analyze the original bitstream to determine the service feature data of the original bitstream and to determine the type of the service feature data. The second processing module is used to determine a first identification method for identifying the business in the original code stream based on the type of the business feature data. The third processing module is used to process the business feature data using the first identification method to obtain the business identification result of the original code stream; The second processing module is further configured to determine, when the business feature data is a key plaintext field, that the first identification method includes a method for business identification using a Natural Language Processing (NLP) model; the key plaintext field is a field that uses plaintext to characterize the business type of the original bitstream; If the business feature data is the bitstream information of the original bitstream, the first identification method is determined to include a method of business identification using a convolutional neural network (CNN) model; or, if the protocol type of the original bitstream is HTTPS and the business feature data is content in the original bitstream for which plaintext information cannot be obtained, the first identification method is determined to include a method of business identification using a CNN model. When the business feature data is a target field, it is determined whether the target field contains the key plaintext field, and a determination result is obtained; the first identification method is determined based on the determination result; the target field is a predefined field.

8. An electronic device, characterized in that, Includes a processor and memory for storing computer programs that can run on the processor; wherein, The processor is used to run the computer program to perform the service identification method according to any one of claims 1 to 6.

9. A computer storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the service identification method according to any one of claims 1 to 6.

Images