A method and apparatus for identifying encrypted data streams

Abstract

The application describes a method and device for identifying encrypted data flow, the method comprising: parsing a plurality of handshake messages of an encrypted data flow according to a secure encrypted transmission protocol corresponding to the encrypted data flow, to obtain a plurality of fields contained in the plurality of handshake messages; determining a rule set matched by the plurality of handshake messages from a plurality of rule sets according to the plurality of fields, and determining an application corresponding to the encrypted data flow according to a mapping relationship between the matched rule set and the application. The method can more accurately identify encrypted traffic in the network, i.e., can more accurately identify encrypted data flow.

Description

[0001] This application is a divisional application. The original application has the application number 201711141610.3 and the original application date is November 17, 2017. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This invention relates to the field of computers, and more particularly to a method and apparatus for identifying encrypted data streams. Background Technology

[0003] The Internet has rapidly spread and grown due to its openness and sharing characteristics, giving rise to a growing number of new network applications. The openness of the Internet also means that any device or software that meets its technical standards can access it without restrictions. To effectively monitor and manage networks, enhance network controllability (e.g., effectively utilize bandwidth and provide better quality of service (QoS), or improve network security and reduce cybercrime), a key issue to address is identifying network traffic. This includes classifying traffic from a data flow perspective, and more specifically, determining which application a data flow belongs to—that is, which application's data a data flow carries.

[0004] Common secure encrypted transport protocols below the application layer include Secure Socket Layer (SSL), Transport Layer Security (TLS), and Datagram Transport Layer Security (DTLS). TLS can be considered an upgrade of SSL. DTLS, based on TLS, is used to protect the security of data transmission over UDP connections. Data streams transmitted over a network using secure encrypted transport protocols are called encrypted data streams (or simply encrypted streams), and the traffic of these encrypted streams is also called encrypted traffic. Encrypted streams are difficult to parse at the sending and receiving ends of the data stream; therefore, identifying the application to which encrypted traffic belongs in a network is currently a major technical challenge in the industry.

[0005] Taking the TLS protocol as an example, encrypted traffic can be identified by parsing the SNI (Server Name Indication) field in the handshake message. This SNI field is located in the ClientHello handshake message and indicates the host name of the encrypted stream containing the ClientHello handshake message. Alternatively, it can be identified by parsing the Common Name field in the handshake message. This Common Name field is located in the subject field of the Certificate handshake message and includes information indicating the host name of the encrypted stream containing the Certificate handshake message.

[0006] However, in many scenarios, the above fields can be set to carry incorrect or ambiguous information (such as wildcards), and the above fields may not exist in TLS messages. Using a single SNI field or Common Name field for identification makes it impossible to correctly identify the corresponding application for encrypted traffic. Summary of the Invention

[0007] This application provides a method and apparatus for identifying encrypted data streams, which can more accurately identify encrypted traffic in the network, that is, can identify more encrypted data streams more accurately. Here, identification refers to identifying the application corresponding to the encrypted data stream.

[0008] In a first aspect, this application discloses a method for identifying an encrypted data stream. The method includes: parsing multiple handshake messages of the encrypted data stream according to a secure encrypted transmission protocol corresponding to the encrypted data stream to obtain multiple fields contained in the multiple handshake messages; determining a set of rules matching the multiple handshake messages from multiple set of rules based on the multiple fields, wherein each set of rules includes at least one rule of field rules and order rules, the field rules indicating the characteristics of a field, the order rules indicating the order of multiple fields in a handshake message, and the fields in the multiple handshake messages satisfying the rules in the matching set of rules; and determining the application corresponding to the encrypted data stream based on the mapping relationship between the matching set of rules and the application.

[0009] By parsing fields in multiple handshake messages within an encrypted data stream and matching these fields with multiple rule sets, the rule sets corresponding to these multiple data packets can be found. Then, the application mapped to the rule set can be determined. This eliminates the need to rely on easily tampered or inaccurate domain name information in the handshake messages, enabling more accurate identification of which application the encrypted data stream corresponds to and improving the accuracy of encrypted data stream identification.

[0010] It should be understood that these multiple fields come from these multiple handshake messages.

[0011] It should be understood that multiple rule sets refer to more than one set of rules. Therefore, "determining the rule set matching the multiple handshake messages from multiple rule sets" means determining the rule set matching the multiple handshake messages from these multiple rule sets. In other words, these multiple rule sets also include at least one type of rule: field rules and sequence rules.

[0012] It should be understood that "parsing multiple handshake messages of an encrypted data stream according to a secure encrypted transmission protocol corresponding to the encrypted data stream" is something that those skilled in the art should understand. For example, in one implementation, multiple fields in the multiple handshake messages of the encrypted data stream are parsed by referring to the definitions of fields in the handshake messages in the secure encrypted transmission protocol corresponding to the encrypted data stream.

[0013] In one implementation, each rule set in the multiple rule sets corresponds to a unique application.

[0014] In one implementation, each of the multiple rule sets is mapped to an application.

[0015] In one implementation, the multiple handshake messages satisfy all rules in the matching rule set. For example, the rule set matching the multiple handshake messages includes only at least one rule among field rules and sequence rules, and the fields in the multiple handshake messages satisfy all rules in the matching rule set. In another implementation, the multiple handshake messages satisfy some rules in the matching rule set. This implementation addresses situations where, due to the special settings of certain rule sets, it is not necessary to match all rules in a rule set (e.g., some rules are optional) to determine the corresponding rule set of the multiple handshake messages, and thus determine the application corresponding to the encrypted data stream.

[0016] In one implementation, the plurality of fields include a plurality of groups, each group corresponding to a handshake message. Determining the rule set matching the plurality of handshake messages from a plurality of rule sets based on the plurality of fields includes: matching the plurality of groups with rules in the plurality of rule sets according to the receiving order of the plurality of handshake messages, to obtain the rule set matching the plurality of handshake messages from the plurality of rule sets. Similarly, in another implementation, the plurality of groups can be matched with rules in the plurality of rule sets according to the timing of the plurality of handshake messages, to obtain the rule set matching the plurality of handshake messages from the plurality of rule sets. The timing refers to the defined interaction order of the handshake messages during the handshake phase. This allows for the identification of encrypted streams without waiting for subsequent handshake messages to arrive, and because different rule sets include different handshake messages, applications where some rules are only distributed in the first few handshake messages can be identified more quickly. As one implementation of the apparatus described in the third aspect, the matching module in the apparatus described in the third aspect can be used to implement this step.

[0017] In one implementation, the plurality of rule sets include field rules and order rules. Determining the rule set matching the plurality of handshake messages from the plurality of rule sets based on the plurality of fields includes: matching the plurality of fields with the field rules and order rules in the plurality of rule sets respectively, to determine the rule set matching the plurality of handshake messages from the plurality of rule sets. This categorized matching of rules in the rule sets allows for batch processing, improving matching speed. As one implementation of the apparatus described in the third aspect, the matching module in the apparatus described in the third aspect can be used to implement this step.

[0018] In one implementation, the matching rule set includes multiple subsets, each subset corresponding to at least one application. Determining the application corresponding to the encrypted data stream based on the mapping relationship between the matching rule set and the application includes: obtaining multiple application sets corresponding to the multiple subsets based on the mapping relationship between each subset and the application; and finding the intersection of the multiple application sets to obtain a unique application corresponding to the rule set, where the application is the application corresponding to the encrypted data stream. In another implementation, each rule set may include multiple subsets, where the data packets corresponding to rules in each subset have the same name, and the data packets corresponding to different subsets within the same rule set have different names. That is, the representation of the mapping relationship between the rule set and the application can be varied, allowing for parallel or serial matching of rules and mapping of applications to these multiple data packets, thus accelerating the identification of the encrypted stream. As one implementation of the apparatus described in the third aspect, the determining module in the apparatus described in the third aspect can be used to implement this step.

[0019] In one implementation, the multiple rule sets include several rules stored in a linked list, where each node in the linked list stores a rule corresponding to the name of a handshake message. This allows for better and more orderly management of the rules in the rule sets, facilitates matching handshake messages one by one, and enables faster identification of encrypted streams.

[0020] In one implementation, parsing multiple handshake messages of an encrypted data stream includes: for each handshake message in the encrypted data stream, performing the following operations: parsing the name of each handshake message; determining at least one rule corresponding to the name of each handshake message from the multiple rule sets based on the name of each handshake message; and parsing the fields indicated by the at least one rule in each handshake message. This allows only the fields needed for subsequent steps to be parsed, reducing the processing load of the parsing steps, making the identification process faster, and saving the resources consumed by the parsing steps. As one implementation of the apparatus described in the third aspect, the parsing module in the apparatus described in the third aspect can be used to implement this step.

[0021] In one implementation, the method further includes: training multiple samples from multiple encrypted data streams using a machine learning algorithm to obtain at least one rule set corresponding to the target application, wherein the multiple samples are known to be handshake messages of the encrypted data streams corresponding to the target application, and the multiple samples include handshake messages of the encrypted data streams corresponding to the target application. That is, the rule set, and the mapping relationship between the rule set and the application, can be obtained through machine learning algorithm training. In this way, the obtained information can more accurately describe the correspondence between data packets and applications. Additionally, the rule set and the mapping relationship between the rule set and the application can also be updated in this manner. As one implementation of the apparatus described in the third aspect, the apparatus may further include a training module, which can be used to implement this step.

[0022] Secondly, this application describes another method for identifying encrypted data streams. The method includes: parsing a handshake message of the encrypted data stream according to a secure encrypted transmission protocol corresponding to the encrypted data stream to obtain multiple fields contained in the handshake message; determining a set of rules matching the handshake message from multiple set of rules based on the multiple fields, wherein each set of rules includes at least one rule selected from field rules and order rules, the field rules indicating the characteristics of a field, and the order rules indicating the order of multiple fields in a data packet, the fields in the handshake message satisfying the rules in the matching set of rules; and determining the application corresponding to the encrypted data stream based on the mapping relationship between the matching set of rules and the application. This second method, by using at least one of the characteristics and order of multiple fields, eliminates the reliance on easily tampered or inaccurate domain name information in the handshake message, enabling more accurate identification of which application the encrypted data stream corresponds to.

[0023] As can be seen, the method described in the second aspect is very similar to that described in the first aspect. The difference is that the second aspect can identify the application corresponding to the encrypted data stream using only a handshake message. Therefore, the explanation and technical effects in the first aspect can also be applied to the corresponding parts of the second aspect. Furthermore, the description of the implementation methods for parsing a data packet, determining the rule set for matching handshake messages, and identifying the application corresponding to the encrypted data stream in the second aspect is also applicable to the processing of multiple data packets in the first aspect.

[0024] In one implementation, the plurality of rule sets are rule sets corresponding to the names of the handshake messages. In another implementation, parsing a handshake message of an encrypted data stream includes: determining at least one rule corresponding to the name of each handshake message from the plurality of rule sets; and parsing the fields indicated by the at least one rule in each handshake message. This allows only the fields needed for subsequent steps to be parsed, reducing the processing load of the parsing steps, making the identification process faster, and saving resources consumed in the parsing steps. As one implementation of the apparatus described in the fourth aspect, the parsing module in the apparatus described in the fourth aspect can be used to implement this step.

[0025] In one implementation, the plurality of rule sets include field rules and sequence rules. Determining the rule set matching the handshake message from the plurality of rule sets based on the plurality of fields includes: matching the plurality of fields with the field rules and sequence rules in the plurality of rule sets respectively, to determine the rule set matching the handshake message from the plurality of rule sets. This categorized matching of rules in the rule sets allows for batch processing, improving matching speed. As one implementation of the apparatus described in the fourth aspect, the matching module in the apparatus of the fourth aspect can be used to implement this step.

[0026] In one implementation, the method further includes training multiple samples using a machine learning algorithm to obtain at least one rule set corresponding to the target application, wherein the multiple samples are known to be handshake messages of the encrypted data stream corresponding to the target application, and the multiple samples include handshake messages of the encrypted data stream corresponding to the target application. As one implementation of the apparatus described in the fourth aspect, the apparatus may further include a training module that can be used to implement this step.

[0027] The following are some implementation methods that are applicable to both the first and second aspects.

[0028] In the two methods described above, the handshake messages can be actively captured by the device executing the method, or they can be sent by other network nodes. These multiple handshake messages can also be copied from the handshake messages transmitted in the network.

[0029] The parsing steps in the two aspects mentioned above involve using the secure encrypted transport protocol corresponding to the encrypted data stream to parse the handshake message of the encrypted data stream in order to obtain the multiple fields contained in the handshake message. The secure encrypted transport protocol corresponding to the encrypted data stream is SSL, TLS, DTLS, or other secure encrypted transport protocols running at the application layer. Accordingly, in one implementation, the multiple handshake messages are Secure Sockets Layer (SSL) handshake messages, Transport Layer Security (TLS) handshake messages, or Datagram Transport Layer Security (DTLS) handshake messages.

[0030] In one implementation, multiple rule sets and their mapping relationships to applications are stored in an identification information database. This identification information database can be located in the device executing the method for identifying encrypted data streams, or in other devices. That is, as one implementation, the apparatus described in the third and fourth aspects may further include a storage module for storing the information in the identification information database.

[0031] In one implementation, for a field rule, the characteristics of the indicated field are either the field length, the field type, the field length and type, the field type and value, or the field length, type, and value. In another implementation, the field characteristics are at least one of the field length, type, and value.

[0032] In one implementation, the plurality of rule sets include multiple rules. The step of determining the rule set matching the handshake message (or multiple handshake messages) from the plurality of rule sets based on the plurality of fields includes: matching the plurality of fields with the plurality of rules to obtain a set of rules corresponding to the plurality of fields; and matching the set of rules with the plurality of rule sets to obtain the rule set matching the handshake message (or multiple handshake messages). As one implementation of the apparatus described in the third and fourth aspects, the matching module in the apparatus of the third and fourth aspects can be used to implement this step.

[0033] In one implementation, determining the rule set matching the handshake message (or multiple handshake messages) from multiple rule sets based on the multiple fields includes: matching the multiple rule sets with the multiple fields until a rule set matching the handshake message (or multiple handshake messages) is obtained. As an implementation of the apparatus described in the third and fourth aspects, the matching module in the apparatus of the third and fourth aspects can be used to implement this step.

[0034] Thirdly, this application discloses an apparatus for identifying encrypted data streams, the apparatus comprising: a parsing module, configured to parse multiple handshake messages of an encrypted data stream according to a secure encrypted transmission protocol corresponding to the encrypted data stream, to obtain multiple fields contained in the multiple handshake messages; a matching module, configured to determine a set of rules matching the multiple handshake messages from multiple set of rules based on the multiple fields, wherein each set of rules includes at least one rule of field rules and order rules, the field rules indicating the characteristics of a field, the order rules indicating the order of multiple fields in a handshake message, and the fields in the multiple handshake messages satisfying the rules in the matching set of rules; and a determining module, configured to determine the application corresponding to the encrypted data stream based on the mapping relationship between the matching set of rules and the application.

[0035] By parsing fields in multiple handshake messages within an encrypted data stream and matching these fields with multiple rule sets, the rule sets corresponding to these multiple data packets can be found. Then, the application mapped to the rule set can be determined. This eliminates the need to rely on easily tampered or inaccurate domain name information in the handshake messages, enabling more accurate identification of which application the encrypted data stream corresponds to and improving the accuracy of encrypted data stream identification.

[0036] In one implementation, the device for identifying encrypted data streams stores the multiple rule sets and the mapping relationship between the rule sets and the application.

[0037] Since the third aspect is the apparatus corresponding to the first aspect, for the various definitions, implementation methods and technical effects of the third aspect, please refer to the description in the first aspect, and this application will not repeat them.

[0038] Fourthly, this application discloses an apparatus for identifying encrypted data streams, the apparatus comprising: a parsing module, configured to parse a handshake message of an encrypted data stream according to a secure encrypted transmission protocol corresponding to the encrypted data stream, to obtain multiple fields contained in the handshake message; a matching module, configured to determine a set of rules matching the handshake message from multiple set of rules based on the multiple fields, wherein each set of rules includes at least one rule of field rules and order rules, the field rules indicating the characteristics of a field, the order rules indicating the order of multiple fields in a data packet, and the fields in the handshake message satisfying the rules in the matching set of rules; and a determining module, configured to determine the application corresponding to the encrypted data stream based on the mapping relationship between the matching set of rules and the application.

[0039] In one implementation, the device for identifying encrypted data streams stores the multiple rule sets and the mapping relationship between the rule sets and the application.

[0040] Since the fourth aspect is the apparatus corresponding to the second aspect, for the various definitions, implementation methods and technical effects of the fourth aspect, please refer to the description in the second aspect, and this application will not repeat them.

[0041] Fifthly, this application discloses an apparatus for identifying encrypted data streams, the apparatus comprising a processing circuit, an interface circuit, and a storage medium, the interface circuit being used to send and receive data packets with other devices via instructions in the storage medium, and the processing circuit being used to execute instructions in the storage medium to control the interface circuit to implement various methods as described in the first aspect and its implementation.

[0042] Since the fifth aspect is the device corresponding to the first aspect, please refer to the description in the first aspect for various definitions, implementation methods, and technical effects related to the fifth aspect; this application will not repeat them here. In one implementation, the device described in the third aspect operates within the device of the fifth aspect.

[0043] Sixthly, this application discloses another device for identifying encrypted data streams, the device comprising a processing circuit, an interface circuit, and a storage medium, the interface circuit being used to send and receive data packets with other devices via instructions in the storage medium, and the processing circuit being used to execute instructions in the storage medium to control the interface circuit to implement various methods as described in the second aspect and its implementation.

[0044] Since the sixth aspect is the device corresponding to the second aspect, please refer to the description in the second aspect for various definitions, implementation methods, and technical effects related to the sixth aspect; this application will not repeat them here. In one implementation, the device described in the fourth aspect operates within the device of the sixth aspect.

[0045] Another aspect of this application provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the methods described in the above aspects.

[0046] Another aspect of this application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the methods described in the above aspects.

[0047] For details on the definitions, implementation methods, and technical effects of the various terms mentioned above, please refer to the relevant descriptions of the technical effects of the corresponding aspects above, which will not be repeated here. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention.

[0049] Figure 1 This is a schematic diagram of a data stream transmission as described in this application;

[0050] Figure 2 This is a network architecture diagram described in this application;

[0051] Figure 3 This is an architecture diagram of a monitoring device described in this application;

[0052] Figure 4a This is a schematic diagram of a handshake process for a TLS stream as described in this application;

[0053] Figure 4b This is a schematic diagram of yet another handshake process for the TLS stream described in this application;

[0054] Figure 5a This is a schematic diagram of a method for identifying encrypted streams as described in this application;

[0055] Figure 5b This is a schematic diagram of another method for identifying encrypted streams as described in this application;

[0056] Figure 6 This is a schematic diagram illustrating the matching process of a TLS stream handshake message as described in this application;

[0057] Figure 7 This is a schematic diagram of a device for identifying encrypted streams as described in this application;

[0058] Figure 8 This is a schematic diagram of a physical machine that can be used as a device for identifying encrypted streams, as described in this application. Detailed Implementation

[0059] In this article, the character " / " generally indicates that the objects before and after it are in an "or" relationship. For example, A / B can be understood as A or B.

[0060] The terms "first" and "second," etc., used in the specification and claims of this invention are used to distinguish different objects, rather than to describe a specific order of objects.

[0061] In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0062] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0063] In the following description, specific details such as particular system architectures, device architectures, and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known devices, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0064] The technical solutions of the present invention will now be described in detail with reference to the accompanying drawings.

[0065] To better understand the various implementations of the embodiments of the present invention, the technical terms involved in the embodiments of the present invention will be defined / explained below.

[0066] Secure encrypted transport protocols: SSL, TLS, DTLS, or other secure encrypted transport protocols running at the application layer mentioned in the background section. Due to technological evolution, each secure encrypted transport protocol can have multiple versions.

[0067] Handshake packet: A packet used between the client and user to establish a connection via a secure, encrypted transport protocol. Handshake packets appear during the handshake phase of connections such as TLS, SSL, or DTLS. Various versions of TLS, SSL, and DTLS (e.g., TLS 1.0–TLS 1.3, SSL 1.0–SSL 3.0, and DTLS 1.0–DTLS 1.3, etc.) provide descriptions of the handshake phase and handshake packets; see RFC 5246, RFC 6176, RFC 6083, RFC 6347, etc. For example, the following packets are handshake packets: ClientHello, HelloVerifyRequest, ServerHello, Certificate, ServerKeyExchange, ServerHelloDone, ClientKeyExchange, HelloRequest, ChangeCipherSpec, EncryptedHandshakeMessage, NewSessionTicket, Alert, Finished, Certificatetatus, CertificateVerify, CertificateRequest.

[0068] It should be understood that in some cases, due to limitations of the network environment or communication protocols, the information in a single handshake message can be carried by multiple packets or messages. That is, multiple packets or messages can be parsed to form the information included in a complete handshake message. It should be understood that the name of the handshake message or message in this application refers to the value of the Handshake Type field parsed by the corresponding secure encryption transmission protocol, or the content recorded after the colon in the "Handshake Protocol:" line after parsing the handshake message, such as "HandshakeProtocol: Client Hello," where Client Hello is the name of the handshake message; or the value of a field used to represent the same meaning in other versions of encryption protocols. For example, Client Hello, Certificate, and ServerKeyExchange are all names of handshake messages.

[0069] Message (packet): Sometimes also translated as packet, for example, a data packet is also a type of message. In this application, the encrypted data stream between the user and the server includes multiple messages. The identification of the application corresponding to the encrypted data stream in this application is actually achieved by identifying the application corresponding to the handshake message within the encrypted data stream.

[0070] Data stream: A set of bytes in an ordered sequence with a start and an end.

[0071] SSL, TLS, and DTLS are all widely used application-layer secure encryption protocols. TLS is an upgraded version of SSL; both are used to protect data over TCP connections, hence they are sometimes used together in the industry. DTLS, based on TLS, is used to protect data over UDP connections. For example, according to data from some international research institutions, nearly 60% of network traffic is now protected by SSL / TLS encryption. These secure encryption protocols ensure the security of data transmission; by using data encryption technology, they ensure that data is not intercepted or eavesdropped on during transmission over the network.

[0072] Using the aforementioned secure encrypted transmission protocol, data from various applications can be transmitted over a network. The technical solution described in this application is used to identify the application corresponding to the data stream (hereinafter referred to as the encrypted stream or data stream) using the aforementioned secure encrypted transmission protocol.

[0073] The sender and receiver of an encrypted stream can be described using a client / server (C / S) software system architecture. An encrypted stream is transmitted between the client and server of an application. In this architecture, both the client and server refer to applications; specifically, the client application requests services, and the server application provides services to the client application. The client and server of the encrypted stream described in this application are the same type of application. This allows multiple applications to be deployed on the same or different physical devices, thus fully utilizing the advantages of the hardware environment where the client and server reside. In other words, the method for identifying encrypted data streams described in this application does not limit the devices on which the client and server of the data stream are deployed; they can be terminals, servers, or cloud platforms. That is, the data stream can be transmitted between a terminal and a cloud, or between clouds or end-to-end. The SSL / TLS protocol sits between the TCP / IP protocol and various application layer protocols, providing security support for data communication. Application data needs to be processed by the SSL layer before it can be sent out through the TCP / IP layer. In other words, a TCP connection needs to be established before a TLS connection can be established between the client and server. The DTLS protocol sits between the UDP protocol and various application layer protocols. Taking data stream transmission between the terminal and the server as an example... Figure 1This diagram illustrates the transmission of a data stream. The application is deployed at the application layer, and monitoring devices may exist between the terminal and the server. However, because the data is encrypted, the information available for identifying the encrypted data stream is very limited. In some embodiments of this application, information from the handshake message is used to identify an encrypted stream. In one scenario of this application, the user terminal and the server are the same version of the same application; in another scenario, the user terminal and the server are different versions of the same application.

[0074] On the other hand, the method of this application can be applied to monitoring devices in a network, which are used to identify traffic. This application does not limit the specific networking method of the network; for example, the monitoring device can be configured as follows: Figure 2 The setup shown is within the link to be monitored, such as a gateway, or it can be a bypass device on the link to be monitored, such as a bypass analysis device with traffic identification capabilities, or a server running software with traffic identification capabilities, in which case the server can be considered a monitoring device. That is, the data stream to be identified is copied from a node in the link to be monitored and input to the monitoring device. For example, Figure 2 This is an architecture diagram of a network that can use the method described in this application. In this diagram, the terminal used by the internet user connects to the access gateway through the local network. The access gateway sends the user's data to the routers in the backbone network. The backbone network is a high-speed network used to connect multiple regions or areas. Each backbone network has at least one node that interconnects with other backbone networks. For example, different network providers have their own backbone networks to connect their networks located in different regions. Generally, various servers on the internet also connect to the routers in the backbone network through access gateways. Multiple hop routers exist between the access gateways corresponding to internet users and servers, and traffic monitoring devices can be added between these access gateways and routers. These traffic monitoring devices can then analyze the received packets to obtain identification results.

[0075] The following is combined Figure 3 This describes the architecture of a monitoring device in one implementation. The monitoring device is used to execute the method for identifying encrypted streams described in this application; that is, it is used to identify encrypted streams. Its software portion can be considered as the device for identifying encrypted streams described in this application (e.g., Figure 7 One implementation method is as follows: The monitoring device includes an input interface and an output interface. Data flow from the network can flow into the monitoring device through the input interface and then flow out through the input interface. Figure 3In this text, for ease of understanding, the input and output interfaces are separated. However, in actual devices, the input and output interfaces can be the same interface, such as in I / O (input / output) devices. The input and output interfaces can be antennas, interface circuits, etc. This monitoring device can analyze the data stream (actually multiple packets) flowing in from the input interface, or it can copy or mirror the data stream flowing in from the input interface for analysis. The data stream flowing in from the input interface flows out from the output interface. For example... Figure 3 Within the two dashed elliptical boxes, a device schematically represented as multiple switches indicates a unit used to control flow rate, with both switches in the vertical position. Figure 3 The data flow from the input interface is identified by modules such as the packet processor described below; when both switches are in the horizontal position, Figure 3 The data flow entering through the input interface is not analyzed; it flows directly out through the output interface. In this case... Figure 3 The data stream not shown in the image is analyzed through the following modules.

[0076] The following is combined Figure 3 Briefly describe the main steps in analyzing a data stream under one implementation: A packet processor parses the packets in the data stream, using either the TCP / IP or UDP protocol stack to obtain the fields within the packets. It can parse a portion of the packets and a subset of their fields. The parsed packets are then processed by a stream processor to determine which data stream they belong to, i.e., which packets belong to the same stream. This can be determined, for example, by using the five-tuple (source IP address, destination IP address, and port number) in the parsed packets, assigning the same stream identifier to packets with the same five-tuple. The monitoring device can use a hash table to record which parsed packets belong to the same stream. The traffic identification engine analyzes the parsed packets on a stream-by-stream basis, i.e., analyzing one or more packets within a stream. This includes matching these packets with a set of rules used to identify encrypted data streams to determine the matching rule set. Based on this rule set and the mapping between the rule set and applications, the application corresponding to the stream is determined.

[0077] In this context, a rule set is simply a collection of rules, and an identification information database is used to store the multiple rule sets and their mapping relationships to applications. The packet processor, stream processor, traffic identification engine, and identification information database described above are all software modules. This classification is for ease of understanding and from a functional perspective. However, in actual implementation, as long as the method described in this application can be implemented, the division of software modules in the monitoring device is not limited. Of course, in one implementation, the monitoring device can cooperate with other devices to complete the above functions. For example, the packet processor, stream processor, traffic identification engine, and identification information database can be cloud-based or distributed across multiple nodes. In this case, the series of modules implementing the above functions can be considered as a monitoring system to achieve the above monitoring functions, or the entire monitoring system can be considered as a cloud-based or distributed monitoring device. That is, the monitoring device in this paper is not limited to a single physical node. For example, the identification information database storing multiple rule sets and their mapping relationships to applications can be stored on other servers or storage nodes, and the monitoring device may only load a portion or even not store the information in the identification information database. The monitoring device uses the identification information database by communicating with other servers or storage nodes.

[0078] A connection using the TLS protocol is called a TLS connection, and a connection using the DTLS protocol is called a DTLS connection.

[0079] Before transmitting data, each TLS or DTLS connection must establish a connection by transmitting a handshake message between the client and server. Data transmission only occurs between the client and server after the handshake is complete.

[0080] The following example illustrates the handshake process between the client and server when transmitting data using the SSL / TLS protocol. For the handshake process of the DTLS protocol, please refer to the relevant content of standard RFC6347, which will not be elaborated here.

[0081] It should be understood that there are multiple versions of the TLS protocol, and it is applicable to various scenarios. Further details can be found in the relevant sections of different versions of the TLS protocol. For example... Figure 4a This example illustrates an ECDHE-ECDSA scenario in TLS 1.2, where ECDHE-ECDSA represents the TLS handshake process during the initial connection between a server and a client. Figure 4bThe image shows an example of a PSIC scenario in TLS 1.2. PSIC represents the TLS handshake process when a server and a client are not making their first connection (e.g., a connection that is subsequently closed, or a new connection being negotiated without the previous connection being closed). It is clear that the TLS handshake process differs depending on the scenario.

[0082] Figure 4a In this process, the client (user end) and server (server end) exchange handshake messages according to a time sequence. These include:

[0083] The client sends a Client Hello message to the server. Figure 4a and Figure 4b (This is translated as a client greeting message), which can be used to inform the server of the encryption algorithms supported by the client.

[0084] The server replies to the client with a Server Hello message. Figure 4a and Figure 4b Both are translated as server greeting message, Certificate message. Figure 4a Translated into Chinese as certificate message, Server Key Exchange message ( Figure 4a Both are translated as Server Key Exchange Message and Server Hello Done Message. Figure 4a and Figure 4b (All are translated as Server Hello Complete Message). The Server Hello message informs the client of the encryption algorithm used in this handshake; the Certificate message confirms the legitimacy and authenticity of the server, indicating it is not an imposter; the Server Key Exchange message sends the key to the client for key exchange; and the Server Hello Done message informs the client that this set of messages has been sent and the client can send subsequent messages.

[0085] After receiving these messages, the client sends a Client Key Exchange message to the server. Figure 4a and Figure 4b Both are translated as User Key Exchange Message and Change Cipher Spec Message. Figure 4a and Figure 4bBoth are translated as "Change Cipher Spec" messages. After receiving these two messages, the server sends a Change Cipher Spec message and an Encrypted Handshake Message message to the client. Figure 4a and Figure 4b (Both are translated as Encrypted Handshake Message). ClientKey Exchange, Change Cipher Spec, and Change Cipher Spec are used by the client and server to negotiate the key to be used in subsequent data transmission. The Encrypted Handshake Message sent by the server can be used to inform the client that the handshake process is complete.

[0086] Figure 4b The handshake interaction process in the described PSIC scenario is relatively simple. The client sends a ClientHello message to the server, which can be used to inform the server of the encryption algorithms supported by the client.

[0087] The server responds to the client with Server Hello and Server Hello Done messages. The Server Hello message is used to inform the client of the encryption algorithm used in this handshake.

[0088] After receiving these messages, the client sends a Client Key Exchange and Change CipherSpec message to the server. Upon receiving these two messages, the server sends a Change Cipher Spec and an Encrypted Handshake Message to the client. The Client Key Exchange, Change Cipher Spec, and Change Cipher Spec messages are used by the client and server to negotiate the key to be used in subsequent data transmissions. The Encrypted Handshake Message sent by the server informs the client that the handshake process is complete.

[0089] After the client parses the information in these two messages, the handshake process between the client and server ends. Afterwards, the client and server can exchange data, which is transmitted over the network as a TLS stream. From the perspective of command exchange, however... Figure 4a and Figure 4bBoth examples demonstrate one implementation method. That is, the client sends a client request ( Figure 4a and Figure 4b The message (translated as "client request message") requests the server to send data, and the server sends a Server Response. Figure 4a and Figure 4b (Both are translated as server response messages) to send data to the client. Therefore, the handshake message of a TLS stream can indicate the TLS stream.

[0090] Before the TLS handshake message exchange process described above, the client and server need to complete the TCP layer three-way handshake process. Figure 4a and Figure 4b China and Israel TCP START( Figure 4a and Figure 4b This is an illustration of the start of a TCP connection. After the TLS data exchange between the client and server is complete, the TCL connection on which this TLS connection depends can be disconnected. Figure 4a and Figure 4b TCP FINISH (China) Figure 4a and Figure 4b This is a diagram illustrating the end of a TCP connection.

[0091] The messages used to identify encrypted data streams as described in this application come from the handshake phase. It is evident that the handshake messages included in the handshake phase differ in different scenarios. For example, in the TLS 1.2 protocol, most of the handshake messages in the aforementioned handshake phase are in plaintext (i.e., the monitoring device can parse most of the field information after receiving the handshake message), while in the TLS 1.3 protocol, the plaintext messages in the handshake phase are only the Client Hello message and the Server Hello message.

[0092] For example, in existing technologies, the domain name sending the TLS stream is determined by the SNI or Common Name field in the handshake message of a TLS stream, thereby identifying the application corresponding to the TLS stream. This is clearly because these fields carry the domain name (Host name), and since the domain name carries information about the server sending the data in the stream, a portion of the domain name can indicate the application. In other words, the fields carry application-related information, and existing technologies indicate the application by parsing the meaning of the field values ​​(domain names).

[0093] However, existing technologies can only identify a limited number of data streams, and the accuracy of identification is low, for the following reasons.

[0094] 1. The domain name information in the above fields is often incorrect. For example, fraudulent, VPN, and virus applications may deliberately set incorrect SNIs to evade application identification.

[0095] 2. Increasingly, domain names no longer carry information about the application corresponding to the encrypted stream, or the domain name is insufficient to identify the application. For example, the domain name in the Common Name is not an exact domain name; in most cases, it is a domain with a wildcard "*". This ensures that a certificate can support a series of domain names simultaneously. However, it also makes application identification more difficult.

[0096] 3. Much of the transmitted data (such as images and videos) is stored on cloud platforms (such as Amazon S3). In this case, the domain name can only indicate information about the cloud platform, such as a248e.akamai.net or s3.amazon.com. Different data is distinguished by the specific path following the domain name. For example, applications A and B both deploy their image content on s3.amazon.com, but when users access applications A and B using HTTPS, the traffic generated uses the SNI of s3.amazon.com. In this case, this technology cannot distinguish between applications A and B.

[0097] 4. Furthermore, the SNI or Common Name field may not be included in the message or may not be parsed, making it impossible to identify the application corresponding to the encrypted data stream by domain name. The SNI field is an optional field in the ClientHello message. For example, in scenarios using secure encrypted transport protocols to transmit FTP, email, etc., the SNI field is generally not set. Even in HTTPS scenarios using secure encrypted transport protocols to transmit HTTP content, SNI is often absent. For example, IE6, as the browser built into Windows XP, does not have an SNI field in its HTTPS communication traffic. As... Figure 4a and Figure 4b As shown, the Certificate message does not appear in every TLS stream. TLS has a handshake reuse mechanism; in the case of handshake reuse, the certificate does not need to be transmitted, and the information from the previous handshake is used directly. Furthermore, the next-generation TLS standard, TLS 1.3, will adjust the encryption mechanism of the TLS protocol, where all messages after the ClientHello and ServerHello messages will be transmitted encrypted. The Certificate handshake message is now transmitted encrypted, and gateways or monitoring devices on network devices cannot parse the domain name carried in the Common Name field of this handshake message.

[0098] The aforementioned shortcomings stem from existing technologies relying solely on the SNI or Common Name fields for encrypted stream identification. However, these fields can be omitted, incorrectly set, or set to ambiguous values, leading to either failure to identify or incorrect identification. This application, however, utilizes other fields to identify encrypted streams, independent of the SNI or Common Name fields. These other fields do not carry information indicating the application. Therefore, this application's solution does not depend on the meaning of field values ​​but rather on a set of matching rules and the mapping relationship between these rules and applications. This allows for the identification of more applications corresponding to data streams, significantly improving accuracy.

[0099] In fact, in this application, the handshake message used to identify the encrypted data stream can be at least one of the following messages: ClientHello, HelloVerifyRequest, ServerHello, Certificate, ServerKeyExchange, ServerHelloDone, ClientKeyExchange, HelloRequest, ChangeCipherSpec, EncryptedHandshakeMessage, NewSessionTicket, Alert, Finished, Certificatetatus, CertificateVerify, and CertificateRequest. That is to say, the various rules described below for identifying the encrypted data stream identify the encrypted stream corresponding to the handshake message by detecting whether fields in one or more of the above handshake messages match.

[0100] Secure encrypted transmission protocols are TLV (Type, Length, Value) format protocols. Therefore, after parsing, the fields in a handshake message generally include three parts: Type, Length, and Value. Each of these three parts, or any combination thereof, can be considered a characteristic of that field.

[0101] The type can have different levels, such as for extended fields.

[0102] In the `extension:ec_point_formats(len=2)` directive, `extension` is the type, and `ec_point_formats` is also the type. Typically, the type of a field is used to represent another field; for example, `extension:ec_point_formats(len=2)` is called an extension field, or more specifically, the `ec_point_formats` field within an extension field. Of course, in actual packets, some parts of a field may be omitted. Network devices can parse the type, length, and value of unencrypted fields in received packets. For example, in a Client Hello packet, 11 represents the type of an extension field; by looking up a table, 11 represents `ec_point_formats`. When matching a field rule, if the rule includes restrictions on these three parts, and the matched packet omits one or more parts of that field, the omits can be assumed to satisfy the corresponding rule. These omits could be due to a field's definition in the protocol being missing a part, or certain parts being fixed values ​​in the protocol and therefore not carried in the packet. For example, in a Certificate message, the id-at-countryName=US field only contains the type (id-at-countryName) and the value (US), while the algorithm(rsaEncryption) field only carries the field value (rsaEncryption) because the TLS protocol has already specified that this type of field appears in a fixed position and the field length is fixed.

[0103] On the other hand, the fields in the handshake message can be nested, as reflected in the nested use of TLV format. For example, an Extension field in the Client Hello message has a Type of...

[0104] ec_point_formats(11), where 11 is the number used to represent the type of this field in the unparsed message, ec_point_formats is the specific type obtained from parsing, the length of this field is 2, and its value is EC pointformats Length:1 and Elliptic curves point formats(1), where Elliptic curves pointformats(1) is the type in the TLV format of the next level field, and EC point formats Length:1 is the length in the TLV format of the next level field. The above information can be recorded in the parsed field in the following way:

[0105] ◢Extension:ec_point_formats(len=2)

[0106] Type: ec_point_formats(11)

[0107] Length: 2

[0108] EC point formats Length: 1

[0109] Elliptic curves point formats(1)

[0110] Where ◢ indicates expansion of the content of the extension field. This indicates that there is more content under the Elliptic curves pointformats(1) field that has not been expanded.

[0111] Figure 5a and Figure 5b Each of the following methods describes a way to identify encrypted data streams. Figure 5a The method identifies the encrypted data stream through multiple handshake messages within the encrypted data stream. Figure 5b The method identifies an encrypted data stream through a handshake message of an encrypted data stream. Figure 5a Including steps S501a, S502a and S503a; Figure 5b This includes steps S501b, S502b, and S503b. It should be understood that the sequence numbers and connections between steps in the figures do not limit the execution order of the steps. Furthermore, this application does not limit... Figure 5a Methods and Figure 5b The specific implementation method.

[0112] For example Figure 5a In some implementations, S501a and S502a are executed alternately. This alternate execution can be implemented in various ways, which are only illustrated in this application and not listed individually. In one implementation, a message can be parsed, and multiple matching rule sets can be determined from the multiple rule sets based on its fields. Then, another message can be parsed, and multiple matching rule sets can be determined from the multiple rule sets again. The intersection is then calculated. If the intersection is not a unique rule set, the message parsing and matching continue until a rule set matching all the messages is obtained.

[0113] In one implementation, a message can be parsed, and multiple matching rule sets can be determined from a set of rules based on its fields. Then, another message can be parsed, and the same rule sets matching the two messages can be determined from the set of matching rules. If the set of matching rules is not unique, the process continues, parsing messages and matching them against the set of matching rules until a set of rules matching all the messages is obtained. Alternatively, in one of the matching steps, if a rule in the matched rule set is not satisfied by other messages in the same flow, the matched rule set is discarded, and the process restarts matching from the set of rules or from the previously matched rule set.

[0114] In another implementation, a subset of fields in the multiple messages can be parsed, and these fields can be matched against the multiple rule sets to obtain a rule set that matches these fields. Then, another subset of fields in the multiple messages can be parsed, and these other fields can be matched against the obtained rule set.

[0115] Another implementation approach involves first parsing the attributes of the fields in the message. Based on the attributes and order of multiple fields, it matches the sequence rules in the rule set (sequence rules may or may not exist). Then, it parses other features of the fields in the message to match the field rules. Of course, this implementation approach is more suitable for situations where multiple rule sets include sequence rules.

[0116] For example Figure 5a S502a and S503a in the above may first match a message to obtain the rule set corresponding to the message, and the rule subset that the message satisfies in the rule set corresponding to the message. Based on the matching relationship between the satisfied rule subset and the application, the application set corresponding to the message is obtained. Then, other messages of the encrypted data stream are matched, and the application set is continuously narrowed or several application sets are obtained by intersection (the specifics are not limited), and finally the application matched by the encrypted data stream is obtained.

[0117] For example Figure 5a In the S501a, S502a, and S503a steps, for multiple messages, the steps can be performed in parallel. That is, while S501a is being executed for some messages, S502a is being executed for another message, and S503a is being executed for yet another message. In one implementation, one could parse one of the multiple messages in the encrypted data stream, determine at least one rule set that the message matches from multiple rule sets based on the fields in that message, and obtain the application set corresponding to each subset of rules satisfied by the message in the at least one matching rule set.

[0118] Similarly, for example Figure 5bIn some implementations, S501b and S502b are executed alternately. This alternate execution can be implemented in various ways, which are only illustrated in this application and not listed individually.

[0119] For example, in one implementation, each time a field is parsed, it is matched with field rules in multiple rule sets, and then the corresponding field is parsed based on the fields involved in the other rules in the rule set that matched the parsed field.

[0120] For example, in one implementation, a handshake message can be parsed in segments. For each segment parsed, several fields are matched with rules in multiple rule sets to obtain at least one rule set that matches those fields. Then, the fields of other segments are parsed and matched to obtain the rule set that matches the message.

[0121] about Figure 5a and Figure 5b The method described in the invention content section of this application also includes some implementation methods, and the following description uses TLS stream as an example. Those skilled in the art can learn from these implementation methods the corresponding implementation methods for identifying encrypted streams of other secure encrypted transmission protocols, so this application will not repeat them here.

[0122] The various implementation methods described above can be used in different scenarios, such as identifying which application an encrypted stream in the network corresponds to, or identifying an encrypted stream corresponding to a certain application from a data stream transmitted from the network. In different scenarios, the implementation method with faster matching speed may vary.

[0123] The following describes the terms and specific implementation methods used in the illustrations and the above implementation methods. It should be understood that this application describes multiple implementation methods for processing a single handshake message, which can be used for identifying multiple handshake messages or for identifying a single handshake message; these will not be described separately here.

[0124] Those skilled in the art will understand that, in this application, using a secure encrypted transmission protocol to parse the fields of a message can not only reveal the characteristics of these fields, but also their order.

[0125] It should be noted that the description above of determining a set of rules for matching one or more messages from multiple rule sets based on multiple fields means that the matched message is matched against a set of rules, and the matched set of rules is the same as one of the multiple rule sets. In other words, this matching not only constrains the message to match several rules, but also constrains the combination of these rules to be one of the multiple rule sets. For example, if the multiple messages match rules numbered 1, 3, 5, 17, 18, 23, and 37, but the multiple rule sets only contain the rule set {1, 5, 18, 23, 37}, then the matched rule set is {1, 5, 18, 23, 37}, not {1, 3, 5, 17, 18, 23, 37}.

[0126] A field rule is used to indicate the characteristics of a field. A field's characteristics can be at least one of the following: field type, length, and values. For example, it can be a field's length, a field's type and length, or a field's type, length, and values.

[0127] Handshake messages contain various lengths. In the rules described in this application, length is treated as an integer; matching the length of a field means matching that length. For example, after parsing the length of the field to be matched, it can be compared with the rule used to match that field's length. If the comparison result is "true," the rule is matched; otherwise, it is not.

[0128] The rules for matching the length of a field can be expressed using terms like equal to, not equal to (e.g., not equal to, greater than or equal to, less than), or a range (e.g., represented by an interval or set). For example, a rule could be that the length of a field in a handshake message is equal to a certain value; or that the length of a field in a handshake message is greater than or less than a certain value; or that the length of a field in a handshake message falls within a certain range.

[0129] In the rules described in this application, the value of a field is treated as a string. Matching the value of a field can be either matching the entire string or matching a substring.

[0130] A complete string match means that all values ​​of a field must match the values ​​defined in the rule to be considered a match. Complete string matches can take many forms. For nested TLV structures, one implementation only matches values ​​at the same level as the field's length and attributes. Taking the Extension in the Client Hello message mentioned above as an example, a complete string match matches the string composed of "EC point formats Length:1" and "Elliptic curves point formats(1)". This string does not include the expanded content of "Elliptic curves point formats(1)," such as "EC point formats Length:1Elliptic curves point formats(1)". In other words, it does not include the content of other lines after expanding "Elliptic curves point formats(1)" as the string to be matched. Another implementation, for nested TLV structures, requires not only matching values ​​at the same level as the field's length and attributes, but also matching any nested, unexpanded fields within that value. Taking the Extension in the Client Hello message mentioned above as an example, the whole string match is the string composed of EC point formats Length:1 and Elliptic curves point formats(1), as well as the expanded content of Elliptic curves point formats(1).

[0131] In a string, any sequence of consecutive characters is called a substring. Substring matching means that the value of a field only needs to include a substring that matches the value indicated in the field rules. For example, if the field value has multiple lines, it can be one or several lines that match the rules, a substring within a line, or a substring consisting of characters spanning multiple lines; this application does not impose any restrictions.

[0132] This application also describes a rule called a sequence rule. A sequence rule is used to indicate the order of multiple fields in a handshake message. In a sequence rule, a field is indicated by its attribute (Type), regardless of its length (Length) and value (Value). That is, to match a sequence rule, the message indicated by the sequence rule must include a set of fields whose attributes and the order in which these attributes are arranged in the message are the same as described in the sequence rule; otherwise, the message does not match (or is considered to have missed) the sequence rule.

[0133] In one implementation, these multiple fields are often a group of fields with certain similarities within a message, such as multiple extension fields in a message. For example, the six extension fields in a Client Hello message can be considered a group of fields; the multiple cipher suits fields in a Client Hello message can be considered a group of fields, storing the names of various encryption algorithms; the multiple extension fields in a Server Hello message can be considered a group of fields. For example, five extension fields in a Certificate message containing eight extension fields (extension:8items) can be considered a group of fields. In other words, the order rule can specify the order of a subset of fields among multiple fields with certain similarities within a message. Furthermore, multiple RDNSequence item fields of the subject type in a Certificate message can also be considered a group of fields, as can multiple RDNSequence item fields of the issuer type in a Certificate message, and so on.

[0134] It should be noted that order rules can take many forms, and this application does not limit them. Any rule that specifies the order of multiple fields in a message is an order rule. Below are examples of several types of order rules:

[0135] 1. For Client Hello messages with 6 extension fields, specify these 6 extensions.

[0136] The order of the (Extension) fields. That is, the order of all fields in a message that have a certain number of fields.

[0137] 2. For a Certificate message with 8 extension fields (extension:8items), specify the data type of the 1st, 3rd, 4th, 5th, and 7th fields among the 8 extension fields (extension:8items) in the Certificate message. That is, specify the data type of certain fields in a specific position within a certain type of field in a message.

[0138] 3. For a Certificate message with 8 extension fields (extension:8items), specify the type of 5 consecutive fields among the 8 extension fields (extension:8items), without specifying which 5 extension fields are among the 8 extension fields. In other words, specify the type of a portion of fields with fixed relative positions within a certain type of field in a message.

[0139] 4. Specify the order of multiple RDNSequence item fields under the subject type in the Certificate message, as well as multiple RDNSequence item fields under the issuer type in the Certificate message.

[0140] 5. Specify the types of multiple cipher suits fields and multiple extension fields in the ClientHello message.

[0141] It should be noted that the above rules may include wildcard rules. For a message with a specific name corresponding to a wildcard rule, all valid messages with that name will satisfy the rule. On the other hand, during the matching of a field rule, if the matched message contains a field whose corresponding field lacks one or more parts of its type, length, or value, and the field rule includes instructions for the missing parts, then the missing parts can be assumed to satisfy the corresponding rule. These default cases could be due to a field being missing a part in the protocol definition, or certain parts being fixed values ​​in the protocol and therefore not carried in the message.

[0142] In one implementation, each of the multiple rule sets corresponds to an application, and an application can correspond to multiple rule sets. This application does not restrict how the mapping relationship between rule sets and applications is represented; various data structures can be used to maintain it, such as tables. Furthermore, the mapping relationship between rule sets and applications can be represented as a mapping between a rule set and an application, or as a mapping between several subsets of a rule set and multiple application sets. Subsets can be divided according to message names, rule types, etc., and this application does not impose any restrictions.

[0143] The rules mentioned above, and the mapping relationship between rule sets and applications, can be pre-configured in the monitoring device as a library (such as the matching information library mentioned above), obtained through machine learning. In one implementation, new rule sets and application mapping relationships can also be input into the monitoring device periodically or irregularly from other interfaces, or the monitoring device can periodically or irregularly sample data for machine learning to update the library.

[0144] In fact, a set of rules corresponding to a specific application can be obtained through machine learning. For ease of description, this application will be referred to as the target application. This machine learning process can be offline. The following is a simplified description of the process of obtaining the set of rules corresponding to the target application through machine learning algorithms. This process uses known handshake messages of the target application (i.e., messages transmitted between the client and server during the handshake phase), as well as messages from other applications, as samples for learning. The scheme described in this application uses a client / server model, meaning that the application corresponding to a sample should be unique. It should be understood that samples can be obtained in various ways, and this application does not impose any restrictions. For example, in a test environment, a data stream is transmitted between the client and server of the target application, including establishing a handshake using the target application. The packets in this data stream are then copied or intercepted for analysis. Of course, messages from other applications and the handshake messages of the target application are also needed as samples to be input into the machine learning platform for training, such as the positive and negative samples mentioned below. In one implementation, to enable the learned rule set to more accurately and comprehensively identify the messages of the target application, the messages transmitted in the test environment can be made closer to the conditions under which the actual messages to be identified are generated. For example, messages can be generated using multiple possible versions of the target application in the actual network; the target application can be run on the operating system that the application may run in the actual network to collect samples; samples can be sent or received using the local area network of the country or region where the target application may be located in the actual network, etc.

[0145] For example, in a real-world networking environment, multiple packets can be intercepted or copied. Based on experience, the handshake packets of the target application can be manually identified and marked, while packets from other applications can be marked differently for processing by the machine learning platform. A convenient approach is to copy the sent and received packets at the terminal. The samples used to train the rules need to reach a certain number, typically hundreds or thousands of packets from various flows. For example, in one implementation, the samples used to train the rules might consist of 1000 packets from the target application's flows and 1200 packets from other application flows. In another implementation, to improve the efficiency of machine learning, the obtained samples can be manually filtered based on experience.

[0146] The above samples are parsed using a secure encrypted transmission protocol to obtain the length, attributes, and values ​​of the fields in the message, as well as the order of grouped fields in the handshake message. This parsed information (i.e., the length, attributes, and values ​​of the fields in the handshake message, and the order of grouped fields in the handshake message) is processed using a machine learning platform to train one or more rule sets corresponding to the target application. Each rule set includes one or more rules; a data stream corresponds to the target application only if the handshake messages in the data stream satisfy all the rules in that rule set. Furthermore, the rule sets for the target application are related by an OR relationship; that is, if the handshake messages in a data stream satisfy any one of the rule sets, the data stream corresponds to the application. This application does not limit the machine learning algorithms used; for example, decision tree-based machine learning algorithms can be used, specifically random forest algorithms, C4.5 algorithms, and their variants. Using decision tree-based machine learning algorithms outputs one or more tree-like models. In one implementation, a tree-like model is used, where each node represents a set of applications. The root node represents the set of applications that a sample without rule matching might correspond to. This set of applications can be pre-defined or set based on the samples input for the machine learning process. A connection between two nodes in the tree model represents one or more rules. That is, from the root node to a leaf node, two connected nodes can be interpreted as: a subset of applications selected from the application set corresponding to a node, based on the rules corresponding to that connection. This subset of applications can be stored at the node at the other end of the connection. Reaching the leaf node, only one application remains. An identifier can be set on the leaf node to indicate the leaf node corresponding to the target application. Thus, each leaf node corresponds to a set of rules, including the rules represented by each connection from that leaf node to the root node. The set of rules corresponding to all nodes traversed from a leaf node representing the target application to the root node constitutes a set of rules corresponding to that target application. Therefore, if a tree model includes multiple leaf nodes representing the target application, the tree model can obtain multiple sets of rules corresponding to that target application.

[0147] Of course, machine learning can be performed using multiple sets of samples, including handshake messages from different flows of the target application. The resulting rule sets may differ. These different rule sets can all be used as the rule set for identifying the application, making the determined rule set more comprehensive and improving the recognition rate when identifying messages in the network.

[0148] In one implementation, the information parsed from multiple messages can be represented in matrix form for training machine learning algorithms. For example, the rows of the matrix can describe one or more handshake messages in a flow, and the columns can describe the length, type, value, or order of a field in a message. The sample includes handshake messages from the target application (e.g., application A) and messages from other applications (e.g., application B, application E). Handshake messages from application A are marked as positive samples, and messages from other applications are marked as negative samples. Correspondingly, a dedicated column in the matrix records whether each row's message is a positive or negative sample.

[0149] On the other hand, when adding multiple rule sets obtained through machine learning methods and their mapping relationships to the library, some filtering and management can be performed to make the identification of data streams more accurate and reduce the need for subsequent re-identification and re-judgment, thus making the above method more practical. For example, if a learned rule set and its mapping relationship overlap with an existing rule set in the library, it can be omitted. Similarly, if the same rule set learns different applications in different learning iterations, it can be omitted. Furthermore, if a learned rule set is a subset of an existing rule set in the library, it can be omitted. Finally, if a learned rule set includes a subset identical to an existing rule set in the library, it can replace the corresponding rule set in the original library with the learned rule set and its mapping relationship. This application does not restrict how to filter and manage these rules.

[0150] In one implementation, each rule set in the maintained library corresponds to only one application, each rule set is unique, and there is no mutual inclusion relationship between the rule sets. Such a library can obviously more accurately indicate the application corresponding to the data flow through the rule sets.

[0151] The aforementioned machine learning methods are not limited to processing packets from any particular application. Since samples can be obtained through various methods, intercepted or copied handshake packets in the network can be matched using the rules in the defined rule set. This can significantly improve the recognition rate and accuracy of encrypted data streams from applications that existing technologies cannot identify or have low accuracy in identifying. For example, dozens of applications currently use TLS tunneling technology called Meek to obfuscate traffic and evade firewall detection. These applications carry arbitrary SNI values ​​in the packets transmitted in the network. CDN (Content Delivery Network) servers deploy data from various applications, and the domain name of the CDN server (usually carried in the Common Name field) cannot indicate the application. On the other hand, because CDN servers deploy data from various applications, when a CDN server receives a handshake packet from a client, even if the SNI in the packet is incorrect or inaccurate, the CDN server may still proceed with establishing a connection and sending data. Therefore, methods that determine the application corresponding to a packet by parsing the information carried in the SNI or Common Name cannot identify or accurately identify packets from such applications transmitted in the network. Among these applications, common ones such as Tor and Psiphon are used as VPN tools to evade internet censorship.

[0152] The following describes a method for identifying encrypted data streams, using the TLS protocol as an example, along with related technical details. Here, "TLS stream" may be used to refer to an encrypted data stream. In conjunction with the technical details, methods, and apparatus described below, the methods and implementations for identifying encrypted streams using other secure encryption protocols, such as SSL or DTLS, are similar and can be readily understood by those skilled in the art. For space constraints, this application will not describe in detail the identification of encrypted streams using other secure encryption protocols.

[0153] Taking Psiphon as an example, machine learning is performed on the handshake messages sent by a version of Psiphon using the method described above to extract the rule set corresponding to the application. In this example, the Psiphon application transmits data via TLS streams. The extracted rule set includes the following eight rules. That is, if a TLS stream handshake message satisfies the following eight rules, then this TLS stream is a Psiphon data stream. It should be understood that the various rule expressions described below (such as format, symbols used to connect attributes between different layers, symbols used to connect values ​​of different fields, etc.) are merely examples for ease of understanding, and this application does not limit the expression of the rules. Among them, for the Client Hello message, there are the following three rules:

[0154] (1) ClientHello_len = 185

[0155] This rule indicates that the value of the field of type len in the Client Hello message is 185. Here, the field of type len represents the length of the Client Hello message, so this rule means that the length of the Client Hello message is 185 bytes. (2)

[0157] ClientHello_cipher_suits=0xc02b_0xc00a_0xc009_0xc023_0xc007_0xc02f_0xc014_0xc013_0xc011_0 xc005_0xc004_0xc003_0x0039_0x0033_0x003d_0x0035_0x003c_0x002f_0x000a_0x0005_0x0004_0x00ff

[0158] This rule states that the value of a field of type cipher_suits in a Client Hello message is equal to the string on the right side of the equals sign, i.e., equal to...

[0159] 0xc02b_0xc00a_0xc009_0xc023_0xc007_0xc02_0xc014_0xc013_0xc011_0xc005_0xc004_ 0xc003_0x0039_0x0033_0x003d_0x0035_0x003c_0x002f_0x000a_0x0005_0x0004_0x00ff

[0160] (3), ClientHello_extension_0x0023=0

[0161] This rule indicates that the value of the extension field of type 0x0023 in the Client Hello message is 0. Here, the message type is indicated by information carried in the unparsed message, rather than being converted to the parsed type indicated by 0x0023.

[0162] For ServerHello messages, there is a rule:

[0163] (4) ServerHello_len = 80

[0164] This rule indicates that the length of the Server Hello message is 80.

[0165] For Certificate messages, there are the following three rules:

[0166] (5)、Certificate_subject_CN=ssl334326.cloudflaressl.com

[0167] This rule is similar to the traditional scheme, where CN is an abbreviation for the Common Name field in the Certificate message, indicating that the Common Name field in the Certificate message has a specific value, namely ssl334326.cloudflaressl.com.

[0168] (6), Certificate_subject_publickey_algorithm=RSA

[0169] This rule states that the value of the publickey algorithm field under the subject type in the Certificate message is RSA.

[0170] In fact, the publickey algorithm in this condition is similar to the version number field; it's a fixed field with a fixed type and length, and it's the default in the handshake message. Therefore, this rule applies to the default field described above. (7)

[0172] Certificate_extension_order=authKeyIdExt_subjKeyIdExt_basicConstraintsExt_key UsageExt_crlDistPointsExt_certPoliciesExt

[0173] This rule represents a sequence rule, meaning the Certificate message includes six extension fields that appear in the order specified in the rule: authKeyIdExt, subjKeyIdExt, basicConstraintsExt, keyUsageExt, crlDistPointsExt, and certPoliciesExt.

[0174] For the ServerHelloDone message, there is a rule:

[0175] (8)、ServerHelloDone_ver=0x0303

[0176] This rule indicates that the version (ver) field in the ServerHelloDone message must have a value of 0x0303. This field is a mandatory field by default in the protocol, with a fixed type and a fixed length of 2 bytes. This also explains why this condition only matches the version and value of this field.

[0177] The aforementioned eight rules are contained within a rule set. This means that a TLS stream is considered to correspond to or match a Psiphon application only if all eight rules are satisfied in the handshake message. Existing solutions parse the SNI or Common Name field from the handshake message to obtain the TLS stream's domain name. However, since Psiphon's SNI varies randomly, it's impossible to determine whether a TLS stream matches a Psiphon application simply by parsing the SNI content. On the other hand, this solution also uses the Common Name as one of the conditions, but since Psiphon uses a general CDN as its server, and its certificates are not Psiphon-specific, relying solely on the Common Name in existing solutions could easily misidentify a TLS stream corresponding to Psiphon as belonging to another application, resulting in low accuracy. In this application's solution, the Common Name content is included in a rule set along with the other seven rules. Only a TLS stream that satisfies all eight rules is considered a match for Psiphon, thus enabling accurate identification of streams matching Psiphon applications.

[0178] By analyzing the handshake messages of multiple TLS streams from Psiphon applications using the machine learning method described in this application, multiple rule sets corresponding to Psiphon applications can be obtained. For example, using the aforementioned rule set containing eight rules, the effectiveness of using this rule set to identify streams corresponding to Psiphon applications in the network can be tested under various different network environments. That is, using Psiphon applications to transmit TLS streams under various network environments, and using this rule set to match the TLS streams transmitted by Psiphon applications in these network environments. For example, the network environments include: using Psiphon applications running on five different operating systems (Windows 7, Windows 10, Android 4.4.2, Android 5.1.1, and Android 6.0.0) to transmit TLS streams; using multiple Psiphon client versions (65, 91, 103, 108, 112, 113, 114, 123, 125, 128, 130, and 133); and using six different network access points to transmit TLS streams. Under the aforementioned network environments, the accuracy of identifying Psiphon application flows is 100%, and the recall rate is 99.19%. Accuracy represents the proportion of identified Psiphon application flows that are indeed Psiphon application flows, i.e., whether the identification is correct; recall represents the proportion of identified Psiphon application flows in the network relative to the Psiphon application flows in the intercepted or copied data streams during the identification process, i.e., how many outflows containing the target application were identified. It is evident that the determined rule set reflects the mapping relationship between the combination of fields in the TLS handshake message (values, lengths, attributes, order of multiple fields, etc.) and Psiphon applications. Therefore, compared to existing solutions, using the rule set described in this application to identify TLS flows can significantly improve the identification rate and accuracy of TLS flows, achieving the goal of accurate identification.

[0179] Of course, as mentioned above and Figure 5b In some cases, matching only one handshake message is sufficient to identify whether the encrypted stream of that handshake message corresponds to a specific application. For example, matching only the ClientHello message, or only the ServerHello message, or only the Certificate message. In such cases, all rules in a rule set for a particular application originate from the same message. Below is a description of a rule set for an application where all rules originate from ClientHello messages. That is, for a data stream, if its ClientHello message matches the rule set below, the data stream can be identified as corresponding to that application. This rule set includes the following rules:

[0180] ClientHello_ver = 0x0303

[0181] ClientHello_CipherSuitesLength=32

[0182] ClientHello_CipherSuites=cca8cca9c02fc030c02bc02cc013c009c014c00a009c009d002f0035c012000a

[0183] ClientHello_CompressionMethodsLength=1

[0184] ClientHello_CompressionMethods=0

[0185] ClientHello_ExtensionsNumbers=7

[0186] The above six rules are the field rules in this rule set. The meaning of each rule will not be described in detail here. You can refer to the definition of each field of the ClientHello message in the TLS protocol and the explanation of the rules in the Psiphon application mentioned above for understanding. The following rule is the order rule, which specifies the order of a set of extension fields (Extensions) in the ClientHello message, where each extension field is represented by its type (Type).

[0187] Extensions_order=0x0005_0x000a_0x000b_0x0023_0x000d_0xff01_0x0012

[0188] For example, if a ClientHello message includes the following set of Extensions fields, then the ClientHello message matches this ordering rule.

[0189] Extensions Items(Type,Length,Value):0x0005 5 0100000000

[0190] Extensions Items(Type,Length,Value):0x000a 10 0008001d001700180019

[0191] Extensions Items(Type,Length,Value):0x000b 2 0100

[0192] Extensions Items(Type,Length,Value):0x0023

[0193] Extensions Items(Type,Length,Value):0x000d 14

[0194] 000c040104030501050302010203

[0195] Extensions Items(Type,Length,Value):0xff01 1 00

[0196] Extensions Items(Type,Length,Value):0x0012 0

[0197] On the other hand, a rule set within an application can also contain only one type of rule; that is, the rule set consists entirely of sequential rules or entirely of field rules. In other words, certain flows can be identified solely by either sequential rules or field rules. In the implementation where the rule set includes only field rules, there are multiple field rules, and at least one field rule includes a length constraint. The following describes an example where the rule set contains only one rule, which is a sequential rule. The rule set is as follows:

[0198] Certificate_subject_order=CountryName_StateOrProvinceName_LocalityName_OrganizationName_OrganizationUnitName_CommonName

[0199] If a flow matches this rule, it means that the order of the subject field in the Certificate message within that flow is CountryName_StateOrProvinceName_LocalityName_OrganizationName_OrganizationUnitName_CommonName.

[0200] For example, a Certificate message that includes the following set of subject fields satisfies the above set of rules.

[0201] subject:rdnSequence(0)

[0202] When malware or rogue software generates certificates in bulk, the Common Name value in the Certificate message can be tampered with or forged. With current technology, identifying software by recognizing the domain name (such as "microsoft" in the example above) in the Common Name value is prone to error. However, using a rule set similar to the example above, where the rules restrict the order of the subject field in the Certificate message, the order remains unchanged regardless of how these fields are modified, thus allowing the correct application to be identified.

[0203] The following uses a TLS handshake message as an example to describe the process by which a monitoring device identifies encrypted data streams. All encrypted streams received by the monitoring device use the TLS protocol. It should be understood that encrypted streams using other secure encryption protocols can also be implemented in a similar manner, which will not be elaborated upon in this application.

[0204] Since a single field rule or sequence rule applies to a single handshake message, in one implementation, once the monitoring device determines that it has intercepted a handshake message (e.g., by parsing the handshake message), it matches that handshake message without waiting for other handshake messages in the data stream. As described in the previous example of the handshake phase, the transmission of handshake messages is sequential. Therefore, after parsing the received handshake message, it can be matched against a set of rules in a database. If, based on the mapping relationship between the rule set in the database and applications, the rule set matched by the handshake message uniquely maps to an application, then the stream containing the handshake message corresponds to that application, the matching process ends, and subsequent handshake messages from that stream are no longer matched. Instead, all messages from that stream received by the monitoring device from now on are considered to correspond to that application. This allows for faster stream identification and saves monitoring device resources. In another implementation, multiple intercepted handshake messages can be parsed before matching. The matching process can be referenced in the following explanation of matching a single handshake message, such as matching multiple handshake messages according to rule types. This simplifies the matching process and enables batch processing, eliminating the need to perform a matching process for each handshake message separately.

[0205] Since the ClientHello message is the first handshake message transmitted during the TLS handshake process and is guaranteed to exist during the TLS handshake, in one implementation, it can be determined by parsing whether a ClientHello message for a stream has been received. This allows us to determine whether a TLS stream message has been received. If it is a TLS stream (i.e., the ClientHello message for that stream has been obtained), then rule matching begins from the ClientHello message. This obviously allows for faster identification of some applications corresponding to streams, as these applications have rule sets that contain rules specifically for matching ClientHello messages. If no ClientHello message for a stream has been obtained, then the stream is not a TLS stream, and it can be directly determined that it cannot be identified using the rule set, i.e., the match fails. In fact, in the technical solution described in this application, the rule sets corresponding to many applications include a subset of rules corresponding to the fields in the ClientHello message. Of course, the rule sets corresponding to some applications do not include a subset of rules corresponding to the fields in the ClientHello message, which means that for these applications, the matching of the ClientHello message can be skipped.

[0206] Specifically, after receiving a handshake message (referred to as an A message, such as a ClientHello message), it can be parsed to obtain information such as field lengths, attributes, values, and the order of multiple fields. Then, rules from a database are used to match this information. This can involve parsing all fields of the handshake message or only some fields. In one implementation, after parsing the name of the handshake message, i.e., determining what type of handshake message it is, only the fields related to the rules for that type of handshake message stored in the database can be parsed. That is, the fields corresponding to multiple rules for that type of handshake message can be obtained from the stored rule set and the application mapping. Parsing the fields in the message that correspond to the rules allows for targeted selection of the parts of the message to be parsed. This reduces the burden on monitoring equipment and improves message processing efficiency in real-world network environments with multiple streams transmitting in parallel. Similarly, when multiple handshake messages are received and parsed together, after determining the names of the multiple handshake messages, the messages corresponding to the rules in these rule sets, as well as the fields in the messages, can be selectively parsed based on the rule sets stored in the database, or based on the rule sets corresponding to the applications to be identified as indicated by the user or business requirements.

[0207] In one implementation, matching can be performed rule by rule. For each rule match, the corresponding application subset is queried. After all rule matching for the packet is complete, the intersection (∩) of the resulting application subsets is obtained, yielding the application set matched by packet A. If this application set contains only one element, then that element is the application corresponding to the TLS stream containing packet A, and the identification process ends. If the application set includes multiple applications, another packet (referred to as packet B, such as a ServerHello packet) in the same data stream as packet A can be matched using the above method or other methods described below. Alternatively, a Certificate message can be used. For example, only the rule set corresponding to the above application set is matched. The rule related to message A can be matched with the rule set of message A, thereby further narrowing down the above application set. If there is only one element in the narrowed application set, then that element is the application corresponding to the TLS stream where message A and message B are located, and the identification process ends. If the narrowed application set still includes multiple applications, then a similar matching process can be performed on other handshake messages in the TLS stream where message A and message B are located until the application corresponding to the TLS stream where message A is located is identified.

[0208] Of course, if an empty subset of applications appears during the above process, the matching process can be terminated, and the application in the library is considered to have failed to match.

[0209] In one implementation, matching can be performed rule by rule. One implementation divides the rules into two categories: field rules and sequence rules, as described above. Another implementation divides the rules into three categories: sequence rules, field rules for matching length, and field rules other than those for matching length. Of course, this application does not limit how the rules are categorized. Taking message A as an example, multiple rules can be matched in parallel, each yielding a matching subset. This subset can be empty, indicating that no rule in that category matched successfully. These subsets are then joined (∪) to obtain the rule set corresponding to the message. Alternatively, rules can be matched in a certain order, continuously merging the matched subsets into the rule set until the rule set matching message A is obtained. For example... First, match one type of rule (referred to as rule type A, which can be any of the rules mentioned above), and filter out a subset A of rules that match rule type A for message A. Then match another type of rule (referred to as rule type B, which can be any of the rules mentioned above that are different from rule type A), and add the rule type B that matches message A to rule subset A. Alternatively, when matching this other type of rule, based on the previously obtained rule subset A and the rule set in the library, obtain rule type B that has a combination relationship with this rule subset, such as the rule set that includes rule subset A: {{A}, {B}, {C}}, {{A}, {D}}, {{B}, {C}, {F}}. In this system, rule subsets {B} and {D} are both type B rules for message A, and {C} is a type of rule for message B. When rule subsets {B} and {D} are used to match message A, rule subset {B} is found to match message A. This yields the possible rule set {{A}, {B}, {C}} for message A. Then, the rules in rule subset {C} are used to match the received message B. If a match is successful, the application corresponding to the flow containing messages A and B is the application corresponding to the rule set {{A}, {B}, {C}}. If a match fails, the flow identification for messages A and B fails.

[0210] It should be noted that matching can be performed sequentially, either according to the order in which the handshake messages are received or according to the interaction order of the handshake messages during the handshake process, as described above. The advantage of this is that once a handshake message with an earlier timing sequence is received, matching can begin without waiting for subsequent handshake messages. If a match is successful, the matching application can be determined without considering subsequent handshake messages. This obviously saves monitoring equipment resources and allows for faster and more timely identification of a flow. For example, matching can begin after receiving the ClientHello message. Alternatively, matching can begin after receiving multiple handshake messages simultaneously, such as after receiving the ClientHello and ServerHello messages, or after receiving the ClientHello and Certificate messages, or after receiving the ClientHello, ServerHello, and Certificate messages. This allows for parallel matching of the rules for each of the multiple handshake messages, and then, based on the matching rules of the multiple handshake messages and the rule set in the matching information database, a rule set for matching the multiple handshake messages is obtained, leading to the application corresponding to the encrypted flow. Alternatively, the rules for each message can be matched in parallel to obtain multiple application subsets corresponding to multiple rule subsets for each message, and then the intersection of these application subsets can be calculated. This is because many rule sets are applicable to these three messages, and parallel processing can improve efficiency. Of course, matching can also be performed after all handshake messages have been received, and so on.

[0211] In summary, monitoring devices can process multiple packets from the same flow in parallel, or process one or more packets using various rules in parallel. Referring to the above, those skilled in the art should understand that there are multiple implementation methods for the matching process. This application does not limit the specific process and timing of how the monitoring device matches one or more packets from a flow. On the other hand, monitoring devices may receive handshake packets from multiple flows simultaneously. Monitoring devices can process handshake packets from different flows in parallel. Please refer to the description above of the implementation method for matching handshake packets from a single flow. This application also does not limit this and will not elaborate further.

[0212] To facilitate understanding, the following will be combined with... Figure 6 To illustrate the matching process for a TLS stream handshake message, let's take another example. It should be understood that for data encrypted streams using other secure encryption protocols, the same process and matching information management methods can be used, namely, linked lists, arrays, and tables. Of course, this application does not limit how the matching information in the library is stored or managed.

[0213] During the handshake process, the ClientHello (CH), ServerHello (SH), and Certificate (CER) messages, which appear earlier in the sequence, often have corresponding rule subsets in the rule set. In the example described below, in addition to the rule subsets with explicit labels for these three handshake messages, rules for other handshake messages can also have their own explicitly labeled rule subsets. The monitoring device stores multiple rule subsets for each of the three handshake messages, as well as rule sets that include at least one of these rule subsets, and the mapping relationship between rule sets and applications. For example, a linked list can be used to store the rule sets, where the rule subsets for CH, SH, and CER can each be stored in a first-level linked list. However, for an application, its rule set often includes rule subsets for certain handshake messages from various subsequent handshake messages. Therefore, rule subsets for other handshake messages can be managed in the same linked list, or these rule subsets can be managed using another data structure, such as an array.

[0214] In this way, upon receiving the handshake message of a flow, it can be matched against a subset of rules, returning the number of the matched subset. For each handshake message matched from a flow, it checks whether the union of the matched subsets of rules in that flow's handshake message uniquely corresponds to an application. If so, the flow is successfully identified, and the matching process ends. If it corresponds to a set of applications, other handshake messages for that flow are obtained, and matching continues. If there is no corresponding application, the matching fails, or if the handshake message is a CH, SH, or CER packet followed by a handshake message, other handshake messages for that flow are matched until a corresponding application is found or all messages in that flow are matched. Specifically, after matching a subset of rules for a subsequent single handshake message, the system queries the rule set stored in the monitoring device that includes that subset of rules. It then determines whether these rule sets contain multiple messages that can match the flow. If so, the matching is successful; otherwise, the subsequent message needs to be replaced for further matching.

[0215] Furthermore, the monitoring equipment uses the SeqInfo data structure to manage rule sets, which means that a subset of rules in a rule set is arranged in a linked list according to the interaction order of the handshake messages during the handshake process. For example, the linked list CH_SH indicates that the rule set includes the rule subsets corresponding to CH and SH, but excludes the rule subset corresponding to CER. The linked list CH_CER indicates that the rule set includes the rule subsets corresponding to CH and CER, but excludes the rule subset corresponding to SH. The linked list CH_SH_CER indicates that the rule set includes the rule subsets corresponding to CH, CER, and SH. In this example, the CH_SH, CH_CER, and CH_SH_CER linked lists attached after the rule subset labels corresponding to different CHs are different and can be at least one of them. Of course, the CH_SH, CH_CER, and CH_SH_CER linked lists can also be part of the linked list attached after the rule subset. They can also be followed by other handshake messages or other rules. For example, the other rules can be rules used to limit the order of multiple handshake messages in a flow. Alternatively, no linked list can be attached, or it can include linked lists corresponding to other handshake messages. Due to the diverse range of applications, the linked list format of the corresponding rule set is also diverse. This application does not restrict the possible linked list format of the rule set. In this way, after parsing a subset of rules corresponding to a CH, when other messages are received, only a portion of the linked list attached to that rule subset can be matched based on the name of the handshake message, narrowing the query scope and more quickly matching the final corresponding application.

[0216] The following describes the process of matching a data stream when maintaining the mapping relationship between rule sets and applications using the data structure described above. In this process, rule subsets corresponding to CH are numbered starting with 1, and the number following 1 is the number of that rule subset within the corresponding CH's rule subsets. It should be understood that this number can be consecutive or non-consecutive. Similarly, rule subsets corresponding to SH are numbered starting with 2, and rule subsets corresponding to CER are numbered starting with 3. For example, in this case, the monitoring device stores the mapping relationship between rule sets and applications as shown in Table 1. Table 1 only lists rule sets including rule subsets 10+20+30. Each column represents a rule subset of a handshake message, with the column name indicating the corresponding handshake message. Each row represents a rule set, denoted by SeqRule in the table, with the number following it being the index of the rule set represented by that row. NA indicates that the rule set in that row does not include the rule subset corresponding to the handshake message in that column. Table 1 is for ease of understanding only. In actual implementation, the relationship between rule sets and rule subsets is not necessarily managed in the form of a table. This application does not limit the management form of the mapping relationship between rule sets and rule subsets.

[0217] Table 1

[0218] SeqRule0 10 20 30 40 50 60 SeqRule1 10 20 30 41 50 61 SeqRule2 10 20 30 41 51 61 SeqRule3 10 20 30 41 50 NA SeqRule4 10 20 31 NA NA NA

[0219] Figure 6 The diagram illustrates subsets of rules that may be involved in the matching process, where arrows indicate a path from a node represented by a rectangle to a node that may be matched next (also represented by a rectangle). The numbers 1, 2, 3, 7, 8, and 9 in the table represent the indices of a rule set.

[0220] like Figure 6 As shown, after querying, a CH packet of a stream matches rule subset 10, represented as CH=10 in the figure. Then, we find the SeqInfo data structure where CH=10 is located. It has three linked lists attached to it: CH_SH, CH_CER, and CH_SH_CER. Obviously, the rule sets represented by these three linked lists include the same rule subset corresponding to CH. After receiving the SH packet from the stream, the monitoring device parses it and matches the corresponding SH portions in the CH_SH and CH_SH_CER linked lists. It finds that the SH packet matches rule subset 20, represented as SH=20 in the diagram. In reality, this matches the node in the linked list where CH=10 and SH=20 (meaning the node satisfies the condition CH=10 and SH=20). The monitoring device then receives the CER packet from the same stream. After parsing the CER packet, it matches the CH_SH_CER linked list following that node, finding that the CER packet matches rule subset 30, represented as CER=30 in the diagram. This matches the node in the CH_SH_CER linked list where CH=10, SH=20, and CER=30. Therefore, it should be understood that maintaining two rule sets like 10+20+30 and 10+20+30+40 simultaneously is not advisable, as a stream satisfying 10+20+30 might match two different applications. Furthermore, the diagram also shows that the CH_SH linked list following CH=10 can carry multiple SH nodes, allowing queries to be made among the nodes representing SH. For example, as shown in the diagram, the node representing SH=20 can be accessed via NEXT to find the node representing SH=21. Another example is the CH_SH_CER linked list, where the node representing SH=20 and CER=30 can be accessed via NEXT to find the node representing SH=20 and CER=31.

[0221] In this example, the rule subsets corresponding to other types of handshake messages are stored in array form, such as the ExtSeqInfo array. The nodes corresponding to CH=10, SH=20, and CER=30 also have an array structure, meaning that the rule set including the three rule subsets CH=10, SH=20, and CER=30 includes other rules or rule subsets. The matching process for this flow is not yet complete. The monitoring device receives the handshake message for this flow and then parses and matches it to determine a rule subset that meets the conditions. It then determines whether there is a rule set that is the same as the union of these multiple rule subsets. It should be noted that since there are many handshake messages, and a rule set often only includes subsets of subsequent handshake messages, after querying the rule subset corresponding to a single handshake message after CH, SH, and CER, if the monitoring device does not have a rule set whose union is the same as the rule subsets matching this flow, it can discard this rule subset of the handshake message and continue matching subsequent handshake messages.

[0222] For example, upon receiving a ServerKeyExchange packet (SKE) from this stream, the SKE matches rule subset 40. The ExtSeqInfo array is then traversed to find rule sets containing rule subset 40. Rule sets with indices 0 and 2 are found to include this rule subset. However, neither of these rule sets also includes rule subsets 10+20+30. Therefore, this stream does not match rule sets 0 and 2, and matching continues. The indices of rule sets 0 and 2 can be recorded first. It should be understood that in this example, matching the CER packet followed by the SKE packet means that the CER packet is transmitted followed by the SKE packet in the handshake process of this stream.

[0223] Similarly, parsing the received ServerHelloDone packet (SHD) of the stream, we match the SKE-matched rule subset 50. Traversing the ExtSeqInfo array, we search for rule sets containing rule subset 50. We find rule sets with indices 0 and 1 that include this rule subset. However, we also find that neither of these rule sets includes rule subset 10+20+30. Therefore, the stream did not match rule sets 0 and 1, and matching continues. We can record the indices of rule sets 0 and 1. It should be understood that we are confirming whether the rule set includes rule subset 10+20+30, not 10+20+30+40. As mentioned earlier, since no rule set including 10+20+30+40 was found when matching SKE, the rule subset matched by SKE is ignored when matching SHD. Of course, if the rule set used in this example does not include rule subsets corresponding to certain handshake messages, the matching process can skip these handshake messages.

[0224] Next, the received ClientKeyExchange packet (CKE) of the stream is parsed, and the CKE matches the rule subset 60. The ExtSeqInfo array is traversed to find the rule set containing the rule subset 60. The rule sets with index values ​​of 0 and 3 contain the rule subset. Rule set 3 also includes rule subsets 10+20+30. Therefore, the stream matches rule set 3.

[0225] Since in the prior art, the order in which the handshake messages of a TLS stream are transmitted can be used as a rule to identify the stream, the rule set described in this application may also include rules that indicate the transmission order of multiple handshake messages, such as the MsgSeq rule. The figure also shows a MsgSeq rule numbered 122334.

[0226] Therefore, if rule set 3 includes rules indicating the transmission order of multiple handshake messages, the order of the handshake messages in the stream should be further checked to see if it satisfies the MsgSeq rule included in rule set 3. If it does, the TLS stream matches rule set 3, and the application corresponding to rule set 3 is the application corresponding to the TLS stream. If it does not, rule set 3 is not matched, and other handshake messages received in the stream should be parsed. Of course, if rule set 3 does not include the MsgSeq rule, then the TLS stream matches rule set 3, and the application corresponding to rule set 3 is the application corresponding to the TLS stream.

[0227] Figure 7 A schematic diagram of an apparatus 700 for identifying encrypted data streams is described. The apparatus includes a parsing module 701, a matching module 702, and a determining module 703. This apparatus 700 can identify the application corresponding to the encrypted data stream through one or more handshake messages of the encrypted data stream. That is, the apparatus 700 can implement any of the methods described above in this application. Figure 3 The corresponding monitoring equipment can also be seen as an implementation of this device, in which the packet processor corresponds to the parsing module 701, the traffic identification engine corresponds to the matching module 702 and the determination module 703.

[0228] On one hand, the parsing module 701 is used to parse multiple handshake messages of the encrypted data stream according to the secure encryption transmission protocol corresponding to the encrypted data stream, so as to obtain multiple fields contained in the multiple handshake messages; the matching module 702 is used to determine the set of rules that the multiple handshake messages match from multiple set of rules according to the multiple fields, wherein each set of rules in the multiple set of rules includes at least one rule of field rules and order rules, the field rules are used to indicate the characteristics of a field, the order rules are used to indicate the order of multiple fields in a handshake message, and the fields in the multiple handshake messages satisfy the rules in the matching set of rules; the determining module 703 is used to determine the application corresponding to the encrypted data stream according to the mapping relationship between the matching set of rules and the application.

[0229] For explanations of terms, implementation methods, and beneficial effects in this area, please refer to the corresponding methods mentioned above. For example... Figure 5a The method is shown. The matching module 702 can be used to execute various implementations of the step described above, "determining the set of rules that match the multiple handshake messages from multiple rule sets based on the multiple fields." The determining module 703 can be used to execute various implementations of the step described above, corresponding to the step "determining the application corresponding to the encrypted data stream based on the mapping relationship between the matched rule set and the application."

[0230] On the other hand, the parsing module 701 is used to parse a handshake message of the encrypted data stream according to the secure encryption transmission protocol corresponding to the encrypted data stream, so as to obtain multiple fields contained in the handshake message; the matching module 702 is used to determine the rule set matched by the handshake message from multiple rule sets according to the multiple fields, wherein each rule set in the multiple rule sets includes at least one rule of field rules and order rules, the field rules are used to indicate the characteristics of a field, the order rules are used to indicate the order of multiple fields in a message, and the fields in the handshake message satisfy the rules in the matching rule set; the determining module 703 is used to determine the application corresponding to the encrypted data stream according to the mapping relationship between the matching rule set and the application.

[0231] For explanations of terms, implementation methods, and beneficial effects in this area, please refer to the corresponding methods mentioned above. For example... Figure 5bThe method is shown. The matching module 702 can be used to execute various implementations of the step described above, "determining the set of rules that match the multiple handshake messages from multiple rule sets based on the multiple fields." The determining module 703 can be used to execute various implementations of the step described above, corresponding to the step "determining the application corresponding to the encrypted data stream based on the mapping relationship between the matched rule set and the application."

[0232] Furthermore, both of the aforementioned devices may also include a training module, which is not embodied in... Figure 7 In this module, multiple samples are trained using machine learning algorithms to obtain at least one set of rules corresponding to the target application. The multiple samples are known to be handshake messages of the encrypted data stream corresponding to the target application. The multiple samples include handshake messages of the encrypted data stream corresponding to the target application.

[0233] Furthermore, both of the aforementioned devices can store multiple rule sets as described above, as well as the mapping relationship between the rule sets and applications. For example, the identification information database described above.

[0234] like Figure 8 As shown, this embodiment of the invention provides a physical machine that can be used to execute any of the methods described above, for example... Figure 5a , Figure 5b The corresponding methods, etc. The physical machine includes processing circuitry 40, interface circuitry 41, storage medium 42, and system bus 43. And, Figure 7 The corresponding device can also be installed on this physical machine, and the processing circuit 40 implements the operation by running the instructions in the storage medium 42. Figure 7 The corresponding modules of device 700. Of course, in one implementation, the storage medium 42 also stores the identification information database mentioned above. And, the physical machine can be as follows: Figure 3 The monitoring equipment shown is... Figure 3 The input and output interfaces can be provided by Figure 8 The interface circuit 41 in the middle is implemented. Figure 3 The packet processor, stream processor, traffic identification engine, and identification information database in the processing circuit 40 can be implemented by running instructions in the storage medium 42.

[0235] Storage medium 42 is used to store computer execution instructions. Processing circuit 40, interface circuit 41 and storage medium 42 are interconnected through system bus 43. When the physical machine is running, processing circuit 40 executes the computer execution instructions stored in storage medium 42 so that the physical machine executes the method provided in the embodiments of the present invention.

[0236] The processing circuit 40 can be implemented by one or more processors. Figure 8 The following description uses only one processor as an example. The processing circuit 40 can be a central processing unit (CPU). The processing circuit 40 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0237] The interface circuit 41 can specifically be a communication interface on the physical machine. This communication interface can be a wireless communication interface. For example, the wireless communication interface can be a wireless module of the physical machine. The processing circuit 40 communicates with other devices, such as other physical machines, through the interface circuit 41 to send and receive data.

[0238] Storage medium 42 may include volatile memory, such as random-access memory (RAM); storage medium 42 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); storage medium 42 may also include combinations of the above types of memory.

[0239] Storage medium 42 may include underlying storage medium and memory. The memory is coupled to the underlying storage medium and serves as a cache for the underlying storage medium.

[0240] System bus 43 may include a data bus, power bus, control bus, and signal status bus, etc. In this embodiment, for clarity, ... Figure 8 The various buses are all illustrated as system bus 43.

[0241] Optionally, this embodiment also provides a readable storage medium, which includes computer-executable instructions. When the physical machine is running, the processor of the physical machine executes the computer-executable instructions to cause the physical machine to perform any of the methods provided in this embodiment of the invention.

[0242] Optionally, the readable storage medium in this embodiment can be as described above. Figure 8 The storage medium 42 shown.

[0243] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0244] In the several embodiments provided in this application, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the module division is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. For example, the matching module 702 and the determining module 703 may be a single module, as described above. Figure 3 The traffic identification engine mentioned above. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device, or unit, and can be electrical, mechanical, or other forms.

[0245] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD), phase change memory), etc.

[0246] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is merely an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and modules described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0247] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, apparatuses, or modules.

[0248] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0249] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in the form of software functional modules.

[0250] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The storage medium is a non-transitory medium, including flash memory, portable hard drive, read-only memory, random access memory, magnetic disk, or optical disk, and other media capable of storing program code.

[0251] The above description is merely a specific embodiment of the invention as described in this application, but the scope of protection of this invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this invention should be included within the scope of protection of this invention. Therefore, the scope of protection of this invention should be determined by the scope of the claims.

Claims

1. A method for identifying encrypted data streams, characterized in that, The method includes: Parse multiple handshake messages of the encrypted data stream to obtain multiple fields contained in the multiple handshake messages, the multiple fields being fields other than the Server Name Indicator (SNI) field and the Common Name field; Based on the multiple fields, a set of rules matching the multiple handshake messages is obtained, wherein the set of rules consists of multiple rules, the set of rules includes field rules and order rules, or the set of rules includes the order rules, the field rules are used to indicate the characteristics of a field, the order rules are used to indicate the order of the types of multiple fields carried in a handshake message, and the fields in the multiple handshake messages satisfy all the rules in the matching set of rules; Based on the mapping relationship between the matching rule set and the application, the application corresponding to the encrypted data stream is obtained.

2. The method according to claim 1, characterized in that, For a field rule, the characteristics of the indicated field are the field length, or the field type, or the field length and type, or the field type and value, or the field length, type and value.

3. The method according to claim 1 or 2, characterized in that, The plurality of fields include a plurality of groups, each of the plurality of groups corresponds to a handshake message, and the rule set matched by the plurality of handshake messages is a rule set obtained by matching the plurality of groups with the rules in the plurality of rule sets according to the receiving order of the plurality of handshake messages.

4. The method according to claim 1 or 2, characterized in that, When the matching rule set includes field rules and order rules, the rule set matched by the multiple handshake messages is a rule set obtained by matching the multiple fields with the field rules and order rules in the multiple rule sets respectively.

5. The method according to claim 1 or 2, characterized in that, The matching rule set includes multiple subsets, each subset in the multiple subsets corresponds to at least one application, and the application corresponding to the encrypted data stream includes applications in the intersection of multiple application sets. The multiple application sets are multiple application sets corresponding to the multiple subsets obtained according to the mapping relationship between each subset in the multiple subsets and the application.

6. The method according to claim 1 or 2, characterized in that, The rule set includes multiple rules stored in the form of a linked list, wherein each node in the linked list stores a rule corresponding to the name of a handshake message.

7. The method according to claim 1 or 2, characterized in that, The multiple handshake messages for parsing the encrypted data stream include: For each of the multiple handshake messages in the encrypted data stream, perform the following operations: Parse the name of each handshake message; Based on the name of each handshake message, at least one rule corresponding to the name of each handshake message is determined from multiple rule sets; Parse the fields indicated by at least one rule in each handshake message.

8. The method according to claim 1 or 2, characterized in that, The multiple handshake messages are Secure Sockets Layer (SSL) handshake messages, Transport Layer Security (TLS) handshake messages, or Datagram Transport Layer Security (DTLS) handshake messages.

9. The method according to claim 1 or 2, characterized in that, The method further includes: Multiple samples from multiple encrypted data streams are trained using machine learning algorithms to obtain at least one set of rules corresponding to the target application. The multiple samples are known to be handshake messages of the encrypted data streams corresponding to the target application, and the multiple samples include handshake messages of the encrypted data streams corresponding to the target application.

10. A method for identifying encrypted data streams, characterized in that, The method includes: Parse the handshake message of the encrypted data stream to obtain multiple fields contained in the handshake message, which are fields other than the Server Name Indicator (SNI) field and the Common Name field; Based on the multiple fields, a set of rules matching the handshake message is obtained, wherein the set of rules consists of multiple rules, including field rules and order rules, or the set of rules includes the order rules. The field rules are used to indicate the characteristics of a field, and the order rules are used to indicate the order of the types of multiple fields carried in a handshake message. The fields in the handshake message satisfy all the rules in the matching set of rules. Based on the mapping relationship between the matching rule set and the application, the application corresponding to the encrypted data stream is obtained.

11. The method according to claim 10, characterized in that, For a field rule, the characteristics of the indicated field are the field length, or the field type, or the field length and type, or the field type and value, or the field length, type and value.

12. The method according to claim 10 or 11, characterized in that, The matching rule set is the rule set corresponding to the name of the handshake message.

13. The method according to claim 10 or 11, characterized in that, When the matching rule set includes field rules and order rules, the rule set matched by the handshake message is a rule set obtained by matching the multiple fields with the field rules and order rules in the multiple rule sets respectively.

14. The method according to claim 10 or 11, characterized in that, The handshake message is a Secure Sockets Layer (SSL) handshake message, a Transport Layer Security (TLS) handshake message, or a Datagram Transport Layer Security (DTLS) handshake message.

15. The method according to claim 10 or 11, characterized in that, The method further includes: Multiple samples are trained using machine learning algorithms to obtain at least one set of rules corresponding to the target application. The multiple samples are known to be handshake messages of the encrypted data stream corresponding to the target application. The multiple samples include handshake messages of the encrypted data stream corresponding to the target application.

16. An apparatus for identifying encrypted data streams, characterized in that, The device includes: The parsing module is used to parse multiple handshake messages of the encrypted data stream to obtain multiple fields contained in the multiple handshake messages, the multiple fields being fields other than the Server Name Indicator (SNI) field and the Common Name field; A matching module is used to obtain a set of rules that match the multiple handshake messages based on the multiple fields. The set of rules consists of multiple rules, including field rules and order rules, or the set of rules includes the order rules. The field rules are used to indicate the characteristics of a field, and the order rules are used to indicate the order of the types of multiple fields carried in a handshake message. The fields in the multiple handshake messages satisfy all the rules in the matching set of rules. The determination module is used to determine the application corresponding to the encrypted data stream based on the mapping relationship between the matching rule set and the application.

17. The apparatus according to claim 16, characterized in that, For a field rule, the characteristics of the indicated field are the field length, or the field type, or the field length and type, or the field type and value, or the field length, type and value.

18. The apparatus according to claim 16 or 17, characterized in that, The plurality of fields include a plurality of groups, each of the plurality of groups corresponds to a handshake message, and the rule set matched by the plurality of handshake messages is a rule set obtained by matching the plurality of groups with the rules in the plurality of rule sets according to the receiving order of the plurality of handshake messages.

19. The apparatus according to claim 16 or 17, characterized in that, When the matching rule set includes field rules and order rules, the rule set matched by the multiple handshake messages is a rule set obtained by matching the multiple fields with the field rules and order rules in the multiple rule sets respectively.

20. The apparatus according to claim 16 or 17, characterized in that, The matching rule set includes multiple subsets, each subset in the multiple subsets corresponds to at least one application, and the application corresponding to the encrypted data stream includes applications in the intersection of multiple application sets. The multiple application sets are multiple application sets corresponding to the multiple subsets obtained according to the mapping relationship between each subset in the multiple subsets and the application.

21. The apparatus according to claim 16 or 17, characterized in that, The rule set includes multiple rules stored in the form of a linked list, wherein each node in the linked list stores a rule corresponding to the name of a handshake message.

22. The apparatus according to claim 16 or 17, characterized in that, In parsing multiple handshake messages of an encrypted data stream, the parsing module is used for: For each of the multiple handshake messages in the encrypted data stream, perform the following operations: Parse the name of each handshake message; Based on the name of each handshake message, at least one rule corresponding to the name of each handshake message is determined from multiple rule sets; Parse the fields indicated by at least one rule in each handshake message.

23. The apparatus according to claim 16 or 17, wherein the plurality of handshake messages are Secure Sockets Layer (SSL) handshake messages, Transport Layer Security (TLS) handshake messages, or Datagram Transport Layer Security (DTLS) handshake messages.

24. The apparatus according to claim 16 or 17, characterized in that, The device further includes a training module, which is used to train multiple samples of multiple encrypted data streams through a machine learning algorithm to obtain at least one rule set corresponding to the target application. The multiple samples are known to be handshake messages of the encrypted data streams corresponding to the target application, and the multiple samples include handshake messages of the encrypted data streams corresponding to the target application.

25. An apparatus for identifying encrypted data streams, characterized in that, The device includes: The parsing module is used to parse the handshake message of the encrypted data stream to obtain multiple fields contained in the handshake message, which are fields other than the Server Name Indicator (SNI) field and the Common Name field; A matching module is configured to obtain a set of rules that match the handshake message based on the plurality of fields. The set of rules consists of multiple rules, including field rules and order rules, or the set of rules includes the order rules. The field rules are used to indicate the characteristics of a field, and the order rules are used to indicate the order of the types of multiple fields in a handshake message. The fields in the handshake message satisfy all the rules in the matching set of rules. The determination module is used to determine the application corresponding to the encrypted data stream based on the mapping relationship between the matching rule set and the application.

26. The apparatus according to claim 25, characterized in that, For a field rule, the characteristics of the indicated field are the field length, or the field type, or the field length and type, or the field type and value, or the field length, type and value.

27. The apparatus according to claim 25 or 26, characterized in that, The matching rule set is the rule set corresponding to the name of the handshake message.

28. The apparatus according to claim 25 or 26, characterized in that, When the matching rule set includes field rules and order rules, the rule set matched by the handshake message is a rule set obtained by matching the multiple fields with the field rules and order rules in the multiple rule sets respectively.

29. The apparatus according to claim 25 or 26, characterized in that, The handshake message is a Secure Sockets Layer (SSL) handshake message, a Transport Layer Security (TLS) handshake message, or a Datagram Transport Layer Security (DTLS) handshake message.

30. The apparatus according to claim 25 or 26, characterized in that, The device further includes a training module, which is used to train multiple samples using a machine learning algorithm to obtain at least one set of rules corresponding to the target application. The multiple samples are known to be handshake messages of the encrypted data stream corresponding to the target application, and the multiple samples include handshake messages of the encrypted data stream corresponding to the target application.

31. A device for identifying encrypted data streams, characterized in that, The device includes a processing circuit, an interface circuit, and a storage medium. The interface circuit is used to send and receive data packets with other devices through instructions in the storage medium. The processing circuit is used to execute the instructions in the storage medium to control the interface circuit, so as to implement the method as described in any one of claims 1 to 9.

32. A device for identifying encrypted data streams, characterized in that, The device includes a processing circuit, an interface circuit, and a storage medium. The interface circuit is used to send and receive data packets with other devices via instructions in the storage medium. The processing circuit is used to execute the instructions in the storage medium to control the interface circuit, so as to implement the method as described in any one of claims 10 to 15.

Images