An Application Feature Extraction Method Based on the Packet Filtering Framework

Abstract

The present invention discloses an application feature extraction method based on a packet filtering framework, which includes: initially screening relevant information of a target application by issuing a blocking policy, taking the domain name information of the intercepted target application as the initial domain name feature, and taking the IP information of the un-intercepted target application as the initial IP feature after establishing an association with the domain name; obtaining an IP feature set through the initial domain name feature or the initial IP feature, and re-screening the IP feature set through the blocking policy until the application is successfully intercepted. This solution initially screens and re-screens the target application by intercepting the application, and efficiently extracts the key domain name information and IP information in the target application.

Description

Technical Field

[0001] The present invention relates to the field of communications, and in particular to an application feature extraction method based on a packet filtering framework. Background Art

[0002] Existing application features mainly include port identification methods and traditional manual interception methods, among which:

[0003] Port identification methods primarily use protocol identification technology to examine only the five-tuple information in the message, extracting the port number and protocol from the message as application features. While protocol identification technology is highly efficient, its scope of application is shrinking. Furthermore, it's impossible to differentiate between different applications using the same protocol and port number. For example, if a web game and a web video both use the same protocol and port number for data transmission, the port number and protocol alone cannot effectively distinguish the two applications.

[0004] Traditional manual interception methods primarily rely on manually intercepting and organizing network data, extracting domain names and IP addresses from application messages, and then manually analyzing and extracting signature information. This is then tested using interception tools such as AdGuard. Different domain names and IP addresses appear randomly each time data is captured, resulting in incomplete and inefficient domain and IP address signatures. The excessive use of tools results in inefficiency and complexity.

[0005] In the process of application data analysis, feature information comes from the domain name and IP data in the message. The information is reliable and provides a solid foundation for application feature analysis. Summary of the Invention

[0006] Therefore, in order to solve the above problems, the present invention provides an application feature extraction method based on a packet filtering framework.

[0007] The present invention is achieved through the following technical solutions:

[0008] An application feature extraction method based on a packet filtering framework, the method comprising:

[0009] By issuing a blocking policy, the relevant information of the target application is preliminarily screened. The domain name information of the blocked target application is used as the preliminary domain name feature. The IP information of the target application that is not blocked is used as the preliminary IP feature after establishing an association with the domain name.

[0010] The IP feature set is obtained through preliminary domain name features or preliminary IP features, and the IP feature set is re-screened through the blocking strategy until the application is successfully intercepted.

[0011] Preferably, it further includes, before initially screening the relevant information of the target application by issuing a blocking policy, capturing packets for each application to obtain the corresponding relationship between the domain names and IP information it accesses, forming a basic application database, and simultaneously extracting the domain name data with intersections among multiple applications to generate a blacklist database.

[0012] Preferably, the step of initially screening the relevant information of the target application by issuing a blocking policy, using the domain name information of the intercepted application as the initial domain name feature, and associating the IP information of the un-intercepted application with the domain name as the initial IP feature includes:

[0013] Setting an interception instruction for the domain name feature of the target application when the target application and the server establish a connection through the Iptables tool, and determining whether the target application is intercepted successfully;

[0014] If the target application is intercepted, obtain the domain name information of the target application, compare the domain name information with the data in the blacklist database, and after excluding the domain name information corresponding to the data in the blacklist database, use the remaining domain name information as the initial domain name feature of the target application;

[0015] If the target application is not intercepted, obtain the IP information of the target application, establish the association between the IP information and the domain name, and use the IP information associated with the domain name as the initial IP feature.

[0016] Preferably, the step of obtaining the IP information of the target application, establishing the association between the IP information and the domain name, and using the IP information associated with the domain name as the initial IP feature includes the following steps:

[0017] Extract all the IP information in the target application;

[0018] Filter out the IP information in the target application that has a corresponding relationship with the domain name;

[0019] Parse the SNI field in the HTTPS protocol, associate the SNI field with the remaining IP information after filtering, generate the relationship feature between the IP and the SNI, and match to obtain the domain name information associated with the IP information.

[0020] Preferably, the step of filtering out the IP information in the target application that has a corresponding relationship with the domain name includes the following steps:

[0021] Compare the domain name information in the target application with the data in the basic application database to find the domain name information corresponding to the data in the basic application database;

[0022] Parse the domain name information obtained by comparing with the data in the basic application database to generate the corresponding IP information;

[0023] Screen out the IP information generated by domain name information resolution from all the IP information of the target application.

[0024] Preferably, if there is still IP information that is not associated with the domain name after parsing the HTTPS protocol, directly use this IP information as the preliminary IP feature.

[0025] Preferably, in the preliminary screening, if the target application is intercepted, then the IP feature set obtained through the preliminary domain name feature or the preliminary IP feature, and the re-screening of the IP feature set through the blocking policy includes:

[0026] Write the preliminary domain name feature of the intercepted target application into the dnsmasq file, start the dnsmasq service, and resolve the corresponding IP information;

[0027] Store the resolved IP information in the Ipset set as the IP feature of the target application.

[0028] Preferably, in the preliminary screening, if the target application is intercepted, then the IP feature set obtained through the preliminary domain name feature or the preliminary IP feature, and the re-screening of the IP feature set through the blocking policy includes:

[0029] Store the IP information corresponding to the domain name pre-obtained from the packet capture data, the IP information associated with the domain name obtained by parsing the HTTPS protocol, and the IP information not associated with the domain name in the Ipset set;

[0030] Issue a blocking policy to the Ipset set through the Iptables tool.

[0031] Preferably, after issuing a blocking policy to the Ipset set through the Iptables tool,

[0032] If the target application cannot communicate normally, then use the Ipset set as the IP feature set of the target application; if the target application can still communicate, then expand the screening scope of the target application, and continuously screen through the blocking policy until the application is intercepted successfully, and then use the Ipset set as the IP feature set of the target application.

[0033] Preferably, the expansion of the screening scope of the target application means: re-obtain the IP information and domain name information of the target application, and establish the association between the IP information and the domain name.

[0034] The beneficial effects of the technical solution of the present invention are mainly reflected in:

[0035] 1. Initially screen and re-screen the target application by intercepting the application, efficiently extract the key domain name information and IP information in the target application. By intercepting the application, it can accurately and quickly determine whether the key features of the target application are extracted. At the same time, the continuous interception and screening method can achieve efficient, accurate and comprehensive extraction of application features.

[0036] 2. Capture packets for each application in advance, refine the data where the domains generated by multiple applications have intersections, generate a blacklist database, use it to compare the domain name information of subsequent target applications, and extract the application features that distinguish the target application from multiple other applications in the same series, which can improve the accuracy of application feature extraction.

[0037] 3. For the target application that cannot be intercepted, after filtering the IP information resolved by the domain name, establish the relationship feature between the SNI field and the IP field in the HTTPS protocol establishment packet, and further associate the IP information with the domain name. For the IP information that cannot be associated with the domain name, it is also stored in the feature set together to prevent omission, which can ensure the complete extraction of application features. Brief Description of the Drawings

[0038] Figure 1 : It is a schematic diagram of the steps of an application feature extraction method based on a packet filtering framework. Detailed Embodiment

[0039] To make the purpose, advantages and features of the present invention be shown more clearly and in detail, it will be illustrated and explained through the non-restrictive description of the following preferred embodiments. This embodiment is only a typical example of applying the technical solution of the present invention. Any technical solution formed by equivalent replacement or equivalent transformation falls within the scope of protection required by the present invention.

[0040] At the same time, it is stated that in the description of the solution, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "front", "rear", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of description and simplification of description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, so it cannot be understood as a limitation to the present invention.

[0041] In addition, the terms "first" and "second" in this solution are only used for descriptive purposes, and cannot be understood as indicating or implying the ranking of importance, or implicitly indicating the quantity of the technical features shown. Therefore, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0042] The present invention discloses, as Figure 1 shown, an application feature extraction method based on a packet filtering framework, the method comprising:

[0043] Pre-screening relevant information of a target application by issuing a blocking policy, taking the domain name information of the intercepted target application as the preliminary domain name feature, and taking the IP information of the non-intercepted target application as the preliminary IP feature after establishing the association with the domain name;

[0044] In an embodiment, the method further comprises, before pre-screening relevant information of a target application by issuing a blocking policy, capturing packets for each application to obtain the corresponding relationship between the domain name and IP information accessed by it, forming a basic application database, and at the same time refining the domain name data with intersections among multiple applications to generate a blacklist database.

[0045] Specifically, in this embodiment, the application features of the target application mainly include the domain name feature and IP feature of the target application. The pre-screening first intercepts the target application by issuing a blocking policy. Here, the blocking policy is mainly to confirm whether the IP of the domain name of the target application and the key service connection of the application are associated. If the target application is successfully intercepted, it means that the IPs of the key services of the application are all generated by domain name resolution, and the domain name information and IP information are associated with each other. Therefore, the corresponding IP information can be directly obtained by using the parsing method after obtaining the domain name feature of the target application; if the target application is not intercepted, it means that the target application still has IPs not associated with the domain name. Therefore, after collecting all the IP information of the target application, the association between the IP information and the domain name information needs to be established.

[0046] Secondly, when the target is successfully intercepted and it is determined that the IPs of the key services of the target application are all generated by domain name resolution, through traversal analysis, all the domain name information in the target application is compared with the data with intersections among multiple applications in the same series. The data with intersections represents the common application features possessed by all applications in the same series. After excluding this part of the common application features, the remaining domain name information can be used as the preliminary domain name feature of the target application.

[0047] The blacklist database is formed by capturing packets for all applications to obtain the corresponding relationship between the domain name and IP information accessed by each application, forming a basic application database by aggregating the corresponding relationships between the domain name and IP information accessed by all applications, and then aggregating the common application features with intersections among different applications in the basic application database.

[0048] Specifically, the step of pre-screening relevant information of a target application by issuing a blocking policy, taking the domain name information of the intercepted application as the preliminary domain name feature, and associating the IP information of the non-intercepted application with the domain name as the preliminary IP feature includes:

[0049] Set an interception instruction for the domain name feature of the target application when the target application and the server establish a connection through the Iptables tool, and determine whether the target application is intercepted successfully;

[0050] If the target application is intercepted, obtain the domain name information of the target application, compare the domain name information with the data in the blacklist database, and exclude the domain name information corresponding to the data in the blacklist database, and use the remaining domain name information as the preliminary domain name feature of the target application;

[0051] Specifically, Iptables is the most commonly used firewall tool on Linux. Iptables interacts with the hook with packet filtering function in the protocol stack to complete the work. These kernel hooks constitute the netfilter framework. Every packet entering the network system and passing through the protocol stack when being received or sent by the network system will trigger these hooks. The program can process network traffic at some key paths by registering hook functions. The kernel modules related to Iptables register processing functions at these hook points. Therefore, the network traffic can be made to conform to the firewall rules by configuring Iptables rules. In one embodiment, the Iptables tool realizes DNS interception by registering hook functions. DNS (Domain Name System) is a server for domain name resolution, which can realize the mutual mapping between domain names and IP addresses. Through DNS interception, it can intercept applications where the IPs of key service connections are generated through domain name resolution. That is, if the target application is intercepted, it means that the IPs of the key service connections of the target application are all generated through domain name resolution. At this time, only the domain name information of the target application needs to be extracted to obtain the corresponding IP information through resolution.

[0052] Subsequently, through traversal and analysis, compare the domain name information in the target application with the general application features in the blacklist database one by one, exclude the parts of the domain name information in the target application that are the same as the general application features, and use the remaining domain name information as the preliminary domain name feature of the target application.

[0053] If the target application is not intercepted, obtain the IP information of the target application, establish the association between the IP information and the domain name, and use the IP information associated with the domain name as the preliminary IP feature.

[0054] Specifically, the steps of obtaining the IP information of the target application, establishing the association between the IP information and the domain name, and using the IP information associated with the domain name as the preliminary IP feature include the following steps:

[0055] Extract all the IP information in the target application;

[0056] Filter the pre-acquired IP information corresponding to the domain name in the captured data;

[0057] Specifically, in this step, by obtaining all the IP information of the target application, after establishing the association between the IP information and the domain name, the IP information and the domain name information of the target application are obtained simultaneously. Among the IP information in the target application, there is a part of the IP information that has already been associated with the domain name, and this part of the IP information does not need to be associated with the domain name again. Therefore, it can be filtered out first.

[0058] The filtering of the IP information in the target application that has a corresponding relationship with the domain name includes the following steps:

[0059] Compare the domain name information in the target application with the data in the basic application database to find the corresponding domain name information in the basic application database;

[0060] Resolve the domain name information obtained by comparing with the data in the basic application database to generate the corresponding IP information;

[0061] Filter out the IP information generated by resolving the domain name information from all the IP information of the target application.

[0062] Since the basic application database has been obtained by capturing packets of each application before the preliminary screening, and the basic application database is composed of the set of the corresponding relationships between the domain names and IP information accessed by each application, therefore, all the domain name information that can be found in the data of the basic application database in the target application can be resolved to obtain the corresponding IP information, and this part of the IP information can be filtered out first.

[0063] Parse the SNI field in the HTTPS protocol, associate the SNI field with the remaining IP information after filtering, generate the relationship feature of IP and SNI, and match to obtain the domain name information associated with the IP information;

[0064] SNI (Server name indication) is a set of fields (or a subset) in the HTTPS message. SNI is an extension to solve the problem of a server using multiple domain names and certificates for SSL / TLS. It allows the server to support multiple certificates under the same IP address and TCP port number, thus allowing multiple HTTPS websites, or other services on TLS websites, to be served by the same IP address without all websites using the same certificate. The method of finally obtaining the domain name information associated with the IP information by parsing the SNI field in the HTTPS protocol can refer to the method for identifying the accessed domain name disclosed in the invention patent with the patent application number: CN201811381022.1 and the patent name "Method and System for Identifying Accessed Domain Name". This method is prior art and will not be elaborated here.

[0065] If there is still IP information that is not associated with a domain name after parsing the HTTPS protocol, then directly use this IP information as the preliminary IP feature.

[0066] After the above steps, the target application is preliminarily screened, and the corresponding preliminary domain name features or preliminary IP features are obtained. Therefore, it is also necessary to parse and expand the features of the target application and intercept the target application again to verify whether the key feature information of the target application has been extracted. This is mainly achieved through the following steps:

[0067] Obtain the IP feature set through the preliminary domain name features or preliminary IP features, and re-screen the IP feature set through the blocking policy until the application interception is successful.

[0068] Specifically, in the preliminary screening, there are two situations where the target application is intercepted and not intercepted, and the target application features obtained in the two situations are also different. Therefore, in the re-screening step, it is also necessary to conduct targeted screening according to these two different situations.

[0069] Among them, in the preliminary screening, if the target application is intercepted, then the obtaining of the IP feature set through the preliminary domain name features or preliminary IP features and the re-screening of the IP feature set through the blocking policy include:

[0070] Write the preliminary domain name features of the successfully intercepted target application into the dnsmasq file, start the dnsmasq service, and resolve the corresponding IP information;

[0071] Store the resolved IP information in the Ipset set as the IP feature of the target application.

[0072] Specifically, in the preliminary screening, since the target application has been intercepted, it means that the IPs of the key services of the target application are all generated through domain name resolution, and the domain name information and IP information are mutually associated. It is only necessary to extract this part of the associated IP information through parsing, and there is no need to intercept the target application again.

[0073] On the contrary, in the preliminary screening, if the target application is not intercepted, then the obtaining of the IP feature set through the preliminary domain name features or preliminary IP features and the re-screening of the IP feature set through the blocking policy include:

[0074] Store the IP information corresponding to the domain name pre-obtained in the packet capture data, the IP information associated with the domain name obtained by parsing the HTTPS protocol, and the IP information that is not associated with the domain name in the Ipset set;

[0075] Issue the blocking policy to the Ipset set through the Iptables tool.

[0076] After issuing a blocking policy to the Ipset set through the Iptables tool,

[0077] If the target application cannot communicate normally, the Ipset set is used as the IP feature set of the target application;

[0078] If the target application can still communicate, the part that the target application can access normally is expanded, and preliminary screening and re-screening are continuously performed through the blocking policy until the application is successfully intercepted. Then, the Ipset sets obtained from each screening are integrated as the IP feature set of the target application.

[0079] There are still many implementation manners of the present invention. All technical solutions formed by equivalent transformation or equivalent substitution fall within the protection scope of the present invention.

Claims

1. An application feature extraction method based on a packet filtering framework, characterized in that: The method includes: By issuing a blocking policy, the relevant information of the target application is preliminarily screened. The domain name information of the blocked target application is used as the preliminary domain name feature. The IP information of the target application that is not blocked is used as the preliminary IP feature after establishing an association with the domain name. Obtain an IP feature set based on preliminary domain name features or preliminary IP features, and re-screen the IP feature set using blocking strategies until the application is successfully intercepted. In the preliminary screening, if the target application is blocked, obtaining an IP feature set through preliminary domain name features or preliminary IP features and re-screening the IP feature set through a blocking strategy includes: Write the preliminary domain name characteristics of the successfully intercepted target application into the dnsmasq file, start the dnsmasq service, and resolve the corresponding IP information; The parsed IP information is stored in the Ipset collection as the IP feature of the target application; As well as including: The IP information corresponding to the domain name obtained in advance in the captured packet data, the IP information associated with the domain name obtained by parsing the HTTPS protocol, and the IP information not associated with the domain name are all stored in the Ipset collection; Use the Iptables tool to issue blocking policies to the Ipset collection; After issuing the blocking policy to the Ipset collection through the Iptables tool, If the target application cannot communicate normally, the Ipset collection will be used as the IP feature set of the target application; if the target application can still communicate, the screening range of the target application will be expanded, and the blocking strategy will be continued to filter until the application is successfully intercepted, and the Ipset collection will be used as the IP feature set of the target application.

2. The application feature extraction method based on a packet filtering framework according to claim 1, wherein: It also includes capturing packets for each application before conducting preliminary screening of relevant information of the target application by issuing a blocking policy, obtaining the correspondence between the domain name and IP information it accesses, forming a basic application database, and extracting domain name data that intersects in multiple applications to generate a blacklist database.

3. The application feature extraction method based on a packet filtering framework according to claim 2, wherein: The steps of preliminarily screening the relevant information of the target application by issuing a blocking policy, using the domain name information of the blocked application as a preliminary domain name feature, and associating the IP information of the unblocked application with the domain name as a preliminary IP feature include: Use the Iptables tool to set up interception instructions for the target application's domain name characteristics when establishing a connection between the target application and the server, and determine whether the target application is successfully intercepted; If the target application is blocked, the domain name information of the target application is obtained and compared with the data in the blacklist database. After excluding the domain name information corresponding to the data in the blacklist database, the remaining domain name information is used as the preliminary domain name feature of the target application; If the target application is not intercepted, the IP information of the target application is obtained, an association between the IP information and the domain name is established, and the IP information associated with the domain name is used as a preliminary IP feature.

4. The application feature extraction method based on a packet filtering framework according to claim 3, wherein: The process of obtaining the IP information of the target application, associating the IP information with the domain name, and using the IP information associated with the domain name as a preliminary IP feature includes the following steps: Extract all IP information from the target application; Filter out IP information that corresponds to the domain name in the target application; Parse the SNI field in the HTTPS protocol, associate the SNI field with the remaining IP information after filtering, generate the relationship characteristics between the IP and the SNI, and match to obtain the domain name information associated with the IP information.

5. The application feature extraction method based on a packet filtering framework according to claim 4, wherein: The filtering of the IP information corresponding to the domain name in the target application includes the following steps: Compare the domain name information in the target application with the data in the basic application database to find the corresponding domain name information in the basic application database; Resolve the domain name information obtained by comparing with the data in the basic application database to generate the corresponding IP information; Filter out the IP information generated by resolving the domain name information from all the IP information in the target application.

6. The application feature extraction method based on a packet filtering framework according to claim 4, wherein: If there is still IP information that is not associated with the domain name after parsing the HTTPS protocol, directly use this IP information as the preliminary IP feature.

7. A method for extracting application features based on a packet filtering framework according to claim 1, characterized in that: The expansion of the screening scope of the target application means: re-obtain the IP information and domain name information of the target application and establish the association between the IP information and the domain name.

Images