Information statistical system and method

Abstract

This application relates to the field of internet communication, providing an information statistics system and method. The system includes a router, a DPI system, a first server, and a second server. The first server is connected to the router via the DPI system deployed at the provincial network's exit point, and the second server is connected to the router via the backbone network. The router is used to mirror DOH requests sent by terminals and / or domain name resolution results returned by the second server based on the DOH requests to the first server via the DPI system. The first server is used to unpack the DOH requests and / or domain name resolution results, and to perform statistical analysis of HttpDns information based on the unpacking results. This application embodiment can conveniently and in real-time obtain DOH requests and domain name resolution results on the operator's side, and can conveniently and accurately perform statistical analysis of HttpDns information, thereby providing users with better services.

Description

Technical Field

[0001] This application relates to the field of Internet communication technology, specifically to an information statistics system and method. Background Technology

[0002] HttpDns is a domain name resolution service that interacts with DNS (Domain Name System) authoritative servers via the HTTP (Hypertext Transfer Protocol) protocol, encapsulating the query message in the HTTP protocol and returning it to the user.

[0003] Because HttpDns is a relatively new service, DNS request and response messages transmitted via the HttpDns protocol are directly exchanged between the client and the service provider. Furthermore, existing service providers can only track their service resolution volume and other data using the version counter mode of BIND software (the most commonly used DNS server software) on their DNS servers. Due to competition and commercial confidentiality mechanisms, service providers cannot share DNS resolution logs; they can only access their own individual DNS resolution log information. This prevents operators from statistically analyzing HttpDns information, i.e., DNS-related request and resolution information.

[0004] In summary, for telecom operators, because HttpDns bypasses the operator's DNS and allows the service provider to communicate directly with the client, the operator cannot accurately collect and analyze HttpDns information, resulting in limited services provided to users. Summary of the Invention

[0005] This application provides an information statistics system and method to solve the technical problem that HttpDns bypasses the operator's DNS and allows the service provider to communicate directly with the client, resulting in the operator being unable to accurately collect and analyze HttpDns information and thus unable to provide better services to users.

[0006] In a first aspect, embodiments of this application provide an information statistics system, including a router, a deep packet inspection (DPI) system, a first server, and a second server;

[0007] The DPI system is deployed at the provincial network's exit point;

[0008] The first server is connected to the router via the DPI system, and the second server is connected to the router via the backbone network;

[0009] The router is used to receive Domain Name Service (DOH) requests based on Hypertext Transfer Protocol sent by the terminal, forward the DOH requests to the second server through the backbone network, and receive the domain name resolution results returned by the second server based on the DOH requests;

[0010] The router is also used to mirror the DOH request and / or the domain name resolution result to the first server through the DPI system;

[0011] The first server is used to unpack the DOH request and / or the domain name resolution result, and to perform statistics on the request information of the DOH request and / or the resolution information of the domain name resolution result based on the unpacking result.

[0012] In one embodiment, the first server includes a listening module, a data packet analysis module, and a query statistics module;

[0013] The monitoring module is used to monitor the DPI system, and if it is determined that the DOH request and / or domain name resolution result exists in the data stream of the DPI system image, it performs protocol conversion on the DOH request and / or domain name resolution result and writes it into the target file;

[0014] The data packet analysis module is used to unpack the DOH request and / or domain name resolution result in the target file;

[0015] The query and statistics module is used to query and statistically analyze the request information of the DOH request and / or the resolution information of the domain name resolution result based on the packet unpacking result, and to return the information statistics result of the DOH request and / or the domain name resolution result when it receives an instruction from the user to query the information statistics result of the DOH request and / or the domain name resolution result.

[0016] In one embodiment, when the unpacking result is the result of unpacking the DOH request, the query statistics module is specifically used to perform statistics on the request information in the DOH request based on the unpacking result, and obtain request information statistics results;

[0017] The request information statistics include one or more combinations of the following: the number of requests corresponding to the source Internet Protocol (IP) address, the number of requests corresponding to the destination Internet Protocol (IP) address, the type of DOH address, and the number of requests for each type of DOH address.

[0018] In one embodiment, when the unpacking result is the result of unpacking the domain name resolution result, the query statistics module is specifically used to perform statistics on the resolution information of the domain name resolution result based on the unpacking result, and obtain the resolution information statistics result;

[0019] The parsing information statistics include one or more combinations of the following: the number of parsing failures, the number of parsing successes, the parsing rate, and the total number of parsing attempts.

[0020] In one embodiment, the first server includes a main process module and a multi-threaded module;

[0021] The main process module is used to provide the first server with a main program for statistically analyzing the request information of the DOH request and / or the resolution information of the domain name resolution result;

[0022] The multi-threaded module is used to provide the first server with multiple threads for parallel statistical analysis of request information for the DOH request and / or resolution information of the domain name resolution result.

[0023] In one embodiment, the router includes a first router, a second router, and a third router;

[0024] Both the second router and the third router are connected to the first router;

[0025] The second router is the router at the provincial network exit point;

[0026] The third router is a router on the backbone network;

[0027] The first router is used to forward the DOH request sent by the terminal hop by hop to the second router, and then forward it to the third router.

[0028] In one embodiment, the second server includes a first Domain Name System (DNS) server and a second DNS server; wherein the first DNS server is a DNS server based on the Hypertext Transfer Protocol (HTTP); wherein the first DNS server is used to receive the DOH request transmitted by the backbone network, and search for the domain name resolution result of the DOH request in the local domain name resolution record; if the domain name resolution result is found, the domain name resolution result is encapsulated and returned to the terminal through the router; if the domain name resolution result is not found, a query request for the domain name information of the DOH request is initiated to the second DNS server;

[0029] The second DNS server is used to query and resolve the domain name information according to the query request of the domain name information, obtain the domain name resolution result, and return the domain name resolution result to the terminal after passing through the first DNS server and the router in sequence.

[0030] In one embodiment, the first server includes an optimization module;

[0031] The optimization module is used to optimize the service strategy provided to users based on the request information of the DOH request and / or the statistical results of the domain name resolution results.

[0032] Secondly, embodiments of this application provide a statistical method based on the information statistics system described in the first aspect, comprising:

[0033] Based on the router, the DOH request sent by the terminal is forwarded to the second server through the backbone network. The second server returns the domain name resolution result according to the DOH request. The DOH request and / or the domain name resolution result are then mirrored to the first server through the DPI system. The DPI system is deployed at the exit of the provincial network.

[0034] Based on the first server, the DOH request and / or the domain name resolution result are unpacked, and based on the unpacking result, the request information of the DOH request and / or the resolution information of the domain name resolution result are statistically analyzed.

[0035] In one embodiment, the step of unpacking the DOH request and / or the domain name resolution result based on the first server, and statistically analyzing the request information of the DOH request and / or the resolution information of the domain name resolution result based on the unpacking result, includes:

[0036] Based on the listening module in the first server, the DPI system is monitored. If it is determined that the DOH request and / or domain name resolution result exists in the data stream of the DPI system image, the DOH request and / or domain name resolution result is converted into a protocol and written to the target file.

[0037] Based on the packet analysis module in the first server, the DOH request and / or domain name resolution result in the target file are unpacked.

[0038] Based on the query statistics module in the first server, statistics are performed on the request information of the DOH request and / or the resolution information of the domain name resolution result according to the packet unpacking result.

[0039] The information statistics system and method provided in this application deploy a DPI system at the provincial network exit to mirror the DOH request sent by the receiving terminal and the domain name resolution result returned by the second server on the service provider side based on the DOH request to the first server on the operator side. The first server unpacks the DOH request and / or the domain name resolution result and performs statistics on the request information of the DOH request and / or the resolution information of the domain name resolution result to obtain the statistical results of HttpDns information. On the one hand, by deploying the DPI system at the provincial network exit, the operator side can conveniently and in real time obtain the DOH request sent by the terminal and the domain name resolution result returned by the second server on the service provider side. On the other hand, the operator side can conveniently and accurately realize the statistics of HttpDns information based on the first server on the operator side, thereby providing users with better services. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a schematic diagram of the structure of an information statistics system provided by existing technology;

[0042] Figure 2 This is a schematic diagram of the structure of the information statistics system provided in the embodiments of this application;

[0043] Figure 3 This is one of the flowcharts illustrating the information statistics method provided in the embodiments of this application;

[0044] Figure 4 This is the second flowchart illustrating the information statistics method provided in the embodiments of this application.

[0045] Figure label:

[0046] 201: Terminal; 202: Router; 203: First Server; 204: First DNS Server; 205: Second DNS Server. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] HTTP-based DNS (HttpDns) is a standard still in the proposal stage, published by the IETF (Internet Engineering Task Force) under RFC 8484. It uses both HTTP and HTTPS (Hypertext Transfer Protocol over Secure Socket Layer) and supports wired DNS response data, such as response data returned in existing UDP (User Datagram Protocol) responses, in HTTPS payloads with MIME (Multipurpose Internet Mail Extensions) types or DNS logs. If HTTP is used, the server can also use HTTP server push to send useful values ​​that it expects the client to discover in advance.

[0049] The major HttpDns service providers in China include two internet giants: First Service Provider and Service Provider.

[0050] (1) HttpDns of the first service provider

[0051] The primary service provider, HttpDns, offers free services and a free API (Application Programming Interface). Both its data requests and responses use the HTTP protocol.

[0052] The format of a DOH (Domain Name System over Hyper Text Transfer Protocol) request is:

[0053] http: / / 119.29.29.29 / d?dn=www.dnspod.cn.&ip=1.1.1.1;

[0054] Where dn is the domain name to be queried; IP (Internet Protocol Address) is the user's IP address. When the IP address is an internal network IP address or an invalid IP address, the source IP address of the HTTP message is taken as the user's IP address by default. In this case, HttpDns will resolve the IP address according to the default line set by the user.

[0055] (2) HttpDns of the second service provider

[0056] Using HttpDns from a second service provider is more cumbersome. It requires activating HttpDns and adding the domain name to be resolved to the domain list in the console. HttpDns is a paid service that provides an open API. For paid cloud DNS users, there is a free domain name resolution limit of 3,000,000 times per calendar month; for other users, there is a free domain name resolution limit of 1,500,000 times per calendar month, with additional charges for exceeding this limit.

[0057] HttpDns provides domain name resolution services to the outside world through an HTTP interface, and the IP address is used directly after the service is connected. There are multiple IP addresses for this service; this description uses the IP address 203.107.1.33 as an example to illustrate how to access the HttpDns service of the second service provider.

[0058] The request method is: HTTP GET

[0059] The URL (Uniform Resource Locator) for the HTTP service is: http: / / 203.107.1.33 / {account_id} / d;

[0060] Table 1. URL Parameter Description

[0061]

[0062] Example 1: With the source IP address set to default, the format of a DOH request is as follows:

[0063] http: / / 203.107.1.33 / 100000 / d?host=www.aliyun.com;

[0064] Here, {account_id} is the HttpDns account that needs to be replaced with the user's ID, which can be obtained from the HttpDns console.

[0065] The parameters of the URL are described in Table 1.

[0066] It should be noted that when accessing the HttpDns service, only one domain name can be resolved per request.

[0067] like Figure 1 As mentioned above, in the prior art, the terminal directly transmits DOH request and response information to the HttpDns server on the service provider side, without going through the DNS server on the operator side. Therefore, the operator side cannot obtain the DOH request and response information, nor can it perform statistics on HttpDns information.

[0068] Furthermore, current carrier standards do not define such data statistics in the DPI (Deep Packet Inspection) specification, and other platforms do not provide complete data mirroring statistics. Therefore, carriers cannot perform operations such as message decoding for these types of DNS resolution requests.

[0069] Furthermore, current technology only performs HttpDns information statistics on the service provider side. Specifically, for request volume and resolution volume statistics, service providers currently mainly use the BIND software on the DNS server to perform the counting. The syntax in the mainstream domain counting software BIND is rndc stats, which is used to write statistical information to a statistics file. The file path name to which the statistical information is appended is defined in the statistics-file section of the detection request options.

[0070] The statistics files generated by BIND9 are similar to, but not exactly the same as, those generated by BIND8. A statistic begins with the line +++Statistics Dump+++(973798949), where the number is a standard UNIX-style timestamp in seconds, starting from January 1, 1970. Following this is a series of lines, including a counter type, counter value, arbitrary domain name, and arbitrary view name. All rows listing views and domains represent overall statistics for the entire server. Rows with domains and views are named with the given view and domain (the view name is defaulted for the default view). The statistic ends with the line ---Statistics Dump---(973798949), where the number is the same as the number on the starting line. Success indicates a successful query on the server or domain.

[0071] As shown in Table 2, a successful query is defined as one that returns a non-error response instead of a recommended response.

[0072] In summary, existing service providers can only use the BIND software version counter mode on DNS servers to count the resolution volume and other data of their respective services. Due to competition and commercial confidentiality mechanisms, service providers cannot share DNS resolution logs and can only have their own DNS resolution log information. This makes it impossible for operators to statistically analyze HttpDns information, resulting in restrictions on the services provided by operators to users.

[0073] Table 2 Definitions of various query commands

[0074] Referral This leads to a recommendation response query. Nxrrset A response to a non-error query that results in no data. Nxdomain This leads to an increase in the number of queries for NXDOMAIN. Recursion The number of queries that cause the server to run recursively to find the final answer. Failure Number of queries that failed

[0075] To address the aforementioned problems, this application provides an information statistics system. See details below. Figure 2 This is a schematic diagram of the structure of an information statistics system provided in this application embodiment, which may include a router 202, a DPI system, a first server 203, and a second server; wherein, the DPI system is deployed at the exit of the provincial network; the first server 203 is connected to the router 202 through the DPI system, and the second server is connected to the router 202 through the backbone network;

[0076] The terminal 201 involved in this application embodiment may be a device that provides voice and / or data connectivity to a user, a handheld device with wireless connectivity, or other processing devices connected to a wireless modem. The name of the terminal device may differ in different systems; for example, in a 5G system, the terminal device may be called User Equipment (UE).

[0077] The terminal 201 can be a mobile terminal or a non-mobile terminal. For example, a mobile terminal can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, wearable device, etc., while a non-mobile terminal can be a server, network attached storage device, personal computer, television set, etc. The embodiments of this application do not make specific limitations.

[0078] The number of routers can be one or more, and this embodiment does not specifically limit this.

[0079] The DPI system is deployed at the exit of the provincial network and is a system that can monitor, analyze and manage Internet traffic in the operator's network channels.

[0080] The first server 203 is a server on the operator's side. There can be one or more servers, and this embodiment does not specifically limit the number. The functions of the first server 203 include, but are not limited to, data statistics and calculation, data parsing, data processing, data transmission and reception, and data management.

[0081] The second server is a server on the service provider's side, and there can be one or more of them. The functions of the first server 203 include, but are not limited to, domain name resolution, data processing, data sending and receiving, and data management.

[0082] Optionally, on the operator's side, terminal 201 is connected to router 202, and router 202 is connected to the first server 203 through the DPI system;

[0083] On the service provider side, terminal 201 is connected to router 202, and router 202 is connected to the second server through the backbone network.

[0084] The router 202 is used to receive the DOH request sent by the terminal 201, forward the DOH request to the second server through the backbone network, and receive the domain name resolution result returned by the second server according to the DOH request;

[0085] In this case, DOH sends DNS query requests via HTTPS.

[0086] Optionally, when router 202 receives a DOH request sent by a user through terminal 201, it can forward the DOH request to the backbone network. The specific forwarding method can be hop-by-hop forwarding, etc. This embodiment does not specifically limit this.

[0087] After receiving a DOH request, the backbone network can forward the DOH request to the service provider's second server;

[0088] After receiving the DOH request, the second server responds to the DOH request, obtains the domain name resolution result corresponding to the DOH request, and returns the domain name resolution result corresponding to the DOH request to router 202 through the backbone network.

[0089] The second server obtains the domain name resolution result corresponding to the DOH request in various ways, including directly searching for it in the local domain name resolution records or sending requests to the root server and authoritative server. This embodiment does not specifically limit the specific methods used in this regard.

[0090] The router 202 is also used to mirror the DOH request and / or the domain name resolution result to the first server 203 through the DPI system.

[0091] Optionally, router 202 can receive both the DOH request sent by terminal 201 and the domain name resolution result returned by the second server. Therefore, router 202 can either mirror the DOH request sent by terminal 201 to the first server 203 through the DPI system, or mirror the domain name resolution result returned by the second server to the first server 203 through the DPI system. This embodiment does not specifically limit this.

[0092] Through the above methods, the operator can obtain the DOH requests sent by the terminal and the domain name resolution results returned by the service provider's second server in real time. This allows for accurate statistical analysis of HttpDns information, namely the request information of the DOH request and / or the resolution information of the domain name resolution results, thus providing users with better services.

[0093] The first server 203 is used to unpack the DOH request and / or the domain name resolution result, and to perform statistics on the request information of the DOH request and / or the resolution information of the domain name resolution result based on the unpacking result.

[0094] Optionally, after the first server 203 obtains the DOH request, it can unpack and parse the DOH request to obtain the data content in the DOH request, including but not limited to the source IP address, destination IP address, service provider type, DoH type and domain name type contained in the DoH request. This embodiment does not specifically limit this.

[0095] Then, based on the data content in the DOH request obtained after unpacking, the request information of the DOH request is statistically analyzed to obtain the request information statistical results.

[0096] The statistical results of the request information include, but are not limited to, the number of requests corresponding to the source IP address, the number of requests corresponding to the destination IP address, the types of DOH addresses, and the number of requests for each type of DOH address. This embodiment does not specifically limit these.

[0097] And / or, after the first server 203 obtains the domain name resolution result, it can unpack and parse the domain name resolution result to obtain the data content in the domain name resolution result, including but not limited to resolution information, service provider information and domain name information, etc. This embodiment does not specifically limit this.

[0098] Then, based on the data content in the domain name resolution results obtained after unpacking, the resolution information of the domain name resolution results is statistically analyzed.

[0099] It should be noted that the HttpDns information statistics method for domain name resolution results and the HttpDns information statistics method for DOH requests can be the same or different, and this embodiment does not make specific limitations on this.

[0100] This embodiment deploys a DPI system at the provincial network's exit point to mirror the DOH requests sent by the receiving terminal and the domain name resolution results returned by the service provider's second server based on the DOH requests to the operator's first server. The first server then unpacks the DOH requests and / or domain name resolution results and performs statistical analysis on the request information of the DOH requests and / or the resolution information of the domain name resolution results to obtain statistical results of HttpDns information. On the one hand, by deploying the DPI system at the provincial network's exit point, the operator can conveniently and in real-time obtain the DOH requests sent by the terminal and the domain name resolution results returned by the service provider's second server. On the other hand, based on the operator's first server, HttpDns information statistics can be conveniently and accurately implemented, thereby providing users with better services.

[0101] Based on the above embodiments, in this embodiment, the first server 203 includes a listening module, a data packet analysis module, and a query statistics module. The listening module is used to listen to the DPI system, and when it is determined that the DOH request and / or domain name resolution result exists in the data stream of the DPI system image, it performs protocol conversion on the DOH request and / or domain name resolution result and writes it into a target file. The data packet analysis module is used to unpack the DOH request and / or domain name resolution result in the target file. The query statistics module is used to query and statistically analyze the request information of the DOH request and / or the resolution information of the domain name resolution result based on the unpacking result, and when it receives an instruction from a user to query the information statistics result of the DOH request and / or the domain name resolution result, it returns the information statistics result of the DOH request and / or the domain name resolution result.

[0102] The monitoring module is a management tool for monitoring network status, data flow, and information transmission on the network.

[0103] The data packet analysis module has a variety of built-in data parsing algorithms for parsing and depackaging data.

[0104] The query statistics module incorporates various data processing algorithms and data query interfaces for data statistics and query services. Specifically, it may include a DNS query statistics module, used for querying and statistically analyzing information related to domain names.

[0105] The first server 203 is equipped with a listening module that can monitor the network card messages of the DPI system in real time; and determine in real time whether there are DOH requests and domain name resolution results in the data stream of the DPI system image based on the network card messages; specifically, it can determine whether the data stream of the DPI system image contains strings related to DOH and strings related to domain name resolution.

[0106] If it is determined that there is a DOH request and / or domain name resolution result in the data stream of the DPI system image, the DOH request and / or domain name resolution result are converted into a data format that can be recognized by the first server 203, and the converted DOH request and / or domain name resolution result is written to the target file.

[0107] Because data needs to be packaged or encapsulated during transmission, i.e., data protocols are added to its header or trailer, a packet analysis module is set up in the first server 203 to unpack the DOH requests and / or domain name resolution results in the target file in order to obtain the valid data content of the DOH requests and / or domain name resolution results. That is, the unpacking result only contains valid data content.

[0108] After obtaining valid data content of the DOH request and / or domain name resolution result, the first server 203 uses the query statistics module to perform statistics on the request information of the DOH request and / or the resolution information of the domain name resolution result based on the packet splitting result, and obtains the statistical results of HttpDns information.

[0109] When the first server 203 receives a user's instruction to query the statistical results of HttpDns information, it can query the corresponding statistical results according to the instruction and return the statistical results to the user. Specifically, the first server 203 can return the statistical results to the terminal 201 via router 202.

[0110] This embodiment, by setting up a listening module, a packet analysis module, and a query statistics module in the first server, can not only monitor network card information in real time, but also, when it is determined that there are DOH requests and / or domain name resolution results in the data stream of the DPI system image, perform real-time and accurate parsing of DOH requests and / or domain name resolution results and statistical analysis of the request information of DOH requests and / or the parsing information of the domain name resolution results.

[0111] Based on the above embodiments, in this embodiment, when the unpacking result is the result of unpacking the DOH request, the query statistics module is specifically used to perform statistics on the request information in the DOH request based on the unpacking result, and obtain request information statistics results; wherein, the request information statistics results include one or more combinations of the following: the number of requests corresponding to the source Internet Protocol IP address, the number of requests corresponding to the destination Internet Protocol IP address, the type of DOH address, and the number of requests for each type of DOH address.

[0112] The statistical results of the request information include, but are not limited to, the number of requests corresponding to the source IP address (i.e., the IP address of the sender of the DOH request) and the number of requests corresponding to the destination IP address (i.e., the IP address of the receiver of the DOH request) within a preset time period, as well as the type of DOH address and the number of requests for each type of DOH address. This embodiment does not specifically limit this, but may also include the total number of DOH requests from HttpDns within the preset time period.

[0113] Optionally, if the unpacking result is the result of unpacking the DOH request, the query and statistics module can query data that meets the statistical conditions of the request information based on the unpacking result. If the corresponding data is found, the request information statistics result is updated; if the corresponding data is not found, the above query and statistics steps are repeated to obtain the final statistical result of the request information, and the final statistical result of the request information is printed out.

[0114] When counting the total number of DOH requests for HttpDns within a preset time period, a script, such as one written in Python, can be used to count the total number of HttpDns DOH requests within that period, thus obtaining the ratio of HttpDns access requests to traditional DNS requests. Alternatively, the count can be based on the destination address of an internet mirror, for example, by counting the number of HTTP GET requests destined for 119.29.29.29.

[0115] The following description uses the example of counting the number of HTTP GET requests destined for 119.29.29.29 to illustrate the HttpDns information statistics method in this embodiment.

[0116] First, the main process is started. The main program is primarily built upon modules loaded from class and function packages, and constructed using the `main()` function. An example of such code is shown below:

[0117] from scapy.all import * / / Load all packages in the scapy package for data processing;

[0118] import re / / Load all packages in the regular expression re package;

[0119] import os / / Load OS packages;

[0120] if __name__ == '__main__':

[0121] main()

[0122] Then, monitor the network card information to determine whether the DOH request exists in the data stream of the DPI system image;

[0123] After locating the HTTP GET request with the destination address 119.29.29.29 in the DOH request, write the DOH request to the target file; an example code is: def GET_print(packet):

[0124] domain = str(packet)

[0125] http_method = domain.split('\r\n')[0] / / Packet splitting;

[0126] relink='GET / d\? dn=(.*)&ip='

[0127] relink1 = '119.29.29.29'

[0128] domain1 = re.findall(relink, domain) / / Returns all DOH request data in the packet;

[0129] newhost = re.findall(relink1,domain) / / Returns all data in the DOH request with a destination address of 119.29.29.29;

[0130] fo = open("tongji.txt", "r+") / / Open the target file;

[0131] print("filename:", fo.name) / / Output the filename;

[0132] # Write a line at the end of the file;

[0133] fo.seek(0,2)

[0134] line = fo.write(str(newhost)) / / Write to the file;

[0135] fo.seek(0,2)

[0136] line = fo.write(str(domain1)) / / Write to file;

[0137] dns_tongji() / / Calls the query statistics function;

[0138] Finally, count the number of HTTP GET requests destined for 119.29.29.29 in the target file; an example code is as follows:

[0139] def dns_tongji(): / / Definition of query statistics function;

[0140] ipstr = "119.29.29.29"

[0141] fo = open("tongji.txt", "r+") / / Open the target file;

[0142] print("File name:", fo.name) / / Outputs the target file name;

[0143] line = fo.read(-1) / / Read the target file;

[0144] print("String read: %s"%(line)) / / Output the string read;

[0145] cnt = line.count(ipstr) / / Count the number of addresses with a destination address of 119.29.29.29;

[0146] print("Statistical count is",cnt) / / Output statistical results;

[0147] In this embodiment, the query statistics module can conveniently and accurately obtain rich statistical results of DOH request information and domain name resolution information from DOH requests, thereby providing users with better services.

[0148] Based on the above embodiments, in this embodiment, when the unpacking result is the result of unpacking the domain name resolution result, the query statistics module is specifically used to perform statistics on the resolution information of the domain name resolution result according to the unpacking result, and obtain the resolution information statistics result; wherein, the resolution information statistics result includes one or more combinations of the following: the number of resolution failures, the number of resolution successes, the resolution rate, and the total number of resolutions.

[0149] The statistical results of the resolution information include, but are not limited to, the number of times the resolution failed, the number of times the resolution succeeded, the resolution rate, and the total number of resolutions during the resolution process of the domain name within a preset time period. This embodiment does not specifically limit these, but may also include the number of times all domain names are resolved by the service provider side that provides domain name resolution services within the preset time period.

[0150] Optionally, if the unpacking result is the result of unpacking the domain name resolution result, the query statistics module can query data that meets the statistical conditions of the resolution information based on the unpacking result. If the corresponding data is found, the resolution information statistics result is updated; if the corresponding data is not found, the above query statistics steps are repeated to obtain the final statistical result of the resolution information, and the final statistical result of the resolution information is printed out.

[0151] In this embodiment, the query statistics module can conveniently and accurately obtain rich statistical results of resolution information from the domain name resolution results, thereby providing users with better services.

[0152] Based on the above embodiments, in this embodiment, the first server 203 includes a main process module and a multi-threaded module; wherein, the main process module is used to provide the first server 203 with a main program for statistically analyzing the request information of the DOH request and / or the resolution information of the domain name resolution result; the multi-threaded module is used to provide the first server 203 with multiple threads for parallel statistical analysis of the request information of the DOH request and / or the resolution information of the domain name resolution result.

[0153] Optionally, the first server 203 may also include a main process module and a multi-threaded module.

[0154] The main process module is a main program used to provide the first server 203 with the request information of the DOH request and / or the resolution information of the domain name resolution result for statistics.

[0155] To accelerate the efficiency of data statistics, a multi-threaded module can be set up in the first server 203 to provide multiple threads for parallel statistics on the request information of DOH requests and / or the resolution information of the domain name resolution results for the first server 203, thereby quickly obtaining the information statistics results.

[0156] Based on the above embodiments, the router 202 in this embodiment includes a first router, a second router, and a third router; wherein the second router and the third router are both connected to the first router; the second router is a router at the provincial network exit; the third router is a router on the backbone network; the first router is used to forward the DOH request sent by the terminal 201 hop by hop to the second router, i.e., the PB router (provincial network exit router), and then forward it to the third router, i.e., the BC (backbone) router.

[0157] Optionally, the first router is set on the user side to receive the DOH request sent by the terminal 201, and forward the DOH request sent by the terminal 201 hop by hop to the second router, and then forward it to the third router;

[0158] The second router is the router at the provincial network exit, used to mirror the DOH request forwarded by the first router to the first server 203 on the operator side through the DPI system;

[0159] The third router is a router on the backbone network, used to forward the DOH request to the service provider's second server via the backbone network after receiving the DOH request forwarded by the first router.

[0160] Accordingly, when transmitting the domain name resolution result, the third router is used to forward the domain name resolution result to the second router via the backbone network after receiving the domain name resolution result returned by the second server, and then forward it to the first router.

[0161] After receiving the domain name resolution result forwarded by the third router, the second router mirrors it to the first server 203 on the operator's side through the DPI system.

[0162] This embodiment enables efficient transmission of data streams between the service provider and operator sides by setting up multiple routers, allowing both the service provider and operator sides to obtain domain name resolution results and DOH requests in real time.

[0163] Based on the above embodiments, in this embodiment, the second server includes a first DNS server 204 and a second DNS server 205; wherein, the first DNS server 204 is used to receive the DOH request transmitted by the backbone network, and search for the domain name resolution result of the DOH request in the local domain name resolution record. If the domain name resolution result is found, the domain name resolution result is encapsulated and returned to the terminal 201 through the router 202; if the domain name resolution result is not found, a query request for the domain name information of the DOH request is initiated to the second DNS server 205; the second DNS server 205 is used to query and resolve the domain name information according to the query request for the domain name information, obtain the domain name resolution result, and return the domain name resolution result to the terminal 201 after passing through the first DNS server 204 and the router 202 in sequence.

[0164] The first DNS server is a new type of DNS server based on the Hypertext Transfer Protocol, also known as HttpDns server; the second DNS server is a traditional DNS server, such as a DNS recursive server.

[0165] Optionally, according to the API (Application Programming Interface) open specification, the DOH resolution function is encapsulated in the APP (Application). When the terminal 201 application needs to resolve the domain name, it needs to call the DOH API interface to send a request to the first DNS server 204, and the first DNS server 204 returns the domain name resolution result to the terminal 201.

[0166] like Figure 2 In the case where the second server includes both the first DNS server 204 and the second DNS server, the specific business process for the information statistics system to collect statistics on HttpDns information, i.e., request information of DOH requests and / or resolution information of the domain name resolution results, is as follows:

[0167] For the service provider side:

[0168] (1) The user sends a DOH request through the terminal, which is forwarded hop by hop by the first router to the second router, i.e., the PB router, and then to the third router, i.e., the BC router.

[0169] (2) The third router transmits the DOH request to the service provider's first DNS server 204 through the backbone network. After receiving the DOH request, the first DNS 204 first searches for the domain name resolution result of the DOH request in the local domain name resolution record, that is, checks whether there is a cache of the domain name resolution record of the DOH request locally. If the domain name resolution result is found, the domain name resolution result is directly encapsulated into the HTTP protocol and then returned to the terminal directly through the router.

[0170] (3) If the first DNS server 204 does not cache the domain name resolution result of the DOH request, that is, if no domain name resolution result is found, the first DNS server 204 calls its own DNS service configuration information to initiate a domain name query request for the DOH request on UDP port 53 to the second DNS server (DNS recursive server); the second DNS server checks whether it has cached the domain name information to be queried for the DOH request locally. If the domain name information to be queried is found, it will provide a normal resolution response for the domain name to be queried for the DOH request; if the domain name information to be queried does not exist, it will initiate a recursive query to query the IP address corresponding to the domain name to obtain the domain name resolution result of the DOH request from the root server and the authoritative server.

[0171] (4) The second DNS server 205 encapsulates the domain name resolution result of the DOH request obtained from the query into a DNS response request and sends it back to the first DNS server 204.

[0172] (5) The first DNS server encapsulates the DNS response request in the HTTP encapsulation format and returns the domain name resolution result to the Internet.

[0173] (6) The router sends the domain name resolution results back to the customer.

[0174] For the operator's side:

[0175] (1') The user sends a DOH request through the terminal, which is forwarded hop by hop by the first router to the second router.

[0176] (2') The second router mirrors the HTTP request (HTTP GET) traffic of the specific IP address in the DOH request to the first server through the DPI system deployed at the provincial network exit in the middle.

[0177] (3') The first server obtains statistical data of the request information of the DOH request by unpacking the application layer data (i.e., DOH request).

[0178] (4') The first DNS server returns the domain name resolution result, which is then received by the second router. The second router mirrors the domain name resolution result to the first server via the DPI system deployed at the provincial network exit.

[0179] (5') The first server obtains statistical data on the resolution information of the domain name resolution results by disassembling the domain name resolution results.

[0180] This embodiment can obtain the domain name resolution results of DOH requests in real time through the second server, and can mirror the domain name resolution results of DOH requests to the operator side in real time through the DPI system, effectively making up for the shortcomings of the existing operator side in being unable to obtain HttpDns-type DNS resolution data and unable to perform statistics on DNS resolution data.

[0181] Based on the above embodiments, the first server in this embodiment further includes an optimization module; the optimization module is used to optimize the service strategy provided to the user based on the request information of the DOH request and / or the statistical results of the resolution information of the domain name resolution result.

[0182] Optionally, the first server may also include an optimization module;

[0183] The optimization module can optimize the service strategy provided to users based on the request information of the DOH request and / or the statistical results of the domain name resolution results.

[0184] For example, based on the request information of the DOH request and / or the statistical results of the domain name resolution results, the development trend of HttpDns-type services and their development ratio with traditional DNS services can be obtained. Then, based on the development trend and development ratio, it can be determined whether the current service strategy needs to be optimized and the development of HttpDns-type services should be expanded.

[0185] For example, operators can also optimize service policies by blocking specific domain names based on the request information of DOH requests and / or the statistical results of the domain name resolution results, thereby improving network security.

[0186] Specifically, for domain name blocking with specific requirements, the operator can analyze such DOH request packets to obtain the domain name requested by the DOH request; if the domain name of the DOH request is detected to be an anti-fraud domain name, an RST (ReSeT) response can be released to block it.

[0187] In this embodiment, the operator can not only obtain the request information of DOH requests and / or the statistical results of the domain name resolution results in real time, but also optimize the service strategy provided to users based on the request information of DOH requests and / or the statistical results of the domain name resolution results, thereby providing users with more secure and reliable services.

[0188] The information statistics method provided in the embodiments of this application is described below. The information statistics method described below can be referred to in correspondence with the information statistics system described above.

[0189] like Figure 3 The image shows an information statistics method provided in an embodiment of this application. The method includes the following steps:

[0190] Step 301: Based on the router, forward the DOH request sent by the terminal to the second server through the backbone network, receive the domain name resolution result returned by the second server according to the DOH request, and mirror the DOH request and / or the domain name resolution result to the first server through the DPI system; wherein, the DPI system is deployed at the exit of the provincial network.

[0191] Optionally, when the router receives a DOH request sent by a user through a terminal, it can forward the DOH request to the backbone network. The specific forwarding method can be hop-by-hop forwarding, etc. This embodiment does not specifically limit this.

[0192] After receiving a DOH request, the backbone network can forward the DOH request to the service provider's second server;

[0193] After receiving the DOH request, the second server responds to the DOH request, obtains the domain name resolution result corresponding to the DOH request, and returns the domain name resolution result corresponding to the DOH request to the router through the backbone network.

[0194] The second server obtains the domain name resolution result corresponding to the DOH request in various ways, including directly searching for it in the local domain name resolution records or sending requests to the root server and authoritative server. This embodiment does not specifically limit the specific methods used in this regard.

[0195] Optionally, the router can receive both the DOH request sent by the terminal and the domain name resolution result returned by the second server. Therefore, the router can either mirror the DOH request sent by the terminal to the first server through the DPI system, or mirror the domain name resolution result returned by the second server to the first server through the DPI system. This embodiment does not specifically limit this.

[0196] Through the above methods, the operator can obtain the DOH requests sent by the terminal and the domain name resolution results returned by the service provider's second server in real time. This allows for accurate statistical analysis of HttpDns information, namely the request information of the DOH request and / or the resolution information of the domain name resolution results, thus providing users with better services.

[0197] Step 302: Based on the first server, the DOH request and / or the domain name resolution result are unpacked, and the request information of the DOH request and / or the resolution information of the domain name resolution result are statistically analyzed according to the unpacking result.

[0198] Optionally, after the first server obtains the DOH request, it can unpack and parse the DOH request to obtain the data content in the DOH request, including but not limited to the source IP address, destination IP address, service provider type, DoH type and domain name type contained in the DoH request. This embodiment does not specifically limit this.

[0199] Then, based on the data content in the DOH request obtained after unpacking, the request information of the DOH request is statistically analyzed to obtain the request information statistical results.

[0200] The statistical results of the request information include, but are not limited to, the number of requests corresponding to the source IP address, the number of requests corresponding to the destination IP address, the types of DOH addresses, and the number of requests for each type of DOH address. This embodiment does not specifically limit these.

[0201] And / or, after the first server obtains the domain name resolution result, it can unpack and parse the domain name resolution result to obtain the data content in the domain name resolution result, including but not limited to resolution information, service provider information and domain name information, etc. This embodiment does not specifically limit this.

[0202] Then, based on the data content in the domain name resolution results obtained after unpacking, the resolution information of the domain name resolution results is statistically analyzed.

[0203] This embodiment deploys a DPI system at the provincial network's exit point to mirror the DOH requests sent by the receiving terminal and the domain name resolution results returned by the service provider's second server based on the DOH requests to the operator's first server. The first server then unpacks the DOH requests and / or domain name resolution results and performs statistical analysis on the request information of the DOH requests and / or the resolution information of the domain name resolution results to obtain statistical results of HttpDns information. On the one hand, by deploying the DPI system at the provincial network's exit point, the operator can conveniently and in real-time obtain the DOH requests sent by the terminal and the domain name resolution results returned by the service provider's second server. On the other hand, based on the operator's first server, HttpDns information statistics can be conveniently and accurately implemented, thereby providing users with better services.

[0204] Based on the above embodiments, in this embodiment, the process of unpacking the DOH request and / or the domain name resolution result based on the first server, and statistically analyzing the request information of the DOH request and / or the resolution information of the domain name resolution result based on the unpacking result, includes: monitoring the DPI system based on the monitoring module in the first server; if the DOH request and / or domain name resolution result are found in the data stream of the DPI system image, performing protocol conversion on the DOH request and / or domain name resolution result and writing it into a target file; unpacking the DOH request and / or domain name resolution result in the target file based on the data packet analysis module in the first server; and statistically analyzing the request information of the DOH request and / or the resolution information of the domain name resolution result based on the query statistics module in the first server.

[0205] like Figure 4 As shown, the specific steps for statistical analysis of HttpDns information based on the first server include:

[0206] Step 401: Start the main process;

[0207] Step 402: Listen to the network card messages of the DPI system; specifically, based on the listening module in the first server, it can listen to the network card messages of the DPI system in real time.

[0208] Step 403: Based on the network card message, determine in real time whether there is a DOH request and a domain name resolution result in the data stream of the DPI system image; specifically, determine whether the data stream of the DPI system image contains strings related to DOH and strings related to domain name resolution.

[0209] Step 404: If it is determined that there is a DOH request and / or domain name resolution result in the data stream of the DPI system image, perform protocol conversion on the DOH request and / or domain name resolution result to convert the DOH request and / or domain name resolution result into a data format that the first server can recognize, and write the protocol-converted DOH request and / or domain name resolution result into the target file.

[0210] Because data needs to be packaged or encapsulated during transmission, i.e., data protocols are added to its header or trailer, the packet analysis module in the first server can be used to unpack the DOH requests and / or domain name resolution results in the target file to obtain the valid data content of the DOH requests and / or domain name resolution results. In other words, the unpacking result only contains valid data content.

[0211] Step 405: After obtaining the valid data content of the DOH request and / or domain name resolution result, the query and statistics module in the first server can perform statistics on the request information of the DOH request and / or the resolution information of the domain name resolution result, i.e., HttpDns information, based on the packet unpacking results, such as counting the number of each DOH address, to obtain the statistical results of HttpDns information.

[0212] Step 406: Print the statistical results of the HttpDns information.

[0213] It should be noted that when the first server receives a user's instruction to query the statistical results of HttpDns information, it can query the corresponding statistical results according to the instruction and return the results to the user. Specifically, the first server can return the statistical results to the terminal via a router.

[0214] This embodiment, by setting up a listening module, a packet analysis module, and a query statistics module in the first server, can not only monitor network card information in real time, but also, when it is determined that there are DOH requests and / or domain name resolution results in the data stream of the DPI system image, accurately parse the DOH requests and / or domain name resolution results and perform HttpDns information statistics in real time.

[0215] It should be noted that the information statistics method provided in this application embodiment can achieve all the functions implemented in the above system embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the system embodiment and the beneficial effects will not be described in detail.

[0216] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0217] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0218] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An information statistics system, characterized in that, This includes routers, a deep packet inspection (DPI) system, a first server, and a second server. The DPI system is deployed at the provincial network's exit point; The first server is a carrier-side server, connected to the router through the DPI system; the second server is a service provider-side server, connected to the router through the backbone network. The router is used to receive Domain Name Service (DOH) requests based on Hypertext Transfer Protocol sent by the terminal, forward the DOH requests to the second server through the backbone network, and receive the domain name resolution results returned by the second server based on the DOH requests; The router is also used to mirror the DOH request and / or the domain name resolution result to the first server through the DPI system; The first server is used to perform protocol conversion on the DOH request and / or the domain name resolution result, then unpack the data, and statistically analyze the request information of the DOH request and / or the resolution information of the domain name resolution result based on the unpacking result.

2. The information statistics system according to claim 1, characterized in that, The first server includes a listening module, a data packet analysis module, and a query statistics module; The monitoring module is used to monitor the DPI system, and if it is determined that the DOH request and / or domain name resolution result exists in the data stream of the DPI system image, it performs protocol conversion on the DOH request and / or domain name resolution result and writes it into the target file; The data packet analysis module is used to unpack the DOH request and / or domain name resolution result in the target file; The query and statistics module is used to query and statistically analyze the request information of the DOH request and / or the resolution information of the domain name resolution result based on the packet unpacking result, and to return the information statistics result of the DOH request and / or the domain name resolution result when it receives an instruction from the user to query the information statistics result of the DOH request and / or the domain name resolution result.

3. The information statistics system according to claim 2, characterized in that, When the unpacking result is the result of unpacking the DOH request, the query statistics module is specifically used to perform statistics on the request information in the DOH request based on the unpacking result, and obtain request information statistics results; The request information statistics include one or more combinations of the following: the number of requests corresponding to the source Internet Protocol (IP) address, the number of requests corresponding to the destination Internet Protocol (IP) address, the type of DOH address, and the number of requests for each type of DOH address.

4. The information statistics system according to claim 2, characterized in that, When the unpacking result is the result of unpacking the domain name resolution result, the query statistics module is specifically used to perform statistics on the resolution information of the domain name resolution result based on the unpacking result, and obtain the resolution information statistics result; The parsing information statistics include one or more combinations of the following: the number of parsing failures, the number of parsing successes, the parsing rate, and the total number of parsing attempts.

5. The information statistics system according to any one of claims 1-4, characterized in that, The first server includes a main process module and a multi-threaded module; The main process module is used to provide the first server with a main program for statistically analyzing the request information of the DOH request and / or the resolution information of the domain name resolution result; The multi-threaded module is used to provide the first server with multiple threads for parallel statistical analysis of request information for the DOH request and / or resolution information of the domain name resolution result.

6. The information statistics system according to any one of claims 1-4, characterized in that, The router includes a first router, a second router, and a third router; Both the second router and the third router are connected to the first router; The second router is the router at the provincial network exit point; The third router is a router on the backbone network; The first router is used to forward the DOH request sent by the terminal hop-by-hop to the second router, and then forward it to the third router.

7. The information statistics system according to any one of claims 1-4, characterized in that, The second server includes a first Domain Name System (DNS) server and a second Domain Name System (DNS) server; The first DNS server is a DNS server based on the Hypertext Transfer Protocol; The first DNS server is used to receive the DOH request transmitted by the backbone network, and search for the domain name resolution result of the DOH request in the local domain name resolution record. If the domain name resolution result is found, the domain name resolution result is encapsulated and returned to the terminal through the router; if the domain name resolution result is not found, a query request for the domain name information of the DOH request is initiated to the second DNS server. The second DNS server is used to query and resolve the domain name information according to the query request of the domain name information, obtain the domain name resolution result, and return the domain name resolution result to the terminal after passing through the first DNS server and the router in sequence.

8. The information statistics system according to any one of claims 1-4, characterized in that, The first server includes an optimization module; The optimization module is used to optimize the service strategy provided to users based on the request information of the DOH request and / or the statistical results of the domain name resolution results.

9. A statistical method based on the information statistics system according to any one of claims 1-8, characterized in that, include: Based on the router, the DOH request sent by the terminal is forwarded to the second server through the backbone network. The second server returns the domain name resolution result according to the DOH request. The DOH request and / or the domain name resolution result are then mirrored to the first server through the DPI system. The DPI system is deployed at the exit of the provincial network. Based on the first server, the DOH request and / or the domain name resolution result are unpacked, and based on the unpacking result, the request information of the DOH request and / or the resolution information of the domain name resolution result are statistically analyzed.

10. The statistical method according to claim 9, characterized in that, The step involves, based on the first server, disassembling the DOH request and / or the domain name resolution result, and, based on the disassembly result, statistically analyzing the request information of the DOH request and / or the resolution information of the domain name resolution result, including: Based on the listening module in the first server, the DPI system is monitored. If it is determined that the DOH request and / or domain name resolution result exists in the data stream of the DPI system image, the DOH request and / or domain name resolution result is converted into a protocol and written to the target file. Based on the packet analysis module in the first server, the DOH request and / or domain name resolution result in the target file are unpacked. Based on the query statistics module in the first server, statistics are performed on the request information of the DOH request and / or the resolution information of the domain name resolution result according to the packet unpacking result.

Images