A method and system for mining industry relationship chains based on protocol traffic analysis

Abstract

The present invention discloses a method and system for mining industry relationship chains based on protocol traffic analysis, which specifically includes the following steps: (1) obtaining traffic to be analyzed; (2) constructing a regular expression capable of identifying traffic containing industry product numbers; using the regular expression to perform preliminary filtering on the obtained traffic; (3) performing secondary manual screening according to HTTP protocol, HTTPS protocol and private protocol to obtain the server information corresponding to each traffic; through the server's IP address, domain name information or keywords in the traffic, it can be analyzed and determined whether each traffic is generated by equipment belonging to the industry; (4) further verifying the equipment assets that respond to the traffic based on APN, IPC filing, message load content and action trajectory; (5) drawing an industry relationship map based on the verified equipment assets and their servers. This method can discover the mutual correlation between business traffic between terminals, platforms and users in the Internet of Things industry from massive traffic and obtain an industry relationship map.

Description

Technical Field

[0001] The present invention relates to the technical field of the Internet of Things and Internet of Things traffic analysis and restoration, and in particular to an industry relationship chain mining method and system based on protocol traffic analysis. Background Art

[0002] Currently, existing techniques connect the device or app to be analyzed to a packet capture environment. Using packet capture tools like Wireshark or Fiddler, the corresponding network card is selected to capture and analyze the application packets. Due to the unique operating environments of IoT devices, IoT devices cannot be directly connected to the packet capture environment. Consequently, traditional analysis methods cannot directly analyze IoT device traffic. Even with existing network traffic analysis, they cannot accurately locate IoT devices within the massive amount of traffic, making traditional analysis methods ineffective in the IoT field. Summary of the Invention

[0003] Purpose of the Invention: To address the above shortcomings, the present invention provides an industry relationship chain mining method based on protocol traffic analysis. This method can discover the interrelationships between business traffic between IoT terminals, platforms, and users from massive amounts of traffic, thereby obtaining valuable industry relationship chain information. The present invention also provides an industry relationship chain mining system based on protocol traffic analysis for obtaining valuable industry relationship chain information.

[0004] Technical solution: To solve the above problems, the present invention provides an industry relationship chain mining method based on protocol traffic analysis, which specifically includes the following steps:

[0005] (1) Obtain the traffic to be analyzed through mirroring;

[0006] (2) Analyze the product number of the industry as needed and construct a regular expression; the regular expression can identify the traffic containing the product number; use the regular expression to perform preliminary filtering on the obtained traffic;

[0007] (3) For the traffic after preliminary filtering, a second manual screening is performed based on the HTTP protocol, HTTPS protocol and private protocol to obtain the server information corresponding to each flow; the server IP address, domain name information or keywords in the traffic can be used to analyze and determine whether each flow is generated by the equipment in the industry;

[0008] (4) Further verifying the equipment assets reflected in each flow based on the APN, IPC record, message payload content, and action trajectory to determine whether they belong to the industry; the action trajectory is obtained based on the base station data contained in the signaling packet in the flow;

[0009] (5) Draw an industry relationship map based on the equipment assets reflected by each verified flow and its corresponding server information.

[0010] Furthermore, the method further includes step (6) of recording the source IP address of the verified device asset or a type of device asset, filtering the source IP address as a filtering condition, filtering out all traffic of the device or the type of device, and then analyzing the cooperative relationship with the manufacturer.

[0011] Furthermore, the secondary manual screening specifically obtains the server corresponding to each flow based on the host of the HTTP protocol, the sni of the HTTPS protocol and the destination IP address of the private protocol.

[0012] Furthermore, all traffic of the device or the device of this type is obtained by tracing the IMSI of the device.

[0013] Furthermore, the analysis of the cooperative relationship with the manufacturer specifically obtains the visited website data based on the device's Internet access log, obtains the data of the device's integrated applications and interfaces through the access records, and then analyzes the cooperative relationship between the device and the manufacturer.

[0014] Furthermore, the product number is a frame number or a courier cabinet number.

[0015] Beneficial Effects: Compared with the existing technology, the present invention has the following significant advantages: 1. By setting regular expressions and various protocols to screen traffic, it is possible to obtain traffic information generated by equipment in the industry to be analyzed. After further verification, it can be determined that the analyzed traffic is generated by equipment in the industry chain; then, based on the device asset information and corresponding server information included in the traffic, a corresponding industry map can be drawn; 2. By designing the conditions of the source IP address, all traffic of a verified device or a type of device can be extracted, and then the cooperating manufacturers can be obtained.

[0016] In addition, the present invention also provides an industry relationship chain mining system based on protocol traffic analysis, which specifically includes:

[0017] Traffic acquisition module, used to obtain the traffic to be analyzed through mirroring;

[0018] A preliminary screening module is used to analyze the product number of the industry as needed and construct a regular expression; the regular expression can identify the traffic containing the product number; and the regular expression is used to perform preliminary filtering on the obtained traffic;

[0019] The manual screening and statistics module is used to perform secondary manual screening on the traffic after preliminary filtering based on HTTP, HTTPS and private protocols to obtain the server information corresponding to each flow. The module can analyze and determine whether each flow is generated by the equipment in the industry through the server's IP address, domain name information or keywords in the flow.

[0020] The device asset verification module is used to further verify the device assets reflected in each flow based on the APN, IPC record, message payload content, and action trajectory to determine whether they belong to the industry; the action trajectory is obtained based on the base station data contained in the signaling packet in the flow;

[0021] The relationship spectrum drawing module is used to draw the industry relationship map based on the device assets reflected in each verified flow and their corresponding server information.

[0022] Furthermore, it also includes: a cooperative relationship analysis module, which is used to record the source IP address of a verified device asset or a type of device asset, use the source IP address as a filtering condition, filter out all traffic of the device or the type of device, and then analyze the cooperative relationship with the manufacturer.

[0023] Furthermore, the secondary manual screening in the manual screening statistics module specifically obtains the server information corresponding to each flow based on the host of the HTTP protocol, the sni of the HTTPS protocol and the destination IP address of the private protocol.

[0024] Furthermore, all traffic of the device or this type of device described in the partnership analysis module is obtained by tracing the device's IMSI; the analysis of the partnership with the manufacturer described in the partnership analysis module specifically obtains the visited website data based on the device's Internet log, obtains the data of the device's integrated applications and interfaces through the access records, and analyzes the partnership with the manufacturer.

[0025] Beneficial effect: Compared with the existing technology, the significant advantage of the present invention is that it uses this system to obtain valuable industry relationship chain information from massive traffic. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 Shown is a flow chart of the method of the present invention;

[0027] Figure 2 Shown is a schematic diagram of the results of filtering using regular expressions according to the present invention;

[0028] Figure 3 The figure shows a schematic diagram of the result obtained from the server using the HTTP protocol in the present invention;

[0029] Figure 4The figure shows a schematic diagram of the result of obtaining the server end by using the HTTPS protocol in the present invention;

[0030] Figure 5 The figure shows the relationship map of the Internet of Vehicles industry;

[0031] Figure 6 Shown is a relationship map of manufacturers cooperating with Lynk & Co. DETAILED DESCRIPTION

[0032] The method of the present invention will be further described below with reference to the accompanying drawings.

[0033] like Figure 1 As shown, the industry relationship chain mining method based on protocol traffic analysis described in the present invention specifically includes the following steps:

[0034] (1) Obtaining the traffic to be analyzed; the traffic to be analyzed is obtained by mirroring the operator's traffic; wherein, the traffic generated by the equipment or application of the industry to be analyzed is ensured;

[0035] (2) Analyze the product number of the industry as needed and construct a regular expression; the regular expression can identify traffic containing the product number;

[0036] Specifically, the product number of the Internet of Vehicles is the vehicle frame number, which is unique. The vehicle frame number is a 17-digit code consisting of letters and numbers. It is a unique number used on the car, which can identify the car's manufacturer, engine, chassis serial number and other performance information. The first digit of the frame number represents the country. China is L and Germany is W. In order to avoid confusion with the numbers 1 and 0, the English letters "I", "O" and "Q" are not used. If it is only for domestic cars, the frame number expression is L[0-9A-HJ-NPR-Z]{16}; Express delivery The product number that can be uniquely authenticated by the industry is the express delivery number; the first two digits of the express delivery number are numbers or subtitles, and the rest are all numbers. The first two digits of the delivery number are different for different express delivery companies. For example, JD.com is JD, Jitu is JT, China Post is 11 or 98, and SF Express is SF. The total length of the delivery number for different express delivery companies is also different. JD.com has 15 digits, Jitu has 15 digits, China Post has 13 digits, and SF Express has 15 digits. Based on the first two digits plus the length of the express delivery number, we can describe the regular expression of the express delivery number of different manufacturers. Taking JD.com as an example, the regular expression for the express delivery number is JD[\d]{13}. In addition, the product number that can be uniquely authenticated by express lockers is the express locker number, the product number that can be uniquely authenticated by express barcode scanners is the PDA number, and the product that can be uniquely authenticated by POS machines is the bank card number. Based on the representation of these product numbers, corresponding regular expressions can be constructed. These expressions can identify traffic containing product numbers.

[0037] (3) Use regular expressions to perform preliminary filtering on the acquired traffic; for example, use the regular expression L[0-9A-HJ-NPR-Z]{16} to perform preliminary filtering on the traffic, i.e., the message information. The filtered messages are all message information containing the vehicle frame number. The filtering results are as follows: Figure 2 As shown;

[0038] (4) For the traffic after preliminary filtering, a second manual screening is performed based on the host of the HTTP protocol, the SNI of the HTTPS protocol, and the destination IP address of the private protocol to obtain the server information corresponding to each flow; the server IP address, domain name information, or keywords in the traffic can be used to analyze and determine whether each flow is generated by the equipment in this industry;

[0039] like Figure 3 As shown in the figure, for a flow, using the HTTP protocol host, we can confirm that the server is Ecarx Technology, a manufacturer of the Internet of Vehicles platform, that is, we can determine that the flow is generated by the Internet of Vehicles platform. Figure 4 As shown, for a piece of traffic, based on the sni of https, it can be confirmed that the server is Volkswagen, a car networking enterprise, and the traffic is generated by the car networking enterprise and the brand is Volkswagen; for a piece of traffic that is screened through a private protocol, the corresponding server is analyzed to determine whether it is generated by industry equipment; for example: through the domain name of the server, it can be known that it is Shanghai Guotai Company, and the company's main business is information navigation; because the generated navigation is a GPS device or a central control device, since the GPS device does not generate traffic, it can be analyzed that the traffic is generated by the vehicle central control; for example: the MQTT protocol of Huawei Cloud's 18835 port, the MQTT protocol client contains the T3 group field, and the MQTT payload contains the T3 BOX field, so it can be analyzed that the traffic is generated by the T-BOX device of the T3 car.

[0040] (5) Verify the asset information of the device that responds to traffic based on the APN, IPC record, message payload content, and action trajectory;

[0041] After the above screening, the analysis and judgment results are inaccurate. For example, the system for recording the chassis number in the vehicle management office contains chassis number information, but it is not generated by the vehicle, nor by the Internet of Vehicles platform or enterprise. Therefore, it is necessary to further verify the asset information reflected by the traffic. According to the base station information in the signaling packet in the traffic, such as APN, CGI, latitude and longitude of the base station, the movement trajectory is finally determined. If the movement trajectory has not changed, it is not the traffic generated by the device in the car, and the asset equipment information is not the equipment of the Internet of Vehicles industry; if the movement trajectory changes very quickly, it can be determined that it is the equipment in the car. Another example: verification of the asset information of the express cabinet. Since a express cabinet device contains N express information and the location information of the device is unchanged, that is, the trajectory remains unchanged, the above characteristics can be verified based on the content of the message payload.

[0042] (6) Draw the industry relationship map through the verified equipment asset information results and their corresponding service-side information. Figure 5 As shown, for example, based on the assets reflected in the traffic information including the vehicle frame number: car central control, vehicle GPS, vehicle T-BOX, etc., combined with the information of the server corresponding to the corresponding equipment assets, a map of the Internet of Vehicles industry chain is drawn; among them, the car central control will report the vehicle frame number, driving data and vehicle status to the parent company, and will also perform NTP clock synchronization, and the server is various Volkswagen, Mercedes-Benz and other automobile companies; the on-board GPS will report the vehicle frame number, real-time latitude and longitude and vehicle speed to the GPS manufacturer, and the server is the GPS manufacturer; due to the diverse functions of T-BOX, it can be used for vehicle network platform monitoring of vehicles, daily leisure and entertainment, and online car-hailing orders, etc. In addition to the vehicle frame number, driving data and location, the reported data will also transmit multimedia data and even report personal privacy, so the server is iFlytek, Guanglian League News, Kugou Music, etc.

[0043] (7) For a verified device or a type of device asset, record the source IP address, use the source IP address as a filtering condition, filter out all traffic obtained by tracing the device or the type of device using IMSI, export the device's Internet log, and analyze what websites each device visits. Take the intersection to determine what websites this type of device visits. Through the access records, you can know which applications and interfaces the device integrates, and thus determine which manufacturers the device has cooperative relationships with. As shown in the attached Figure 6 As shown, during the tracing process, it was found that Lynk & Co's T-BOX integrated the following applications when it left the factory: driving information interface Ecarx Technology, navigation system Baidu Maps, music search QQ Music, and mobile phone projection Baidu Carlife. This shows that Lynk & Co has a clear cooperative relationship with the above companies, drawing a product manufacturer cooperation map.

[0044] In addition, the present invention also provides an industry relationship chain mining system based on protocol traffic analysis, which specifically includes:

[0045] (1) Traffic acquisition module, used to acquire the traffic to be analyzed by mirroring;

[0046] (2) A preliminary screening module, which is used to analyze the product number of the industry as needed and construct a regular expression; the regular expression can identify the traffic containing the product number; and use the regular expression to perform preliminary filtering on the obtained traffic;

[0047] (3) A manual screening and statistics module is used to perform a secondary manual screening of the traffic after preliminary filtering based on the host of the HTTP protocol, the sni of the HTTPS protocol, and the destination IP address of the private protocol to obtain the server information corresponding to each flow; the server IP address, domain name information, or keywords in the traffic can be used to analyze and determine whether each flow is generated by the equipment in the industry; the movement trajectory is obtained based on the base station data contained in the signaling packet in the traffic;

[0048] (4) Equipment asset verification module, used to further verify the equipment assets reflected by each flow based on the APN, IPC record, message payload content and action trajectory, and determine whether they belong to the industry;

[0049] (5) A relationship spectrum drawing module is used to draw an industry relationship map based on the device assets reflected in each verified flow and their corresponding server information.

[0050] (6) Partnership analysis module, which is used to record the source IP address of a verified device asset or a type of device asset, and use the source IP address as a filtering condition to filter out all traffic obtained by the device or the type of device using the IMSI, obtain the visited website data based on the device's Internet access log, obtain the data of the device's integrated applications and interfaces through the access records, and then analyze the partnership relationship between the product device and the manufacturer.

Claims

1. A method for mining industry relationship chains based on protocol traffic analysis, characterized in that: The specific steps include: (1) Obtain the traffic to be analyzed through mirroring; (2) Analyze the product number of the industry as needed and construct a regular expression; the regular expression can identify traffic containing the product number; Use regular expressions to perform preliminary filtering on the acquired traffic; (3) For the traffic after preliminary filtering, perform a secondary manual screening based on HTTP protocol, HTTPS protocol and private protocol to obtain the server information corresponding to each traffic; Through the server's IP address, domain name information or keywords in the traffic, it can be analyzed to determine whether each traffic flow is generated by devices in the industry; (4) Further verify whether the equipment assets reflected by each flow belong to the industry based on the APN, IPC record, message payload content and action trajectory; the action trajectory is obtained based on the base station data contained in the signaling packet in the flow; (5) Draw an industry relationship map based on the equipment assets reflected in each verified flow and their corresponding server information; (6) For a verified device asset or a type of device asset, record the source IP address, use the source IP address as a filtering condition, filter out all traffic of the device or the type of device, and then analyze the cooperative relationship with the manufacturer.

2. The industry relationship chain mining method based on protocol traffic analysis according to claim 1 is characterized in that: The secondary manual screening specifically obtains the server information corresponding to each flow based on the host of the HTTP protocol, the sni of the HTTPS protocol and the destination IP address of the private protocol.

3. The industry relationship chain mining method based on protocol traffic analysis according to claim 1 is characterized in that: All traffic of the device or this type of device is obtained by tracing the IMSI of the device.

4. The industry relationship chain mining method based on protocol traffic analysis according to claim 1 is characterized in that: The analysis of the cooperative relationship with the manufacturer specifically obtains the visited website data based on the device's Internet log, obtains the data of the device's integrated applications and interfaces through the access records, and analyzes the cooperative relationship with the manufacturer.

5. The industry relationship chain mining method based on protocol traffic analysis according to claim 1 is characterized in that: The product number mentioned is the frame number or express cabinet number.

6. An industry relationship chain mining system based on protocol traffic analysis, characterized in that: Specifically include: Traffic acquisition module, used to obtain the traffic to be analyzed through mirroring; A preliminary screening module is used to analyze the product number of the industry as needed and construct a regular expression; the regular expression can identify traffic containing the product number; Use regular expressions to perform preliminary filtering on the acquired traffic; The manual screening and statistics module is used to perform secondary manual screening on the traffic after preliminary filtering according to the HTTP protocol, HTTPS protocol and private protocol to obtain the server information corresponding to each traffic; The server's IP address, domain name information, or keywords in the traffic can be used to analyze and determine whether each flow is generated by devices in the industry. The device asset verification module is used to further verify the device assets reflected in each flow based on the APN, IPC record, message payload content, and action trajectory to determine whether they belong to the industry; the action trajectory is obtained based on the base station data contained in the signaling packet in the flow; The relationship spectrum drawing module is used to draw the industry relationship map based on the device assets reflected in each verified flow and their corresponding server information; The partnership analysis module is used to record the source IP address of a verified device asset or a type of device asset, use the source IP address as a filtering condition, filter out all traffic of the device or the type of device, and then analyze the partnership with the manufacturer.

7. The industry relationship chain mining system based on protocol traffic analysis according to claim 6 is characterized in that: The secondary manual screening in the manual screening statistics module specifically obtains the server information corresponding to each flow based on the host of the HTTP protocol, the sni of the HTTPS protocol and the destination IP address of the private protocol.

8. The industry relationship chain mining system based on protocol traffic analysis according to claim 6 is characterized in that: All traffic of the device or this type of device described in the partnership analysis module is obtained by tracing the device's IMSI. The analysis of the partnership with the manufacturer described in the partnership analysis module specifically obtains the visited website data based on the device's Internet log, obtains the data of the device's integrated applications and interfaces through the access records, and analyzes the partnership with the manufacturer.

Images