A method and device for identifying traffic flow of intelligent connected vehicles

Abstract

A method and device for identifying intelligent networked vehicle traffic relate to the field of information technology. The present invention comprises the steps of calculating a traffic measurement model and calculating a traffic type using the traffic measurement model. The method combines the identification methods of network traffic class features and device information class features with the addition of a method based on traffic slicing and machine learning classification technology to eliminate non-networked vehicle traffic mixed in networked vehicle traffic, thereby greatly improving the accuracy of traffic identification while reducing the underreporting rate of networked vehicle traffic identification.

Description

Technical Field

[0001] The present invention relates to the field of information technology. Background Art

[0002] According to the data from the "China Intelligent Connected Vehicle Industry Development Report (2021)" released by the China Association of Automotive Engineers, in 2021, 4.766 million vehicles with L2 intelligent driving functions were sold in my country's new passenger car market, with a penetration rate of 23.5%. Intelligent connected vehicle technology supports vehicle access to the Internet, which can obtain data and services through the Internet. Since communications are transmitted through public networks, the Internet of Vehicles is vulnerable to different types of network attacks, such as denial of service, deception, and network sniffing. At present, the security measures of vehicle manufacturers have not kept up in time. In addition to strengthening security protection measures on vehicle terminals or Internet of Vehicles service terminals, it is also necessary to monitor the security of the Internet of Vehicles network in real time on the mobile network pipeline side and keep abreast of the dynamics of the Internet of Vehicles network.

[0003] In the field of security supervision of connected cars through mobile pipelines, the current traffic identification technology for intelligent connected cars mainly uses an identification method based on a feature fingerprint library. For example, the invention patent number [CN112367326A] analyzes the network characteristics of known Internet of Vehicles application data to form an Internet of Vehicles traffic template, and then matches the unknown traffic data with the Internet of Vehicles traffic template to identify whether the traffic is Internet of Vehicles traffic. This method is essentially a feature fingerprint-based identification method with high accuracy, but it is completely dependent on the network traffic feature fingerprint database, and its generalization is reduced. With the continuous emergence of new Internet of Vehicles application data, the library needs to be continuously updated, so it is impossible to timely identify the new Internet of Vehicles application traffic data, and there is a high rate of missed reports; Patent invention number [CN10794 8172A], by combining the use of one or more features of APN access point name, number segment, and Internet of Vehicles application, the mobile network traffic is screened to obtain traffic related to Internet of Vehicles communication. In addition to using the Internet of Vehicles network traffic features, this method also combines APN features and number segment features. These two types of features have strong generalization and can effectively reduce the underreporting rate of Internet of Vehicles traffic identification. However, in actual situations, there are situations where non-networked automotive devices are connected to the on-board wireless network. Under this method, the traffic generated by non-Internet of Vehicles devices in the mixed Internet of Vehicles traffic cannot be effectively distinguished using this method, and thus there is a problem of insufficient recognition accuracy.

[0004] A method and device for identifying intelligent connected vehicle traffic proposes a method for accurately identifying connected vehicle traffic in mobile network traffic. First, by combining the use of network traffic features and device information features, the false negative rate can be effectively reduced to achieve a more comprehensive effect of identifying vehicle network traffic. Then, the traffic is segmented and features are extracted by using traffic slicing. Then, the non-connected vehicle traffic mixed in the vehicle network traffic is filtered out by combining the use of machine learning methods, thereby improving the accuracy and precision of traffic identification. A method and device for identifying intelligent connected vehicle traffic can comprehensively and accurately identify connected vehicle traffic in mobile network traffic, providing strong data support for subsequent vehicle network situation supervision and network security monitoring.

[0005] Description of the prior art used

[0006] APN (Access Point Name): APN is the name of the gateway between a mobile network such as GPRS and another computer network, and is used to define the network path for all mobile data connections on a mobile device. A mobile device must be set with the access point name provided by the operator to create a data connection. The operator uses this name to distinguish the type of network connection to be created. The access point name consists of two parts, including a network identifier and an operator identifier, such as ctnet.mnc011.mcc460.gprs. In IoT scenarios or IoV scenarios, a specific network identifier is usually used to indicate that the APN is used for a specific scenario. Therefore, by collecting and extracting feature characters from such APNs, and identifying the dedicated IoV scenario APN, the purpose of identifying IoV terminals can be achieved.

[0007] IMEI International Mobile Equipment Identity: IMEI number, commonly known as the "serial number", is used to identify each independent mobile communication device in the mobile network, which is equivalent to the ID card of the mobile device. The serial number has 15 digits. The first 6 digits TAC are the model approval number, which represents the device type. The next 2 digits FAC are the final assembly number, which represents the place of origin. The type of mobile device can be determined by the first 8 digits of IMEI. Based on this feature, we can establish a feature library of the device type corresponding to the first 8 digits of IMEI, and can accurately identify the Internet of Vehicles devices in the mobile network traffic through this information.

[0008] The MSISDN number is equivalent to the user telephone number of the fixed network and is a public number for users to dial. Since the telephone number structure of mobile users in the GSM system is based on the ISDN numbering method, it is called MSISDN, which is the international ISDN number of mobile users.

[0009] Number segment. Different operators have different SIM card number segments. In order to connect to various smart hardware, there are also special number segments for IoT and IoV scenarios, which are dedicated to IoT scenarios. Therefore, the number segment information of the MSISDN number can be used to distinguish whether it belongs to IoV traffic.

[0010] Protocol type. There are dedicated IoV protocols in IoV scenarios. For example, the IoV protocols in my country mainly include JT / T808, JT / T905, GB / T32960 and other protocols. JT / T808 is a standard protocol formulated by the Ministry of Transport, which stipulates the communication protocol and data format between the Beidou-compatible vehicle terminal of the road transport vehicle satellite positioning system and the supervision / monitoring platform. By identifying these types of protocols in the traffic log data, the IoV traffic data can be effectively distinguished.

[0011] Destination IP. Some Internet of Vehicles TSP platform servers use fixed IP addresses. After analyzing these IP address information, the traffic connected to the destination IP can be filtered out from the mobile traffic and regarded as Internet of Vehicles traffic. The overall Internet ecosystem can be divided into TSP platforms, vehicle manufacturers, application / content service providers, telecom operators, vehicle parts manufacturers, chip manufacturers, automotive backward service providers, traditional offline manufacturers, traffic management bureaus, etc. TSP is an automotive remote service provider. It occupies a core position in the Telematics industry chain, connecting to automobiles, vehicle equipment manufacturers, network operators, and content providers.

[0012] Domain name. A certain model of connected cars usually access the same connected car server. The domain name in the service address can be used as a feature of connected car traffic access.

[0013] URL address. A certain model of connected car usually accesses a URL that satisfies a pattern or characteristic, and the pattern of the URL address can be used as a characteristic.

[0014] LAC location area identification code. It uniquely identifies each location area in my country's digital PLMN and is a 2-byte hexadecimal BCD code.

[0015] CELL ID is the number of the cell in the network.

[0016] IMSI generally refers to International Mobile Subscriber Identity.

[0017] IP refers to Internet Protocol address.

[0018] HTTP user-agent is used to check what operating system the visitor is using to browse the page.

[0019] http content-type comparison table, content-type content type, generally refers to the content-type existing in a web page, which is used to define the type of network files and the encoding of the web page, and determines the form and encoding in which the browser will read the file.

[0020] URL generally refers to Uniform Resource Locator System, which is a method used to specify the location of information on the World Wide Web service program of the Internet.

[0021] A domain name, also known as a network domain, is the name of a computer or a group of computers on the Internet consisting of a string of names separated by dots, and is used to locate and identify computers during data transmission.

[0022] XGBOOST, also known as the extreme gradient boosting tree, is an implementation of the boosting algorithm that can effectively handle classification problems. The training process of the XGBOOST model is roughly as follows: Since XGBOOST consists of multiple decision trees, it needs to go through multiple rounds of training, and each round of training generates a new tree model. In the process of generating new trees in each round of training, the most basic operation is node splitting, that is, to find the optimal features and optimal cut-off points of the node, and split the leaf node according to the optimal features and optimal cut-off points. After that, when the leaf node is no longer split, it is necessary to calculate the optimal leaf node weight. According to this process, a new tree model can be generated. According to the above method, multiple tree models are generated in sequence until the condition of no longer generating new tree models is met. Finally, the multiple tree models generated constitute an XGBOOST model.

[0023] By querying the TAC database, you can obtain the trademark information of the terminal device and the sales target information of the terminal device. The IMEI database is the only complete TAC (first eight digits of IMEI) database in the world. Its TAC data is only open to review agencies, operators, customs and other agencies. Currently, no online channels for querying TAC information are open. The official TAC certificate is the only certificate provided to enterprises, and each enterprise can only view its own TAC data.

[0024] The IP address table of the Internet of Vehicles TSP platform server. Some Internet of Vehicles TSP platform servers use fixed IP addresses. The Internet of Vehicles TSP platform server IP addresses publicly disclosed in the industry are collected to form the Internet of Vehicles TSP platform server IP address table.

[0025] Based on the public Internet of Vehicles services, commonly used Internet of Vehicles server domain names are collected to form an Internet of Vehicles server domain name table.

[0026] Collect URLs frequently visited by Internet of Vehicles devices to form an Internet of Vehicles service URL table.

[0027] In the process of machine learning modeling, regularization is an important means to control model complexity, combat overfitting, and pursue better prediction results. L2 regularization is a commonly used technical means in machine learning. Summary of the invention

[0028] In view of the shortcomings of the prior art, the present invention provides a method and device for identifying the flow of intelligent connected vehicles, including a step of calculating a flow measurement model and a step of calculating the flow type using the flow measurement model;

[0029] 1) Calculation of traffic measurement model steps

[0030] ⑴ Obtain mobile network traffic data through packet capture tools;

[0031] ⑵ After the traffic is parsed by the traffic parser, the traffic parsing log is obtained. The network traffic data parameters obtained by the traffic parsing log include: MSISDN number, LAC, CELL ID, IMSI, IMEI, APN, IP, start time, end time, duration, source IP, source port, destination IP, destination port, protocol type, upstream traffic size, downstream traffic size, upstream packet number, downstream packet number, total traffic, return status code, http user-agent, http content-type, domain name, URL;

[0032] ⑶ The traffic filter queries the IMEI database, and according to the type of Internet of Vehicles equipment recorded in the IMEI database, extracts and retains the network traffic of the relevant Internet of Vehicles equipment; the traffic filter extracts and retains the network traffic belonging to the Internet of Things segment according to the segment information of the MSISDN number; the traffic filter extracts and saves the network traffic that complies with the Internet of Vehicles protocol, and the Internet of Vehicles network protocol includes JT / T808, JT / T905, and GB / T32960; the traffic filter extracts and retains the network traffic whose destination IP belongs to the IP address table of the Internet of Vehicles TSP platform server according to the Internet of Vehicles TSP platform server IP address table as the Internet of Vehicles traffic; the traffic filter extracts and retains the network traffic whose access domain name belongs to the Internet of Vehicles server domain name table according to the Internet of Vehicles server domain name table as the Internet of Vehicles traffic; the traffic filter extracts and retains the network traffic whose access URL belongs to the Internet of Vehicles service URL table according to the Internet of Vehicles service URL table as the Internet of Vehicles traffic; the traffic filter sends all the extracted and retained network traffic to the traffic slicer;

[0033] (4) The network traffic received by the traffic slicer includes all Internet of Vehicles traffic and non-Internet of Vehicles traffic generated by devices connected to the Internet through the vehicle-mounted mobile Internet devices. The network traffic received by the traffic slicer is sliced ​​to further filter the Internet of Vehicles traffic; the slicing method is to use the IMEI in the network traffic as the device identifier plus the source IP and source port to generate a slice identifier, and the network traffic is cut into multiple traffic blocks based on the slice identifier; the traffic slicer sends the traffic block to the feature extractor;

[0034] (5) The feature extractor extracts the network features of the segmented traffic blocks, including: the number of upstream visits per minute, the number of downstream visits per minute, the size of upstream traffic per minute, the size of downstream traffic per minute, the number of duplicated access destination IPs per minute, and the proportion of the top five access times of different destination IPs per minute;

[0035] (6) The traffic classifier classifies the traffic blocks into IoV traffic and non-IoV traffic according to the traffic block features extracted by the feature extractor; when the traffic change rate of the uplink traffic size per minute and the downlink traffic size per minute is 10% or less, the corresponding traffic block is marked as IoV traffic; when the traffic change rate of the uplink traffic size per minute and the downlink traffic size per minute exceeds 10%, the corresponding traffic block is marked as non-IoV traffic; when the change rate of the uplink access times per minute and the downlink access times per minute is 10% or less, the corresponding traffic block is marked as IoV traffic; when the change rate of the uplink access times per minute and the downlink access times per minute is 10% or less, the corresponding traffic block is marked as IoV traffic; When the access count change rate exceeds 10%, the corresponding traffic block is marked as non-Internet of Vehicles traffic; when the number of deduplicated destination IPs accessed per minute is less than or equal to 2, the corresponding traffic block is marked as Internet of Vehicles traffic; when the number of deduplicated destination IPs accessed per minute is greater than 3, the corresponding traffic block is marked as non-Internet of Vehicles traffic; when the fourth and fifth positions of the top five IP access counts per minute are zero, the corresponding traffic block is marked as Internet of Vehicles traffic; when the fourth and fifth positions of the top five IP access counts per minute are greater than zero, the corresponding traffic block is marked as non-Internet of Vehicles traffic;

[0036] The traffic classifier collects 100 IoV traffic and 100 non-IoV traffic, and sends the network features of the 100 IoV traffic and the 100 non-IoV traffic to the model trainer;

[0037] ⑺ The model trainer uses 100 IoV traffic features and 100 non-IoV traffic features as sample data to train the XGBOOST extreme gradient boosting tree. When the sample data is non-IoV traffic, the sample target value is 0, and when the sample data is IoV traffic, the sample target value is 1; there are m samples, each sample has n features, m=200 samples, n=6, expressed as ,in are all n eigenvalues ​​of the i-th sample, R is a real number set, is the target value of the i-th sample; the exact greedy algorithm and Gain gain function are used to realize the node splitting of the first node of the tree model of the XGBOOST extreme gradient boosting tree, that is, to select the optimal feature and optimal splitting point of the node; the Gain gain function can calculate the information gain effect brought by the current node splitting compared with the current node not splitting, and the calculation method is as follows:

[0038] ; In the formula, I L It is the sample set contained in the left node after splitting according to the specified features and the specified split point. R It is the sample set contained in the right node after splitting according to the specified features and the specified split point, I=I L And I R ; represents the left subtree of the node, represents the right subtree of the node, represents the branch tree that can be obtained without splitting; γ represents the complexity cost introduced by adding a new leaf node, and λ is the L2 regularization coefficient; represents the coefficient of the first-order expansion term; represents the coefficient of the second-order expansion term; represents the target value of the i-th sample, represents the predicted value of the i-th sample in the t-1th iteration; traverse all candidate features and all candidate split points, that is, input all Information, obtain its corresponding Gain function value, and select the feature with the largest Gain function value and its split point as the optimal feature and optimal split point, so that the current node is split according to the optimal feature and optimal split point;

[0039] The model trainer calculates the weights of the leaf nodes according to the loss function formula defined in the XGBOOST model:

[0040] ,in,

[0041] definition is the sample in leaf node j, ,

[0042] is the space of decision trees; q represents the tree structure that maps samples to leaf nodes, T is the number of leaves per tree, and each Corresponding to independent tree structures q and leaf weights w, represents the weight of the i-th leaf; for a fixed structure q(x), the optimal weight of leaf node j is derived and calculated by the above formula :

[0043] ;

[0044] The model trainer iterates continuously until it reaches the preset target and stops iterating; the preset target is that when the sample data is non-Internet of Vehicles traffic, the sample target value is 0, and when the sample data is Internet of Vehicles traffic, the sample target value is 1; the model trainer completes the process of the XGBOOST model through continuous iteration, that is, completes the steps of calculating the traffic measurement model, and obtains multiple decision trees in the traffic measurement model, including the structure of each decision tree, the judgment features and judgment cut points of each leaf node, and the weight of each leaf node;

[0045] 2) Calculate traffic type using traffic measurement model

[0046] (1) When a traffic measurement model already exists, the feature extractor sends the network features of the traffic block and the slice identifier of the traffic block to the traffic measurement model;

[0047] (2) The traffic measurement model calculates the prediction values ​​of multiple sub-decision trees of the traffic block to generate a multi-decision tree prediction value data set, where one network feature corresponds to one sub-decision tree; the traffic measurement model sends the multi-decision tree prediction value data set to the traffic type judgment module;

[0048] ⑶ The traffic type judgment module performs normalization calculation on all prediction values ​​in the multi-decision tree prediction value data set:

[0049] , where x is the value to be processed. This function is used to map the previous prediction result value to a real number in the (0,1) interval, and then make a judgment based on this value. When the S(x) value is greater than the threshold, it is judged as a vehicle network traffic block; when the S(x) value is less than the threshold, it is judged as a non-vehicle network traffic block and the traffic block is deleted; the threshold value is a real number between 0 and 1. The larger the threshold value, the stricter the judgment of the vehicle network traffic block. The user can change the threshold setting as needed to adjust the strictness of the automatic judgment of the vehicle network traffic.

[0050] Beneficial Effects

[0051] Compared with previous methods, this method combines the identification methods of network traffic features and device information features, and adds a method based on traffic slicing and machine learning classification technology to eliminate non-networked car traffic mixed in the Internet of Vehicles traffic. While reducing the underreporting rate of networked car traffic identification, it greatly improves the accuracy of traffic identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a system structure diagram and a flow chart of the present invention. DETAILED DESCRIPTION

[0053] See also Figure 1 , a method and device for identifying the flow of intelligent connected vehicles provided by the present invention includes a step of calculating a flow measurement model and a step of calculating a flow type using the flow measurement model;

[0054] 1) Calculation of traffic measurement model steps

[0055] ⑴ Obtain the Internet traffic data of the mobile network through the packet capture tool 1;

[0056] (2) After traffic parsing by traffic parser 2, traffic parsing log is obtained. The network traffic data parameters obtained by the traffic parsing log include: MSISDN number, LAC, CELL ID, IMSI, IMEI, APN, IP, start time, end time, duration, source IP, source port, destination IP, destination port, protocol type, upstream traffic size, downstream traffic size, upstream packet number, downstream packet number, total traffic, return status code, http user-agent, http content-type, domain name, URL;

[0057] ⑶ The traffic filter 3 queries the IMEI database, and according to the type of Internet of Vehicles equipment recorded in the IMEI database, extracts and retains the network traffic of the relevant Internet of Vehicles equipment; the traffic filter 3 extracts and retains the network traffic belonging to the Internet of Things segment according to the segment information of the MSISDN number; the traffic filter 3 extracts and saves the network traffic that complies with the Internet of Vehicles protocol, and the Internet of Vehicles network protocol includes JT / T808, JT / T905, and GB / T32960; the traffic filter 3 extracts and retains the network traffic whose destination IP belongs to the IP address table of the Internet of Vehicles TSP platform server according to the Internet of Vehicles TSP platform server IP address table as the Internet of Vehicles traffic; the traffic filter 3 extracts and retains the network traffic whose access domain name belongs to the Internet of Vehicles server domain name table according to the Internet of Vehicles server domain name table as the Internet of Vehicles traffic; the traffic filter 3 extracts and retains the network traffic whose access URL belongs to the Internet of Vehicles service URL table according to the Internet of Vehicles service URL table as the Internet of Vehicles traffic; the traffic filter 3 sends all the extracted and retained network traffic to the traffic slicer 4;

[0058] (4) The network traffic received by the traffic slicer 4 includes all Internet of Vehicles traffic and non-Internet of Vehicles traffic generated by devices connected to the Internet through the vehicle-mounted mobile Internet devices. The network traffic received by the traffic slicer 4 is sliced ​​to further filter the Internet of Vehicles traffic; the slicing method is to use the IMEI in the network traffic as the device identifier plus the source IP and source port to generate a slice identifier, and the network traffic is cut into multiple traffic blocks based on the slice identifier; the traffic slicer sends the traffic block to the feature extractor 5;

[0059] ⑸ The feature extractor 5 extracts the network features of the segmented traffic blocks, and the network features include: the number of uplink visits per minute, the number of downlink visits per minute, the size of the uplink traffic per minute, the size of the downlink traffic per minute, the number of deduplicated destination IPs visited per minute, and the proportion of the top five in the distribution of the number of visits to different destination IPs per minute; the basis for the feature extractor 5 to extract the traffic block features is: a) the number of uplink visits per minute and the number of downlink visits per minute: Since the connected car contains heartbeat traffic, its uplink visits per minute and the number of downlink visits per minute remain in a relatively stable numerical range, while the traffic of non-connected cars, since there is no heartbeat traffic behavior, its uplink visits per minute are relatively random; b) the size of the uplink traffic per minute and the size of the downlink traffic per minute: Since the network behavior pattern of the connected car is relatively fixed, the change trend of the size of the uplink traffic per minute and the size of the downlink traffic per minute is relatively stable or its trend is relatively fixed, while the traffic size of the non-connected car, due to its relatively unstable behavior pattern, presents a relatively random trend. c) The number of deduplicated destination IP addresses visited per minute: The main interactive objects of connected cars are the connected car TSP platform, the connected car audio and video entertainment IVI service platform, etc. Therefore, the number of deduplicated destination IP addresses visited per minute remains in a relatively small range. However, for non-connected car traffic, due to its diversified functions and more dispersed access purposes, the number of deduplicated destination IP addresses visited per minute will be larger in magnitude. d) The proportion of the top five IP addresses visited per minute: that is, the proportion of the top five IP addresses visited per minute with the highest number of IP addresses visited per minute in a single traffic block is calculated. If the number of IP addresses visited is less than 5, the proportion of the last few IP addresses is taken as 0 by default. Since the server IP addresses visited by connected cars are concentrated in a few ranges, the proportion of the top five IP addresses visited shows a high proportion of the first few and a low proportion of the last few IP addresses visited. However, the network behavior of non-connected cars is usually more dispersed, so the proportion of the top five IP addresses visited will not show uniform characteristics.

[0060] (6) The traffic classifier 6 classifies the traffic blocks into Internet of Vehicles traffic and non-Internet of Vehicles traffic according to the traffic block features extracted by the feature extractor 5; when the traffic change rate of the uplink traffic size per minute and the downlink traffic size per minute is 10% or less, the corresponding traffic block is marked as Internet of Vehicles traffic; when the traffic change rate of the uplink traffic size per minute and the downlink traffic size per minute exceeds 10%, the corresponding traffic block is marked as non-Internet of Vehicles traffic; when the change rate of the number of uplink visits per minute and the number of downlink visits per minute is 10% or less, the corresponding traffic block is marked as Internet of Vehicles traffic; when the change rate of the number of uplink visits per minute and the number of downlink visits per minute is 10% or less, the corresponding traffic block is marked as Internet of Vehicles traffic; When the change rate of the number of visits exceeds 10%, the corresponding traffic block is marked as non-Internet of Vehicles traffic; when the number of deduplicated destination IPs accessed per minute is less than or equal to 2, the corresponding traffic block is marked as Internet of Vehicles traffic; when the number of deduplicated destination IPs accessed per minute is greater than 3, the corresponding traffic block is marked as non-Internet of Vehicles traffic; when the fourth and fifth positions of the top five in the distribution of the number of visits to different IPs per minute are zero, the corresponding traffic block is marked as Internet of Vehicles traffic; when the fourth and fifth positions of the top five in the distribution of the number of visits to different IPs per minute are greater than zero, the corresponding traffic block is marked as non-Internet of Vehicles traffic;

[0061] The traffic classifier 6 collects 100 IoV traffic and 100 non-IoV traffic, and sends the network features of the 100 IoV traffic and the 100 non-IoV traffic to the model trainer 7;

[0062] ⑺ The model trainer 7 uses 100 IoV traffic features and 100 non-IoV traffic features as sample data to train the XGBOOST extreme gradient boosting tree. When the sample data is non-IoV traffic, the sample target value is 0, and when the sample data is IoV traffic, the sample target value is 1; there are m samples, each sample has n features, m=200 samples, n=6, expressed as ,in are all n eigenvalues ​​of the i-th sample, R is a real number set, is the target value of the i-th sample; the exact greedy algorithm and Gain gain function are used to realize the node splitting of the first node of the tree model of the XGBOOST extreme gradient boosting tree, that is, to select the optimal feature and optimal splitting point of the node; the Gain gain function can calculate the information gain effect brought by the current node splitting compared with the current node not splitting, and the calculation method is as follows:

[0063] ; In the formula, I L It is the sample set contained in the left node after splitting according to the specified features and the specified split point. RIt is the sample set contained in the right node after splitting according to the specified features and the specified split point, I=I L And I R ; represents the left subtree of the node, represents the right subtree of the node, represents the branch tree that can be obtained without splitting; γ represents the complexity cost introduced by adding a new leaf node, and λ is the L2 regularization coefficient; represents the coefficient of the first-order expansion term; represents the coefficient of the second-order expansion term; represents the target value of the i-th sample, represents the predicted value of the i-th sample in the t-1th iteration; traverse all candidate features and all candidate split points, that is, input all Information, obtain its corresponding Gain function value, and select the feature with the largest Gain function value and its split point as the optimal feature and optimal split point, so that the current node is split according to the optimal feature and optimal split point;

[0064] The weights of the leaf nodes are calculated by the model trainer 7 according to the loss function formula defined in the XGBOOST model:

[0065] ,in,

[0066] definition is the sample in leaf node j, ,

[0067] is the space of decision trees; q represents the tree structure that maps samples to leaf nodes, T is the number of leaves per tree, and each Corresponding to independent tree structures q and leaf weights w, represents the weight of the i-th leaf; for a fixed structure q(x), the optimal weight of leaf node j is derived and calculated by the above formula :

[0068] ;

[0069] The model trainer 7 continuously iterates until a preset target is reached and stops iterating; the preset target is that when the sample data is non-Internet of Vehicles traffic, the sample target value is 0, and when the sample data is Internet of Vehicles traffic, the sample target value is 1; the model trainer 7 continuously iterates to complete the process of the XGBOOST model, that is, completes the step of calculating the traffic measurement model, and obtains multiple decision trees in the traffic measurement model 8, including the structure of each decision tree, the judgment features and judgment cut points of each leaf node, and the weight of each leaf node;

[0070] 2) Calculate traffic type using traffic measurement model

[0071] ⑴ Obtain the Internet traffic data of the mobile network through the packet capture tool 1;

[0072] (2) After traffic parsing by traffic parser 2, traffic parsing log is obtained. The network traffic data parameters obtained by the traffic parsing log include: MSISDN number, LAC, CELL ID, IMSI, IMEI, APN, IP, start time, end time, duration, source IP, source port, destination IP, destination port, protocol type, upstream traffic size, downstream traffic size, upstream packet number, downstream packet number, total traffic, return status code, http user-agent, http content-type, domain name, URL;

[0073] ⑶ The traffic filter 3 queries the IMEI database, and according to the type of Internet of Vehicles equipment recorded in the IMEI database, extracts and retains the network traffic of the relevant Internet of Vehicles equipment; the traffic filter 3 extracts and retains the network traffic belonging to the Internet of Things segment according to the segment information of the MSISDN number; the traffic filter 3 extracts and saves the network traffic that complies with the Internet of Vehicles protocol, and the Internet of Vehicles network protocol includes JT / T808, JT / T905, and GB / T32960; the traffic filter 3 extracts and retains the network traffic whose destination IP belongs to the IP address table of the Internet of Vehicles TSP platform server according to the Internet of Vehicles TSP platform server IP address table as the Internet of Vehicles traffic; the traffic filter 3 extracts and retains the network traffic whose access domain name belongs to the Internet of Vehicles server domain name table according to the Internet of Vehicles server domain name table as the Internet of Vehicles traffic; the traffic filter 3 extracts and retains the network traffic whose access URL belongs to the Internet of Vehicles service URL table according to the Internet of Vehicles service URL table as the Internet of Vehicles traffic; the traffic filter 3 sends all the extracted and retained network traffic to the traffic slicer 4;

[0074] (4) The network traffic received by the traffic slicer 4 includes all Internet of Vehicles traffic and non-Internet of Vehicles traffic generated by devices connected to the Internet through the vehicle-mounted mobile Internet devices. The network traffic received by the traffic slicer 4 is sliced ​​to further filter the Internet of Vehicles traffic; the slicing method is to use the IMEI in the network traffic as the device identifier plus the source IP and source port to generate a slice identifier, and the network traffic is cut into multiple traffic blocks based on the slice identifier; the traffic slicer sends the traffic block to the feature extractor;

[0075] (5) The feature extractor 5 extracts the network features of the segmented traffic block, the network features including: the number of uplink visits per minute, the number of downlink visits per minute, the size of uplink traffic per minute, the size of downlink traffic per minute, the number of deduplicated destination IP addresses visited per minute, and the proportion of the top five IP addresses visited per minute with different destinations; the feature extractor 5 sends the network features of the traffic block and the slice identifier of the traffic block to the traffic measurement model 8;

[0076] (6) The traffic measurement model 8 calculates the prediction values ​​of multiple sub-decision trees of the traffic block to generate a multi-decision tree prediction value data set, where one network feature corresponds to one sub-decision tree; the traffic measurement model 8 sends the multi-decision tree prediction value data set to the traffic type judgment module 9;

[0077] (7) The traffic type judgment module 9 performs normalization calculation on all the prediction values ​​in the multi-decision tree prediction value data set:

[0078] , where x is the value to be processed. This function is used to map the previous prediction result value to a real number in the (0,1) interval, and then make a judgment based on this value. When the S(x) value is greater than the threshold, it is judged as a vehicle network traffic block; when the S(x) value is less than the threshold, it is judged as a non-vehicle network traffic block and the traffic block is deleted; the threshold value is a real number between 0 and 1. The larger the threshold value, the stricter the judgment of the vehicle network traffic block. The user can change the threshold setting as needed to adjust the strictness of the automatic judgment of the vehicle network traffic.

Claims

1. A method for identifying traffic flow of intelligent connected vehicles, characterized in that The method comprises the steps of calculating a flow measurement model and calculating a flow type using the flow measurement model; 1) Calculation of traffic measurement model steps ⑴ Obtain mobile network traffic data through packet capture tools; ⑵ After the traffic is parsed by the traffic parser, the traffic parsing log is obtained. The network traffic data parameters obtained by the traffic parsing log include: MSISDN number, LAC, CELL ID, IMSI, IMEI, APN, IP, start time, end time, duration, source IP, source port, destination IP, destination port, protocol type, upstream traffic size, downstream traffic size, upstream packet number, downstream packet number, total traffic, return status code, http user-agent, http content-type, domain name, URL; ⑶ The traffic filter queries the IMEI database, and according to the type of Internet of Vehicles equipment recorded in the IMEI database, extracts and retains the network traffic of the relevant Internet of Vehicles equipment; the traffic filter extracts and retains the network traffic belonging to the Internet of Things segment according to the segment information of the MSISDN number; the traffic filter extracts and saves the network traffic that complies with the Internet of Vehicles protocol, and the Internet of Vehicles network protocol includes JT / T808, JT / T905, and GB / T32960; the traffic filter extracts and retains the network traffic whose destination IP belongs to the IP address table of the Internet of Vehicles TSP platform server according to the Internet of Vehicles TSP platform server IP address table as the Internet of Vehicles traffic; the traffic filter extracts and retains the network traffic whose access domain name belongs to the Internet of Vehicles server domain name table according to the Internet of Vehicles server domain name table as the Internet of Vehicles traffic; the traffic filter extracts and retains the network traffic whose access URL belongs to the Internet of Vehicles service URL table according to the Internet of Vehicles service URL table as the Internet of Vehicles traffic; the traffic filter sends all the extracted and retained network traffic to the traffic slicer; (4) The network traffic received by the traffic slicer includes all Internet of Vehicles traffic and non-Internet of Vehicles traffic generated by devices connected to the Internet through the vehicle-mounted mobile Internet devices. The network traffic received by the traffic slicer is sliced ​​to further filter the Internet of Vehicles traffic; the slicing method is to use the IMEI in the network traffic as the device identifier plus the source IP and source port to generate a slice identifier, and the network traffic is cut into multiple traffic blocks based on the slice identifier; the traffic slicer sends the traffic block to the feature extractor; (5) The feature extractor extracts the network features of the segmented traffic blocks, including: the number of upstream visits per minute, the number of downstream visits per minute, the size of upstream traffic per minute, the size of downstream traffic per minute, the number of duplicated access destination IPs per minute, and the proportion of the top five access times of different destination IPs per minute; (6) The traffic classifier classifies the traffic blocks into IoV traffic and non-IoV traffic according to the traffic block features extracted by the feature extractor; when the traffic change rate of the uplink traffic size per minute and the downlink traffic size per minute is 10% or less, the corresponding traffic block is marked as IoV traffic; when the traffic change rate of the uplink traffic size per minute and the downlink traffic size per minute exceeds 10%, the corresponding traffic block is marked as non-IoV traffic; when the change rate of the uplink access times per minute and the downlink access times per minute is 10% or less, the corresponding traffic block is marked as IoV traffic; when the change rate of the uplink access times per minute and the downlink access times per minute is 10% or less, the corresponding traffic block is marked as IoV traffic; When the access count change rate exceeds 10%, the corresponding traffic block is marked as non-Internet of Vehicles traffic; when the number of deduplicated destination IPs accessed per minute is less than or equal to 2, the corresponding traffic block is marked as Internet of Vehicles traffic; when the number of deduplicated destination IPs accessed per minute is greater than 3, the corresponding traffic block is marked as non-Internet of Vehicles traffic; when the fourth and fifth positions of the top five IP access counts per minute are zero, the corresponding traffic block is marked as Internet of Vehicles traffic; when the fourth and fifth positions of the top five IP access counts per minute are greater than zero, the corresponding traffic block is marked as non-Internet of Vehicles traffic; The traffic classifier collects 100 IoV traffic and 100 non-IoV traffic, and sends the network features of the 100 IoV traffic and the 100 non-IoV traffic to the model trainer; ⑺ The model trainer uses 100 IoV traffic features and 100 non-IoV traffic features as sample data to train the XGBOOST extreme gradient boosting tree. When the sample data is non-IoV traffic, the sample target value is 0, and when the sample data is IoV traffic, the sample target value is 1; there are m samples, each sample has n features, m=200 samples, n=6, expressed as ,in are all n eigenvalues ​​of the i-th sample, R is a real number set, is the target value of the i-th sample; the exact greedy algorithm and Gain gain function are used to realize the node splitting of the first node of the tree model of the XGBOOST extreme gradient boosting tree, that is, to select the optimal feature and optimal splitting point of the node; the Gain gain function can calculate the information gain effect brought by the current node splitting compared with the current node not splitting, and the calculation method is as follows: ; In the formula, I L It is the sample set contained in the left node after splitting according to the specified features and the specified split point. R It is the sample set contained in the right node after splitting according to the specified features and the specified split point, I=I L And I R ; represents the left subtree of the node, represents the right subtree of the node, represents the branch tree that can be obtained without splitting; γ represents the complexity cost introduced by adding a new leaf node, and λ is the L2 regularization coefficient; represents the coefficient of the first-order expansion term; represents the coefficient of the second-order expansion term; represents the target value of the i-th sample, represents the predicted value of the i-th sample in the t-1th iteration; traverse all candidate features and all candidate split points, that is, input all Information, obtain its corresponding Gain function value, and select the feature with the largest Gain function value and its split point as the optimal feature and optimal split point, so that the current node is split according to the optimal feature and optimal split point; The model trainer calculates the weights of the leaf nodes according to the loss function formula defined in the XGBOOST model: ,in, definition is the sample in leaf node j, , is the space of decision trees; q represents the tree structure that maps samples to leaf nodes, T is the number of leaves per tree, and each Corresponding to independent tree structures q and leaf weights w, represents the weight of the i-th leaf; for a fixed structure q(x), the optimal weight of leaf node j is derived and calculated by the above formula : ； The model trainer iterates continuously until it reaches the preset target and stops iterating; the preset target is that when the sample data is non-Internet of Vehicles traffic, the sample target value is 0, and when the sample data is Internet of Vehicles traffic, the sample target value is 1; the model trainer completes the process of the XGBOOST model through continuous iteration, that is, completes the steps of calculating the traffic measurement model, and obtains multiple decision trees in the traffic measurement model, including the structure of each decision tree, the judgment features and judgment cut points of each leaf node, and the weight of each leaf node; 2) Calculate traffic type using traffic measurement model (1) When a traffic measurement model already exists, the feature extractor sends the network features of the traffic block and the slice identifier of the traffic block to the traffic measurement model; (2) The traffic measurement model calculates the prediction values ​​of multiple sub-decision trees of the traffic block to generate a multi-decision tree prediction value data set, where one network feature corresponds to one sub-decision tree; the traffic measurement model sends the multi-decision tree prediction value data set to the traffic type judgment module; ⑶ The traffic type judgment module performs normalization calculation on all prediction values ​​in the multi-decision tree prediction value data set: , where x is the value to be processed. This function is used to map the previous prediction result value to a real number in the (0,1) interval, and then make a judgment based on this value. When the S(x) value is greater than the threshold, it is judged as a vehicle network traffic block; when the S(x) value is less than the threshold, it is judged as a non-vehicle network traffic block and the traffic block is deleted; the threshold value is a real number between 0 and 1. The larger the threshold value, the stricter the judgment of the vehicle network traffic block. The user can change the threshold setting as needed to adjust the strictness of the automatic judgment of the vehicle network traffic.

Images