A method for analyzing mobile network air interface traffic based on USRP

Abstract

The application discloses a mobile network air interface traffic analysis method based on USRP and relates to the field of computer network security.The method comprises a mobile network air interface relay module, an automatic traffic collection module and a traffic analysis module, wherein the mobile network air interface relay module directly acquires the communication traffic of a mobile network device in a mobile network air interface, and no additional unit needs to be added on the mobile device side or the base station side; the automatic traffic collection module collects traffic, pre-processes data, extracts features and constructs a data set on the basis of the mobile network air interface relay module, and manual operation of the mobile device is not needed; and the traffic analysis module trains a model according to the constructed data set and classifies and analyzes the mobile network traffic.

Description

Technical Field

[0001] This invention relates to the field of computer network security, and in particular to a method for analyzing air interface traffic of mobile networks based on USRP. Background Technology

[0002] As cyberspace becomes increasingly intertwined with human life, its governance has become a major issue concerning national security and social stability. The basic mode of network information transmission involves network devices transmitting data in the form of network packets, using different network protocols to achieve content transmission across different networks.

[0003] With the development of internet technology, users have raised higher demands for network security. In order to protect users' data privacy, more and more network communications have adopted traffic encryption technology. The proportion of encrypted traffic in the internet continues to grow. There are huge differences between encrypted and unencrypted traffic. The text and stream characteristics that are consistent after encryption have changed. Traditional network security monitoring and analysis methods can no longer identify network behavior based on traffic. Malicious behaviors hidden in encrypted traffic can easily escape supervision. It is necessary to study encrypted traffic identification and detection technology and use new methods to achieve encrypted traffic identification and analysis.

[0004] In the current internet environment, a large amount of traffic originates from applications running on mobile devices. These applications encrypt their communications to ensure the privacy and security of user data. Encryption technologies such as Tor, SSL, VPN, and DoH are widely used, and large internet companies like Google have been actively promoting their deployment. However, it has been discovered that the quantity and timing characteristics of communication data packets from mobile devices can reveal a wealth of information that can be used to analyze user behavior. For example, this can be achieved by installing special applications on mobile devices to capture and analyze the encrypted traffic of other applications, or by obtaining and analyzing application-layer encrypted traffic from VPN gateways and proxy servers. However, in these scenarios, the encrypted traffic that can be captured is specifically for application-layer data, and as users become more security-conscious and application vendors continuously upgrade their security service levels, obtaining traffic at the device, proxy, and server levels is becoming increasingly difficult.

[0005] When users communicate with the Internet using mobile networks, because mobile networks are a type of wireless network, any user with the appropriate hardware and wireless technology can access the mobile communication traffic of all users in the vicinity of the mobile network air interface. To protect user data privacy, data encryption is also employed in mobile networks. As a widely covered infrastructure, mobile networks perform different synchronization processes for mobile devices in different physical locations during wireless signal transmission, making passive information capture methods insufficient to fully obtain user information. Furthermore, unlike Ethernet networks, where encrypted traffic is encrypted at the application layer, in mobile networks, encryption is performed at the protocol layer level within the mobile network protocol stack.

[0006] Therefore, those skilled in the art are dedicated to developing a mobile network air interface traffic analysis method based on USRP. Instead of capturing data by adding any modules or applications to the mobile device or server, this method uses a special protocol stack run on USRP hardware to obtain all data from the mobile device's interaction with the internet via the mobile network air interface. Then, machine learning algorithms are used to analyze the encrypted traffic within the mobile network air interface. Summary of the Invention

[0007] In view of the above-mentioned deficiencies of the prior art, the technical problem to be solved by the present invention is to obtain all traffic of communication between mobile devices and networks in a transparent manner to users and servers, and then to analyze the encrypted traffic of mobile network air interface with the help of machine learning algorithms.

[0008] To achieve the above objectives, this invention provides a method for analyzing mobile network air interface traffic based on USRP, comprising the following steps:

[0009] Step 1: Use a USRP-based mobile network air interface relay module to obtain actual communication traffic;

[0010] Step 2: Use the Selenium automation testing framework to design an automated traffic collection module to automate data collection and build a dataset;

[0011] Step 3: Design a traffic analysis module using a combination of the random forest algorithm and the K-nearest neighbor algorithm, and use the advantages of each algorithm to classify encrypted traffic.

[0012] Furthermore, it includes a mobile network air interface relay module, an automated traffic collection module, and a traffic analysis module.

[0013] Furthermore, the mobile network air interface relay module enables mobile devices to communicate indirectly with the base station instead of directly with it when accessing the network. The mobile network air interface relay module can obtain all traffic from the communication between the mobile device and the base station.

[0014] Furthermore, the automated traffic collection module is implemented based on the mobile network air interface relay module to construct a dataset for mobile network traffic analysis.

[0015] Furthermore, the automated traffic collection module first uses aiohttp to filter the original URL file, and then uses the Selenium framework to collect traffic from the websites to be collected in sequence.

[0016] Furthermore, the traffic analysis module filters and processes the raw data collected by the automated traffic collection module, extracts corresponding data features to establish a dataset, establishes a traffic analysis model based on the dataset, and enables the traffic analysis model to obtain the uplink and downlink traffic of mobile devices output by the mobile network air interface relay module. The established traffic analysis model is used to detect and determine the classification of the current network data traffic.

[0017] Further, step 1 includes the following steps:

[0018] Step 1.1: Select an available mobile network frequency band (DL_EARFCN), and the target Mobile Country Code (MCC) and Mobile Network Number (MNC) under this frequency band. Run the mobile network air interface relay module to enable the relay module to perform the functions of the mobile device to camp on the target mobile network vendor's mobile cell (CELL).

[0019] Step 1.2: Run the base station function of the mobile network air interface relay module, and select the same mobile network frequency band (DL_EARFCN) as in Step 1.1;

[0020] Step 1.3: The mobile device will search for a suitable mobile cell based on its power level. If it selects the base station part of the mobile network air interface relay module, it will send registration information representing the device to the mobile network air interface relay module to complete the registration and network access process.

[0021] Step 1.4: If no mobile device selects to access the base station part of the mobile network air interface relay module, the user part of the relay module maintains a finite state machine loop, searching for a state in the public land mobile network, finding a cell and parking the state, and looping between the registration failure state.

[0022] Step 1.5: The mobile network air interface relay module does not modify the requests and responses from mobile devices and base stations, but only forwards them accordingly;

[0023] Step 1.6: After the mobile network air interface relay module forwards all the device access information and authentication information, the mobile device will receive an IP address for accessing the network. All subsequent user data will communicate indirectly with the base station through the mobile network air interface relay module.

[0024] Furthermore, step 2 includes the following steps:

[0025] Step 2.1: Select appropriate URLs to facilitate subsequent dataset construction. The URLs used for ranking on ranking websites were integrated, deduplicated, and used for future reference.

[0026] Step 2.2: Collect traffic from the websites to be collected in sequence and add tags; when collecting data for training the model, the existing network conditions are fully considered. Considering the wide variety of websites accessed by users, a monitoring dataset and an unmonitored dataset are constructed. During encrypted traffic analysis, the URLs of interest are placed in the monitoring dataset, while an unmonitored dataset is set up to improve the recall rate of classification; considering that different server settings will lead to different protocols used by devices for communication and thus different traffic characteristics, the traffic is placed into different models for analysis according to the data packet length of the encrypted traffic to improve the accuracy of classification.

[0027] Further, in step 3, feature extraction is performed on the traffic data, data feature preprocessing is performed, categorical features are extracted and encoded as traffic features, numerical features are standardized, and feature vectors are constructed to process the collected traffic files to obtain the corresponding dataset.

[0028] Furthermore, in step 3, the traffic features used in the classification model include the time features and length features of the traffic. The algorithm used combines the random forest algorithm and the K-nearest neighbor algorithm to take advantage of the advantages of each of the two algorithms. The output of all decision trees in the random forest is used as a fingerprint, which is input into the K-nearest neighbor algorithm to find the category of the nearest fingerprint and output the result.

[0029] In a preferred embodiment of the present invention, the objective of the present invention is to propose a mobile network air interface traffic analysis method based on the Universal Software Radio Peripheral (USRP). By running a modified mobile network protocol stack through the USRP hardware device, all traffic between the mobile device and the network is obtained in a transparent manner to users and servers. Then, with the help of machine learning algorithms, the encrypted traffic of the mobile network air interface is analyzed.

[0030] This invention proposes a mobile network air interface traffic analysis method based on the Universal Software Radio Peripheral (USRP), which consists of three parts: a mobile network air interface relay module, an automated traffic collection module, and a traffic analysis module.

[0031] 1) Implement a relay module for the mobile network air interface, enabling mobile devices to communicate indirectly with the base station via the relay module instead of directly communicating with it. The relay module receives all traffic from the communication between the mobile device and the base station. The complete mobile network protocol stack is very large; for the required mobile network air interface relay module, only a portion of the protocol stack needs modification. For layers with complex operating principles, the entities implemented in the protocol stack are used directly, treating them as black boxes, ensuring that the inputs and outputs of these layers remain unchanged and conform to 3GPP protocol specifications.

[0032] 2) Implement an automated traffic collection module. Based on the mobile network air interface relay module, implement an automated traffic collection module to construct a dataset for mobile network traffic analysis. Since the relay module can acquire all traffic between the mobile device and the base station, it can export the traffic to an interface using inter-system communication methods such as SOCKET, and then use tools like TCPDUMP to collect the traffic and record tags. However, constructing the dataset requires a large amount of data; therefore, an automated traffic collection module is needed.

[0033] 3) Traffic analysis module: Filters and processes the raw data collected by the automated traffic collection module, extracts corresponding data features to build a dataset, builds a traffic analysis model based on the dataset, and enables the traffic analysis model to obtain the uplink and downlink traffic of mobile devices output by the mobile network air interface relay module. The established traffic analysis model is used to detect and determine the classification of the current network data traffic.

[0034] In the relay module of the mobile network air interface, this invention designs a traffic relay implementation that is transparent to both the mobile device and the base station core network. The mobile network air interface traffic relay module simultaneously incorporates functionalities from both the mobile device and the base station. The mobile network air interface traffic relay method implemented by this design includes the following steps:

[0035] Step 1.1: Select an available mobile network frequency band (DL_EARFCN), and the target Mobile Country Code (MCC) and Mobile Network Number (MNC) under this frequency band. Run the mobile network air interface relay module to enable the relay module to perform the functions of the mobile device to camp on the target mobile network vendor's mobile cell (CELL).

[0036] Step 1.2: Run the base station function of the mobile network air interface relay module, and select the same mobile network frequency band (DL_EARFCN) as in Step 1;

[0037] Step 1.3: The mobile device will search for a suitable mobile cell based on its power level. If it selects the base station part of the mobile network air interface relay module, it will send registration information representing the device to the mobile network air interface relay module to complete the registration and network access process.

[0038] Step 1.4: If no mobile device selects to access the base station part of the mobile network air interface relay module, the user part of the relay module maintains a finite state machine loop, searching for a state in the public land mobile network, finding a cell and parking the state, and looping between the registration failure state.

[0039] Step 1.5: The mobile network air interface relay module does not modify the requests and responses from mobile devices and base stations, but only forwards them accordingly;

[0040] Step 1.6: After the mobile network air interface relay module forwards all the device access information and authentication information, the mobile device will receive an IP address for accessing the network. All subsequent user data will communicate indirectly with the base station through the mobile network air interface relay module.

[0041] In the automated traffic collection module, this invention designs a fully automated method for collecting traffic.

[0042] Step 2.1: Select appropriate URLs to facilitate subsequent dataset construction. The URLs used for ranking on ranking websites were integrated, deduplicated, and used for future reference.

[0043] Step 2.2: Collect traffic from the websites to be collected and add tags in sequence.

[0044] Finally, the traffic analysis module of this invention is based on the random forest algorithm and the K-nearest neighbor algorithm of machine learning. It trains and extracts the dataset on the collected data and trains the machine learning model.

[0045] Step 3.1: Extract features from traffic data, preprocess data features, encode categorical features, standardize numerical features, and construct feature vectors to process the collected traffic files and obtain the corresponding dataset.

[0046] Step 3.2: In the classification model, the traffic features used include the time features and length features of the traffic. The algorithm used combines the random forest algorithm and the K-nearest neighbor algorithm to take advantage of the advantages of each of the two algorithms. The output of all decision trees in the random forest is further used as a fingerprint, which is input into the K-nearest neighbor algorithm to find the category of the nearest fingerprint and output the result.

[0047] Compared with the prior art, the present invention has the following obvious substantive features and significant advantages:

[0048] The USRP-based mobile network air interface traffic analysis method of this invention does not require the installation of special applications on mobile devices or any changes to the existing mobile network structure. It can be directly embedded into the existing mobile network in a transparent manner, obtaining all uplink and downlink communication traffic between the mobile device and the base station without the user's awareness. When performing traffic analysis, a combination of the random forest algorithm and the K-nearest neighbor algorithm is selected, making full use of the characteristics of these two algorithms to complete the classification of encrypted traffic in a simpler way than deep learning.

[0049] The following will further explain the concept, specific structure, and technical effects of the present invention in conjunction with the accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Attached Figure Description

[0050] Figure 1 This is a schematic diagram of the overall structure of a USRP-based mobile network air interface traffic analysis method according to a preferred embodiment of the present invention.

[0051] Figure 2 This is a structural diagram of a relay module implementation for a mobile network air interface according to a preferred embodiment of the present invention;

[0052] Figure 3 This is a flowchart of an automated traffic collection module according to a preferred embodiment of the present invention;

[0053] Figure 4 This is a flowchart of a traffic analysis module according to a preferred embodiment of the present invention. Detailed Implementation

[0054] The following description, with reference to the accompanying drawings, illustrates several preferred embodiments of the present invention to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0055] In the accompanying drawings, components with the same structure are indicated by the same numerical designation, and components with similar structures or functions are indicated by similar numerical designations. The dimensions and thicknesses of each component shown in the drawings are arbitrary, and the present invention does not limit the dimensions and thicknesses of each component. To make the illustrations clearer, the thickness of some components has been appropriately exaggerated in the drawings.

[0056] This invention proposes a mobile network air interface traffic analysis method based on the Universal Software Radio Peripheral (USRP), which consists of three parts: a mobile network air interface relay module, an automated traffic collection module, and a traffic analysis module. The overall structure is as follows: Figure 1As shown. The mobile network air interface relay module is responsible for relaying traffic between mobile devices and base stations. The automated traffic collection module controls both the mobile devices and the traffic relay module, controlling the mobile devices' network access and exporting encrypted data from the traffic relay module to local storage, while also tagging each collected traffic file. The traffic analysis module uses the data collected by the automated traffic collection module to train a model, and then feeds the remaining new traffic with unknown tags from the mobile network air interface relay module into the model to obtain classification results.

[0057] 1) Implement a relay module for the mobile network air interface, enabling mobile devices to communicate indirectly with the base station via the relay module instead of directly communicating with it. The relay module receives all traffic from the communication between the mobile device and the base station. The complete mobile network protocol stack is very large; for the required mobile network air interface relay module, only a portion of the mobile network protocol stack needs modification. For layers with complex operating principles, the entities implemented in the protocol stack are directly used, treating them as black boxes, ensuring that the inputs and outputs of these layers remain unchanged and conform to 3GPP protocol specifications. The structure of the relay module is as follows: Figure 2 As shown, it possesses some functions of a base station and some functions of a mobile device. The communication structure is implemented using the libZMQ tool, enabling communication between the base station part and the mobile device part of the relay module.

[0058] Specifically, in order to flexibly and dynamically utilize wireless resources, mobile networks continuously exchange certain physical layer parameters related to wireless connections in encrypted form after a device successfully registers and joins the network. To implement a relatively stable mobile network air interface relay module, a parameter guessing module needs to be implemented in the MAC layer of the relay module's base station section to dynamically adjust indicators such as scheduling requests and channel quality metrics until the relay module observes stable measurements from the mobile device.

[0059] The specific steps for implementing mobile network air interface traffic relay operation in this design are as follows:

[0060] Step 1.1: Select an available mobile network frequency band (DL_EARFCN), and the target Mobile Country Code (MCC) and Mobile Network Number (MNC) under this frequency band. Run the mobile network air interface relay module to enable the relay module to perform the functions of the mobile device to camp on the target mobile network vendor's mobile cell (CELL).

[0061] Step 1.2: Run the base station function of the mobile network air interface relay module, and select the same mobile network frequency band (DL_EARFCN) as in Step 1;

[0062] Step 1.3: The mobile device will search for a suitable mobile cell based on its power level. If it selects the base station part of the mobile network air interface relay module, it will send registration information representing the device to the mobile network air interface relay module to complete the registration and network access process.

[0063] Step 1.4: If no mobile device selects to access the base station part of the mobile network air interface relay module, the user part of the relay module maintains a finite state machine loop, searching for a state in the public land mobile network, finding a cell and parking the state, and looping between the registration failure state.

[0064] Step 1.5: The mobile network air interface relay module does not modify the requests and responses from mobile devices and base stations, but only forwards them accordingly;

[0065] Step 1.6: After the mobile network air interface relay module forwards all the device access information and authentication information, the mobile device will receive an IP address for accessing the network. All subsequent user data will communicate indirectly with the base station through the mobile network air interface relay module.

[0066] 2) Implement an automated traffic collection module. Based on the mobile network air interface relay module, implement an automated traffic collection module to construct a dataset for mobile network traffic analysis. Since the relay module can acquire all traffic between the mobile device and the base station, it can export the traffic to an interface using inter-system communication methods such as SOCKET, and then use tools like TCPDUMP to collect the traffic and record tags. However, constructing the dataset requires a large amount of data; therefore, an automated traffic collection module is needed. The specific process of this module is as follows: Figure 3 As shown, aiohttp is first used to filter the raw URL file, and then the Selenium framework is used to collect traffic from the websites to be collected in sequence.

[0067] Step 2.1: Select appropriate URLs to facilitate subsequent dataset construction. The URLs used for ranking on ranking websites were integrated, deduplicated, and used for future reference.

[0068] Step 2.2: Collect traffic from the websites to be collected in sequence and add labels. When collecting data for training the model, the existing network conditions were fully considered. Given the diverse types of websites users visit, a monitored dataset and an unmonitored dataset were constructed. During encrypted traffic analysis, the URLs of interest were placed in the monitored dataset, while an unmonitored dataset was set up to improve the recall rate of the classification. Considering that different server settings can lead to different protocols used for device communication and thus different traffic characteristics, the traffic was analyzed using different models based on the packet length of the encrypted traffic to improve the accuracy of the classification.

[0069] 3) The traffic analysis module filters and processes the raw data collected by the automated traffic collection module, extracts corresponding data features to build a dataset, establishes a traffic analysis model based on the dataset, and enables the traffic analysis model to acquire the uplink and downlink traffic of mobile devices output from the mobile network air interface relay module. The established traffic analysis model is used to detect and determine the classification of the current network data traffic. The flowchart of the traffic analysis module is shown in Figure 4.

[0070] In the classification model, the traffic features used include the time features and length features of the traffic. The algorithm used combines the random forest algorithm and the K-nearest neighbor algorithm to take advantage of the advantages of each of the two algorithms. The output of all decision trees in the random forest is further used as a fingerprint, which is then input into the K-nearest neighbor algorithm to find the class of the nearest fingerprint and output the result.

[0071] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for analyzing air interface traffic in a mobile network based on USRP, characterized in that, Includes the following steps: Step 1: Use a USRP-based mobile network air interface relay module to obtain actual communication traffic; Step 2: Use the Selenium automation testing framework to design an automated traffic collection module to automate data collection and build a dataset; Step 3: Design a traffic analysis module using a combination of the random forest algorithm and the K-nearest neighbor algorithm, and classify encrypted traffic by leveraging the advantages of each algorithm. Step 1 includes the following steps: Step 1.1: Select an available mobile network frequency band, and the target mobile country code and mobile network number under this frequency band. Run the mobile network air interface relay module to enable the relay module to perform the functions of a mobile device to camp on the mobile cell of the target mobile network vendor. Step 1.2: Run the base station function of the mobile network air interface relay module, and select the same mobile network frequency band as in Step 1.1; Step 1.3: The mobile device will search for a suitable mobile cell based on its power level. If it selects the base station part of the mobile network air interface relay module, it will send registration information representing the device to the mobile network air interface relay module to complete the registration and network access process. Step 1.4: If no mobile device selects to access the base station part of the mobile network air interface relay module, the user part of the relay module maintains a finite state machine loop, searching for a state in the public land mobile network, finding a cell and parking the state, and looping between the registration failure state. Step 1.5: The mobile network air interface relay module does not modify the requests and responses from mobile devices and base stations, but only forwards them accordingly; Step 1.6: After the mobile network air interface relay module forwards all the device access information and authentication information, the mobile device will receive an IP address for accessing the network. All subsequent user data will communicate indirectly with the base station through the mobile network air interface relay module.

2. The USRP-based mobile network air interface traffic analysis method as described in claim 1, characterized in that, It includes a mobile network air interface relay module, an automated traffic collection module, and a traffic analysis module.

3. The USRP-based mobile network air interface traffic analysis method as described in claim 2, characterized in that, The mobile network air interface relay module enables mobile devices to communicate indirectly with the base station instead of directly with it when accessing the network. The mobile network air interface relay module can obtain all traffic from the communication between the mobile device and the base station.

4. The USRP-based mobile network air interface traffic analysis method as described in claim 2, characterized in that, The automated traffic collection module is implemented based on the mobile network air interface relay module, and constructs a dataset for mobile network traffic analysis.

5. The USRP-based mobile network air interface traffic analysis method as described in claim 2, characterized in that, The automated traffic collection module first uses aiohttp to filter the original URL file, and then uses the Selenium framework to collect traffic from the websites to be collected in sequence.

6. The USRP-based mobile network air interface traffic analysis method as described in claim 2, characterized in that, The traffic analysis module filters and processes the raw data collected by the automated traffic collection module, extracts corresponding data features to establish a dataset, establishes a traffic analysis model based on the dataset, and enables the traffic analysis model to obtain the uplink and downlink traffic of mobile devices output by the mobile network air interface relay module. The established traffic analysis model is used to detect and determine the classification of the current network data traffic.

7. The USRP-based mobile network air interface traffic analysis method as described in claim 1, characterized in that, Step 2 includes the following steps: Step 2.1: Select appropriate URLs to facilitate subsequent dataset construction. The URLs used for ranking on ranking websites were integrated, deduplicated, and used for future reference. Step 2.2: Collect traffic from the websites to be collected in sequence and add tags; when collecting data for training the model, the existing network conditions are fully considered. Considering the wide variety of websites accessed by users, a monitoring dataset and an unmonitored dataset are constructed. During encrypted traffic analysis, the URLs of interest are placed in the monitoring dataset, while an unmonitored dataset is set up to improve the recall rate of classification; considering that different server settings will lead to different protocols used by devices for communication and thus different traffic characteristics, the traffic is placed into different models for analysis according to the data packet length of the encrypted traffic to improve the accuracy of classification.

8. The USRP-based mobile network air interface traffic analysis method as described in claim 1, characterized in that, In step 3, feature extraction is performed on the traffic data, data feature preprocessing is performed, categorical features are extracted and encoded as traffic features, numerical features are standardized, and feature vectors are constructed to process the collected traffic files to obtain the corresponding dataset.

9. The USRP-based mobile network air interface traffic analysis method as described in claim 1, characterized in that, In step 3, the traffic features used in the classification model include the time features and length features of the traffic. The algorithm used combines the random forest algorithm and the K-nearest neighbor algorithm to take advantage of the advantages of each of the two algorithms. The output of all decision trees in the random forest is used as a fingerprint, which is input into the K-nearest neighbor algorithm to find the category of the nearest fingerprint and output the result.

Images