Method and system for constructing user network data fingerprint based on distributed processing and dpi data

Abstract

The invention discloses a method and system for constructing user network data fingerprints based on a distributed processing framework and DPI data, so as to extract user mobile terminal online behavior characteristics and preferences. The system includes: data preprocessing module: face the demand of network data fingerprint to clean and remove the redundancy of original data; rule extraction module: select M commonly used mobile APPs, capture packets to get the domain name of each APP and regular match, will match The formula is used as the identification rule of each APP and forms a rule file; the user set extraction module: extracts the user set that needs to be counted by the network data fingerprint system; the user behavior extraction module: counts the user's access to M APPs per unit time period; the data Storage module: save the result partitions to the data warehouse, create indexes and back them up. The present invention establishes the corresponding relationship between network space and real life by describing the online behavior of the user's mobile terminal, provides convenience for analyzing mobile Internet user behavior, and saves space and time resources.

Description

technical field [0001] The invention discloses a method and system for constructing user network data fingerprints based on a distributed processing framework and DPI data, so as to extract user behavior characteristics and preferences when surfing the Internet at a mobile terminal. Background technique [0002] Through the analysis and feature extraction of data such as network access records, data characteristics and pattern rules with significant signs and distinctions are obtained, and based on this, a research system for network personality and behavior is established. We call this method data fingerprinting . In view of the background of massive mobile Internet data, the present invention can establish the corresponding relationship between network space and real life through the accumulation and research of network data fingerprints based on the distributed framework processing method, and clearly describe the user mobile terminal network. The access behavior provide...

AI Technical Summary

Problems solved by technology

Telecom operators generate a huge amount of DPI data every day, simply recording the sending / receiving information of each data packet, making a lot of effective information buried in the massive data, and the fields in the original DPI data are complicated, too many fields will seriously Consume time resources and space resources in the process of data analysis

Images