User online behavior data acquisition method in backbone link and system

Abstract

The invention discloses a user online behavior data acquisition method in a backbone link. The method comprises the following steps of: capturing a data packet containing a HTTP (Hyper Text Transport Protocol) request message in the backbone link, extracting web page URLs (Uniform Resource Locator) accessed by a user and HTTP field information reflecting user online behavior from the data packet according to the HTTP analysis by adopting parallel processing; filtering the URLs and acquiring web page URLs of which page contents needed to be captured according to a URL mode characteristic tree;and capturing the page contents of the filtered web page URLs in an asynchronous mode. The method and the system of the invention can capture an original data pocket containing a HTTP request messagein a gigabit link in real time, extract a user IP (Internet Protocol) address, web page URL addresses and other related user online information data from the original data pocket in real time, filterthe mass web page URL addresses in real time and capture the page contents in quasi real time.

Description

technical field

[0001] The invention relates to network technology, in particular to a method and system for collecting user online behavior data in a backbone network link. Background technique

[0002] Based on the needs of various aspects such as statistics and applications, it is necessary to comprehensively collect and obtain the online behavior activity data of Internet users on the transmission link of the backbone network of the telecom operator. Generally speaking, the backbone networks of telecom operators mostly use 10G bandwidth (10Gbps) network transmission links. How to quickly and real-time obtain the online behavior data of network users from the transmission links of the backbone network of telecom operators has many technical problems. great challenges and difficulties. At present, there is no solution for collecting user online behavior data based on the backbone network level of telecom operators in the industry.

[0003] The present applicant provides ...

Examples

Embodiment Construction

[0039] The present invention will be described more fully hereinafter with reference to the accompanying drawings, in which exemplary embodiments of the invention are illustrated.

[0040] figure 1 A flowchart showing an embodiment of the method for collecting user online behavior data in the backbone network link of the present invention.

[0041] Such as figure 1 As shown, in step 102, a data packet including an HTTP (Hypertext Transfer Protocol, Hypertext Transfer Protocol) request message in a backbone network link is captured. For example, on the low-speed backbone network link, the system interrupt method can be used to obtain the serial network link frame data flow from the network card device, and on the high-speed backbone network network link, the high-speed serial network link frame data stream can be obtained from the network card device by software polling. Network link frame data flow, with DMA ( direct Memory Access , direct memory access) to store network...

technical field

[0001] The invention relates to network technology, in particular to a method and system for collecting user online behavior data in a backbone network link. Background technique

[0002] Based on the needs of various aspects such as statistics and applications, it is necessary to comprehensively collect and obtain the online behavior activity data of Internet users on the transmission link of the backbone network of the telecom operator. Generally speaking, the backbone networks of telecom operators mostly use 10G bandwidth (10Gbps) network transmission links. How to quickly and real-time obtain the online behavior data of network users from the transmission links of the backbone network of telecom operators has many technical problems. great challenges and difficulties. At present, there is no solution for collecting user online behavior data based on the backbone network level of telecom operators in the industry.

[0003] The present applicant provides ...

Examples

Embodiment Construction

[0039] The present invention will be described more fully hereinafter with reference to the accompanying drawings, in which exemplary embodiments of the invention are illustrated.

[0040] figure 1 A flowchart showing an embodiment of the method for collecting user online behavior data in the backbone network link of the present invention.

[0041] Such as figure 1 As shown, in step 102, a data packet including an HTTP (Hypertext Transfer Protocol, Hypertext Transfer Protocol) request message in a backbone network link is captured. For example, on the low-speed backbone network link, the system interrupt method can be used to obtain the serial network link frame data flow from the network card device, and on the high-speed backbone network network link, the high-speed serial network link frame data stream can be obtained from the network card device by software polling. Network link frame data flow, with DMA ( direct Memory Access , direct memory access) to store network...

Images