Efficient internet dynamic data automatic screening and grasping method and system

Abstract

The invention provides an efficient internet dynamic data automatic screening and grasping method and system. Firstly, a browser is started, inputting, clicking and skipping operation of pages is simulated, internet dynamic data is automatically screened and grasped, and the data is stored in a classified mode. The method includes the steps that 1, certain data is positioned, a major element list on which data switching depends is loaded, and all major elements start to be traversed; 2, all sub-element lists under the major elements are searched for and positioned, all sub-elements start to be traversed, and names of the major elements to which the selected sub-elements belong are acquired; 3, S2 is cycled till no any sub-element exists in all the major elements; 4, dynamically loaded internet data starts to be automatically grasped according to the major elements and the sub-elements of the dynamically simulated and screened data, the internet dynamic data is automatically screened and grasped through a design method, and classified storage is carried out. By means of the method and system, the dynamically loaded internet data can be efficiently and accurately grasped, the efficiency of internet dynamic data grasping is greatly improved, and the correction rate of internet dynamic data grasping is greatly increased.

Description

technical field [0001] The invention relates to the technical field of network data capture, in particular to a network dynamic data capture method and system. Background technique [0002] With the advent of the information age, the Internet is rich in public data resources, and various academic, educational, commodity and other information have spread across various network platforms. Considering security, timeliness, and rapidity, most Internet data is presented to users through Web dynamic loading technology. At the same time, some important resources can only be accessed after users log in. fetching becomes more difficult. [0003] The traditional Internet data capture is basically based on the static html content of the specified URL, and the data content is downloaded through the crawler tool to analyze and extract the data. The method of parsing web pages through URLs can only obtain given data, and cannot achieve the purpose of interaction with users to achieve th...

AI Technical Summary

Problems solved by technology

The method of parsing webpages through URLs can only obtain given data, and cannot achieve the purpose of interaction with users to achieve the purpose of screening. At the same time, traditional data capture systems have no way to dynamically load html content with js and ajax technology

Images