Internet key data information acquisition and extraction method

Abstract

The invention relates to an internet key data information acquisition and extraction method, which comprises the steps of introducing a newly designed data logic association relationship, taking a target webpage as a basic screening object, and realizing progressive logic combination of a high-frequency vocabulary search mode and a preset key information word search mode, so that on the one hand,information in a specified target direction is considered, and on the other hand, the method adapts to a big data updating direction, network data searching work becomes more comprehensive and objective, and key information in the basic screening object is obtained through comprehensive screening; and taking the basic screening object as a starting point, performing step-by-step one-to-one analysis on each webpage under direct reference, indirect reference and multi-level reference of the basic screening object to obtain each level of key information related to related topics and subjects in the basic screening object, and constructing a topological structure of the key information under the multi-level associated webpage, so that key information in the webpage is accurately, objectively and comprehensively screened, and the searching and mining efficiency of actual network data is improved.

Description

technical field [0001] The invention relates to a method for collecting and extracting key Internet data information, and belongs to the technical field of web page key information extraction. Background technique [0002] With the popularization of the Internet and various digital terminal devices, a world of Internet of Everything is taking shape, data is showing explosive exponential growth, and digitalization has become the basic force for building a modern society. And with the popularity of Internet of Things infrastructure, smart phones, and wearable devices, each of us is generating a large amount of data all the time, and a large amount of data is updated on the network every day, and it is full of various types of data. How to deal with massive data Performing quick analysis and interpreting the important information is the best use of network data at present. Most of the methods in the prior art use the comparison method, that is, for the data in the target webpa...

AI Technical Summary

Problems solved by technology

Most of the methods in the prior art use the comparison method, that is, for the data in the target webpage, perform matching with the preset keywords, and find the content that matches the keywords in the webpage. Such a method searches for the obtained information It is limited, and only through direct matching with each other, many other related information in the webpage will be missed, that is, it is impossible to really carry out effective data mining on the target webpage, so the value of finding the obtained data is not great

Traditional data acquisition methods are too weak in the face of unstructured and high-speed big data processing requirements, and it is necessary to innovate and develop data acquisition methods that meet the requirements of new big data processing

Images