A method for collecting and extracting key Internet data information

Abstract

The invention relates to a method for collecting and extracting key data information from the Internet, which introduces a newly designed data logic association relationship, takes the target web page as the basic screening object, and realizes progressive logic for the high-frequency vocabulary search method and the preset key information word search method Combining, on the one hand, it considers the information of the specified target direction, and on the other hand, it adapts to the direction of big data update, making the network data search work more comprehensive and objective, and the key information in the basic screening objects can be obtained through comprehensive screening; Based on the screening object as the starting point, each web page under its direct reference, indirect reference and multi-level reference is analyzed step by step to obtain key information at all levels related to the relevant topics and themes in the basic screening object, and construct a multi-level Associate the topological structure of the key information under the webpage, accurately, objectively and comprehensively screen the key information in the webpage, and improve the search and mining efficiency of actual network data.

Description

technical field [0001] The invention relates to a method for collecting and extracting key Internet data information, and belongs to the technical field of web page key information extraction. Background technique [0002] With the popularization of the Internet and various digital terminal devices, a world of Internet of Everything is taking shape, data is showing explosive exponential growth, and digitalization has become the basic force for building a modern society. And with the popularity of Internet of Things infrastructure, smart phones, and wearable devices, each of us is generating a large amount of data all the time, and a large amount of data is updated on the network every day, and it is full of various types of data. How to deal with massive data Performing quick analysis and interpreting the important information is the best use of network data at present. Most of the methods in the prior art use the comparison method, that is, to match the data in the target w...

AI Technical Summary

Problems solved by technology

And with the popularity of Internet of Things infrastructure, smart phones, and wearable devices, each of us is generating a large amount of data all the time, and a large amount of data is updated on the network every day, and it is full of various types of data. How to deal with massive data Performing quick analysis and interpreting the important information is the best use of network data at present. Most of the methods in the prior art use the comparison method, that is, to match the data in the target web page with the preset keywords. Find the content that matches the keyword in the webpage. The information obtained by such a method is limited, and only through direct matching with each other, many other related information in the webpage will be missed, that is, the target webpage cannot be truly analyzed. Effective data mining, so finding the data obtained is not of much value

Images