Webpage data analyzing and processing method

Abstract

The invention discloses a webpage data analyzing and processing method which is implemented on the basis of a webpage data service platform. The webpage data service platform comprises a client, a content server and a word segmentation cloud server, and a webpage capturing system, a content extraction system, a content analyzing system and a database are installed on the content server. The method specifically includes the steps of S1, webpage capturing; S2, content extracting; S3, Chinese word segmentation; S4, content analyzing; S5, result displaying, namely the client calls a data result form the database and displays to users. By the adoption of the reading habit based webpage content extraction technology, subject contents of a webpage can be recognized rapidly and extracted, Chinese word segmentation is effectively performed by adopting the cloud segmentation technology, fundamental guarantee is provided for big-data analysis, investment of software and hardware resources by the users is not needed, and requirements on low cost and orientation of big-data analysis service of small and medium-sized enterprises and ordinary individual users can be met.

Description

technical field [0001] The invention relates to the technical field of Internet data mining and analysis, in particular to a method for analyzing and processing webpage data. Background technique [0002] At present, the Internet is flooded with a large amount of various information, and people are in an era of huge data and massive information. These large amounts of data require certain discovery methods in order to realize in-depth mining of meaningful information for enterprise or social development. [0003] In the past, information mining methods were generally carried out through information retrieval or mathematical statistics. For example, Baidu, Google and other search engines used by ordinary individual users can retrieve relevant content of entries, but most of them are useless information. It is difficult to obtain the desired accurate results under the condition of a large amount of data. However, its in-depth mining and analysis functions are often oriented ...

AI Technical Summary

Problems solved by technology

[0003] In the past, information mining methods were generally carried out through information retrieval or mathematical statistics. For example, Baidu, Google and other search engines used by ordinary individual users can retrieve relevant content of entries, but most of them are useless information. It is difficult to obtain the desired accurate results under the condition of a large amount of data

And its in-depth mining and analysis functions are often oriented to large enterprises or institutions, which are extremely expensive compared to the majority of small and medium-sized enterprises or ordinary individual users

Images