Monitoring system for integrating social network site information

Abstract

The present invention relates to the technical field of Internet communications, and particularly to a monitoring system for integrating social network site information, used for effectively classifying and analyzing network information and then providing exact search. The monitoring system is characterized by comprising a data processing server, wherein the data processing server is provided with a data acquisition unit, a data preprocessing unit, a data mining unit and a result output unit. The data mining module comprises a clustering module, a classification module, an association rule module, and a social relation network analysis module. A K-means algorithm model is arranged in the clustering module; the data preprocessing module adopts a parallel data preprocessing module which is provided with a Map / Reduce processing model. By invoking a Map function, the present invention effectively improves the data processing efficiency in a parallel data processing mode, and has the obvious advantages of rational structure, fast processing speed, high efficiency, etc.

Description

Technical field: [0001] The invention relates to the technical field of Internet communication, in particular to an integrated social network information monitoring system that effectively classifies and analyzes network information and provides accurate search. Background technique: [0002] With the development of science and technology, the Internet has been applied on a large scale. While there are a lot of valuable information in Web pages, there are also a lot of noise information that has nothing to do with the text, such as navigation, copyright, and advertisements. The existence of noise information makes the data quality of subsequent information services unable to be guaranteed. Extracting valuable content from webpages to ensure data quality can undoubtedly lay the foundation for subsequent webpage information processing applications. In applications such as information search, information filtering, text classification, text clustering, and summarization, it is...

AI Technical Summary

Problems solved by technology

[0004] Existing big data systems still use the serial method on a stand-alone node to process data, and its data processing capacity and algorithm load depend on the performance of a single execution node, and because big data processing systems often have to process massive data , the existing stand-alone node serial mechanism obviously has the problems of low efficiency and low computational load

Images