Cloud cluster extraction method of network information

Abstract

The invention provides a cloud cluster extraction method of network information. The cloud cluster extraction method comprises the following steps of: performing file writing, data storage and access to network information by a distributed file system; performing seamless combination on calculation models Map / Reduce of SOM (Self-Organizing Maps), a Kmeans clustering algorithm and cloud calculation to obtain a Map / Reduce SOM and Kmeans clustering algorithm based on the cloud calculation; performing control on the whole Map / Reduce by JobTracker, and distributing Map tasks or Reduce tasks by free TaskTracker; executing an instruction sent from the JobTracker and processing movement of data between Map and Reduce phases at the same time by the TaskTracker; periodically reporting finished work and state updating by each TaskTracker node; and if one TaskTracker node keeps silent for longer than a pre-set time interval, recording that the state of the node is dead and sending data distributed to the node to the other nodes by the JobTracker. The cloud cluster extraction method of the network information has good characteristic extracting performance and overcomes the disadvantage of too strong subjectivity in the existing network flow time sequence analyzing and predicating algorithm.

Description

technical field [0001] The invention relates to cloud computing and data mining technology, in particular to a cloud clustering extraction method for network information content. Background technique [0002] With the exponential growth of the number of websites and the number of webpages existing on the Internet, and the extensive development of e-government and e-commerce, these have greatly promoted the country's informatization construction and brought more and more benefits to people's study, work, and life. More and more convenience. However, at the same time, the Internet has also become a place for the dissemination of pornographic, cult, reactionary, Taiwan independence, and violent information. Therefore, how to prevent the dissemination and browsing of illegal information on the Internet, supervise and control the content of online information, protect the security of network information, effectively prevent the illegal dissemination of bad information in our cou...

AI Technical Summary

Problems solved by technology

The research work in the field of network content monitoring in our country is in its infancy. Some common network content monitoring software currently appearing are mostly passive working modes, usually running on the gateway. When illegal words are found, the web pages containing the words will be blocked. This method generally controls the network card, grabs network data packets, and analyzes the content of the data packets. This type of software can protect users from the influence of illegal words, but cannot obtain the source of illegal content, and cannot conduct overall network analysis. monitoring

[0005] Moreover, with the exponential growth of the number of websites and the number of web pages existing on the Internet, it is difficult for traditional serial algorithm-based techniques to handle such massive data.

Images