Data mining method and system

Abstract

The invention discloses a data mining method and system. The data mining method comprises the following steps of: obtaining original data packets from a plurality of data sources and putting the original data packets into different distributed message queues; extracting the original data packets from the message queues and preprocessing extracted data; creating different distributed processing tasks according to data type, wherein the tasks include an original data packet task for analyzing the original data packets to generate index information and description information; storing the data in the original data packets and the index information obtained by analysis to HBASE and a search server respectively, and storing the data in the original data packets and the description information to a database; and extracting the data from the database and performing data mining. The data mining method and system, based on network data, can efficiently perform information collection, linkage, aggregation, storage and mining, so that network threats and source-tracing attacks can be discovered in time and the security of the network data is ensured.

Description

technical field [0001] The invention relates to network data, in particular to a data mining method and system. Background technique [0002] In recent years, the security situation in cyberspace has undergone tremendous changes, and the growth trend of cyber attacks has developed exponentially, and has gradually evolved into a comprehensive attack of various social engineering attacks and various 0day exploits, becoming the most threatening cyber attack The advanced nature, complexity, concealment and persistence of new security threat technology means have exceeded the response ability of traditional network security technology. In order to adapt to the new security situation, it is necessary to build an information collection, linkage, aggregation, storage, and mining system based on network data to detect network threats in a timely manner, trace the source of attacks, and ensure enterprise security. Contents of the invention [0003] The technical problem to be solve...

AI Technical Summary

Problems solved by technology

[0003] The technical problem to be solved by the present invention is to overcome the defect in the prior art that lacks efficient means for collecting, associating, gathering, storing, and mining information based on network data so as to discover network threats in time and ensure network data security, and to provide a data mining method and system

Images