Hadoop malicious node detection system based on network behavior analysis

Abstract

The invention provides a Hadoop malicious node detection system based on network behavior analysis for protecting the internal security of a cluster in view of a present situation that most malicious programs have certain network behaviors and even permeate inside the cluster to perform malicious behaviors. The Hadoop malicious node detection system comprises a network behavior monitoring module, a node log analyzing module, a node load analyzing module, and a training evaluation module malicious detection module. The network behavior monitoring module, the node log analyzing module, and the node load analyzing module operate on each node to acquire, monitor and preliminarily analyze information. The training evaluation module malicious detection module operates in an analysis host, performs model training and malicious detection after receiving information acquired by each node, and updates and stores the model regularly.

Description

Technical field [0001] The invention relates to the field of network behaviors, in particular to a Hadoop malicious node detection system based on network behavior analysis. Background technique [0002] Based on malicious analysis of network behavior, first establish a monitoring module in the cluster to monitor the behavior of each node, and set the monitoring center to be responsible for recording the key behaviors of these nodes, and use these behaviors to train the evaluation model. Through continuous iterative training, Maintain a good evaluation model to evaluate the status of nodes in the cluster in real time. In a cluster, if a node is attacked and a malicious program is run, a large number of behaviors different from other nodes will be generated, which is the core condition of judgment. [0003] At present, the malicious program monitoring technology based on network behavior analysis has had many phased results, such as NICTER system, TrumanBox system, AMCAS system, et...

AI Technical Summary

Problems solved by technology

The main idea of ​​these systems is almost based on network analysis, constructing a virtual network to trigger malicious programs, thereby locating malicious programs. However, there are relatively few researches on the detection of malicious nodes inside Hadoop. The current security assurance framework with relatively complete services mainly includes SecureMR. VIFA, etc., but all have certain limitations

[0004] 1) SecureMR: Secure MapReduce is a framework for enhancing and improving the mapping and simplification work in the MapReduce computing model, adding the Secure Committer and Secure Verifier modules, and designing and implementing the communication protocols of these modules, which can ensure non-collusion work The correctness and mapping of node calculation results, and the security of simplified nodes, but the framework cannot effectively detect the malicious behavior of a group of colluding malicious nodes.

[0005] 2) VIFA: Verification-based Integrity Assurance Framework. A verification-based service integrity assurance framework, which introduces high-level security and credible verification work nodes in cloud computing, and assumes that the simplified work n

Images