Detection and analysis method for abnormal behaviors of user in big data environment

Abstract

The invention relates to a detection and analysis method for the abnormal behaviors of a user in a big data environment. The method is characterized in that the method comprises the following steps: enabling a user abnormal behavior detection system to carry out the abnormality analysis of user access behaviors in an offline mode through machine learning according to the log record of the user in HDFS in one historical statistical period, and building a user behavior model; enabling the user abnormal behavior detection system to carry out the online comparison of real-time behaviors and historical behaviors based on the current real-time user's operation behavior in Storm; transmitting safety early-warning information to Kaffka and displaying the safety early-warning information at a Stream interface if the difference between real-time behaviors and historical behaviors is big, or else judging that the behavior is a legal safe behavior. Compared with the prior art, the method supports the definition of a behavior mode or a user portrait according to the historical use behavior habit of the user at a Hadoop platform through a machine learning algorithm. A training system updates a model each month in a default manner, and the granularity of the model is one minute.

Description

technical field [0001] The invention relates to a method for detecting and analyzing abnormal user behaviors in a big data environment. Background technique [0002] In the era of big data, data as an asset has become an industry consensus. Therefore, the issue of data security has risen to the strategic height of the enterprise. For enterprises, data security is very important. Once the data is hacked, the consequences will be very bad. [0003] Generally, data security products generally protect data from brutal intrusion through access control, security isolation, data classification, data encryption, and other methods. In the traditional small data environment, administrators rely on setting some basic security policies, such as single event execution policy (user accesses sensitive data columns, or moves 1TB of data from safe area A to unsafe area B, etc.), window-based Policy (the user accesses the restricted data more than 5 times within 10 minutes), which can basi...

AI Technical Summary

Problems solved by technology

In the face of massive real-time data streams and diversified use case scenarios, security policies need to have great flexibility and scalability, and a limited number of rules can no longer guarantee the absolute security of data

Images