Website operation state monitoring and abnormal detection based on MapReduce

Abstract

The invention discloses website operation state monitoring and abnormal detection based on MapReduce. By utilizing a MapReduce parallel programming model, the state monitoring and abnormal detection can be finished, an optimal information point is captured from massive journal files, and abnormal behaviours in the access process are effectively and accurately captured by adopting an effective strategy. A website journal is analyzed by adopting four parallel strategies, namely state monitoring, characteristic abnormal detection, flow peak value detection and decision tree learning access rules, so that the monitoring and abnormal access detection of the website operation state can be achieved.

Description

technical field [0001] The monitoring and anomaly detection of website running status based on MapReduce belong to the technical field of improving website service performance by mining and processing website logs. Background technique [0002] A website, a fixed place on the Internet to publish news to the world, is composed of a domain name (that is, a website address) and a website space, usually including a homepage and other pages with hyperlinked files. Its appearance has brought an irreplaceable role to modern society, such as promoting self-image, providing rich and convenient information, expanding business channels for businesses, etc. In order to allow the website to run stably on the Internet for a long time, let users change rapidly In order to seize more network business opportunities in the information society, the maintenance and improvement of the website has become a crucial link. [0003] The maintenance of the website is mostly based on the website’s acc...

AI Technical Summary

Problems solved by technology

There are also good works in this area in China. For example, Professor Liu Dayou and Professor Yang Bo of Jilin University use machine learning methods to identify repetitive tasks in logs, thereby saving computing time (Li Jiafei et al., 2007); Professor Chen Guolong of Fuzhou University uses Machine learning and optimization methods detect intrusions in log files, however these traditional methods do not take advantage of parallel mechanisms and are destined not to be applied to portals with huge traffic volume

Moreover, Internet information is changing rapidly, and the delay brought by traditional analysis greatly reduces the value of network information.

Images