Cross-time multi-dimensional abnormal data monitoring method and system

Abstract

The invention relates to a cross-time multi-dimensional abnormal data monitoring method and system. The method includes: collecting Web logs and OLA log queries to obtain website information, delivery advertisement information and user information; cleaning obtained heterogeneous data by employing Python in a Spark environment, generating an electronic table corresponding to each dimension, and preparing modeling analysis; establishing a normal advertisement click behavior model by employing a GBDT algorithm; conducting a modeling analysis on an abnormal data sample by employing a support vector machine, and establishing an abnormal advertisement click behavior model; and generating a cheating click monitoring model, and deploying an online anti-cheating model. According to the method and system, the problems of fusion analysis of various heterogeneous data and small sample classification accuracy are solved by employing machine learning and cloud computing technologies through monitoring of cheating Internet advertisement traffic, subsequent Internet advertisement click data can be recorded and analyzed in a whole machine learning chain, the anti-cheating model is improved, and the accuracy of advertisement delivery can be better achieved.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a method and system for monitoring abnormal data across time and in multiple dimensions. Background technique [0002] Various cheating behaviors that are easy to occur after online advertisements are placed, including advertisement clicks generated by illegal traffic caused by maliciously simulating user clicks through programs or scripts, damage the interests of advertisers and advertising platforms. In the prior art, The modeling method for abnormal advertisement click behavior is mainly based on the supervised learning method, which extracts the characteristics of each record from the original log, undergoes manual labeling, and regenerates the model. This method is not only inefficient, but also unable to adapt to business expansion. When When the amount of access data reaches TB or even PB level, it cannot meet business needs in terms of timeliness and processing performa...

AI Technical Summary

Problems solved by technology

[0002] Various cheating behaviors that are easy to occur after online advertisements are placed, including advertisement clicks generated by illegal traffic generated by maliciously simulating user clicks through programs or scripts, damage the interests of advertisers and advertising platforms. In the prior art, The modeling method for abnormal advertisement click behavior is mainly based on the supervised learning method, which extracts the characteristics of each record from the original log, undergoes manual labeling, and regenerates the model. This method is not only inefficient, but also unable to adapt to business expansion. When When the amount of access data reaches TB or even PB level, it cannot meet business needs in terms of timeliness and processing performance

Images