Distributed convergence behavior mining method and system

Abstract

The invention relates to a distributed convergence behavior mining method and system. The method comprises the following steps of: obtaining a data set formed by four groups of data, and dividing thedata set into a plurality of event sets according to event types, wherein the four groups of data comprise objects, events, time points and weights; constructing index trees for the event sets, determining a total time length of the index trees according to the time points in the event sets, splitting the total time length into time periods according to a segment tree division method, and storingthe index trees to a distributed database, wherein each node of the index trees corresponds to a weight sum of all objects in the time periods; obtaining a to-be-queried event and a to-be-queried timeperiod of a to-be-queried object, searching an index tree corresponding to the to-be-queried event from the distributed database to serve as a to-be-queried index tree, querying a weight sum of all objects in the to-be-queried time period in the to-be-queried index tree to serve as a convergence degree of the to-be-queried object, and selecting the object with the highest convergence degree as amining result.

Description

technical field [0001] The invention relates to the field of data mining, in particular to a distributed trend peer behavior mining method and system. Background technique [0002] Any data that can be abstracted into (object, event, time, weight) quadruples can calculate the degree of convergence between objects. Wherein, "object" refers to it in general, and when it corresponds to a specific application, such as detecting a mouse warehouse in the following embodiment, its "object" is "user". The goal of Lockstep BehaviorMining (LBM) is to mine out the object with the highest degree of convergence in the database under a given behavior set of an object. The behavior set refers to all (events, time, weights) belonging to the object )Triad. [0003] If each object splices the vectors on each event (time is the dimension, and the value on each dimension is the weight) into an object vector in order, and the length of each object vector is |E|*|T|, here E is the set of all e...

AI Technical Summary

Problems solved by technology

[0009] In the research of the mouse warehouse behavior mining algorithm, the original algorithm is implemented on a single machine using SQL language, which not only has high algorithm complexity, but also cannot be distributed, resulting in low efficiency. It takes several days to dig out the behavior of the mouse barn, which cannot satisfy the effective supervision of the behavior of the mouse barn

[0010] After researching the related maximized inner product retrieval algorithm, it is found that most of the maximized inner product retrieval algorithms use LSH to find approximate solutions, but their accuracy and efficiency cannot meet the requirements under high-dimensional sparse data.

While the existing tree-based methods are less practical on large-dimensional datasets, even inferior to LSH

Images