Data flow high-utility item set mining system based on historical utility table pruning

Abstract

The invention discloses a data flow high-utility item set mining system based on historical utility table pruning. Data flow high-utility item set mining based on a sliding window is one of the most challenging subjects in the field of data mining, due to the fact that a current algorithm can generate a large number of candidate item sets and redundant items, the performance is reduced when large-scale data flows are mined, and meanwhile, few historical mining results are referenced in the data flow mining process. The innovation point of the method is that the historical utility value table is established, the search space of the data stream is effectively built by using historical data, candidate items and redundant items are reduced, and a data mining system is constructed by using a distributed architecture, so that the historical utility value table is created and updated on the premise of not influencing data stream mining, so the efficient item set mining efficiency of the data stream is effectively improved.

Description

technical field [0001] The invention relates to a frequent pattern mining algorithm and a data stream mining system. [0002] High utility itemset mining is an important branch of frequent pattern mining. Background technique [0003] Frequent itemset mining is an important branch in the field of data mining. It can mine itemsets whose occurrence frequency exceeds the threshold set by the user from all transactions in the data set. With the wide application of frequent itemsets, people found that compared with frequent itemsets, some infrequent itemsets can create higher value. In response to this problem, scholars proposed the concept of high-utility itemsets mining, high-utility itemsets The set overcomes the defects of frequency of occurrence, price, profit, regional distribution and other data item weight information that are not considered in the frequent mining item set, and evaluates the importance of the item set through comprehensive utility indicators. [0004] A...

AI Technical Summary

Problems solved by technology

[0007] The current pattern growth algorithm inevitably has problems such as candidate item sets, too many redundant items, and useless processing of low-utility data, which often leads to high space complexity of the high-utility tree structure constructed, which makes the mining process difficult. Frequent recursive creation of subtrees will eventually cause problems such as memory overflow and low algorithm efficiency

Images