Data mining method for quickly finding utility pattern

Abstract

A data mining method for quickly finding a utility pattern can find a utility pattern which not only has substantial statistical characteristics but also meets user expectations and user goals from massive data, having a wide application in network information search and knowledge discovery. Aiming at solving the present problems of high time overhead and space overhead of existing methods caused by adoption of a two-stage method which generates a candidate pattern, the present invention provides three innovative technologies. The first is data representation based on a sparse matrix and virtual projection, the second is a prefix growth strategy, a prefix growth tree and a tailoring method thereof, and the third is a depth-first dynamic search method. With the three innovative technologies, a novel mining method is designed which has a single stage, causes no candidate pattern, and enables mining the utility pattern. The time efficiency ratio of the data mining method is higher by one to three orders of magnitude than that of other three referential mining methods, and the memory usage is reduced by 40% to 90%. The present data mining method has a high performance and enables various applications such as massive Web mining, multimedia mining and test mining.

Description

technical field [0001] The invention relates to the field of intelligent information processing. The present invention designs a utility mode mining method that can discover not only significant statistical features but also meet user expectations and goals from massive data, especially in massive data mining, especially network information search and knowledge discovery, including Web mining, text mining, multimedia In mining, it has broad application prospects. Background technique [0002] Traditional data mining technology, especially frequent pattern mining technology [1][2], mainly analyzes data based on statistical significance, such as digging out product combinations with high purchase frequency from supermarket sales data, without considering the user's Expectations or goals, such as the user may be interested in a product portfolio with a higher profit return. That is to say, in data mining, not only the statistical significance of the data, but also the user's ...

AI Technical Summary

Problems solved by technology

Not only does this cause excessive storage space overhead, leading to a scalability bottleneck for the first stage, but also for the second stage, and ultimately leads to inefficient runtime

[0006] In order to overcome the defects of previous mining methods, the present invention proposes the following three innovative technologies to get rid of the framework of the two-stage method, and designs "a data mining method for quickly discovering utility patterns", thereby solving the bottleneck of scalability and efficiency question

Images