Method and device for realizing association rule mining algorithm supporting distributed computation

Abstract

The invention discloses a method and a device for realizing an association rule mining algorithm supporting a distributed computation. An HDFS (Hadoop Distributed File System) programming model is used to carry out two-stage analysis of a map function stage and a reduce function stage on the association rule mining algorithm, and the analysis steps comprises the following steps: step 1, a job scheduler is configured; step 2, a data set is read by a prior probability mapping module, and the data of the data set are converted by a map function into a value pair; step 3, the value pair processed in the step 2 is read by the prior probability reduction module, an ordering rule Top N containing an i item set is randomly generated by a reduce function, and the prior probability distribution value of a confidence coefficient is calculated at the same time; step 4, the same data set is read by a rule mapping module, and the data row of the data set is converted by the map function into the value pair; and step 5, the value pair processed in the step 4 and the prior probability distribution value in the step 3 are read by a rule reduction module, and the predication accuracy value of the ordering rule Top N is calculated by the reduce function. The method and the device for realizing the association rule mining algorithm supporting the distributed computation are mainly applied to the PA (Pridictive Apriori)-distribution type computing technology.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a method and device for realizing an association rule mining algorithm supporting distributed computing. Background technique [0002] With the advent of the "big data" era, the amount of enterprise business data has increased sharply. Data analysts are trying various data analysis methods and data mining methods, aiming to discover potential user behaviors with business value from massive data. model. Data Mining: A technique for finding patterns in large amounts of data by analyzing each piece of data. In addition, big data, massive data, and data sets mentioned in the present invention have the same meaning. [0003] Association rule mining is a widely used and influential method in data mining methods, which can be used in various recommendation systems to recommend items of interest to users. Currently, the various versions of association rule mining algorithms that can...

AI Technical Summary

Problems solved by technology

[0011] The present invention aims to solve the problems that the existing technology cannot handle massive data mining, the calculation efficiency is very slow and the rule mining results are not comprehensive enough, and combines the PA association algorithm and the Hadoop distributed computing framework to provide a method that can handle massive data mining and has high calculation efficiency. Very fast, an association rule mining algorithm implementation method and device supporting distributed computing that can more comprehensively, quickly and efficiently mine association rule results for business support from massive business data

Images