Distributed fast frequent item set mining method based on Apriori

A frequent item set mining and distributed technology, which is applied in special data processing applications, instruments, electrical digital data processing, etc., can solve problems such as memory leaks, achieve simplified statistics, meet the needs of frequent item set mining, and avoid frequent scanning Effect

Inactive Publication Date: 2016-12-07
SOUTHEAST UNIV
View PDF0 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Then most Apriori-based parallelization methods still rely on global frequent 1-itemsets, while FP-Growth-based parallelization methods also face the problem of memory leaks

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed fast frequent item set mining method based on Apriori
  • Distributed fast frequent item set mining method based on Apriori
  • Distributed fast frequent item set mining method based on Apriori

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0014] Below in conjunction with specific embodiments, the present invention will be further illustrated, and it should be understood that these embodiments are only used to illustrate the present invention and not to limit the scope of the present invention. The modifications all fall within the scope defined by the appended claims of this application.

[0015] All Apriori-like association rule mining methods mainly include two main processes: frequent itemset generation and pruning. The distributed frequent itemset mining proposed in this paper is no exception. The main similarities and differences are mainly reflected in Spark-based parallelization. The implementation part is described in detail below. In terms of implementation, the whole method can be divided into preprocessing ( image 3 (a)) and iterative ( image 3 (b)) Two steps, both of which include candidate item set generation and pruning.

[0016] The method handles input datasets such as figure 1 shown, the ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a distributed fast frequent item set mining method based on Apriori. A corresponding thing set, where an item set is, is recorded while the item set is recorded; therefore, statistics of the frequency of the item set is realized by utilizing simple thing number statistics at a frequent item set mining pruning part; and thus, the purpose of pruning is achieved. The distributed fast frequent item set mining method based on Apriori disclosed by the invention has the advantages that: frequent scanning to an input data set by the traditional method is overcome; therefore, the method has good expandability; simultaneously, in combined with Spark, the method is realized; and thus, a frequent item set mining task in a big-data era can be processed well.

Description

technical field [0001] The invention relates to a distributed fast frequent itemset mining method based on Apriori, which is used for realizing distributed and efficient frequent itemset mining in big data application scenarios, and belongs to the technical field of association rule mining of data mining. Background technique [0002] Apriori has received extensive attention since its creation, and it mainly consists of two steps: candidate itemset generation and pruning. The generation of candidate item sets is mainly achieved through item order lattices, and the pruning process requires frequent scanning of the input data set to achieve item frequency statistics. Obviously, the frequent scan of the input dataset in the pruning process will inevitably lead to high time complexity. In order to overcome this drawback, researchers proposed FP-Growth, by constructing a hash tree of candidate item sets in memory, so as to realize the statistics of itemset frequency and avoid fr...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/182
Inventor 杨鹏吕培培顾梁董永强
Owner SOUTHEAST UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap