A data analysis method and device based on bitmap data structure

Abstract

The embodiment of the invention provides a bitmap data structure-based data analysis method and device. The method comprises the steps of acquiring a first affair subset distributed by a main node; acquiring a total number and a ranking; determining bitmap data of each objective project according to the ranking; counting a ratio of first data to the total number in the bitmap data of each objective project, and determining a frequent first item set in the objective project according to the counted ratio; broadcasting the objective project of the frequent first item set and the bitmap data forthe objective projects of the frequent first item set to the main node and other distributed sub-nodes; receiving counted projects and the bitmap data of the counted project; based on the received counted projects, the bitmap data of the counted projects and the bitmap data for the objective projects of the frequent first item set, calculating whether the objective item set is the frequent item set; and if the objective item set is the frequent item set, determining an association rule between the projects in the objective item set. Therefore, the association relation and the association relation between the projects can be rapidly obtained.

Description

technical field [0001] The invention relates to the technical field of data mining, in particular to a data analysis method and device based on a bitmap data structure. Background technique [0002] With the explosive growth of data, people are more and more eager to mine valuable information from a large amount of existing data, and then make corresponding decisions based on these valuable information. [0003] For example, for large retail stores, these stores generate tens of thousands of transaction records every year, wherein each transaction record corresponds to an order number, and an order number corresponds to multiple item names. However, before data mining, people cannot get association rules between items in the transaction process (for example, 60% of people who buy coffee also buy pastries). After data mining, the purchase relationship between items in the transaction process can be mined, that is, the association rules between items, and then the marketing d...

AI Technical Summary

Problems solved by technology

However, this kind of matching speed is very slow, which makes the speed of obtaining association rules slow

Images