Check patentability & draft patents in minutes with Patsnap Eureka AI!

Mass data frequent item set mining method

A technology of frequent itemset mining and frequent itemsets, applied in data mining, digital data processing, special data processing applications, etc., can solve problems such as incorrect data structure construction and complex data structure

Pending Publication Date: 2019-09-10
HARBIN INST OF TECH AT WEIHAI
View PDF2 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

By using this data structure, frequent itemsets can be efficiently calculated. However, the construction of data structures by such algorithms is very complicated, and when processing massive data, the memory demand usually exceeds the available memory, resulting in data structures that cannot be stored in memory. Build correctly in

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Mass data frequent item set mining method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0083] figure 1 is a schematic flowchart of a method in one embodiment of the present invention. in, figure 1 The execution subject can be a computing node, a server, or an ordinary PC. The massive data frequent itemset mining method is used to mine the frequent itemsets satisfying the global minimum support degree minsup in the total transaction data set T, and the global minimum support degree minsup is the minimum support on the preset total transaction data set T degree, the total transaction data set T includes the original transaction data set T O and the new transaction data set T Δ .

[0084] see figure 1 , the massive data frequent itemset mining method includes:

[0085] Step 110, use the frequent itemset mining algorithm to analyze the original transaction data set T O Mining to obtain the original transaction data set T O corresponding to all local frequent itemsets.

[0086] Specifically, the original transaction data set T can be read sequentially O and...

Embodiment 2

[0141] Compared with Embodiment 1, the difference is that the massive data frequent itemset mining method described in Embodiment 2, in order to further improve the mining rate of the present invention, between the above-mentioned step P11 and step P12, also includes the step S: for the set LF k,Δ The fine cutting step;

[0142] where, for the set LF k,Δ The refinement steps include:

[0143] According to whether the first (k-1) items of the item set are the same, obtain and divide the set LF k,Δ The grouping of frequent k-itemsets in , obtains a corresponding number of itemset groups, wherein, the first (k-1) items of the frequent k-itemsets in the same itemset group are the same;

[0144] Count the number of frequent k-itemsets in each itemset group separately, and judge whether the counted numbers are equal to 1: if yes, delete the corresponding itemset group, and delete the set LF k,Δ Among the itemsets identical to the frequent k-itemsets in the corresponding itemset ...

Embodiment 3

[0158] Compared with Embodiment 2, the difference is that, in the method for mining frequent itemsets of massive data described in Embodiment 3, when updating the newly added transaction data set T Δ , and upon updating the original transaction dataset T O is the original original transaction data set T O and the original new transaction data set T Δ and the updated new transaction data set T Δ When non-empty, also includes the step of updating mining.

[0159] Specifically, the step of update mining described in this embodiment includes:

[0160] Update the number n of transactions in the total transaction data set T to the original original transaction data set T O and the original new transaction data set T Δ The sum of the number of transactions;

[0161] Obtain the original original transaction data set T obtained above O For all the corresponding local frequent itemsets, calculate the obtained original original transaction data set T O The corresponding local fre...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a mass data frequent item set mining method, which comprises the following steps of: mining an original transaction data set TO by adopting a frequent item set mining algorithmto obtain all local frequent item sets corresponding to the original transaction data set TO; scanning an original transaction data set TO; correspondingly calculating the support degree count of eachobtained local frequent item set on the original transaction data set TO; filtering the obtained local frequent item set, obtaining each local frequent item set of which the support degree is not less than omega, and correspondingly writing each obtained local frequent item set and the calculated corresponding support degree count into a file Fqf; reading the newly added transaction data set T delta, judging whether the newly added transaction data set T delta is empty or not, and then performing frequent item set mining based on whether the newly added transaction data set T delta is empty or not. According to the method, the file Fqf is multiplexed in the whole mining process, the STCAD and the array cnt delta are integrated, the calculation expenditure is reduced to a certain extent, and therefore the mining rate of a frequent item set can be increased.

Description

technical field [0001] The invention relates to the technical field of data mining, in particular to a method for mining frequent itemsets of massive data. Background technique [0002] Frequent itemset mining has always been one of the most active fields in data mining. It has a very wide range of applications in real life, for example, it is widely used in many research fields such as data mining, software error detection, spatiotemporal data analysis, biological analysis, etc. Due to its practical significance, frequent itemset mining has attracted extensive attention. [0003] In the field of data storage, data is usually stored in read-only / add-only mode, and the entire transaction data set can be divided into two parts: the original transaction data set and the new transaction data set. Under a certain time or condition, the data in the new transaction data set is merged into the original transaction data set. At this time, the data in the original transaction data s...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/2458
CPCG06F16/2465G06F2216/03
Inventor 韩希先陈剑赖国骏
Owner HARBIN INST OF TECH AT WEIHAI
Features
  • R&D
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com