Weighted association rule mining method based on data source partition matrix

Abstract

The invention relates to a weighted association rule mining method based on a data source partition matrix, and belongs to the technical field of manual intelligence, machine learning and data mining. By using a manual intelligence method, implied valuable expert knowledge can be mined from massive and complex industrial text data and expressed in an association rule manner, and a support can be provided for establishment of a knowledge base in an expert system. According to the method, a classic association rule mining algorithm is analyzed; a method for rapidly computing a support degree based on the data source partition matrix is provided aiming at the problem that an Apriori algorithm is low in efficiency as a data set is scanned repeatedly; and a weighted calibration method based on probability is used for weighting data item sets during an association rule mining process aiming at the problem that the Apriori algorithm treats data items in the transaction data set equally. Experiments verify that the provided method can reduce space complexity and time complexity of traditional association rule mining.

Description

technical field [0001] The invention uses a data mining method to mine association rules for valuable expert knowledge hidden in massive industry text data to obtain expert knowledge represented by association rules; it belongs to the fields of artificial intelligence, machine learning, and data mining. Background technique [0002] Association rules are used to describe a certain relationship between a thing and other things, which mainly manifests as the mutual influence relationship between things. In most cases, the mutual influence relationship between a large number of things cannot be discovered manually, and the computer artificial intelligence technology is used to organize and analyze the data, and express these relationships in the form of rules, which is association rule mining. For example, in a data set, assuming that there is a relationship between two things, it means that the information of one thing can be used to infer the other thing related to it. In co...

AI Technical Summary

Problems solved by technology

Usually, the amount of data in the database is relatively large, which will take a lot of time for information reading operations, resulting in an excessively high time cost for the algorithm.

[0026] (2) Due to the large amount of data, a large number of candidate frequent itemsets will be generated when the connection operation is performed during the algorithm calculation process, which will also affect the calculation efficiency

[0027] (3) The traditional Apriori algorithm treats the data items in the transaction database equally, and thinks that each data item plays the same role in association rule mining, and cannot reflect the emphasis of the data item on the association rule conclusion, which will lead to Some "trivial" rules may appear in the results of mining, which has certain limitations

Images