Data mining system and application thereof in big data environment

Abstract

The invention discloses a data mining system under a big data environment and an application thereof. The data mining system includes a data preprocessing module, a data mining module, a pattern evaluation module, and a knowledge representation and system background management module. The system can be applied to the internet of things system, geographic information system, the power dispatching automation system, the remote monitoring system, the agricultural information system, the financial and insurance information system and the e-commerce system. The system sets up the standard in advance for the massive data information database, the massive mining algorithm and the compilation of all kinds of visual components, and connects all the components conforming to the standard to the platform, so that other components can be invoked. By using the GEF and RCP technology, users can have a simple and convenient interface to improve the visualization of data mining. The data mining systemand the application thereof can support a large number of data pre-processing algorithms, a variety of complex data sources and many classic mining algorithms, and can be used in multi-industry and multi-domain analysis, search and research decision-making.

Description

technical field

[0001] The invention relates to a data mining system, in particular to a data mining system and its application in a big data environment. Background technique

[0002] With the rapid development of information technology, all walks of life in society continue to generate a variety of massive data, resulting in explosive data growth and entering the era of big data. The massive amounts of data accumulated in various industries generally contain rich knowledge of hidden and usable value, but at the same time, big data is also characterized by unpredictable and diverse content, so powerful data analysis tools are urgently needed to analyze and process the data. So as to provide powerful information support for business decision-making, education and other fields. Data mining technology is a symbol of the information age, which refers to the process of finding useful information from a large amount of complex and complex massive data stored in various informati...

Examples

Embodiment Construction

[0021] Data mining system in big data environment, including:

[0022] The data preprocessing module is used to collect the original data that needs to be mined from the data source, and then use the cleaning algorithm and call the relevant interface services of the model to perform cleaning operations on the original data and output the cleaning results, and then integrate the described data by category Clean the results and select relevant data with useful value, and finally submit the preprocessing results to the control layer after type conversion;

[0023] The data mining module first separates the preprocessing results of the data preprocessing module by category through the controller; secondly, uses the corresponding classification algorithm to analyze and study different types of data, and find out hidden valuable data; finally, the controller according to The type of sampling result selects the optimal mining type and mining algorithm to complete the data mining task...

technical field

[0001] The invention relates to a data mining system, in particular to a data mining system and its application in a big data environment. Background technique

[0002] With the rapid development of information technology, all walks of life in society continue to generate a variety of massive data, resulting in explosive data growth and entering the era of big data. The massive amounts of data accumulated in various industries generally contain rich knowledge of hidden and usable value, but at the same time, big data is also characterized by unpredictable and diverse content, so powerful data analysis tools are urgently needed to analyze and process the data. So as to provide powerful information support for business decision-making, education and other fields. Data mining technology is a symbol of the information age, which refers to the process of finding useful information from a large amount of complex and complex massive data stored in various informati...

Examples

Embodiment Construction

[0021] Data mining system in big data environment, including:

[0022] The data preprocessing module is used to collect the original data that needs to be mined from the data source, and then use the cleaning algorithm and call the relevant interface services of the model to perform cleaning operations on the original data and output the cleaning results, and then integrate the described data by category Clean the results and select relevant data with useful value, and finally submit the preprocessing results to the control layer after type conversion;

[0023] The data mining module first separates the preprocessing results of the data preprocessing module by category through the controller; secondly, uses the corresponding classification algorithm to analyze and study different types of data, and find out hidden valuable data; finally, the controller according to The type of sampling result selects the optimal mining type and mining algorithm to complete the data mining task...