Data mining method and system for big data analysis

Abstract

The invention provides a data mining method and a data mining system for big data analysis. The data mining method comprises the steps of: S1, data collection, which is implemented by acquiring a dataset for data mining from a database according to needs; S2, data processing, which is implemented by checking the integrity and consistency of each piece of data in the data set, performing denoisingprocessing, missing domain filling and invalid data deletion on each piece of data, converting the cleaned data into target data, and extracting the target data to generate preprocessed data; S3, data analysis, which is implemented by correcting a data vector according to an analysis rule, summarizing and performing multi-dimensional analysis on the corrected data to generate a data analysis result; and S4, data management, which is implemented by explaining and evaluating the data analysis result, and generating a data mining report. According to the data mining method and the data mining system, the data is collected, stored, processed and analyzed, then the analyzed data is presented to a user in the modes of characters, pictures, tables and the like, the purpose of data integration isachieved, and the data integration is rapid and visual.

Description

technical field [0001] The invention relates to the technical field of data mining, in particular to a data mining method and system for big data analysis. Background technique [0002] Big data analysis refers to the process of analyzing a large amount of collected data with appropriate statistical analysis methods, extracting useful information and forming conclusions to study and summarize the data in detail. This process is also a supporting process of the quality management system. In practical terms, data analysis can help people make judgments so that appropriate actions can be taken. [0003] In recent years, with the rapid growth of database information and the continuous upgrading of storage devices, the diversification of data has become more and more obvious. There are two main reasons for the formation of data diversification: First, there are many data sources, including search engines, social networks, call records, sensors, etc.; second, there are many data...

AI Technical Summary

Problems solved by technology

[0005] In view of the shortcomings of the prior art described above, the purpose of the present invention is to provide a data mining method and system for big data analysis, which is used to solve the problems of large data volume, multiple structural forms and real-time performance in the prior art. requirements, data collection and integration difficulties

Images