Big data mining tool and method based on dragging process

Abstract

The invention discloses a big data mining tool and method based on a dragging process. The big data mining tool comprises a data management module, an operator library module, a process management module, a data mining engine module, a display design module and a visualization module. The method has the following beneficial effects: (1) logic separation of big data calculation and modeling is realized, advanced encapsulation of a big data mining algorithm library is realized in the aspects of multi-level reduction of a data mining threshold and big data calculation, and an operator does not need to know specific algorithm implementation; in the aspect of big data modeling, a graphical interactive prototype is developed, and a mining process is realized through intuitive dragging connection; (2) the optimal calculation mode can be automatically selected according to the operator content of the mining process, and the calculation efficiency is optimized; (3) in a big data scene, the calculation cost is greatly reduced; and (4) tedious cluster construction and maintenance are not needed, and the big data mining use threshold is reduced.

Description

technical field [0001] The invention belongs to a big data mining tool and method, in particular to a drag-and-drop process-based big data mining tool and method. Background technique [0002] Data mining is a key technology to obtain potential knowledge from massive information, and has become an important force to promote the development of various fields. Data mining has been fully utilized and developed on traditional small data sets, but on large data sets, due to its particularity in storage and calculation, the previous mature data mining algorithms cannot directly integrate with the computing framework of big data. Combined, the application of big data mining has a higher cost. [0003] In the big data scenario, data mining involves algorithm principles, algorithm parallel code implementation, and algorithm combination, which requires high professional knowledge of operators, and mining modeling is a process of repeated iterative optimization, and the operation effi...

AI Technical Summary

Problems solved by technology

Data mining has been fully utilized and developed on traditional small data sets, but on large data sets, due to its particularity in storage and calculation, the previous mature data mining algorithms cannot directly integrate with the computing framework of big data. integration, making big data mining applications have a higher cost

[0003] In the big data scenario, data mining involves algorithm principles, algorithm parallel code implementation, and algorithm combination, which requires high professional knowledge of operators, and mining modeling is a process of repeated iterative optimization, and the operation efficiency at the code level is low and insufficient. Intuitive, unable to satisfy fast big data mining analysis

Images