A big data classification method and system

Abstract

The invention discloses a big data classification method and system. The method includes the steps: training, namely dividing input data into input data blocks, generating classification rules (pattern strings => class marks) of pattern strings from the input data blocks, and writing the classification rules into a Hbase database rule table; testing, namely, reading the input data blocks, constructing pattern strings to be classified, searching for classification rules matching with the pattern strings to be classified, in the Hbase database rule table, and outputting classification results. Therefore, the big data classification method and system based on hypersurface is provided; classification can be achieved by a hypersurface-based covering algorithm on the basis of a Hadoop mapping / simplification programming frame and a Hbase distributed non-relational database, a rule model easy to explain can be constructed at low calculation cost, big data is quickly and efficiently processed, and the requirement for classifying explosively-increasing data in real world is met.

Description

technical field [0001] The invention relates to the field of big data analysis, in particular to a hypersurface-based big data classification method and system. Background technique [0002] Classification is an important form of data analysis used to extract models that characterize important classes of data. Such a model is called a classifier and it is used to predict the class label for classification. Data classification is a two-stage process, including a learning stage and a classification stage. The learning stage is the stage of building a classification model, and the classification stage is to use the model to predict the class label of the given data. For example, we can build a classification model that classifies bank loan applications as safe or dangerous. This analysis can help us better understand the data comprehensively. Many classification and prediction methods come from machine learning, pattern recognition, and statistics. Most of the algorithms ar...

AI Technical Summary

Problems solved by technology

[0024] In order to solve the above problems, the purpose of the present invention is to provide a large data classification method and system based on hypersurfaces, which can solve the problems of unstable prediction accuracy, high calculation cost, slow speed, high model complexity and difficulty in the above-mentioned prior art. Interpretation, the problem of being unable to handle massive data

Images