Intelligent grading method and system for big data and terminal

Abstract

The invention discloses an intelligent classification method for big data. The method comprises the following steps: reading training data and normalizing the training data; Creating a neural network,setting training parameters, and training the neural network; Reading test data, and normalizing the test data; And identifying the test data and outputting a result to realize the hot, and achievingtemperature and cold intelligent classification of the data. Meanwhile, the invention provides an intelligent grading system and a terminal. For three levels of hot, warm and cold data for big data,,deep learning is carried out through a multi-level neural network; The neural network is used as a classifier, the problem of standardization of a cross-industry classification algorithm is solved, big data of different industries can be divided into three categories of hot data, warm data and cold data according to first access time, last access time, access times, industry attribute codes and the like of the data, and preparation is made for intelligent hierarchical storage of the big data. By adopting the technical scheme, the identification accuracy of small sample data in different industries and fields reaches more than 90%.

Description

technical field [0001] The present invention relates to the technical field of big data hierarchical storage, in particular, to an intelligent big data grading method, system and terminal. Background technique [0002] In the era of big data with explosive growth in data volume, data can be divided into hot data, warm data, and cold data according to access frequency. Statistics show that the current cold data volume accounts for 80% of big data, so the storage of big data is mainly to solve the problem of cold data storage. The magneto-optical hybrid method of storing cold data in an optical disk library, storing warm data in a hard disk array, and storing hot data in solid-state disks and memory can combine the advantages of three types of storage media, such as disks, solid-state disks, and optical disks, to achieve long life (more than 50 years) ), large capacity (above PB level), low cost (50% reduction in initial construction cost, 80% reduction in energy consumption)...

AI Technical Summary

Problems solved by technology

Some people in China have used SVM (Support Vector Machine, Support Vector Machine) for classification, but the effect has not been good.

Tried to use BP (Back propagation) neural network, obtained a relatively good classification effect, but the efficiency and accuracy are still very low, unable to meet people's needs for intelligent classification of big data and automatic hierarchical storage

[0007] At present, there is no description or report of the similar technology of the present invention, and no similar data at home and abroad have been collected yet.

Images