Big data storage system

Abstract

The invention discloses a big data storage system, which comprises a target data acquisition module for acquiring target data through multiple ports and sending the acquired data to a data preprocessing module; the data preprocessing module is used for clearing redundant data on the basis of an ISODATA algorithm; the data feature extraction module is used for extracting feature data of the targetdata by adopting an attribute reduction algorithm based on attribute importance; the data classification module is used for realizing classification of target data according to the feature data basedon a naive Bayes classification algorithm; and the data positioning module is used for finding a proper position in the database for the target data in the database according to the classification result of the data classification module, finding similar data points for the target data, and establishing a relationship between the target data and the similar data points. According to the invention,while automatic arrangement and storage of big data can be realized, numerous and diverse mass data can be converted into available data with information and commercial value.

Description

technical field

[0001] The invention relates to the field of big data, in particular to a big data storage system. Background technique

[0002] With the explosive growth and accumulation of information, the era of big data is approaching. The basic characteristics of big data are: large data volume, various types, low value density, fast speed and high timeliness; for big data, correlation is more important than causality. More importantly, this is the conclusion of the era of big data. In the face of massive amounts of information, how to analyze the correlation between certain types of information and analyze the hidden value behind the information can reflect the data at a higher and deeper level. The value of information, but in the face of such a massive amount of data, it is very difficult to quickly and accurately analyze the relationship between the data.

[0003] At present, the traditional data storage system generally has the disadvantages of single function and...

Examples

Embodiment Construction

[0027] In order to make the objects and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the examples. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

[0028] Such as figure 1 As shown, the embodiment of the present invention provides a kind of big data storage system, comprises:

[0029] The target data acquisition module collects the target data through multiple ports, and sends the collected data to the data preprocessing module;

[0030] Data preprocessing module, based on ISODATA algorithm to realize the removal of redundant data;

[0031] The data feature extraction module uses an attribute reduction algorithm based on attribute importance to extract feature data of target data;

[0032] A data classification module, based on the Naive Bayesian classification algorithm, realizes the cl...

technical field

[0001] The invention relates to the field of big data, in particular to a big data storage system. Background technique

[0002] With the explosive growth and accumulation of information, the era of big data is approaching. The basic characteristics of big data are: large data volume, various types, low value density, fast speed and high timeliness; for big data, correlation is more important than causality. More importantly, this is the conclusion of the era of big data. In the face of massive amounts of information, how to analyze the correlation between certain types of information and analyze the hidden value behind the information can reflect the data at a higher and deeper level. The value of information, but in the face of such a massive amount of data, it is very difficult to quickly and accurately analyze the relationship between the data.

[0003] At present, the traditional data storage system generally has the disadvantages of single function and...

Examples

Embodiment Construction

[0027] In order to make the objects and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the examples. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

[0028] Such as figure 1 As shown, the embodiment of the present invention provides a kind of big data storage system, comprises:

[0029] The target data acquisition module collects the target data through multiple ports, and sends the collected data to the data preprocessing module;

[0030] Data preprocessing module, based on ISODATA algorithm to realize the removal of redundant data;

[0031] The data feature extraction module uses an attribute reduction algorithm based on attribute importance to extract feature data of target data;

[0032] A data classification module, based on the Naive Bayesian classification algorithm, realizes the cl...

Images