Data storage, retrieval method, device, server and storage medium

Abstract

The embodiment of the invention discloses a data storage and retrieval method, device, server and storage medium. The data storage method includes: preprocessing the data to be stored; and storing the preprocessed data to be stored in a corresponding database according to the data type of the data to be stored. Through the above technical solution, simultaneous and comprehensive storage of different types of data can be realized, and costs can be reduced.

Description

technical field [0001] The embodiments of the present invention relate to the technical field of big data processing, and in particular to a data storage and retrieval method, device, server and storage medium. Background technique [0002] With the rapid development of big data storage technology and the rapid popularization of enterprise informatization, enterprises have more and more various electronic files and data information, and are facing the challenge of "information explosion", and different data There is a certain correlation between information. For enterprise employees, if they want to find the data information inside the enterprise, they need to search a variety of materials, including structured data stored in the database, as well as unstructured data in emails, web pages, documents, etc., and the data information needs to be analyzed. Correlation and comprehensive analysis can provide perfect search results for different business needs. [0003] At presen...

AI Technical Summary

Problems solved by technology

[0003] At present, most of the solutions for big data storage are large-scale solutions based on KV libraries such as Hadoop Hbase or Cassandra. Processing) support is not perfect, although there is also a SQL-on-Hadoop solution, but its core principle is still offline computing technologies such as MapReduce, and the real-time performance is poor

For example, NoSQLDatabase needs to do calculations based on a large range of data when it is oriented to analytical applications, and even scan the entire table, which is slow, and because the data is queried by key-value, a Key queries a Value, so its The storage method is not conducive to querying complex conditions

Another example is the search engine (Search Engine), which does not have SQL support, it is difficult to connect with enterprise data applications and BI (Business Intelligence, business intelligence) tools, and it cannot store and integrate comprehensive business information

Another example is SQL-on-Hadoop, since there is no index support, it is not conducive to query complex data with filter conditions

Another example is the parallel database (Parallel Database), which does not support the retrieval of full-text data, and cannot meet the needs of comprehensive storage and retrieval of different types of data

[0004] Generally speaking, the data that needs to be stored in the database has different types, and the nature of different types of data is quite different. The existing storage methods cannot comprehensively store various types of data, and lack perfect functional support for business systems. , and usually require different hardware devices to manage different types of data, thus increasing the cost of data storage

Images