A massive data storage method based on mongodb

Abstract

The invention provides a mass data storage method based on mongodb, which is implemented as follows. 1) a data storage req: a data logical distribution processing cent is added between that data storage and the user request; Before each piece of data is stored, a request must be made to the processing center to pass in the required parameters; 2) data cluster storage: according to the returned storage information, finding the corresponding cluster and storing it; 3) data sub-warehouse storage: according to returned storage information, finding the corresponding warehouse under the corresponding cluster and storing the corresponding warehouse; 4) data table storage: accord to that returned storage information, the correspond table is found and the final data storage is carried out; 5) Value: when querying a piece of data, it is necessary to go to the distribution processing center to obtain the position information of the data, that is, the center obtains the position information of thepiece of data according to the stored same rule and returns the position information, and then searches for the piece of data according to the returned position information.

Description

technical field [0001] The invention relates to a massive data storage method, specifically a mongodb-based massive data storage method, and can also be applied to other databases. Background technique [0002] At present, the Mongodb database itself is a non-relational database designed to store information, but there are some problems in actual use: [0003] 1. The amount of data supported by a single data cluster is still limited, and there will be similar problems when using sharding technology; [0004] 2. To speed up the query, it is necessary to add an index, and the response speed of the index will slow down; [0005] 3. Even if multiple clusters are used to store massive data, if there is no good storage solution, there will still be performance bottlenecks, because multiple clusters mean that data processing will be very complicated. Under the multi-cluster architecture, data deployment requires Centralized and unified processing requires corresponding solutions....

AI Technical Summary

Problems solved by technology

[0003] 1. The amount of data supported by a single data cluster is still limited, and there will be similar problems when using sharding technology;

[0004] 2. To speed up the query, it is necessary to add an index, and the response speed of the index will be slower;

[0005] 3. Even if multiple clusters are used to store massive data, if there is no good storage solution, there will still be performance bottlenecks, because multiple clusters mean that data processing will be very complicated. Under the multi-cluster architecture, data deployment requires Centralized and unified processing requires corresponding solutions

Images