Mass data storage and quick retrieval method based on MongoDB and Solr

Abstract

The invention discloses a mass data storage and rapid retrieval method based on MongoDB and Solr. The method includes: for storage of unstructured data, calling a MongoDB database to store the unstructured data; creating a MongoDB resource information unique code corresponding to the unstructured data, calling Solr to create index information of the unstructured data, and storing the index information of the unstructured data and the MongoDB resource information unique code into an index library of the Solr; for rapid retrieval of the unstructured data, carrying out searching in an index library of Solr according to the query statement, and finding out index information and MongoDB resource information unique codes of the unstructured data corresponding to the query statement; and searching unstructured data corresponding to the MongoDB resource information unique code in a MongoDB database. According to the method, the requirements of storage and quick retrieval of various unstructured data in daily business are met, and quick retrieval of mass unstructured data is realized so as to quickly obtain required information.

Description

technical field [0001] The invention relates to the technical field of fast retrieval of big data, in particular to a massive data storage and fast retrieval method based on MongoDB and Solr. Background technique [0002] With the development of computer information technology and the application of big data, more and more data are flooding our daily life. For example, for public security agencies, using urban video surveillance resources to detect cases and form various case materials Information has accumulated a large amount of unstructured data such as pictures and videos. Therefore, how to quickly obtain the required information from these massive unstructured data has become a hot and difficult issue in massive data retrieval. [0003] After searching, the Chinese Patent Publication No. CN106682148A discloses a method and device based on Solr data search. In this patent, it is proposed that the underlying storage database of Hbase or MongoDB is selected, and the rowke...

AI Technical Summary

Problems solved by technology

However, since JVM memory and Solr's memory configuration, disk usage, transaction logs and other related content are relatively cumbersome and complicated, when dealing with massive data, optimization of these aspects requires trial and error and is not easy to implement.

Images