Method and system for building indexes and method and system for retrieving indexes

Abstract

An embodiment of the invention provides a method and a system for building indexes and a method and a system for retrieving the indexes. The method for building the indexes includes enabling an acquisition server to store acquired real data and generating index data according to storage positions of the real data; storing the index data into current buffer blocks according to a key value sequence; continuously arraying the index data in the buffer blocks into index blocks according to the key value sequence; sequentially arraying the index blocks to form index files. The index data comprise key values. The method for retrieving the indexes includes acquiring key values of target data; comparing the key values of the target data to key value ranges of the various index blocks in the index files and determining certain index blocks with the key value ranges covering the key values of the target data as target index blocks; searching the target index blocks to obtain index data with key values equal to the key values of the target data and using the index data as target index data; extracting storage positions of the target data from the target index data and reading the target data at the storage positions.

Description

Technical field [0001] The present invention relates to the technical field of data processing, in particular to an index establishment method and system, and a retrieval method and system. Background technique [0002] The data collection process can be summarized as the process of collecting data from the source server to the target server. For large-scale data collection at this stage, a dedicated collection server is often needed to complete this process. This will inevitably lead to extremely large data traffic in the collection server. Therefore, in scenarios with data query and retrieval requirements such as distributed multi-table associated collection, change data comparison, and business code replacement, the ability to quickly query and retrieve matching data has become a new problem that needs to be solved urgently. [0003] In the prior art, a cache technology is generally used to realize fast query and retrieval, that is, data with a higher query frequency is cached ...

AI Technical Summary

Problems solved by technology

It can be seen that the technical solution of using cache technology to retrieve in the collection server is difficult to meet the actual needs due to its slow speed and low efficiency

Images