Index search method and system of data file

Abstract

The invention provides an index search method and system of a data file. The method comprises the following steps: obtaining a keyword for the search of the data file; reading an index file, and positioning a logic block where the keyword is positioned through a Bloom filter in the index file; and searching to obtain data associated with the positioned logic block, and outputting the data obtained by search. The system comprises a keyword acquisition module, a logic block positioning module and a search module, wherein the keyword acquisition module is used for obtaining the keyword used for carrying out data file search; the logic block positioning module is used for reading the index file and positions the logic block where the keyword is positioned through the Bloom filter in the index file; and the search module is used for searching to obtain the data associated with the positioned logic block and outputs the data obtained by the search. Search speed can be improved on the premise of high writing speed.

Description

technical field [0001] The invention relates to data storage technology, in particular to an index search method and system for data files. Background technique [0002] With the development of Internet applications, more and more business systems have massive log data, and these massive log data will be stored in the hard disk for future query and use when needed. [0003] These massive log data have the characteristics of huge writing volume and relatively low reading frequency. Therefore, most of the traditional log data storage is directly stored in the hard disk without any index to avoid the existence of the index. However, due to the large amount of data when searching the written log data, it usually takes several hours to find the required data, which cannot be improved on the premise of ensuring high writing speed. Find speed. [0004] However, the traditional data indexing algorithm sacrifices the writing speed to achieve the purpose of fast search. Among them, ...

AI Technical Summary

Problems solved by technology

[0003] These massive log data have the characteristics of huge writing volume and relatively low reading frequency. Therefore, most of the traditional log data storage is directly stored in the hard disk without any index to avoid the existence of the index. However, due to the large amount of data when searching the written log data, it usually takes several hours to find the required data, which cannot be improved under the premise of ensuring a high writing speed. search speed

[0004] The traditional data indexing algorithm sacrifices the writing speed to achieve the purpose of fast search. Among them, the traditional data indexing algorithm includes B-tree indexing algorithm, inverted indexing algorithm and hash indexing algorithm, etc., so it cannot guarantee high Improve search speed under the premise of writing speed

Images