Read optimization method for asynchronously updating indexes based on B+ tree

Abstract

The invention relates to a read optimization method for asynchronously updating indexes based on a B+ tree. The read optimization method comprises the steps of 1) generating index cache files, 2) obtaining all the index cache files, 3) filtering and removing incomplete or being-written index cache files, 4) determining whether the condition of initiating asynchronous update is satisfied, and if so, continuing the performing of the step 4), 5) reading index data in memory areas corresponding to the same memory area number of each filtered index cache file, respectively, 6) reading index data in memory areas corresponding to memory area numbers in an index library, 7) organizing the index data into a B+ tree according to the rule of the B+ tree, 8) serializing the B+ tree generated in the step 7) into a binary system stream, and 9) performing coverage and update of the data processed in the step 8) into the memory areas corresponding to the memory area numbers in the index library. Compared with the prior art, the read optimization method for asynchronously updating the indexes based on the B+ tree has the advantage of reducing the index data size inquired and read from mass index records each time.

Description

technical field

[0001] The invention relates to the field of data storage, in particular to a reading optimization method for asynchronously updating indexes based on a B+ tree, which is applicable to scenarios where it is impossible to load all index records into memory due to the existence of a large number of index records. Background technique

[0002] With the continuous development of computers, there are more and more valid data in computers, and the increase of valid data will make it impossible for users to quickly find the data they need in a large amount of valid data; if there is no effective way to quickly find what users need data, the increase in effective data can only increase the burden on users.

[0003] The introduction of indexing technology solves this problem very well. During the data writing process, some important information will be recorded to uniquely identify the data and record the storage location and size information of the data. We call thi...

Examples

Embodiment

[0037] Assuming that the number of records in the index is 100 million, the index database divides the index into 500,000 storage areas according to 500,000. A batch write operation is initiated every time the index records in the memory reach 1 million. Assuming that the total number of all index cache files after excluding incomplete index cache files and the index cache files being written reaches 3, the condition is considered to be satisfied and an asynchronous update operation can be initiated . Such as figure 1 As shown, this example includes the following steps:

[0038] 1. Divert the 1 million index records cached in the memory according to the storage area numbers 0 to 499999.

[0039] 2. Sort the index corresponding to each storage area number in step 1 in ascending order.

[0040] 3. Serialize the sorted indexes corresponding to each storage area number in step 2 into data streams.

[0041] 4. Combine the serialized data streams in step 3 into a unique data str...

technical field

[0001] The invention relates to the field of data storage, in particular to a reading optimization method for asynchronously updating indexes based on a B+ tree, which is applicable to scenarios where it is impossible to load all index records into memory due to the existence of a large number of index records. Background technique

[0002] With the continuous development of computers, there are more and more valid data in computers, and the increase of valid data will make it impossible for users to quickly find the data they need in a large amount of valid data; if there is no effective way to quickly find what users need data, the increase in effective data can only increase the burden on users.

[0003] The introduction of indexing technology solves this problem very well. During the data writing process, some important information will be recorded to uniquely identify the data and record the storage location and size information of the data. We call thi...

Examples

Embodiment

[0037] Assuming that the number of records in the index is 100 million, the index database divides the index into 500,000 storage areas according to 500,000. A batch write operation is initiated every time the index records in the memory reach 1 million. Assuming that the total number of all index cache files after excluding incomplete index cache files and the index cache files being written reaches 3, the condition is considered to be satisfied and an asynchronous update operation can be initiated . Such as figure 1 As shown, this example includes the following steps:

[0038] 1. Divert the 1 million index records cached in the memory according to the storage area numbers 0 to 499999.

[0039] 2. Sort the index corresponding to each storage area number in step 1 in ascending order.

[0040] 3. Serialize the sorted indexes corresponding to each storage area number in step 2 into data streams.

[0041] 4. Combine the serialized data streams in step 3 into a unique data str...

Images