A method and device for merging and sorting LSM data based on sliding window

By using sliding window technology in the LSM database to filter and optimize the merge operation of sorted string tables, the problem of excessive system resource occupation caused by repeated read and write of high-level sorted string tables is solved, and the system's read and write performance is improved.

CN118069891BActive Publication Date: 2025-05-13INSTITUTE OF INFORMATION ENGINEERING CHINESE ACADEMY OF SCIENCES
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410261182.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-03-07
Publication Date
2025-05-13
Estimated Expiration
2044-03-07

AI Technical Summary

Technical Problem

In LSM-based databases, repeated read and write of high-level sorted string tables lead to excessive system resource usage, affecting the system's read and write performance.

Method used

The sliding window technology is used to filter the sorted string table, determine the merge range through merge cost calculation, optimize the merge planning, and reduce the repeated read and write of the sorted string table.

Benefits of technology

It effectively reduces the number of repeated reads and writes of the sorted string table, and improves the system's write rate and read and write performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118069891B_ABST
    Figure CN118069891B_ABST
Patent Text Reader

Abstract

The present invention relates to a sliding window-based LSM data merge sorting method and device. The method proposed by the present invention first constructs a sliding window for a sorted string table at a low level in a merge operation, selects a low-level sorted string table participating in the merge operation based on the sliding window, and simultaneously obtains the two sorted string tables closest to the left and right of the sliding window, and obtains their key control ranges, and then selects a sorted string table in a high-level sorted string table that has an intersection with the key control range of the low-level sorted string table and obtains its key control range, and determines whether to perform a merge sort operation by estimating the key control range of the sorted string table after the merge, and comparing it with the key control ranges of the two sorted string tables closest to the left and right of the sliding window. The present invention reduces the number of times a high-level sorted string table repeatedly participates in the merge process of a low-level sorted string table, speeds up the merge sorting rate, and improves the system read and write performance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of data storage, and in particular relates to a method and device for merging and sorting log structured merge (LogStructured Merge, hereinafter referred to as LSM) data based on a sliding window. Background Art

[0002] In an LSM-based database, data is first written to the lower level, and then migrated upward layer by layer relying on merge sort operations. After each layer completes data migration, the space is released and can be rewritten. During this process, if the writing speed is greater than the speed of the merge sort operation, the writing will be blocked because the space of that layer has not been released. Therefore, the speed of the merge sort operation directly affects the writing speed of the system.

[0003] During merge sort, a sort string table at a lower level is generally selected as the input of the lower level sort string table, and its key control range is obtained. Then, a sort string table at an adjacent higher level whose key control range intersects with the lower level sort string table is selected as the input of the higher level sort string table for the merge sort process. Since the higher level sort string table is larger, its key control range often corresponds to several lower level sort string tables, resulting in repeated reading and writing of the higher level sort string table, which occupies system resources such as disk bandwidth and CPU time slices, greatly affecting the read and write performance of the system. Summary of the invention

[0004] In view of the above problems, the present invention proposes a sliding window-based LSM data merge sorting method and device to alleviate the problem that the repeated reading and writing of the sorting string table under the LSM data structure in the prior art occupies system resources and reduces the system read and write performance. The method uses a sliding window to screen the sorting string table, calculates the merge cost based on the key range metadata of the sorting string table, thereby determining the merge range of the sorting string table, and determining the optimal merge plan with the minimum read and write cost, thereby reducing the number of repeated read and write times of the sorting string table, improving the system write rate, and improving the read and write performance of the LSM-based database.

[0005] The specific technical solutions provided by the present invention are as follows:

[0006] In a first aspect, the present invention proposes a sliding window-based LSM data merge sorting method, comprising the following steps:

[0007] 1) When the size of the low-level sorting string table reaches a threshold, a merge operation with an adjacent higher-level sorting string table is triggered;

[0008] 2) Construct a sliding window for the low-level sorted string table;

[0009] 3) according to the file selection instruction of the merge operation, a group of sorted string tables are selected in the low level by using the sliding window as the low level sorted string table input of the merge operation, and at the same time, the most recent sorted string table before the low level sorted string table input of the merge operation is selected, and the most recent sorted string table after the low level sorted string table input of the merge operation is selected;

[0010] 4) obtaining the key control range of the low-level sorted string table input of the merge operation, the key control range of the most recent sorted string table before the low-level sorted string table input of the merge operation, and the key control range of the most recent sorted string table after the low-level sorted string table input of the merge operation;

[0011] 5) determining a sorted string table in an adjacent higher level that has an intersection with the key control range of the low-level sorted string table input of the merge operation as the high-level sorted string table input of the merge operation;

[0012] 6) obtaining the keying range of the high-level sorted string table input for the merge operation;

[0013] 7) estimating the key range of the merged sorted string table according to the key range of the low-level sorted string table input for the merge operation and the key range of the high-level sorted string table input for the merge operation;

[0014] 8) determining that the left boundary value of the key control range of the merged sorted string table is greater than the right boundary value of the key control range of the most recent sorted string table before the low-level sorted string table of the merge operation is input, and determining that the right boundary value of the key control range of the merged sorted string table is less than the left boundary value of the key control range of the most recent sorted string table after the low-level sorted string table of the merge operation is input;

[0015] 9) performing a merge sort operation on the low-level sorted string table input of the merge operation and the high-level sorted string table input of the merge operation.

[0016] Furthermore, the obtaining of the key control range of the low-level sorted string table input for the merge operation includes:

[0017] If the number of low-level sorting string table inputs is 1, the key control range of the low-level sorting string table input of the merge operation is the key control range of a single file in the low-level sorting string table input; if the number of low-level sorting string table inputs is greater than 1, the key control range of the low-level sorting string table input of the merge operation is the union of the key control ranges of multiple files in the low-level sorting string table input.

[0018] Furthermore, the initial parameters of the sliding window in the low level include: the starting position is 0, the window size is 1, the step size is 1, and after each traversal, the starting position is reset to 0, the step size remains unchanged at 1, the window size increases by 1, and so on.

[0019] Furthermore, the estimating the key control range of the merged sorted string table includes: using the minimum key in the key control range of the low-level sorted string table input and the key control range of the high-level sorted string table input as the left boundary of the key control range of the merged sorted string table, and using the maximum key as the right boundary of the key control range of the merged sorted string table.

[0020] Furthermore, when the most recent sorted string table before the input of the low-level sorted string table of the merge operation and the most recent sorted string table after the input of the low-level sorted string table of the merge operation are both empty, that is, the sliding window selects all the remaining sorted string tables in the lower level, a merge sort operation will be forced at this time.

[0021] Furthermore, if the left boundary value of the key control range of the sorted string table after the merger is not greater than the right boundary value of the key control range of the sorted string table closest to the left side of the sliding window, or if the right boundary value of the key control range of the sorted string table after the merger is not less than the left boundary value of the key control range of the sorted string table closest to the right side of the sliding window, the sliding window parameters are updated; the sliding window parameter update includes: the pointer P1 of the left boundary and the pointer P2 of the right boundary of the sliding window are both moved to the right by the step distance, that is, pointing to the next sorted string table; when there is no sorted string table on the right side of the sliding window, that is, when the next position of the pointer P2 is empty, the pointer P1 is reset to point to the sorted string table with a position index of 0, and the pointer P2 is reset to point to the sorted string table with a position index of N, where N is the number of times the sliding window traverses the sorted string table.

[0022] In a second aspect, the present invention proposes a sliding window-based LSM data merge sorting device, comprising:

[0023] A metadata acquisition module, used to acquire a key range of a single sorted string table or acquire a key range of a group of sorted string tables;

[0024] A low-level sorted string table acquisition module is used to acquire the low-level sorted string table input of the merge operation, acquire the most recent sorted string table before the low-level sorted string table input of the merge operation, acquire the most recent sorted string table after the low-level sorted string table input of the merge operation, and acquire the key control range of the above sorted string tables according to the metadata acquisition module;

[0025] A high-level sorted string table acquisition module, used to acquire a high-level sorted string table input for a merge operation, and at the same time acquire a key control range of the sorted string table in the high-level sorted string table input for the merge operation according to the metadata acquisition module;

[0026] A merge determination module, used to determine whether to perform a merge sort operation on the low-level sort string table input of the merge operation and the high-level sort string table input of the merge operation;

[0027] The merge sort module is used to coordinate the merge sort process, and according to the maintained sliding window, sends a sort string table selection command to the low-level sort string table acquisition module to obtain the key control range of the low-level sort string table input of the merge operation, the key control range of the most recent sort string table before the low-level sort string table input of the merge operation, and the key control range of the most recent sort string table after the low-level sort string table input of the merge operation; then according to the key control range of the low-level sort string table input of the merge operation, sends a sort string table selection command to the high-level sort string table acquisition module to obtain the key control range of the high-level sort string table input of the merge operation; according to the above key control range information, sends a merge sort determination command to the merge determination module, and after the determination is passed, according to the low-level sort string table acquisition module and the high-level sort string table acquisition module, obtains the low-level sort string table input of the merge operation and the high-level sort string table input of the merge operation, and performs a merge sort operation to obtain a merged sort string table, and stores the merged sort string table in the high level.

[0028] The sliding window-based LSM data merge sorting method and device provided by the present invention obtains the key control range information of the sorting string table and uses the sliding window to determine the optimal merge plan with the minimum read and write cost, thereby greatly reducing the number of repeated read and write times of the sorting string table in the merge sorting operation, effectively alleviating the system resource pressure, and thus improving the system's read and write performance.

[0029] The main innovations of the present invention are: ① a sliding window-based LSM data merge sorting method; ② a sliding window-based LSM data merge sorting device structure. The method has strong practicality and application scope in the field of Internet data storage and has a very broad application prospect. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the LSM storage structure of the present invention;

[0031] Figure 2A flowchart of a sliding window-based LSM data merge sorting method proposed in an embodiment of the present invention;

[0032] Figure 3 A schematic diagram of a sliding window-based LSM data merge sorting device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical method and advantages of the present invention more clearly understood, the method of the present invention is further described in detail below in conjunction with the accompanying drawings.

[0034] LSM usually has multiple levels. For the sake of simplicity and intuitiveness, two levels are selected for illustration, namely L1 and L2. Figure 1 As shown, Figure 1 This is a schematic diagram of the LSM storage structure of the present invention. The storage space size of each level in the LSM shows an increasing trend, that is, the storage space of the L2 layer is larger than the storage space of the L1 layer; at the same time, the sorted string tables in the L1 layer and the L2 layer are arranged in order according to the key control range, and there is no intersection between the key control ranges of the sorted string tables in the same level.

[0035] When the data size stored in the L1 level exceeds a preset threshold, a sorted string table of a certain L1 level is selected as the L1 level file input of the merge operation, and then a sorted string table having an intersection with the key range of the L1 level file input of the merge operation is selected in the L2 level as the L2 level file input of the merge operation, and a merge sorting operation is performed on the L1 level file input of the merge operation and the L2 level file input of the merge operation. For example: according to the polling rule, the L1 level file input of the merge operation is determined to be a sorted string table T with a key range of 0-90 11 , determining the L2 level file input of the merge operation in the L2 layer, that is, the sorted string table T with a key range of 10-100 21 , sort the string table T in the storage space of the L1 layer 11 and sorted string table T 21 Perform a merge sort operation. Specifically, for the same key value, only the latest version is retained, all old versions are deleted, and then each key value is sorted and written to the L2 layer to form a sorted string table.

[0036] However, the above method will cause the sorted string table in the L2 layer to repeatedly participate in the merging process of the sorted string table in the L1 layer. Figure 1 As shown, for example, in the storage space of the L1 layer, the sorting string table T 11 and sorted string table T 21Perform a merge sort operation to obtain a sorted string table with a key range of 0-100, and write the sorted string table into the L2 level. When the L1 level file input of the merge operation is determined according to the polling rule to be a sorted string table T with a key range of 95-150 12 When the L2-level file input of the merge operation is determined in the L2 layer, that is, the sorted string table with a key range of 0-100, it can be seen that the sorted string table with a key range of 0-100 again participates in the merging process of the sorted string table in the L1 layer. This problem is more serious especially in the scenario of massive data.

[0037] In order to solve the above problems, the present invention improves the strategy of merge sort operation.

[0038] The technical solution of the present invention is described in detail below with reference to specific embodiments.

[0039] Figure 2 is a flow chart of a sliding window-based LSM data merge sorting method proposed in an embodiment of the present invention, such as Figure 2 As shown, the method of this embodiment includes:

[0040] S101: L1 layer triggering and L2 layer merging operation.

[0041] S102: Construct a sliding window for the L1 level sorting string table.

[0042] Specifically, the sorting string table for the L1 level maintains two pointers P1 and P2, where the position index pointed to by P1 is the left boundary of the sliding window, and the position index pointed to by P2 is the right boundary of the sliding window. The size of the sliding window is (the position index pointed to by P2 - the position index pointed to by P1) + 1. Initially, both P1 and P2 point to the sorting string table with a position index of 0.

[0043] S103: Acquire the L1 level sorted string table and the two sorted string tables closest to the left and right of the sliding window according to the sliding window.

[0044] Specifically, obtain the sorted string table at position [P1, P2] in the L1 level, and the sorted string files at P1-1 and P2+1, and obtain their key range information. For example, initially, the sliding window selects the sorted string table T with a key range of 0-90 at position index 0 in the L1 level. 11 , and get the sorted string table T with the key range 95-150 at position index 1 12 .

[0045] S104: Determine whether the L1 level sorting string table is empty; if not, execute S105; if it is empty, the merging process ends.

[0046] S105: Determine whether the two sorting string tables closest to the left and right of the sliding window are both empty. If both are empty, execute S110; otherwise, execute S106.

[0047] S106: Obtaining the L2 level sorting string table according to the key control range of the L1 level sorting string table.

[0048] Specifically, a sorted string table whose key range in the L2 level and the key range of the sorted string table in the L1 level intersect is obtained. 21 , because its key range is 10-100 and the above sorted string table T with key range 0-90 11 There is a keyed range intersection.

[0049] S107: Estimate the key range of the sorted string table after merging.

[0050] Specifically, the minimum key in the L1 level sorting string table and the L2 level sorting string table is used as the left boundary of the key control range of the merged sorting string table, and the maximum key is used as the right boundary of the key control range of the merged sorting string table. 11 and a L2 hierarchical sorted string table T with key range 10-100 21 The key range of the merged sorted string table is 0-100.

[0051] S108: Determine whether the left boundary value of the key control range of the merged sorted string table is greater than the right boundary value of the key control range of the nearest sorted string table on the left side of the sliding window. If so, execute S109; otherwise, execute S111.

[0052] Specifically, the left boundary value of the key range of the merged sorting string table is 0, and the nearest sorting string table on the left side of the sliding window is empty, so no comparison is performed.

[0053] S109: Determine whether the right boundary value of the key control range of the merged sorted string table is smaller than the left boundary value of the key control range of the nearest sorted string table on the right side of the sliding window. If so, execute S110; otherwise, execute S111.

[0054] Specifically, the right boundary value of the key control range of the merged sorted string table is 100, and the left boundary value of the key control range of the sorted string table closest to the right side of the sliding window is 95. 100 is greater than 95, so S111 is executed.

[0055] S110: performing a merge and sort operation on the L1 level sorting string table and the L2 level sorting string table.

[0056] Specifically, the L1 level sorted string table and the L2 level sorted string table are loaded from the disk into the memory, and their internal key-value pairs are traversed in the memory. For the same key value, only the latest version is retained and all old versions are deleted. Then, each key value is sorted and written into the L2 layer to form a merged sorted string table.

[0057] S111: Sliding window parameter update.

[0058] Specifically, for example, when updating the initial pointer P1 and pointer P2 pointing information, pointer P1 and pointer P2 both move rightward by the step length, that is, point to the next sorted string table. When there is no sorted string table on the right side of the sliding window, that is, when the next position of pointer P2 is empty, pointer P1 is reset to point to the sorted string table with position index 0, and pointer P2 is reset to point to the sorted string table with position index N, where N is the number of times the sliding window traverses the sorted string table.

[0059] Figure 3 Schematic diagram of a sliding window-based LSM data merge sorting device provided in an embodiment of the present invention. Figure 3 As shown, the device of this embodiment includes: a metadata acquisition module, a low-level sorting string table acquisition module, a high-level sorting string table acquisition module, a merging determination module and a merging sorting module.

[0060] The metadata acquisition module is specifically used to obtain the key control range of the sorted string table. Specifically, for a single sorted string table, the minimum key is used as the left boundary value of the key control range of the sorted string table, and the maximum key is used as the right boundary value of the key control range of the sorted string table; for a group of sorted string tables, the key control range of each sorted string table is obtained respectively, and the union of the key control ranges of each sorted string table is taken as the key control range of the group of sorted string tables.

[0061] The low-level sorting string table acquisition module is specifically used to obtain the sorting string table in the sliding window and the two sorting string tables closest to the left and right of the sliding window in the low level according to the sliding window parameters, and at the same time, access the metadata acquisition module to obtain the key control range of the above-mentioned sorting string table.

[0062] The high-level sorting string table acquisition module is specifically used to acquire the sorting string table in the high level that has an intersection with the key control range of the low-level sorting string table according to the key control range of the low-level sorting string table, and at the same time, access the metadata acquisition module to acquire the key control range of the sorting string table.

[0063] The merge determination module is specifically used to determine whether to perform a merge sort operation on the low-level sort string table and the high-level sort string table. Specifically, based on the key control range of the low-level sort string table and the key control range of the high-level sort string table, the key control range of the merged sort string table is estimated, and the minimum key in the low-level sort string table and the high-level sort string table is used as the left boundary of the key control range of the merged sort string table, and the maximum key is used as the right boundary of the key control range of the merged sort string table. If the left boundary of the key control range of the merged sort string table is greater than the right boundary value of the key control range of the nearest sort string table on the left side of the sliding window, and the right boundary of the key control range of the merged sort string table is less than the left boundary value of the key control range of the nearest sort string table on the right side of the sliding window, the determination is passed, otherwise, the determination is not passed.

[0064] The merge sort module is specifically used to select the low-level sort string table and the high-level sort string table and perform a merge sort operation. Specifically, the module maintains a sliding window inside, and according to the sliding window parameters, including the sliding window starting position, the window size and the sliding step length, accesses the low-level sort string table acquisition module to obtain the key control range of the low-level sort string table located in the sliding window and the two sort string tables closest to the left and right of the sliding window, and according to the key control range of the low-level sort string table located in the sliding window, accesses the high-level sort string table acquisition module to obtain the key control range of the high-level sort string table. Furthermore, based on the key control range of the low-level sorted string table, the key control ranges of the two sorted string tables closest to the left and right of the sliding window, and the key control range of the high-level sorted string table, the merge determination module is accessed; if the determination fails, the sliding window parameters are updated; if the determination passes, the low-level sorted string table acquisition module is accessed again to obtain the low-level sorted string table data, and the high-level sorted string table acquisition module is accessed again based on the key control range of the high-level sorted string table to obtain the high-level sorted string table data, and the above-mentioned sorted string table data are all loaded into the memory, and the key-value pairs therein are further traversed. For the same key value, only the latest version is retained based on the timestamp information, and all old versions are deleted, and then each key-value pair is sorted by key, and written into the high level to form a new sorted string table.

[0065] Another embodiment of the present invention provides a computer device (computer, server, smart phone, etc.), which includes a memory and a processor, the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program includes instructions for executing each step in the method of the present invention.

[0066] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, magnetic disk, optical disk), wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the steps of the method of the present invention are implemented.

[0067] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and implement it accordingly. It can be understood by those skilled in the art that various replacements, changes and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the contents disclosed in the embodiments of this specification, and the scope of protection of the present invention shall be subject to the scope defined in the claims.

Claims

1. A sliding window-based LSM data merge sorting method, characterized in that: The following steps are involved: When the size of the low-level sorting string table reaches a threshold, a merge operation with an adjacent higher-level sorting string table is triggered; Construct a sliding window over a low-level sorted string table; According to the file selection instruction of the merge operation, a group of sorted string tables are selected in the low level by using the sliding window as the low level sorted string table input of the merge operation, and at the same time, a most recent sorted string table before the low level sorted string table input of the merge operation is selected, and a most recent sorted string table after the low level sorted string table input of the merge operation is selected; Acquire the key control range of the low-level sorted string table input of the merge operation, the key control range of the most recent sorted string table before the low-level sorted string table input of the merge operation, and the key control range of the most recent sorted string table after the low-level sorted string table input of the merge operation; Determine a sorted string table at an adjacent higher level that has an intersection with the key control range of the low-level sorted string table input of the merge operation as a high-level sorted string table input of the merge operation; Obtaining a keying range of a high-level sorted string table input for the merge operation; estimating the key range of the merged sorted string table according to the key range of the low-level sorted string table input of the merge operation and the key range of the high-level sorted string table input of the merge operation; Determine that the left boundary value of the key control range of the merged sorted string table is greater than the right boundary value of the key control range of the most recent sorted string table before the low-level sorted string table of the merge operation is input, and determine that the right boundary value of the key control range of the merged sorted string table is less than the left boundary value of the key control range of the most recent sorted string table after the low-level sorted string table of the merge operation is input; Performing a merge sort operation on the low-level sorted string table input of the merge operation and the high-level sorted string table input of the merge operation; The method comprises: maintaining two pointers P1 and P2 facing the low-level sorted string table, wherein the position index pointed to by P1 is the left boundary of the sliding window, the position index pointed to by P2 is the right boundary of the sliding window, the size of the sliding window is (the position index pointed to by P2-the position index pointed to by P1)+1, and initially both P1 and P2 point to the sorted string table with the position index of 0; The obtaining of the keying range of the low-level sorted string table input for the merge operation includes: If the number of low-level sorted string table inputs is 1, the keying range of the low-level sorted string table inputs of the merge operation is the keying range of a single file in the low-level sorted string table inputs; If the number of low-level sorted string table inputs is greater than 1, the key range of the low-level sorted string table inputs of the merge operation is the union of the key ranges of multiple files in the low-level sorted string table inputs; The estimating the key control range of the merged sorted string table includes: using the minimum key in the key control range input from the low-level sorted string table and the key control range input from the high-level sorted string table as the left boundary of the key control range of the merged sorted string table, and using the maximum key as the right boundary of the key control range of the merged sorted string table.

2. The method according to claim 1, characterized in that The initial parameters of the sliding window include: the starting position is 0, the window size is 1, and the step length is 1. After each traversal, the starting position is reset to 0, the step length remains unchanged at 1, the window size increases by 1, and so on.

3. The method according to claim 1, characterized in that When the most recent sorted string table before the input of the lower-level sorted string table of the merge operation and the most recent sorted string table after the input of the lower-level sorted string table of the merge operation are both empty, a merge sort operation is forced to be performed.

4. The method according to claim 1, characterized in that: If the left boundary value of the key control range of the sorted string table after the merger is not greater than the right boundary value of the key control range of the sorted string table closest to the left side of the sliding window, or if the right boundary value of the key control range of the sorted string table after the merger is not less than the left boundary value of the key control range of the sorted string table closest to the right side of the sliding window, the sliding window parameter is updated; the sliding window parameter update includes: the pointer P1 of the left boundary and the pointer P2 of the right boundary of the sliding window are both moved to the right by the step distance, that is, pointing to the next sorted string table; when there is no sorted string table on the right side of the sliding window, that is, when the next position of the pointer P2 is empty, the pointer P1 is reset to point to the sorted string table with a position index of 0, and the pointer P2 is reset to point to the sorted string table with a position index of N, where N is the number of times the sliding window traverses the sorted string table.

5. A sliding window-based LSM data merge sorting device using the method described in any one of claims 1 to 4, characterized in that: include: A metadata acquisition module, used to acquire a key range of a single sorted string table or acquire a key range of a group of sorted string tables; A low-level sorted string table acquisition module is used to acquire the low-level sorted string table input of the merge operation, acquire the most recent sorted string table before the low-level sorted string table input of the merge operation, acquire the most recent sorted string table after the low-level sorted string table input of the merge operation, and acquire the key control range of the above sorted string tables according to the metadata acquisition module; A high-level sorted string table acquisition module, used to acquire a high-level sorted string table input for a merge operation, and at the same time acquire a key control range of the sorted string table in the high-level sorted string table input for the merge operation according to the metadata acquisition module; A merge determination module, used to determine whether to perform a merge sort operation on the low-level sort string table input of the merge operation and the high-level sort string table input of the merge operation; The merge sort module is used to coordinate the merge sort process, and according to the maintained sliding window, sends a sort string table selection command to the low-level sort string table acquisition module to obtain the key control range of the low-level sort string table input of the merge operation, the key control range of the most recent sort string table before the low-level sort string table input of the merge operation, and the key control range of the most recent sort string table after the low-level sort string table input of the merge operation; then according to the key control range of the low-level sort string table input of the merge operation, sends a sort string table selection command to the high-level sort string table acquisition module to obtain the key control range of the high-level sort string table input of the merge operation; according to the above key control range information, sends a merge sort determination command to the merge determination module, and after the determination is passed, according to the low-level sort string table acquisition module and the high-level sort string table acquisition module, obtains the low-level sort string table input of the merge operation and the high-level sort string table input of the merge operation, and performs a merge sort operation to obtain a merged sort string table, and stores the merged sort string table in the high level.

6. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the method according to any one of claims 1 to 4 is implemented.

Citation Information

Patent Citations

  • LSM (Log Structured Merge) data compact method and device

    CN105095287A

  • Data processing method and device, electronic equipment and storage medium

    CN114691039A