Data processing method, device, electronic device and storage medium

Abstract

The embodiments of the present application provide a data processing method, device, electronic device, and storage medium, which relate to the field of database technology. The method includes: when a preset merging condition is reached, the deletion record data to be currently merged is determined from the deletion record data stored in the preset storage space as the first deletion record data; the keyword in the deletion operation record contained in the first deletion record data is determined as the keyword to be processed; the write operation record containing the keyword to be processed is determined from the write record data to be currently merged stored in the preset storage space as the write operation record to be processed; based on the write operation record to be processed corresponding to the keyword to be processed, the deletion operation record containing the keyword to be processed in the first deletion record data is merged. The data processing method provided by the present application can improve the efficiency of data reading.

Description

Technical Field

[0001] The present application relates to the field of database technology, and in particular to a data processing method, device, electronic device and storage medium. Background Art

[0002] In related technologies, operations on a KV (Key Value) database can be recorded in a storage space based on an LSM Tree (Log-Structured-Merge-Tree) approach. Subsequently, data in the database can be read based on the operation records. The KV database can be a database such as NessDB (Ness Database), LevelDB (Level Database), or Hbase (Hadoop Database).

[0003] For example, with LevelDB, data write operations and data delete operations can be recorded in the same LSM tree. Since write and delete operation records are mixed and stored in the same LSM tree for different keys, subsequent reading of the value corresponding to a key requires parsing a large number of delete operation records, which increases data read time and reduces data read efficiency. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide a data processing method, device, electronic device, and storage medium to improve the efficiency of data reading. The specific technical solution is as follows:

[0005] In a first aspect of an embodiment of the present application, a data processing method is first provided, the method comprising:

[0006] When a preset merging condition is met, the deletion record data to be merged is determined from the deletion record data stored in the preset storage space as the first deletion record data; wherein the deletion record data includes a deletion operation record of the data deletion operation on the target database;

[0007] determining a keyword in the deletion operation record included in the first deletion record data as a keyword to be processed;

[0008] Determining, from the write record data currently to be merged stored in the preset storage space, a write operation record containing the keyword to be processed as the write operation record to be processed; wherein the write record data includes a write operation record of a data write operation for the target database;

[0009] Based on the pending write operation records corresponding to the pending keyword, the deletion operation records containing the pending keyword in the first deletion record data are merged.

[0010] In some embodiments, merging the deletion operation records containing the keyword to be processed in the first deletion record data based on the pending write operation records corresponding to the pending keyword includes:

[0011] Determine whether the deletion operation record of the keyword to be processed included in the first deletion record data and the operation record with the latest operation time among the pending write operation records corresponding to the keyword to be processed are deletion operation records;

[0012] If the operation record with the latest operation time is a deletion operation record, determining from the first deletion record data the deletion operation record that contains the keyword to be processed and has the latest operation time as the first deletion operation record, and deleting from the first deletion record data the deletion operation records that contain the keyword to be processed except the first deletion operation record;

[0013] If the operation record with the latest corresponding operation time is not a deletion operation record, the deletion operation record containing the to-be-processed keyword is deleted from the first deletion record data.

[0014] In some embodiments, the method further comprises:

[0015] If the operation record with the latest operation time is not a delete operation record, determining the write operation record with the latest operation time from the pending write operation records corresponding to the pending keyword as the first write operation record;

[0016] The other write operation records except the first write operation record in the write operation records to be processed corresponding to the to-be-processed keyword are deleted from the corresponding write record data.

[0017] In some embodiments, the method further comprises:

[0018] If the operation record with the latest corresponding operation time is a deletion operation record, the pending write operation record corresponding to the pending keyword is deleted from the corresponding write record data.

[0019] In some embodiments, the deletion record data currently to be merged belongs to two adjacent layers in the log structure merge LSM tree, and the write record data currently to be merged belongs to the two adjacent layers;

[0020] After merging the deletion operation records containing the to-be-processed keyword in the first deletion record data based on the to-be-processed write operation records corresponding to the to-be-processed keyword, the method further includes:

[0021] A merging result obtained by merging the deletion operation records containing the keyword to be processed in the first deletion record data is written into the latter layer of the two adjacent layers.

[0022] In some embodiments, the deletion record data currently to be merged and the write record data currently to be merged belong to the same layer in the log-structured merge LSM tree;

[0023] After merging the deletion operation records containing the to-be-processed keyword in the first deletion record data based on the to-be-processed write operation records corresponding to the to-be-processed keyword, the method further includes:

[0024] If the current write record data to be merged does not belong to the bottom layer in the log structure merge tree, the merge result obtained by merging the deletion operation records containing the keyword to be processed in the first deletion record data will be written to the next layer of the layer to which the current write record data to be merged belongs.

[0025] In some embodiments, the method further comprises:

[0026] When a first preset storage condition is met, writing the deletion record data stored in the preset cache space into the preset storage space; wherein the deletion record data stored in the preset cache space is generated based on a data deletion operation on the target database;

[0027] When a second preset storage condition is met, the write record data stored in the preset cache space is written into the preset storage space; wherein the write record data stored in the preset cache space is generated based on a data write operation for the target database.

[0028] In some embodiments, the deletion record data stored in the preset storage space includes a range deletion operation record; wherein the range deletion operation record represents a deletion operation for deleting values ​​corresponding to each keyword belonging to a specified keyword range in the target database.

[0029] In a second aspect of the embodiments of the present application, a data processing device is further provided, the device comprising:

[0030] a first deletion record data determining module, configured to determine, when a preset merging condition is met, deletion record data currently to be merged from the deletion record data stored in the preset storage space as first deletion record data; wherein the deletion record data includes a deletion operation record of a data deletion operation on a target database;

[0031] a module for determining keywords to be processed, configured to determine keywords in the deletion operation record contained in the first deletion record data as keywords to be processed;

[0032] a pending write operation record determination module, configured to determine, from the current pending merged write record data stored in the preset storage space, a write operation record containing the pending keyword, as a pending write operation record; wherein the write record data includes a write operation record of a data write operation for the target database;

[0033] The first merging module is configured to merge the deletion operation records containing the keyword to be processed in the first deletion record data based on the write operation records to be processed corresponding to the keyword to be processed.

[0034] In some embodiments, the first merging module includes:

[0035] a judgment submodule, configured to judge whether the deletion operation record containing the to-be-processed keyword in the first deletion record data and the operation record with the latest operation time among the to-be-processed write operation records corresponding to the to-be-processed keyword are deletion operation records;

[0036] a first merging submodule configured to, if the operation record with the latest operation time is a deletion operation record, determine from the first deletion record data the deletion operation record that contains the keyword to be processed and has the latest operation time, as the first deletion operation record, and delete from the first deletion record data the deletion operation records that contain the keyword to be processed except the first deletion operation record;

[0037] a second merging submodule, configured to delete the deletion operation record containing the keyword to be processed from the first deletion record data if the operation record with the latest corresponding operation time is not a deletion operation record;

[0038] The device further comprises:

[0039] a first write operation record determining module configured to determine, if the operation record with the latest operation time is not a delete operation record, the write operation record with the latest operation time from the pending write operation records corresponding to the pending keyword as the first write operation record;

[0040] a second merging module, configured to delete, from the corresponding write record data, the other write operation records except the first write operation record in the write operation records to be processed corresponding to the keyword to be processed;

[0041] The device further comprises:

[0042] a third merging module, configured to delete the pending write operation record corresponding to the pending keyword from the corresponding write record data if the operation record with the latest corresponding operation time is a delete operation record;

[0043] The deletion record data to be merged currently belongs to two adjacent layers in the log structure merge LSM tree, and the write record data to be merged currently belongs to the two adjacent layers;

[0044] The device further comprises:

[0045] a write record data storage module configured to, after merging the deletion operation records containing the keyword to be processed in the first deletion record data based on the write operation records to be processed corresponding to the keyword to be processed, write a merged result obtained by merging the deletion operation records containing the keyword to be processed in the first deletion record data into the latter of the two adjacent layers;

[0046] The deletion record data to be merged and the write record data to be merged belong to the same layer in the log structure merge LSM tree;

[0047] The device further comprises:

[0048] a merge result storage module for, after merging the delete operation records containing the keyword to be processed in the first deletion record data based on the write operation records corresponding to the keyword to be processed, if the current write record data to be merged does not belong to the bottom layer of the log structure merge tree, writing the merge result obtained by merging the delete operation records containing the keyword to be processed in the first deletion record data to a layer after the layer to which the current write record data to be merged belongs;

[0049] The device further comprises:

[0050] A first storage module is configured to write the deletion record data stored in the preset cache space into the preset storage space when a first preset storage condition is met; wherein the deletion record data stored in the preset cache space is generated based on a data deletion operation on the target database;

[0051] a second storage module, configured to write the write record data stored in the preset cache space into the preset storage space when a second preset storage condition is met; wherein the write record data stored in the preset cache space is generated based on a data write operation to the target database;

[0052] The deletion record data stored in the preset storage space includes a range deletion operation record, wherein the range deletion operation record represents a deletion operation for deleting values ​​corresponding to each keyword belonging to a specified keyword range in the target database.

[0053] In a third aspect of the embodiments of the present application, an electronic device is provided, including:

[0054] Memory for storing computer programs;

[0055] The processor is used to implement any of the above-mentioned data processing methods when executing the program stored in the memory.

[0056] In another aspect of the embodiments of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the above-mentioned data processing methods is implemented.

[0057] An embodiment of the present application further provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the above-described data processing methods.

[0058] Beneficial effects of the embodiments of the present application: The embodiments of the present application provide a data processing method, which, when a preset merging condition is reached, determines the deletion record data currently to be merged from the deletion record data stored in the preset storage space as the first deletion record data; wherein the deletion record data includes a deletion operation record of a data deletion operation for a target database; determines the keyword in the deletion operation record contained in the first deletion record data as the keyword to be processed; determines the write operation record containing the keyword to be processed from the write record data currently to be merged stored in the preset storage space as the write operation record to be processed; wherein the write record data includes a write operation record of a data write operation for a target database; based on the write operation record to be processed corresponding to the keyword to be processed, merges the deletion operation record containing the keyword to be processed in the first deletion record data.

[0059] The data processing method provided by the embodiments of the present application can merge deletion operation records containing keywords to be processed within the deletion record data, thereby reducing the number of deletion operation records. Furthermore, this method can reduce the number of deletion operation records that need to be parsed when reading data, shortening the data reading time and improving data reading efficiency. Furthermore, by storing deletion operation records and write operation records separately, when the deletion operation records need to be merged, the number of record data that needs to be parsed can be reduced, reducing the number of I (Input) / O (Output) cycles and thus reducing the waste of computing resources.

[0060] Of course, it is not necessary to achieve all the advantages described above at the same time when implementing any product or method of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0062] Figure 1 A flowchart of a data processing method provided in an embodiment of the present application;

[0063] Figure 2 A schematic diagram of another flow chart of the data processing method provided in an embodiment of the present application;

[0064] Figure 3 A schematic diagram of the structure of a log structure merge tree provided in an embodiment of the present application;

[0065] Figure 4 A schematic diagram of a data processing principle provided in an embodiment of the present application;

[0066] Figure 5 A structural diagram of a data processing device provided in an embodiment of the present application;

[0067] Figure 6 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field based on this application are within the scope of protection of this application.

[0069] In order to improve the efficiency of data reading, the embodiments of the present application provide a data processing method, device, electronic device and storage medium, which are described in detail below.

[0070] An embodiment of the present application provides a data processing method, which can be applied to an electronic device used to manage a database. For example, the electronic device can record a deletion operation record of a data deletion operation on the database in the deletion record data, and record a write operation record of a data write operation on the database in the write record data. For example, the electronic device can record the above operation records in the form of a log structure merge tree. In addition, when the electronic device receives a data query instruction, it can also query according to the recorded data to obtain the query result. The above database can be a KV database. For example, it can be a database such as NessDB, LevelDB, or Hbase. See Figure 1 , Figure 1 A flow chart of a data processing method provided in an embodiment of the present application. The method may include the following steps:

[0071] S101: When a preset merging condition is met, the deletion record data to be currently merged is determined from the deletion record data stored in the preset storage space as the first deletion record data.

[0072] The deletion record data includes a deletion operation record of a data deletion operation on a target database.

[0073] S102: Determine a keyword in the deletion operation record included in the first deletion record data as a keyword to be processed.

[0074] S103: Determine a write operation record containing a keyword to be processed from the write record data to be merged currently stored in the preset storage space as the write operation record to be processed.

[0075] The write record data includes a write operation record of a data write operation on a target database.

[0076] S104: Based on the pending write operation records corresponding to the pending keyword, the deletion operation records containing the pending keyword in the first deletion record data are merged.

[0077] Based on the data processing method provided by the embodiments of the present application, deletion operation records and write operation records can be stored in the deletion record data and write record data of the LSM tree respectively, and the deletion record data and write record data can be merged. In addition, the invalid data recorded in the deletion record data can be reduced, and redundant deletion operation records can be reduced. Subsequently, when the value corresponding to a certain keyword needs to be read, parsing a large number of deletion operation records can be avoided, thereby shortening the data reading time and improving data reading efficiency.

[0078] In addition, compared to the related art that mixes the deletion operation records and the write operation records, that is, mixes them in the same record data. Furthermore, when it is necessary to merge the deletion operation records, for each record data, as long as the record data contains a deletion operation record, all operation records in the record data need to be parsed, which will increase the number of record data that need to be parsed, resulting in a high number of I / O operations and a waste of computing resources. The method provided in the present application stores the deletion operation records and the write operation records separately, which can correspondingly reduce the number of record data containing deletion operation records. Furthermore, when it is necessary to merge the deletion operation records, it can also reduce the number of record data that need to be parsed, reduce the number of I / O operations, and thus reduce the waste of computing resources.

[0079] In one implementation, the electronic device may record deletion operation records in the form of a table, which may be referred to as a deletion record table; or may record write operation records in the form of a table, which may be referred to as a write record table.

[0080] For step S101, the preset storage space can be a disk of the electronic device. In the present application, the deletion operation records of the data deletion operation and the write operation records of the data write operation can be stored separately in the preset storage space. That is, based on the user's operation, deletion record data containing only deletion operation records and write record data containing only write operation records can be generated. The preset merging condition can be pre-set by a technician. For example, an interface function can be set. When the user needs to merge the deletion record data stored in the preset storage space, the interface function can be called. Then, the electronic device can determine that the preset merging condition has been met. Alternatively, for a deletion record data, when the size of the storage space occupied by the deletion operation records recorded in the deletion record data reaches a first preset threshold, the electronic device can determine that the preset merging condition has been met. Alternatively, when the total storage space occupied by all currently recorded deletion record data reaches a second preset threshold, the electronic device can determine that the preset merging condition has been met. Alternatively, a merging period can be set. When the preset merging period is reached, the electronic device can determine that the preset merging condition has been met. That is, the electronic device can periodically merge the deletion record data stored in the preset storage space.

[0081] The deletion operation record may include a deletion mark and a keyword corresponding to the deletion operation. That is, the deletion operation record data indicates the value of the keyword included in the deletion operation record deleted from the above database.

[0082] The deletion record data to be merged can be one or more. If the deletion record data to be merged is one, the deletion record data to be merged is used as the first deletion record data and steps S102-S104 are executed; if the deletion record data to be merged is multiple, steps S102-S104 are executed for each first deletion record data.

[0083] With respect to step S102, the first deletion record data may include at least one deletion operation record for a data deletion operation on the target database. The included deletion operation record may be a data deletion operation for the same keyword or for multiple keywords. Therefore, the determined keywords to be processed may be one or multiple. If there is one keyword to be processed, steps S103-S104 are executed based on the keyword to be processed; if there are multiple keywords to be processed, steps S103-S104 are executed for each keyword to be processed.

[0084] Regarding step S103 , for example, if the size of the total storage space occupied by all currently recorded write record data reaches a preset threshold, the electronic device may determine that a preset merging condition is met.

[0085] The write record data may include at least one write operation record for a data write operation to a target database. Each write operation record may include a keyword corresponding to the write operation and a value for the keyword. In other words, the write operation record data indicates that the keyword included in the write operation record is written to the database, and the value of the keyword is set to the value recorded in the write operation record.

[0086] For each write record data, all write operation records contained in the write record data can be determined. For each write operation record, if the write operation record contains a to-be-processed keyword, the write operation record is regarded as a to-be-processed write operation record.

[0087] In one implementation, when generating each deletion record, the range of keywords contained in the deletion record can be determined based on the keywords in each deletion operation record. Similarly, when generating each write record, the range of keywords contained in the write record can be determined based on the keywords in each write operation record. Furthermore, after determining the keywords to be processed, a query can be performed within each write record based on the keyword range corresponding to each write operation record to determine the write operation record to be processed.

[0088] Regarding step S104, the data deletion operation and the data writing operation can be collectively referred to as data operations. Deleting recorded data and writing recorded data can be collectively referred to as recording data. Accordingly, the recorded data is merged, that is, the recorded data is compacted.

[0089] The pending write operation record corresponding to the pending keyword represents all write operation records containing the pending keyword in the current write record data to be merged. Based on the pending write operation record corresponding to the pending keyword, the deletion operation records containing the pending keyword in the first deletion record data are merged, thereby deleting all deletion operation records containing the pending keyword in the first deletion record data, or deleting some deletion operation records containing the pending keyword in the first deletion record data.

[0090] Based on the above processing, the write operation records containing the pending keyword in each write record data can be determined based on the keyword range of each write record data. Subsequently, based on the pending write operation records corresponding to each pending keyword, the delete operation records containing the pending keyword in the first delete record data can be merged, thereby making the merge target more accurate, reducing the number of I / O operations, and reducing the waste of computing resources.

[0091] See also Figure 2 , Figure 2Another flow chart of the data processing method provided in the embodiment of the present application. Figure 1 Based on this, correspondingly, the above step S104 may include the following steps:

[0092] S1041: Determine whether the deletion operation record containing the pending keyword in the first deletion record data and the operation record with the latest operation time among the pending write operation records corresponding to the pending keyword are deletion operation records.

[0093] S1042: If the operation record with the latest corresponding operation time is a deletion operation record, then determine from the first deletion record data the deletion operation record that contains the keyword to be processed and has the latest corresponding operation time, as the first deletion operation record, and delete the other deletion operation records containing the keyword to be processed except the first deletion operation record from the first deletion record data.

[0094] S1043: If the operation record with the latest corresponding operation time is not a deletion operation record, the deletion operation record containing the keyword to be processed is deleted from the first deletion record data.

[0095] In one implementation, the electronic device may further provide a corresponding identifier for each delete operation record and each write operation record to indicate the chronological order of the operation times of the operation records. The operation time of the operation record indicates the time when the electronic device performed the corresponding data operation. For example, the identifier of the operation record may be the operation time of the data operation. Alternatively, the electronic device may assign a corresponding serial number to each operation record according to the chronological order of the operation times, serving as the identifier of the operation record. For example, a larger serial number indicates a later operation time.

[0096] Accordingly, the electronic device can determine the operation record with the latest operation time from the deletion operation record containing the keyword to be processed in the first deletion record data and the pending write operation record corresponding to the keyword to be processed based on the identifier of the operation record.

[0097] If the operation record with the latest operation time is a deletion operation record, it means that the last data operation performed on the keyword to be processed was a data deletion operation (i.e., the first deletion operation record). In other words, at this time, the operation time is earlier than the first deletion operation record, and all other deletion operation records for the keyword to be processed are invalid. Therefore, only the first deletion operation record can be retained, and the invalid deletion operation records can be deleted.

[0098] If the latest operation record corresponding to the keyword to be processed is not a delete operation record, that is, the latest operation record corresponding to the keyword to be processed is a write operation record, then it means that the last data operation performed on the keyword to be processed was a data write operation. In other words, at this point, all delete operation records for the keyword to be processed are invalid. Therefore, these invalid delete operation records can be deleted from the first delete record data.

[0099] Based on the above processing, invalid deletion operation records in the deletion and write log data can be reduced, reducing redundant deletion operation records. Subsequently, when reading the value corresponding to a keyword, parsing a large number of deletion operation records can be avoided, thereby shortening the data reading time and improving data reading efficiency. Furthermore, deletion operation records in the deletion log data can be preferentially merged, thereby freeing up disk space and improving query efficiency during subsequent queries.

[0100] In one embodiment, if the operation record with the latest corresponding operation time is a deletion operation record, the pending write operation record corresponding to the pending keyword is deleted from the corresponding write record data.

[0101] If the operation record with the latest corresponding operation time is a delete operation record, it means that the last data operation performed on the pending keyword was a data delete operation (i.e., the first delete operation record). In other words, at this time, the operation time is earlier than the first delete operation record, and the write operation records for the pending keyword are all invalid, that is, the pending write operation records corresponding to the pending keyword are all invalid. Therefore, the invalid write operation records can be deleted from the pending write operation records and the corresponding write record data.

[0102] In one embodiment, the method further comprises:

[0103] Step 1: If the operation record with the latest operation time is not a delete operation record, determine the write operation record with the latest operation time from the pending write operation records corresponding to the pending keyword as the first write operation record.

[0104] Step 2: Delete the other write operation records except the first write operation record in the write operation records to be processed corresponding to the keyword to be processed from the corresponding write record data.

[0105] If the operation record with the latest operation time is not a delete operation record, that is, the operation record with the latest operation time corresponding to the keyword to be processed is a write operation record (that is, the first write operation record in this application), then it means that the last data operation performed on the keyword to be processed is a data write operation. In other words, at this time, the operation time is earlier than the first write operation record, and all other write operation records for the keyword to be processed are invalid. Therefore, the invalid write operation records can be deleted, and only the first write operation record can be retained.

[0106] In one implementation, the deletion record data to be merged currently belongs to two adjacent layers in the log structure merge LSM tree, and the write record data to be merged currently belongs to the two adjacent layers. Accordingly, after step S104, the method further includes: writing the merge result obtained by merging the deletion operation records containing the keyword to be processed in the first deletion record data to the latter of the two adjacent layers.

[0107] like Figure 3 As shown, Figure 3 A schematic diagram of the structure of a log structure merge tree provided in an embodiment of the present application. Figure 3 In , the log structure merge tree contains 4 layers, Level1 represents the first layer of the log structure merge tree, Level2 represents the second layer of the log structure merge tree, Level3 represents the third layer of the log structure merge tree, and Level4 represents the fourth layer of the log structure merge tree. That is, the second layer is the next layer after the first layer, the third layer is the next layer after the second layer, and the fourth layer is the next layer after the third layer. Conversely, the first layer is the previous layer of the second layer, the second layer is the previous layer of the third layer, and the third layer is the previous layer of the fourth layer. The first layer is the top layer, and the fourth layer is the bottom layer. According to Figure 3 In the structure, the storage space allocated to each layer gradually increases from top to bottom. For example, the first layer can contain 2 record tables, and the storage space allocated to the first layer is 300 megabytes; the second layer can contain 4 record tables, and the storage space allocated to the second layer is 3000 megabytes; the third layer can contain 6 record tables, and the storage space allocated to the third layer is 30,000 megabytes; the fourth layer can contain 8 record tables, and the storage space allocated to the fourth layer is 300,000 megabytes. Accordingly, for each layer, when the storage space occupied by the record tables recorded in the layer reaches the storage space allocated for the layer, the record tables recorded in the layer can be merged.

[0108] In this application, in a preset storage space, different log structure merge trees can be constructed based on deletion record data and writing record data. The structure of each log structure merge tree can be consistent. The log structure merge tree can contain multiple layers, and each layer can contain multiple record data.

[0109] For each layer except the bottom layer in the log structure merge tree, the electronic device can merge the recorded data of the layer with the recorded data of the layer after the layer, and record the merged result in the layer after the layer. For each layer except the top layer in the log structure merge tree, when the electronic device merges the recorded data of the layer before the layer and the recorded data of the layer, and records the merged result in the layer, it can be determined that the recorded data in the layer meets the preset merge condition.

[0110] In the embodiment of the present application, the deletion record data and the write record data in two adjacent layers can be merged each time. If deletion record data still exists after the merger, the deletion operation record in the merged deletion record data is written to the deletion record data of the later of the two adjacent layers; if write record data still exists after the merger, the write operation record in the merged write record data is written to the write record data of the later of the two adjacent layers.

[0111] For each piece of write-log data in the first of the two adjacent layers, if the write-log data does not contain a write operation record containing the keyword to be processed, and the write-log data does not contain any keyword in the write-log data in the second of the two adjacent layers, that is, the write-log data and the write-log data in the second of the two adjacent layers do not contain the same keyword, then the write operation record contained in the write-log data can be stored in the write-log data in the second of the two adjacent layers. In addition, the write-log data can be deleted, that is, the write-log data is not retained in the first of the two adjacent layers.

[0112] Based on the above processing, there is no need to merge the write record data that does not contain the write operation record containing the keyword to be processed. Therefore, the number of record data that needs to be parsed can be further reduced, the number of I / O times can be reduced, and the waste of computing resources can be reduced.

[0113] For example, if the preset merging condition is determined to be met based on the first-level deletion record data in the log structure merge tree, all the deletion record data in the first level and the level after the first level (i.e., the second level) can be used as the deletion record data to be merged, and all the write record data in the first level and the second level can be used as the write record data to be merged. In other words, all the record data in the first and second levels are merged at the same time, and the merged result is stored in the second level.

[0114] Based on the above processing, for each keyword, only one operation record can be recorded in the record data in the log-structured merge tree except the first layer. This can reduce the number of operation records stored in the log-structured merge tree, shorten the data reading time, and improve the efficiency of data reading.

[0115] With respect to step S1042, if the deletion record data and the write record data in two adjacent layers are merged each time, and the invalid deletion operation records are other deletion operation records with an operation time earlier than the first deletion operation record and for the keyword to be processed, then after deleting the invalid deletion operation records from the first deletion operation record data, the remaining deletion operation records in the first deletion operation record data may be written into the deletion record data of the latter of the two adjacent layers.

[0116] For the above-mentioned step S1043, if the deletion record data and the write record data in two adjacent layers are merged each time, and the invalid deletion operation records are all deletion operation records containing the keyword to be processed in the first deletion record data, then after deleting the invalid deletion operation records from the first deletion record data, the remaining deletion operation records in the first deletion record data can be written into the deletion record data of the latter layer of the two adjacent layers.

[0117] Regarding step 2 above, if the deletion record data and write record data in two adjacent layers are merged each time, and the invalid write operation record is a write operation record with an operation time earlier than the first write operation record and containing the keyword to be processed, then after deleting these invalid write operation records from the write operation record data to be merged, the remaining write operation records in the write operation record data to be merged can be written into the write operation record data of the next layer in the two adjacent layers. Based on this, the write operation record data of the next layer in the two adjacent layers only stores one write operation record containing the keyword to be processed. Correspondingly, the deletion record data of the next layer in the two adjacent layers only stores one deletion operation record containing the keyword to be processed. In other words, in this case, for each keyword in the log-structured merge tree except the first layer, there is only one write operation record containing the keyword in the write operation record data of that layer, that is, there is no overlap between the keywords contained in any two write operation record data in that layer; and there is only one deletion operation record containing the keyword in the deletion operation record data of that layer, that is, there is no overlap between the keywords contained in any two deletion operation record data in that layer. Furthermore, the number of operation records stored in the log structure merge tree can be reduced, the time for data reading can be shortened, and the efficiency of data reading can be improved.

[0118] In addition, deletion operation records can be stored separately from write operation records, that is, they can be stored in different log structure merge trees. Subsequently, deletion operation records in the deletion record data can be merged first, reducing the number of deletion operation records. In turn, this can reduce the number of deletion operation records that need to be parsed when reading data, shortening the data reading time and improving data reading efficiency.

[0119] If the deletion record data to be merged and the write record data to be merged belong to the same layer in the log structure merge LSM tree, then after merging the deletion operation records containing the pending keyword in the first deletion record data based on the pending write operation records corresponding to the pending keyword, the method further includes:

[0120] If the current write record data to be merged does not belong to the bottom layer in the log structure merge tree, the merge result obtained by merging the deletion operation records containing the keyword to be processed in the first deletion record data will be written to the next layer of the layer to which the current write record data to be merged belongs.

[0121] In the embodiment of the present application, the deletion record data and the writing record data belonging to the same layer may be merged each time, and the merged result may be written into the next layer of the layer.

[0122] If the write record data to be merged does not belong to the bottom layer in the log structure merge tree, and there is no write operation record containing the keyword to be processed in the write record data, the write operation record contained in the write record data can be stored in the write record data of the next layer in the log structure merge tree, and the write record data can also be deleted, that is, the write record data is not retained in the previous layer; if the write record data to be merged belongs to the bottom layer in the log structure merge tree, no processing can be done on the write record data to be merged.

[0123] Based on the above processing, there is no need to merge the write record data that does not contain the write operation record containing the keyword to be processed. Therefore, the number of record data that needs to be parsed can be further reduced, the number of I / O times can be reduced, and the waste of computing resources can be reduced.

[0124] If there is still deletion record data after the merge, the deletion operation record in the merged deletion record data will be written to the deletion record data of the next layer in the log structure merge tree; if there is still write record data after the merge, the write operation record in the merged write record data will be written to the write record data of the next layer in the log structure merge tree.

[0125] For example, in step S1042 above, if the deletion record data and the write record data belonging to the same layer are merged each time, and the invalid deletion operation records are other deletion operation records with an operation time earlier than the first deletion operation record and for the keyword to be processed, then after deleting these invalid deletion operation records from the first deletion operation data, the remaining deletion operation records in the first deletion operation data can be written into the deletion record data of the next layer.

[0126] For the above step S1043, if the deletion record data and write record data belonging to the same layer are merged each time, and the invalid deletion operation records are all deletion operation records containing the keyword to be processed in the first deletion record data, then after deleting this part of the invalid deletion operation records from the first deletion record data, the remaining deletion operation records in the first deletion record data can be written into the deletion record data of the next layer.

[0127] Regarding step 2 above, if the delete record data and write record data belonging to the same layer are merged each time, and the invalid write operation records are write operation records whose operation time is earlier than the first write operation record and that contain the keyword to be processed, then after deleting these invalid write operation records from the write record data to be merged, the remaining write operation records in the write record data to be merged can be written into the write record data of the next layer.

[0128] In one embodiment, the method further comprises:

[0129] Step 1: When a first preset storage condition is reached, the deletion record data stored in the preset cache space is written into the preset storage space.

[0130] The deletion record data stored in the preset cache space is generated based on a data deletion operation on the target database.

[0131] Step 2: When a second preset storage condition is met, the write record data stored in the preset cache space is written into the preset storage space.

[0132] The write record data stored in the preset cache space is generated based on a data write operation to the target database. The preset cache space may be a memory of the electronic device.

[0133] In an embodiment of the present application, the write operation records and delete operation records for the database can be stored separately in a preset cache space. That is, in the preset cache space, write record data containing only write operation records and delete record data containing only delete operation records can be generated based on user operations. In other words, the electronic device can record the corresponding write record data and delete record data in the preset cache space respectively according to the data operation on the target database. Furthermore, when the first preset storage condition is reached, the electronic device can write the delete operation record recorded in the delete record data stored in the preset cache space into the preset storage space, and the delete record data in the preset storage space can be obtained. And, when the second preset storage condition is reached, the electronic device can write the write operation record recorded in the write record data stored in the preset cache space into the preset storage space, and the write record data in the preset storage space can be obtained.

[0134] In one implementation, the electronic device may allocate storage space of specified sizes for the deletion record data and the write record data stored in the preset cache space, respectively. Accordingly, when the storage space occupied by the deletion record data stored in the preset cache space reaches a third preset threshold, it may be determined that the first preset storage condition is satisfied. When the storage space occupied by the write record data stored in the preset cache space reaches a fourth preset threshold, it may be determined that the second preset storage condition is satisfied.

[0135] Based on the above processing, data operation records can be cached in the preset cache space. When certain conditions are met, the data operation records in the preset cache space are stored in the preset storage space, thereby reducing the number of times the preset storage space is accessed and reducing the time consumed in recording data operation records.

[0136] In one embodiment, when a keyword value needs to be queried, a query can be performed in a preset cache space. If an operation record corresponding to the keyword is found in the preset cache space, the query result is obtained. If no operation record corresponding to the keyword is found in the preset cache space, the query can be performed in a preset storage space. Because querying in the predictive cache space is faster, query efficiency can be improved to a certain extent.

[0137] In one embodiment, the deletion record data stored in the preset storage space includes range deletion operation records; wherein the range deletion operation record data indicates a deletion operation for deleting values ​​corresponding to each keyword belonging to a specified keyword range in the target database.

[0138] For example, when a "RANGE_DELETE" instruction is received, a corresponding range delete operation record may be recorded.

[0139] A specified keyword range can be represented by multiple keywords. For example, the keyword range can be A, B, and D. Accordingly, the keywords belonging to the specified keyword range include A, B, and D. Alternatively, the specified keyword range can be represented by two keywords, one of which represents the starting keyword and the other represents the ending keyword. Accordingly, the starting keyword, the ending keyword, and all keywords between the starting keyword and the ending keyword are within the specified keyword range. For example, the specified keyword range can be [A, D]. Accordingly, keywords A, B, C, and D all belong to the specified keyword range.

[0140] Based on the above processing, since the range deletion operation record only occupies the space of one deletion operation record, when it is necessary to delete the value of a certain keyword range, the storage space occupied by the deletion operation record can be reduced based on the range deletion operation record, and then, redundant deletion operation records can be reduced, and then, the number of deletion operation records that need to be parsed when reading data can be reduced, the time for data reading can be reduced, and the efficiency of data reading can be improved.

[0141] Correspondingly, if the first deletion record data includes a range deletion operation record, the designated keyword range in the range deletion operation record is determined, and each keyword within the designated keyword range is used as a keyword to be processed, and then steps S103-S104 are executed.

[0142] In one embodiment, the deletion record data to be merged and the write record data to be merged belong to the same layer in the log structure merge tree; accordingly, after step S104, the method further includes:

[0143] If the deletion record data to be merged does not belong to the bottom layer in the log structure merge tree, the deletion operation record containing the keyword to be processed in the first deletion record data and the pending write operation record corresponding to the keyword to be processed will be merged to obtain the merged result, and written to the next layer in the log structure merge tree.

[0144] In one implementation, Figure 4 As shown, Figure 4 A schematic diagram of the data processing principle provided in an embodiment of the present application.

[0145] Figure 4 In the preset cache space, the electronic device can record write operation records for write operations on the database based on Put Memtable (memory write record table, i.e., the write record table in the preset cache space in this application), and record delete operation records for delete operations on the database based on Delete Memtable (memory delete record table, i.e., the delete record table in the preset cache space in this application).

[0146] Correspondingly, when the first preset storage condition in this application is reached, the deletion operation record recorded in the Delete Memtable can be written to the preset storage space to obtain the Delete sstable (disk deletion record table, that is, the deletion record table in the preset storage space in this application), such as Figure 4 sstable 3 (table 3) and sstable 4 (table 4) in.

[0147] When the second preset storage condition in this application is met, the write operation record recorded in the Put Memtable can be written to the preset storage space to obtain the Put sstable (disk write record table, that is, the write record table in the preset storage space in this application), such as Figure 4 sstable 1 (table 1) and sstable 2 (table 2) in.

[0148] In addition, based on the above steps S101 to S104 , the disk write record data and the disk delete record data may be merged.

[0149] Based on the same inventive concept, the present application also provides a data processing device, see Figure 5 , Figure 5 A structural diagram of a data processing device provided in an embodiment of the present application, the device comprising:

[0150] The first deletion record data determining module 501 is configured to determine the deletion record data to be merged from the deletion record data stored in the preset storage space when a preset merging condition is met, as the first deletion record data; wherein the deletion record data includes a deletion operation record of a data deletion operation on a target database;

[0151] A to-be-processed keyword determining module 502 is configured to determine a keyword in a deletion operation record included in the first deletion record data as a to-be-processed keyword;

[0152] a pending write operation record determination module 503, configured to determine, from the current pending write operation record data stored in the preset storage space, a write operation record containing the pending keyword, as a pending write operation record; wherein the write operation record data includes a write operation record of a data write operation to the target database;

[0153] The first merging module 504 is configured to merge the deletion operation records containing the keyword to be processed in the first deletion record data based on the write operation records to be processed corresponding to the keyword to be processed.

[0154] In some embodiments, the first merging module 504 includes:

[0155] a judgment submodule, configured to judge whether the deletion operation record containing the to-be-processed keyword in the first deletion record data and the operation record with the latest operation time among the to-be-processed write operation records corresponding to the to-be-processed keyword are deletion operation records;

[0156] a first merging submodule configured to, if the operation record with the latest operation time is a deletion operation record, determine from the first deletion record data the deletion operation record that contains the keyword to be processed and has the latest operation time, as the first deletion operation record, and delete from the first deletion record data the deletion operation records that contain the keyword to be processed except the first deletion operation record;

[0157] a second merging submodule, configured to delete the deletion operation record containing the keyword to be processed from the first deletion record data if the operation record with the latest corresponding operation time is not a deletion operation record;

[0158] The device further comprises:

[0159] a first write operation record determining module configured to determine, if the operation record with the latest operation time is not a delete operation record, the write operation record with the latest operation time from the pending write operation records corresponding to the pending keyword as the first write operation record;

[0160] a second merging module, configured to delete, from the corresponding write record data, the other write operation records except the first write operation record in the write operation records to be processed corresponding to the keyword to be processed;

[0161] The device further comprises:

[0162] a third merging module, configured to delete the pending write operation record corresponding to the pending keyword from the corresponding write record data if the operation record with the latest corresponding operation time is a delete operation record;

[0163] The deletion record data to be merged currently belongs to two adjacent layers in the log structure merge LSM tree, and the write record data to be merged currently belongs to the two adjacent layers;

[0164] The device further comprises:

[0165] a write record data storage module configured to, after merging the deletion operation records containing the keyword to be processed in the first deletion record data based on the write operation records to be processed corresponding to the keyword to be processed, write a merged result obtained by merging the deletion operation records containing the keyword to be processed in the first deletion record data into the latter of the two adjacent layers;

[0166] The deletion record data to be merged and the write record data to be merged belong to the same layer in the log structure merge LSM tree;

[0167] The device further comprises:

[0168] a merge result storage module for, after merging the delete operation records containing the keyword to be processed in the first deletion record data based on the write operation records corresponding to the keyword to be processed, if the current write record data to be merged does not belong to the bottom layer of the log structure merge tree, writing the merge result obtained by merging the delete operation records containing the keyword to be processed in the first deletion record data to a layer after the layer to which the current write record data to be merged belongs;

[0169] The device further comprises:

[0170] A first storage module is configured to write the deletion record data stored in the preset cache space into the preset storage space when a first preset storage condition is met; wherein the deletion record data stored in the preset cache space is generated based on a data deletion operation on the target database;

[0171] a second storage module, configured to write the write record data stored in the preset cache space into the preset storage space when a second preset storage condition is met; wherein the write record data stored in the preset cache space is generated based on a data write operation to the target database;

[0172] The deletion record data stored in the preset storage space includes a range deletion operation record, wherein the range deletion operation record represents a deletion operation for deleting values ​​corresponding to each keyword belonging to a specified keyword range in the target database.

[0173] The present application also provides an electronic device, such as Figure 6 Shown, including:

[0174] Memory 601, used for storing computer programs;

[0175] The processor 602 is configured to implement the steps of any data processing method in the above embodiments when executing the program stored in the memory 601 .

[0176] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 602, the communication interface, and the memory 601 communicate with each other via the communication bus.

[0177] The communication bus mentioned in the electronic device mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0178] The communication interface is used for communication between the above electronic device and other devices.

[0179] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0180] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0181] In another embodiment provided in the present application, a computer-readable storage medium is further provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps of any of the above-mentioned data processing methods are implemented.

[0182] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any one of the data processing methods in the above embodiments.

[0183] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0184] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0185] Each embodiment in this specification is described in a related manner. Similar portions between the embodiments can be referenced to each other. Each embodiment focuses on the differences between the other embodiments. In particular, since the apparatus, electronic device, computer-readable storage medium, and computer program product are generally similar to the method embodiments, their descriptions are relatively simple. For related portions, reference can be made to the descriptions of the method embodiments.

[0186] The above description is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application are included in the scope of protection of the present application.

Claims

1. A data processing method, characterized in that: The method comprises: When a preset merging condition is met, the deletion record data to be merged is determined from the deletion record data stored in the preset storage space as the first deletion record data; wherein the deletion record data includes a deletion operation record of the data deletion operation on the target database; determining a keyword in the deletion operation record included in the first deletion record data as a keyword to be processed; Determining, from the write record data currently to be merged stored in the preset storage space, a write operation record containing the keyword to be processed as the write operation record to be processed; wherein the write record data includes a write operation record of a data write operation for the target database; Based on the pending write operation record corresponding to the pending keyword, merging the deletion operation records containing the pending keyword in the first deletion record data; The merging of the deletion operation records containing the keyword to be processed in the first deletion record data based on the write operation records to be processed corresponding to the keyword to be processed includes: Determine whether the deletion operation record of the keyword to be processed included in the first deletion record data and the operation record with the latest operation time among the pending write operation records corresponding to the keyword to be processed are deletion operation records; If the operation record with the latest operation time is a deletion operation record, determining from the first deletion record data the deletion operation record that contains the keyword to be processed and has the latest operation time as the first deletion operation record, and deleting from the first deletion record data the deletion operation records that contain the keyword to be processed except the first deletion operation record; If the operation record with the latest corresponding operation time is not a deletion operation record, the deletion operation record containing the to-be-processed keyword is deleted from the first deletion record data.

2. The method according to claim 1, characterized in that The method further comprises: If the operation record with the latest operation time is not a delete operation record, determining the write operation record with the latest operation time from the pending write operation records corresponding to the pending keyword as the first write operation record; The other write operation records except the first write operation record in the write operation records to be processed corresponding to the to-be-processed keyword are deleted from the corresponding write record data.

3. The method according to claim 1, characterized in that The method further comprises: If the operation record with the latest corresponding operation time is a deletion operation record, the pending write operation record corresponding to the pending keyword is deleted from the corresponding write record data.

4. The method according to claim 1, wherein The deletion record data to be merged currently belongs to two adjacent layers in the log structure merge LSM tree, and the write record data to be merged currently belongs to the two adjacent layers; After merging the deletion operation records containing the to-be-processed keyword in the first deletion record data based on the to-be-processed write operation records corresponding to the to-be-processed keyword, the method further includes: A merging result obtained by merging the deletion operation records containing the keyword to be processed in the first deletion record data is written into the latter layer of the two adjacent layers.

5. The method according to claim 1, wherein The deletion record data to be merged and the write record data to be merged belong to the same layer in the log structure merge LSM tree; After merging the deletion operation records containing the to-be-processed keyword in the first deletion record data based on the to-be-processed write operation records corresponding to the to-be-processed keyword, the method further includes: If the current write record data to be merged does not belong to the bottom layer in the log structure merge tree, the merge result obtained by merging the deletion operation records containing the keyword to be processed in the first deletion record data will be written to the next layer of the layer to which the current write record data to be merged belongs.

6. The method according to claim 1, characterized in that The method further comprises: When a first preset storage condition is met, writing the deletion record data stored in the preset cache space into the preset storage space; wherein the deletion record data stored in the preset cache space is generated based on a data deletion operation on the target database; When a second preset storage condition is met, the write record data stored in the preset cache space is written into the preset storage space; wherein the write record data stored in the preset cache space is generated based on a data write operation for the target database.

7. The method according to claim 1, characterized in that The deletion record data stored in the preset storage space includes a range deletion operation record, wherein the range deletion operation record represents a deletion operation for deleting values ​​corresponding to each keyword belonging to a specified keyword range in the target database.

8. A data processing device, characterized in that: The device comprises: a first deletion record data determining module, configured to determine, when a preset merging condition is met, deletion record data currently to be merged from the deletion record data stored in the preset storage space as first deletion record data; wherein the deletion record data includes a deletion operation record of a data deletion operation on a target database; a module for determining keywords to be processed, configured to determine keywords in the deletion operation record contained in the first deletion record data as keywords to be processed; a pending write operation record determination module, configured to determine, from the current pending merged write record data stored in the preset storage space, a write operation record containing the pending keyword, as a pending write operation record; wherein the write record data includes a write operation record of a data write operation for the target database; a first merging module, configured to merge the deletion operation records containing the keyword to be processed in the first deletion record data based on the write operation records to be processed corresponding to the keyword to be processed; The first merging module includes: a judgment submodule, configured to judge whether the deletion operation record containing the to-be-processed keyword in the first deletion record data and the operation record with the latest operation time among the to-be-processed write operation records corresponding to the to-be-processed keyword are deletion operation records; a first merging submodule configured to, if the operation record with the latest operation time is a deletion operation record, determine from the first deletion record data the deletion operation record that contains the keyword to be processed and has the latest operation time, as the first deletion operation record, and delete from the first deletion record data the deletion operation records that contain the keyword to be processed except the first deletion operation record; The second merging submodule is configured to delete the deletion operation record containing the keyword to be processed from the first deletion record data if the operation record with the latest corresponding operation time is not a deletion operation record.

9. The device according to claim 8, characterized in that The device further comprises: a first write operation record determining module configured to determine, if the operation record with the latest operation time is not a delete operation record, the write operation record with the latest operation time from the pending write operation records corresponding to the pending keyword as the first write operation record; a second merging module, configured to delete, from the corresponding write record data, the other write operation records except the first write operation record in the write operation records to be processed corresponding to the keyword to be processed; The device further comprises: a third merging module, configured to delete the pending write operation record corresponding to the pending keyword from the corresponding write record data if the operation record with the latest corresponding operation time is a delete operation record; The deletion record data to be merged currently belongs to two adjacent layers in the log structure merge LSM tree, and the write record data to be merged currently belongs to the two adjacent layers; The device further comprises: a write record data storage module configured to, after merging the deletion operation records containing the keyword to be processed in the first deletion record data based on the write operation records to be processed corresponding to the keyword to be processed, write a merged result obtained by merging the deletion operation records containing the keyword to be processed in the first deletion record data into the latter of the two adjacent layers; The deletion record data to be merged and the write record data to be merged belong to the same layer in the log structure merge LSM tree; The device further comprises: a merge result storage module for, after merging the delete operation records containing the keyword to be processed in the first deletion record data based on the write operation records corresponding to the keyword to be processed, if the current write record data to be merged does not belong to the bottom layer of the log structure merge tree, writing the merge result obtained by merging the delete operation records containing the keyword to be processed in the first deletion record data to a layer after the layer to which the current write record data to be merged belongs; The device further comprises: A first storage module is configured to write the deletion record data stored in the preset cache space into the preset storage space when a first preset storage condition is met; wherein the deletion record data stored in the preset cache space is generated based on a data deletion operation on the target database; a second storage module, configured to write the write record data stored in the preset cache space into the preset storage space when a second preset storage condition is met; wherein the write record data stored in the preset cache space is generated based on a data write operation to the target database; The deletion record data stored in the preset storage space includes a range deletion operation record, wherein the range deletion operation record represents a deletion operation for deleting values ​​corresponding to each keyword belonging to a specified keyword range in the target database.

10. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the method steps described in any one of claims 1 to 7 when executing a program stored in a memory.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps of any one of claims 1 to 7 are implemented.

Images