A synchronization method for data update and related device

Abstract

The application discloses a data update synchronization method and related device, the method comprises the following steps: after B+ tree data table of B+ tree database is synchronized to LSM tree database to obtain LSM tree data table, current B+ tree data update information of the B+ tree data table is acquired; wherein, the LSM tree data table comprises a deletion identifier column and a data version column; on the basis of the current B+ tree data update information, the inserted data, the deletion identifier field value and the data version field value corresponding to the inserted data are configured to obtain LSM tree data insertion information; and the LSM tree data insertion information is used to update the LSM tree data table. It can be seen that the current B+ tree data update information is converted into the LSM tree data insertion information, so that the LSM tree database performs a high-performance data insertion operation on the basis of the LSM tree data insertion information during data update, and the cost of synchronizing data update between the B+ tree database and the LSM tree database is reduced and the efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a data update synchronization method and related device. BACKGROUND

[0002] When the data volume in a B+ tree database increases, the performance of data analysis of the B+ tree relational database will decrease, and it is necessary to synchronize the data in the B+ tree database to a log structured merge (LSM) tree database to perform professional data analysis. When the data in the B+ tree database is updated, the corresponding data in the LSM tree database also needs to be updated.

[0003] In terms of data update, the data in the B+ tree database can be updated in place, which makes the transaction support of the B+ tree database more friendly. However, the in-place update of the corresponding data in the LSM tree database needs to rewrite the entire data block in the LSM tree database, which is costly and inefficient. Therefore, there is a problem of high cost and low efficiency when the data update in the B+ tree database is synchronized to the LSM tree database. SUMMARY

[0004] Therefore, the present application provides a data update synchronization method and related device, which reduces the cost and improves the efficiency when the data update in the B+ tree database is synchronized to the LSM tree database.

[0005] In a first aspect, an embodiment of the present application provides a data update synchronization method, which comprises:

[0006] After the B+ tree data table of the B+ tree database is synchronized to the log structured merge (LSM) tree database to obtain an LSM tree data table, current B+ tree data update information of the B+ tree data table is obtained. The LSM tree data table includes a deletion identifier column and a data version column.

[0007] Based on the current B+ tree data update information, the inserted data, the deletion identifier field value and the data version field value corresponding to the inserted data are configured to obtain LSM tree data insertion information.

[0008] Based on the LSM tree data insertion information, the LSM tree data table is updated.

[0009] Optionally, the current B+ tree data update information includes at least one of current B+ tree data modification information, current B+ tree data deletion information and current B+ tree data insertion information.

[0010] Optionally, the current B+ tree data update information comprises the current B+ tree data modification information, the LSM tree data insertion information is obtained by configuring the insertion data, a deletion identifier field value corresponding to the insertion data and a data version field value corresponding to the insertion data based on the current B+ tree data update information, and the LSM tree data insertion information comprises:

[0011] obtaining old data records, new data records and update times corresponding to the current B+ tree data modification information;

[0012] configuring the insertion data, the deletion identifier field value corresponding to the insertion data and the data version field value corresponding to the insertion data as the old data records, a deletion identifier field value corresponding to the old data records indicating deletion, a data version field value corresponding to the old data records indicating the update times, the new data records, a deletion identifier field value corresponding to the new data records indicating non-deletion and a data version field value corresponding to the new data records indicating the update times, and obtaining the LSM tree data insertion information.

[0013] Optionally, the current B+ tree data update information comprises the current B+ tree data deletion information, the LSM tree data insertion information is obtained by configuring the insertion data, a deletion identifier field value corresponding to the insertion data and a data version field value corresponding to the insertion data based on the current B+ tree data update information, and the LSM tree data insertion information comprises:

[0014] obtaining old data records and update times corresponding to the current B+ tree data deletion information;

[0015] configuring the insertion data, the deletion identifier field value corresponding to the insertion data and the data version field value corresponding to the insertion data as the old data records, a deletion identifier field value corresponding to the old data records indicating deletion and a data version field value corresponding to the old data records indicating the update times, and obtaining the LSM tree data insertion information.

[0016] Optionally, the current B+ tree data update information comprises the current B+ tree data insertion information, the LSM tree data insertion information is obtained by configuring the insertion data, a deletion identifier field value corresponding to the insertion data and a data version field value corresponding to the insertion data based on the current B+ tree data update information, and the LSM tree data insertion information comprises:

[0017] obtaining new data records and update times corresponding to the current B+ tree data insertion information;

[0018] configuring the insertion data, the deletion identifier field value corresponding to the insertion data and the data version field value corresponding to the insertion data as the new data records, a deletion identifier field value corresponding to the new data records indicating non-deletion and a data version field value corresponding to the new data records indicating the update times, and obtaining the LSM tree data insertion information.

[0019] Optionally, the method further comprises:

[0020] In the data record query to the LSM tree data table, the data record query request carries the data record identifier, the deletion identifier field value representing non-deletion, and the data version field value corresponding to the data record identifier representing the latest update times as the query conditions, and the data record in the LSM tree data table is queried.

[0021] Optionally, the method further comprises:

[0022] A timestamp column is added in the LSM tree data table, and the timestamp field value corresponding to the timestamp column is used to record the update time corresponding to the B+ tree data update information;

[0023] After the B+ tree data table data is updated, the data record in the B+ tree data table is read, and the first statistical data corresponding to each field column in the B+ tree data table and the reading time are obtained;

[0024] After a preset time period interval from the reading time, each data record identifier, the deletion identifier field value representing non-deletion, the data version field value corresponding to the data record identifier representing the latest update times, and the timestamp field value representing a time earlier than the reading time are used as the reading conditions to read the data record in the LSM tree data table, and the second statistical data corresponding to the field column in the LSM tree data table is obtained;

[0025] If the second statistical data is inconsistent with the first statistical data, it is prompted that the data between the LSM tree data table and the B+ tree data table is inconsistent.

[0026] In a second aspect, an embodiment of the present application provides a data update synchronization device, and the device comprises:

[0027] An acquisition unit is configured to, after a B+ tree data table of a B+ tree database is synchronized to a log-structured merge LSM tree database to obtain an LSM tree data table, acquire current B+ tree data update information of the B+ tree data table; the LSM tree data table comprises a deletion identifier column and a data version column.

[0028] A first obtaining unit is configured to, based on the current B+ tree data update information, configure inserted data, a deletion identifier field value corresponding to the inserted data, and a data version field value to obtain LSM tree data insertion information.

[0029] An updating unit is configured to, based on the LSM tree data insertion information, update the LSM tree data table.

[0030] Optionally, the current B+ tree data update information comprises at least one of current B+ tree data modification information, current B+ tree data deletion information and current B+ tree data insertion information.

[0031] Optionally, the current B+ tree data update information comprises the current B+ tree data modification information, and the first obtaining unit is configured to:

[0032] obtain old data records, new data records and update times corresponding to the current B+ tree data modification information;

[0033] configure the insertion data, a deletion identifier field value and a data version field value corresponding to the insertion data into the old data records, a deletion identifier field value corresponding to the old data records indicating deletion, a data version field value corresponding to the old data records indicating the update times, the new data records, a deletion identifier field value corresponding to the new data records indicating non-deletion, and a data version field value corresponding to the new data records indicating the update times, to obtain the LSM tree data insertion information.

[0034] Optionally, the current B+ tree data update information comprises the current B+ tree data deletion information, and the first obtaining unit is configured to:

[0035] obtain old data records and update times corresponding to the current B+ tree data deletion information;

[0036] configure the insertion data, a deletion identifier field value and a data version field value corresponding to the insertion data into the old data records, a deletion identifier field value corresponding to the old data records indicating deletion, and a data version field value corresponding to the old data records indicating the update times, to obtain the LSM tree data insertion information.

[0037] Optionally, the current B+ tree data update information comprises the current B+ tree data insertion information, and the first obtaining unit is configured to:

[0038] obtain new data records and update times corresponding to the current B+ tree data insertion information;

[0039] configure the insertion data, a deletion identifier field value and a data version field value corresponding to the insertion data into the new data records, a deletion identifier field value corresponding to the new data records indicating non-deletion, and a data version field value corresponding to the new data records indicating the update times, to obtain the LSM tree data insertion information.

[0040] Optionally, the apparatus further comprises:

[0041] The query unit is configured to, when querying the data record in the LSM tree data table, take the data record identifier, the deletion identifier field value representing non-deletion, and the data version field value corresponding to the data record identifier and representing the latest update times as query conditions to query the data record in the LSM tree data table.

[0042] Optionally, the apparatus further comprises:

[0043] The adding unit is configured to add a timestamp column in the LSM tree data table, and a timestamp field value corresponding to the timestamp column is used to record an update time corresponding to B+ tree data update information.

[0044] The second obtaining unit is configured to, after the B+ tree data table is updated, read the data record in the B+ tree data table to obtain first statistical data corresponding to each field column in the B+ tree data table and a read time.

[0045] The third obtaining unit is configured to, after a preset time period from the read time, read the data record in the LSM tree data table to obtain second statistical data corresponding to each field column in the LSM tree data table, by taking each data record identifier, the deletion identifier field value representing non-deletion, the data version field value corresponding to the data record identifier and representing the latest update times, and a timestamp field value representing a time earlier than the read time as read conditions.

[0046] The prompting unit is configured to, if the second statistical data is inconsistent with the first statistical data, prompt that the data between the LSM tree data table and the B+ tree data table is inconsistent.

[0047] In a third aspect, an embodiment of the present application provides a synchronization apparatus for data update, the apparatus comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors to execute the synchronization method for data update as described in the first aspect.

[0048] In a fourth aspect, an embodiment of the present application provides a machine readable medium having instructions stored thereon, which, when executed by one or more processors, cause an apparatus to perform the synchronization method for data update as described in the first aspect.

[0049] Compared with the prior art, the present application has at least the following advantages:

[0050] The technical scheme of the embodiment of the present application is adopted, after the B+ tree data table of the B+ tree database is synchronized to the LSM tree database to obtain the LSM tree data table, current B+ tree data update information of the B+ tree data table is acquired; wherein, the LSM tree data table comprises a deletion identifier column and a data version column; on the basis of the current B+ tree data update information, the inserted data, the deletion identifier field value and the data version field value corresponding to the inserted data are configured to obtain LSM tree data insertion information; and the LSM tree data table is updated by using the LSM tree data insertion information. It can be seen that, by increasing the deletion identifier column and the data version column in the LSM tree data table, when the current B+ tree data update information of the B+ tree data table is captured, the inserted data and the corresponding deletion identifier field value and data version field value are set in the manner, the current B+ tree data update information can be converted into LSM tree data insertion information, so that the LSM tree database performs high-performance data insertion operation based on the LSM tree data insertion information when data is updated, and the cost is reduced and the efficiency is improved when the data update in the B+ tree database is synchronized to the LSM tree database. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical scheme of the embodiments of the present application, the drawings needed in the description of the embodiments of the present application will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0052] Figure 1 The system framework schematic diagram involved in one application scenario in the embodiments of the present application;

[0053] Figure 2 The flowchart schematic diagram of the data update synchronization method provided by the embodiments of the present application;

[0054] Figure 3 The schematic diagram of the B+ tree data table and the corresponding LSM tree data table provided by the embodiments of the present application;

[0055] Figure 4 The schematic diagram of the current B+ tree data modification information of the B+ tree data table and the LSM tree data insertion information of the corresponding LSM tree data table provided by the embodiments of the present application;

[0056] Figure 5 The schematic diagram of the current B+ tree data deletion information of the B+ tree data table and the LSM tree data insertion information of the corresponding LSM tree data table provided by the embodiments of the present application;

[0057] Figure 6A schematic diagram of current B+ tree data insertion information of a B+ tree data table and corresponding LSM tree data insertion information of an LSM tree data table provided for an embodiment of the present application;

[0058] Figure 7 A schematic diagram of a B+ tree data table after data update and an LSM tree data table after synchronous data update provided for an embodiment of the present application;

[0059] Figure 8 A structural schematic diagram of a synchronous device for data update provided for an embodiment of the present application;

[0060] Figure 9 A structural schematic diagram of a synchronous device for data update provided for an embodiment of the present application;

[0061] Figure 10 A structural schematic diagram of a server provided for an embodiment of the present application. DETAILED DESCRIPTION

[0062] In order to enable personnel in the technical field to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0063] When data in the B tree database is updated, the corresponding data in the LSM tree database also needs to be updated. In terms of data update, the data in the B+ tree database can be updated in place, which makes the transaction support of the B+ tree database more friendly. However, synchronous in-place update of the corresponding data in the LSM tree database needs to rewrite the entire data block in the LSM tree database, which is costly and inefficient. Thus, there is a problem of high cost and low efficiency in updating the data in the B+ tree database and synchronizing it to the LSM tree database.

[0064] To solve this problem, in the embodiment of the present application, after the B+ tree data table of the B+ tree database is synchronized to the LSM tree database to obtain the LSM tree data table, the current B+ tree data update information of the B+ tree data table is obtained; wherein the LSM tree data table comprises a deletion identifier column and a data version column; on the basis of the current B+ tree data update information, the inserted data, the deletion identifier field value and the data version field value corresponding to the inserted data are configured to obtain the LSM tree data insertion information; and the LSM tree data table is updated with the LSM tree data insertion information. As can be seen, by adding the deletion identifier column and the data version column in the LSM tree data table, when the current B+ tree data update information of the B+ tree data table is captured, the inserted data and the corresponding deletion identifier field value and data version field value are set in the manner, which can convert the current B+ tree data update information into the LSM tree data insertion information, so that the LSM tree database performs high-performance data insertion operation based on the LSM tree data insertion information when data is updated, and the cost is reduced and the efficiency is improved when the data update in the B+ tree database is synchronized to the LSM tree database.

[0065] For example, one of the scenarios of the embodiment of the present application can be applied to the scenario as shown in Figure 1 As shown in the figure, the scenario comprises a B+ tree database 101, a processor 102 and an LSM tree database 103, wherein the B+ tree database 101 interacts with the processor 102, and the processor 102 and the LSM tree database 103 interact. After the processor 102 synchronizes the B+ tree data table of the B+ tree database 101 to the LSM tree database 103 to obtain the LSM tree data table, the processor 102 performs the synchronization data update method provided by the embodiment of the present application to realize the synchronization of the data update in the B+ tree database 101 to the LSM tree database 103.

[0066] Firstly, in the above application scenario, although the actions of the embodiment of the present application are described as being executed by the server 101, these actions can also be executed by the client 102, or can be partially executed by the client 102 and partially executed by the server 101. The present application is not limited in the execution subject, as long as the actions disclosed in the embodiment of the present application are executed.

[0067] Secondly, the above scenario is only one scenario example provided by the embodiment of the present application, and the embodiment of the present application is not limited to this scenario.

[0068] The specific implementation of the synchronization method of the data update and the related device in the embodiment of the present application will be described in detail below with reference to the accompanying drawings and by embodiments.

[0069] Exemplary method

[0070] Referring to Figure 2The diagram illustrates a flowchart of a data update synchronization method according to an embodiment of this application. In this embodiment, the method may include, for example, the following steps:

[0071] Step 201: After synchronizing the B+ tree data table in the B+ tree database to the log-structured merged LSM tree database to obtain the LSM tree data table, obtain the current B+ tree data update information of the B+ tree data table; the LSM tree data table includes a deletion identifier column and a data version column.

[0072] As the amount of data in the B+ tree database gradually increases, in order to perform professional data analysis on the data in the B+ tree database, the data in the B+ tree database needs to be synchronized to the LSM tree database. For example, the B+ tree data table in the B+ tree database is synchronized to the LSM tree database to obtain a data table that corresponds to the B+ tree data table. Subsequently, in order to facilitate the updating of this data table, it is also necessary to add a deletion identifier column and a data version column to the data table to obtain the LSM tree data table.

[0073] As an example, such as Figure 3 The diagram illustrates a B+ tree data table and its corresponding LSM tree data table. The LSM tree data table, compared to the B+ tree data table, adds a deletion flag column and a data version column. The deletion flag column indicates that the data has not been deleted; for example, the deletion flag column value is 0. The data version column indicates the initial version; for example, the data version column value is 0.

[0074] When data in the B-tree database is updated, the corresponding data in the LSM-tree database also needs to be updated; that is, when the B+-tree data table is updated, the LSM-tree data table also needs to be updated accordingly. Therefore, in this embodiment, it is first necessary to obtain the current B+-tree data update information of the B+-tree data table.

[0075] Common data update information can be categorized into three types: data modification information, data deletion information, and data insertion information. Therefore, in an optional embodiment of this application, the current B+ tree data update operation includes at least one of the following: current B+ tree data modification information, current B+ tree data deletion information, and current B+ tree data insertion information.

[0076] Step 202: Based on the current B+ tree data update information, configure the inserted data, the deletion identifier field value and the data version field value corresponding to the inserted data to obtain LSM tree data insertion information.

[0077] The B+ tree data table of the B+ tree database can be updated in place, which makes the transaction support of the B+ tree database more friendly. However, the in-place synchronous update of the LSM tree data table of the LSM tree database needs to rewrite the entire corresponding data block in the LSM tree database, which has the problems of high cost and low efficiency. Therefore, in the embodiments of the present application, considering that the LSM tree data table of the LSM tree database has high performance in data insertion operation, the current B+ tree data update information of the B+ tree data table obtained in step 201 can be converted into LSM tree data insertion information of the LSM tree data table. That is, on the basis of the current B+ tree data update information, the insertion data, the deletion identifier field value and the data version field value corresponding to the insertion data are configured to obtain the LSM tree data insertion information.

[0078] On the basis that the current B+ tree data update information can be any one or a combination of the current B+ tree data modification information, the current B+ tree data deletion information and the current B+ tree data insertion information; since the current B+ tree data modification information, the current B+ tree data deletion information and the current B+ tree data insertion information belong to different types of data update information, the specific implementation of converting the current B+ tree data update information into the LSM tree data insertion information is also different. The specific implementation is as follows:

[0079] First, the current B+ tree data modification information of the B+ tree data table indicates that the old data record in the B+ tree data table is modified into a new data record, and the current B+ tree data modification information is converted into LSM tree data insertion information. In fact, on the basis of obtaining the old data record, the new data record and the update times corresponding to the current B+ tree data modification information, the data insertion information indicating that the old data record is deleted in this data update and the data insertion information indicating that the new data record is added (not deleted) in this data update are configured as the LSM tree data insertion information. Among them, the deletion identifier field value is used to indicate deletion or non-deletion, and the data version field value indicates the update times. Therefore, in an optional embodiment of the present application, step 202 may, for example, include the following steps A-B:

[0080] Step A: Obtain the old data record, the new data record and the update times corresponding to the current B+ tree data modification information.

[0081] Step B: configuring the insertion data, the deletion identifier field value and the data version field value corresponding to the insertion data as the old data record, the deletion identifier field value corresponding to the old data record indicating deletion, the data version field value corresponding to the old data record indicating the update times, and the new data record, the deletion identifier field value corresponding to the new data record indicating non-deletion, and the data version field value corresponding to the new data record indicating the update times, to obtain the LSM tree data insertion information.

[0082] As an example, as shown in a current B+ tree data modification information of a B+ tree data table and a corresponding LSM tree data insertion information of an LSM tree data table, Figure 4 Figure 3 On the basis of the current B+ tree data modification information indicating that the old data record

ID=2, name=Li Si

ID=2, name=Er Ya

ID=2, name=Li Si, deletion identifier=1, data version=1

ID=2, name=Er Ya, deletion identifier=0, data version=1

[0083] Secondly, the current B+ tree data deletion information of the B+ tree data table indicates that the old data record in the B+ tree data table is deleted, and the current B+ tree data modification information is converted into LSM tree data insertion information. In fact, on the basis of obtaining the old data record and the update times corresponding to the current B+ tree data deletion information, the data insertion information indicating that the old data record is deleted in this data update is configured as the LSM tree data insertion information. Similarly, the deletion identifier field value is used to indicate deletion or non-deletion, and the data version field value indicates the update times. Therefore, in an optional embodiment of the present application, the step 202 may, for example, include the following steps C-D:

[0084] Step C: obtaining the old data record and the update times corresponding to the current B+ tree data deletion information;

[0085] Step D: configuring the insertion data, the deletion identifier field value and the data version field value corresponding to the insertion data as the old data record, the deletion identifier field value corresponding to the old data record indicating deletion, and the data version field value corresponding to the old data record indicating the update times, to obtain the LSM tree data insertion information.

[0086] As an example, on the basis of the above example, as shown in a current B+ tree data modification information of a B+ tree data table and a corresponding LSM tree data insertion information of an LSM tree data table, Figure 5 ​A diagram of current B+ tree data deletion information of a B+ tree data table and corresponding LSM tree data insertion information of an LSM tree data table. Further, the current B+ tree data deletion information indicates that the old data record

ID=3, name= Wang five

ID=3, name= Wang five, deletion identifier=1, data version=2

[0087] Thirdly, the current B+ tree data insertion information of the B+ tree data table indicates that a new data record is inserted in the B+ tree data table, and the current B+ tree data insertion information is converted into LSM tree data insertion information. In fact, on the basis of obtaining the new data record and the update times corresponding to the current B+ tree data insertion information, the data insertion information indicating that the new data record is added (not deleted) in this data update is configured as the LSM tree data insertion information. Similarly, the deletion identifier field value is used to indicate deletion or non-deletion, and the data version field value indicates the update times. Therefore, in an optional embodiment of the present application, the step 202 may, for example, include the following steps E-F:

[0088] Step E: obtaining the new data record and the update times corresponding to the current B+ tree data insertion information;

[0089] Step F: configuring the insertion data, the deletion identifier field value and the data version field value corresponding to the insertion data into the new data record, the deletion identifier field value corresponding to the new data record indicating non-deletion, and the data version field value corresponding to the new data record indicating the update times, to obtain the LSM tree data insertion information.

[0090] As an example, on the basis of the above example, as Figure 6 A diagram of current B+ tree data insertion information of a B+ tree data table and corresponding LSM tree data insertion information of an LSM tree data table. Wherein, the current B+ tree data insertion information indicates that a new data record

ID=4, name= Zhao six

ID=4, name= Zhao six, deletion identifier=0, data version=3

[0091] Step 203: updating the LSM tree data table based on the LSM tree data insertion information.

[0092] In the embodiments of the present application, after the LSM tree data insertion information is obtained in step 202, the LSM tree data table needs to be updated using the LSM tree data insertion information. Based on the LSM tree data insertion information, high-performance data insertion operations can be performed in the LSM tree data table of the LSM tree database, so that the cost of data update in the LSM tree database is reduced and the efficiency is improved, thereby reducing the cost and improving the efficiency of data update in the B+ tree database when synchronizing to the LSM tree database.

[0093] In addition, in the embodiments of the present application, after the data update in the B+ tree database is synchronized to the LSM tree database, when data record queries are performed on the LSM tree data table based on the data update characteristics of the LSM tree data table of the LSM tree database, instead of querying all data records corresponding to the data record identifier carried by the data record query request, the query condition is obtained by combining the non-deleted deletion identifier field value and the data version field value corresponding to the data record identifier, so that the correct data record after synchronization of data update can be queried from the LSM tree data table.

[0094] Therefore, in an optional embodiment of the present application, the method can further include step G: when data record queries are performed on the LSM tree data table, the data record query request carries a data record identifier, a non-deleted deletion identifier field value, and a data version field value corresponding to the data record identifier, which are used as query conditions to query data records in the LSM tree data table.

[0095] As an example, in the LSM tree data insertion information of the LSM tree data table shown in FIG. 8, when the data record query request carried by the LSM tree data table carries a data record identifier "ID=2", the query condition is "ID=2 & deletion identifier=0 & data version=1", and the data record query result is

ID=2, name=Er Ya, deletion identifier=0, data version=1

ID=2, name=Er Ya, deletion identifier=0, data version=1

[0096] In addition, in the embodiment of the present application, after the data update in the B+ tree database is synchronized to the LSM tree database, it can also be verified whether the data between the B+ tree data table of the B+ tree database and the LSM tree data table of the LSM tree database is consistent after the synchronization of the data update is completed. Specifically, a timestamp column is added in the LSM tree data table of the LSM tree database, and the timestamp field value corresponding to the timestamp column can record the update time corresponding to the B+ tree data update information of the B+ tree data table; then the data record in the B+ tree data table after the data update is read, and the statistical data corresponding to each field column is counted as the first statistical data; a certain time for the synchronization of the data update is given, and after a period of time corresponding to the reading operation, the correct data record in the LSM tree data table after the synchronization of the data update is read, and the statistical data corresponding to each field column is counted as the second statistical data, as described above. Only when it is judged that the second statistical data is consistent with the first statistical data, it is determined that the synchronization of the data update is successful, and the data between the B+ tree data table and the LSM tree data table is consistent; otherwise, the data between the B+ tree data table and the LSM tree data table is inconsistent, and the inconsistent situation needs to be prompted so as to view the B+ tree data table and the LSM tree data table and make relevant adjustment to ensure the data consistency.

[0097] Therefore, in an optional implementation of the embodiment of the present application, the method can further include the following steps H to K, for example:

[0098] Step H: A timestamp column is added in the LSM tree data table, and the timestamp field value corresponding to the timestamp column is used to record the update time corresponding to the B+ tree data update information;

[0099] Step I: After the data update in the B+ tree data table, the data record in the B+ tree data table is read to obtain the first statistical data corresponding to each field column in the B+ tree data table and the reading time;

[0100] Step J: After a preset time period from the reading time, the data record in the LSM tree data table is read with each data record identifier, a non-deleted deletion identifier field value, a data version field value corresponding to the data record identifier and indicating the latest update times, and a timestamp field value indicating a time earlier than the reading time as the reading condition, to obtain the second statistical data corresponding to each field column in the LSM tree data table;

[0101] Step K: If the second statistical data is inconsistent with the first statistical data, it is prompted that the data between the LSM tree data table and the B+ tree data table is inconsistent.

[0102] As an example, on the basis of Figure 4 , in Figure 7An example of a B+ tree data table after data update and a schematic view of an LSM tree data table after synchronization data update, wherein a time stamp column is added in the LSM tree data table, data records in the B+ tree data table are read as

ID = 1, name = Zhang San

ID = 2, name = Er Ya

ID = 3, name = Wang Wu

ID = 1, name = Zhang San, deletion identifier = 0, data version = 0

ID = 2, name = Er Ya, deletion identifier = 0, data version = 1

ID = 3, name = Wang Wu, deletion identifier = 0, data version = 1

[0103] Through various embodiments provided by the embodiment, after the B+ tree data table of the B+ tree database is synchronized to the LSM tree database to obtain the LSM tree data table, the current B+ tree data update information of the B+ tree data table is obtained; wherein the LSM tree data table comprises a deletion identifier column and a data version column; on the basis of the current B+ tree data update information, the inserted data, the deletion identifier field value and the data version field value corresponding to the inserted data are configured to obtain the LSM tree data insertion information; and the LSM tree data table is updated by using the LSM tree data insertion information. It can be seen that, by adding the deletion identifier column and the data version column in the LSM tree data table, when the current B+ tree data update information of the B+ tree data table is captured, the inserted data and the deletion identifier field value and the data version field value corresponding to the inserted data are set, the current B+ tree data update information is converted into the LSM tree data insertion information, the LSM tree database performs high-performance data insertion operation based on the LSM tree data insertion information when data is updated, and the cost is reduced and the efficiency is improved when the data update in the B+ tree database is synchronized to the LSM tree database.

[0104] Exemplary apparatus

[0105] Referring to Figure 8 , a structural schematic diagram of a data update synchronization device in an embodiment of the present application is shown. In the embodiment, the device may, for example, specifically include:

[0106] The obtaining unit 801 is configured to, after a B+ tree data table of a B+ tree database is synchronized to a log-structured merge (LSM) tree database to obtain an LSM tree data table, obtain current B+ tree data update information of the B+ tree data table; the LSM tree data table includes a deletion identifier column and a data version column.

[0107] The first obtaining unit 802 is configured to, based on the current B+ tree data update information, configure inserted data, a deletion identifier field value corresponding to the inserted data, and a data version field value corresponding to the inserted data, to obtain LSM tree data insertion information.

[0108] The updating unit 803 is configured to, based on the LSM tree data insertion information, update the LSM tree data table.

[0109] In an optional implementation of the embodiment of the present application, the current B+ tree data update information includes at least one of current B+ tree data modification information, current B+ tree data deletion information, and current B+ tree data insertion information.

[0110] In an optional implementation of the embodiment of the present application, the current B+ tree data update information includes the current B+ tree data modification information, and the first obtaining unit 802 is configured to:

[0111] obtain an old data record, a new data record, and an update number corresponding to the current B+ tree data modification information;

[0112] configure the inserted data, the deletion identifier field value corresponding to the inserted data, and the data version field value corresponding to the inserted data, to the old data record, a deletion identifier field value corresponding to the old data record indicating deletion, a data version field value corresponding to the old data record indicating the update number, the new data record, a deletion identifier field value corresponding to the new data record indicating non-deletion, and a data version field value corresponding to the new data record indicating the update number, to obtain the LSM tree data insertion information.

[0113] In an optional implementation of the embodiment of the present application, the current B+ tree data update information includes the current B+ tree data deletion information, and the first obtaining unit 802 is configured to:

[0114] obtain an old data record and an update number corresponding to the current B+ tree data deletion information;

[0115] configuring the insertion data, the deletion identifier field value corresponding to the insertion data and the data version field value corresponding to the insertion data as the new data record, the deletion identifier field value corresponding to the new data record indicating non-deletion and the data version field value corresponding to the new data record indicating the update times, to obtain the LSM tree data insertion information.

[0116] In an optional implementation of the embodiments of the present application, the current B+ tree data update information includes the current B+ tree data insertion information, and the first obtaining unit 802 is configured to:

[0117] obtain the new data record corresponding to the current B+ tree data insertion information and the update times;

[0118] configuring the insertion data, the deletion identifier field value corresponding to the insertion data and the data version field value corresponding to the insertion data as the new data record, the deletion identifier field value corresponding to the new data record indicating non-deletion and the data version field value corresponding to the new data record indicating the update times, to obtain the LSM tree data insertion information.

[0119] In an optional implementation of the embodiments of the present application, the apparatus further includes:

[0120] The querying unit is configured to, when querying the data record in the LSM tree data table, take the data record identifier, the deletion identifier field value indicating non-deletion and the data version field value corresponding to the data record identifier and indicating the latest update times carried in the data record query request as the query condition, and query the data record in the LSM tree data table.

[0121] In an optional implementation of the embodiments of the present application, the apparatus further includes:

[0122] The adding unit is configured to add a timestamp column in the LSM tree data table, and the timestamp field value corresponding to the timestamp column is used to record the update time corresponding to the B+ tree data update information.

[0123] The second obtaining unit is configured to, after the B+ tree data table is updated, read the data record in the B+ tree data table, and obtain the first statistical data corresponding to each field column in the B+ tree data table and the reading time.

[0124] The third obtaining unit is configured to, after a preset time period from the reading time, take each data record identifier, the deletion identifier field value indicating non-deletion, the data version field value corresponding to the data record identifier and indicating the latest update times and the timestamp field value indicating a time earlier than the reading time as the reading condition, read the data record in the LSM tree data table, and obtain the second statistical data corresponding to the field columns in the LSM tree data table.

[0125] The prompting unit is configured to prompt that the data between the LSM tree data table and the B+ tree data table is inconsistent if the second statistical data is inconsistent with the first statistical data.

[0126] Through various embodiments provided by the embodiment, after the B+ tree data table of the B+ tree database is synchronized to the LSM tree data table of the LSM tree database, current B+ tree data update information of the B+ tree data table is obtained; wherein the LSM tree data table comprises a deletion identifier column and a data version column; on the basis of the current B+ tree data update information, the inserted data, the deletion identifier field value corresponding to the inserted data and the data version field value are configured to obtain LSM tree data insertion information; and the LSM tree data table is updated by using the LSM tree data insertion information. It can be seen that, by adding the deletion identifier column and the data version column in the LSM tree data table, when the current B+ tree data update information of the B+ tree data table is captured, the inserted data and the corresponding deletion identifier field value and data version field value are set, the current B+ tree data update information can be converted into LSM tree data insertion information, so that the LSM tree database performs high-performance data insertion operation based on the LSM tree data insertion information when data is updated, and the cost is reduced and the efficiency is improved when the data update in the B+ tree database is synchronized to the LSM tree database.

[0127] Figure 9 is a block diagram of a synchronization apparatus 900 for data update according to an example embodiment. The apparatus 900 can be a mobile phone, a computer, a digital broadcast terminal, a message communicator, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like, for example.

[0128] Referring to Figure 9 , the apparatus 900 can include one or more of the following components: a processing component 902, a memory 904, a power supply component 906, a multimedia component 908, an audio component 910, an input / output (I / O) interface 912, a sensor component 914, and a communication component 916.

[0129] The processing component 902 usually controls the overall operation of the apparatus 900, such as operations associated with displaying, making phone calls, data communications, camera operations and recording operations. The processing component 902 can include one or more processors 920 to execute instructions to complete all or part of the steps of the methods described above. In addition, the processing component 902 can include one or more modules to facilitate the interaction between the processing component 902 and other components. For example, the processing component 902 can include a multimedia module to facilitate the interaction between the multimedia component 908 and the processing component 902.

[0130] The memory 904 is configured to store various types of data to support the operation of the device 900. Examples of such data include instructions for any application or method operating on the device 900, contact data, phonebook data, messages, pictures, videos, and the like. The memory 904 can be implemented by any type of volatile or nonvolatile storage devices or a combination thereof such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0131] The power supply component 906 supplies electrical power for the various components of the device 900. The power supply component 906 can include a power supply management system, one or more power supplies, and other components associated with generating, managing and distributing electrical power for the device 900.

[0132] The multimedia component 908 includes a screen providing an output interface between the device 900 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swiping and gestures on the touch panel. The touch sensors can not only sense a boundary of a touching or swiping action, but also detect duration and pressure associated with the touching or swiping action. In some embodiments, the multimedia component 908 includes a front camera and / or a rear camera. The front and / or rear camera can receive external multimedia data when the device 900 is in an operation mode, such as a shooting mode or a video mode. Each of the front and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0133] The audio component 910 is configured to output and / or input audio signals. For example, the audio component 910 includes a microphone (MIC) that is configured to receive an external audio signal when the device 900 is in an operation mode, such as a calling mode, a recording mode and a voice recognition mode. The received audio signal can be further stored in the memory 904 or transmitted via the communication component 916. In some embodiments, the audio component 910 also includes a speaker for outputting audio signals.

[0134] The I / O interface 912 provides an interface between the processing component 902 and peripheral interface modules, which can be a keyboard, a click wheel, buttons, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button and a lock button.

[0135] The sensor component 914 includes one or more sensors for providing status assessments for various aspects of the device 900. For example, the sensor component 914 can detect an open / closed position of the device 900, relative positioning of components, such as a display and keypad of the device 900, changes in position of the device 900 or a component of the device 900, presence or absence of user contact with the device 900, changes in orientation of the device 900 or acceleration / deceleration, and temperature changes of the device 900. The sensor component 914 can include proximity sensor(s) configured to detect presence of objects in a proximity without any physical contact. The sensor component 914 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 914 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0136] The communication component 916 is configured to facilitate wired or wireless communication between the device 900 and another device. The device 900 can access a wireless network based on a communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 916 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 916 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technology.

[0137] In an exemplary embodiment, the device 900 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic components, for performing the above-described methods.

[0138] In an exemplary embodiment, a non-transitory computer-readable storage medium, such as the memory 904 including instructions stored therein, is also provided. The instructions may, for example, be executable by the processor 920 of the device 900 to perform the above-described methods. The non-transitory computer-readable storage medium may, for example, be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, and the like.

[0139] A non-transitory computer-readable storage medium, when instructions stored therein are executed by a processor of a mobile terminal, enables the mobile terminal to perform a method of synchronizing data updates, the method including:

[0140] After the B+ tree data table of the B+ tree database is synchronized to the log-structured merge (LSM) tree database to obtain an LSM tree data table, current B+ tree data update information of the B+ tree data table is acquired; the LSM tree data table includes a deletion identifier column and a data version column;

[0141] Based on the current B+ tree data update information, an LSM tree data insertion information is obtained by configuring insertion data, a deletion identifier field value and a data version field value corresponding to the insertion data.

[0142] Based on the LSM tree data insertion information, the LSM tree data table is updated.

[0143] Figure 10 Fig. 1 is a schematic diagram of a server according to an embodiment of the present application. The server 1000 can be significantly different due to different configurations or performances, and can include one or more central processing units (CPUs) 1022 (e.g., one or more processors) and a memory 1032, one or more storage media 1030 (e.g., one or more mass storage devices) storing application programs 1042 or data 1044. The memory 1032 and the storage media 1030 can be temporary or persistent storage. The programs stored in the storage media 1030 can include one or more modules (not shown in the figure), each of which can include a series of instruction operations in the server. Further, the central processing unit 1022 can be configured to communicate with the storage media 1030 and execute the series of instruction operations in the storage media 1030 on the server 1000.

[0144] The server 1000 can also include one or more power supplies 1026, one or more wired or wireless network interfaces 1050, one or more input / output interfaces 1058, one or more keyboards 1056, and / or one or more operating systems 1041, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0145] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part.

[0146] Those skilled in the art will further appreciate that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or any combination thereof. To clearly illustrate this interchangeability of hardware and software, various examples have been described herein in terms of their functionality, which has been described generally and symbolically in flow charts. Having thus described the functionality of the examples in terms of a process, it is appreciated that this functionality can be implemented by one or more types of electrical circuits or computer software, which are collectively referred to herein as a "circuit" that can carry out a variety of operations described herein. One of ordinary skill in the art can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

[0147] It is to be understood that the terms "including", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements recited, but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0148] The above description is only the preferred embodiment of the present application, and is not intended to limit the present application in any form. Although the present application has been disclosed as above with the preferred embodiment, it is not intended to limit the present application. Any person skilled in the art, without departing from the scope of the present application, can make many possible changes and modifications to the above-mentioned disclosed methods and technical contents, or modify equivalent embodiments. Therefore, any simple modification, equivalent change and modification made to the above-mentioned embodiments, without departing from the scope of the present application, in accordance with the technical essence of the present application, are still within the scope of protection of the present application.

Claims

1. A data update synchronization method, characterized in that, include: After synchronizing the B+ tree data table in the B+ tree database to the log-structured merged LSM tree database to obtain the LSM tree data table, the current B+ tree data update information of the B+ tree data table is obtained. The LSM tree data table includes a deletion identifier column and a data version column, and the current B+ tree data update information includes at least one of the current B+ tree data modification information, current B+ tree data deletion information, and current B+ tree data insertion information. Based on the current B+ tree data update information, the inserted data, the deletion identifier field value and the data version field value corresponding to the inserted data are configured to obtain LSM tree data insertion information. The deletion identifier field value is used to indicate deletion or non-deletion, and the data version field value indicates the number of updates. The LSM tree data table is updated based on the LSM tree data insertion information; The current B+ tree data update information includes the current B+ tree data modification information. Based on the current B+ tree data update information, the process of configuring the inserted data, the deletion identifier field value corresponding to the inserted data, and the data version field value to obtain LSM tree data insertion information includes: Obtain the old data record, new data record, and update count corresponding to the current B+ tree data modification information; The inserted data, the deletion identifier field value corresponding to the inserted data, and the data version field value are configured such that the old data record, the deletion identifier field value corresponding to the old data record indicates deletion, and the data version field value corresponding to the old data record indicates the update count; and the new data record, the deletion identifier field value corresponding to the new data record indicates non-deletion, and the data version field value corresponding to the new data record indicates the update count, thereby obtaining the LSM tree data insertion information.

2. The method according to claim 1, characterized in that, The current B+ tree data update information includes the current B+ tree data deletion information. Based on the current B+ tree data update information, the process of configuring the inserted data, the deletion identifier field value corresponding to the inserted data, and the data version field value to obtain LSM tree data insertion information includes: Obtain the old data record and update count corresponding to the current B+ tree data deletion information; Configure the inserted data, the deletion identifier field value and the data version field value corresponding to the inserted data as follows: the old data record, the deletion identifier field value corresponding to the old data record represents deletion, and the data version field value corresponding to the old data record represents the update count, thereby obtaining the LSM tree data insertion information.

3. The method according to claim 1, characterized in that, The current B+ tree data update information includes the current B+ tree data insertion information. The step of configuring the inserted data, the deletion identifier field value corresponding to the inserted data, and the data version field value based on the current B+ tree data update information to obtain LSM tree data insertion information includes: Obtain the new data record and update count corresponding to the current B+ tree data insertion information; Configure the inserted data, the deletion identifier field value and the data version field value corresponding to the inserted data, so that the new data record, the deletion identifier field value corresponding to the new data record indicates non-deletion, and the data version field value corresponding to the new data record indicates the update number, to obtain the LSM tree data insertion information.

4. The method according to claim 1, characterized in that, The method further includes: When querying data records in the LSM tree data table, the data records in the LSM tree data table are queried using the data record identifier carried in the data record query request, the deletion identifier field value indicating non-deletion, and the data version field value corresponding to the data record identifier indicating the latest update number.

5. The method according to claim 1, characterized in that, The method further includes: Add a timestamp column to the LSM tree data table. The timestamp field value corresponding to the timestamp column is used to record the update time corresponding to the B+ tree data update information. After the B+ tree data table is updated, read the data records in the B+ tree data table to obtain the first statistical data and reading time corresponding to each field column in the B+ tree data table; After a preset time interval from the reading time, the data records in the LSM tree data table are read using each data record identifier, the deletion identifier field value indicating non-deletion, the data version field value corresponding to the data record identifier indicating the latest update count, and the timestamp field value indicating a time earlier than the reading time as reading conditions, to obtain the second statistical data corresponding to each field column in the LSM tree data table; If the second statistical data is inconsistent with the first statistical data, it indicates that the data in the LSM tree data table and the B+ tree data table are inconsistent.

6. A data update synchronization device, characterized in that, include: The acquisition unit is used to acquire the current B+ tree data update information of the B+ tree data table after synchronizing the B+ tree data table of the B+ tree database to the log-structured merged LSM tree database to obtain the LSM tree data table. The LSM tree data table includes a deletion identifier column and a data version column, and the current B+ tree data update information includes at least one of the current B+ tree data modification information, current B+ tree data deletion information, and current B+ tree data insertion information. The first obtaining unit is configured to configure the inserted data, the deletion identifier field value and the data version field value corresponding to the inserted data based on the current B+ tree data update information, and obtain LSM tree data insertion information. The deletion identifier field value is used to indicate deletion or non-deletion, and the data version field value indicates the number of updates. The update unit is used to update the LSM tree data table based on the LSM tree data insertion information; The current B+ tree data update information includes the current B+ tree data modification information, and the first obtaining unit is used for: Obtain the old data record, new data record, and update count corresponding to the current B+ tree data modification information; The inserted data, the deletion identifier field value corresponding to the inserted data, and the data version field value are configured such that the old data record, the deletion identifier field value corresponding to the old data record indicates deletion, and the data version field value corresponding to the old data record indicates the update count; and the new data record, the deletion identifier field value corresponding to the new data record indicates non-deletion, and the data version field value corresponding to the new data record indicates the update count, thereby obtaining the LSM tree data insertion information.

7. The apparatus according to claim 6, characterized in that, The current B+ tree data update information includes the current B+ tree data deletion information, and the first obtaining unit is used for: Obtain the old data record and update count corresponding to the current B+ tree data deletion information; Configure the inserted data, the deletion identifier field value and the data version field value corresponding to the inserted data as follows: the old data record, the deletion identifier field value corresponding to the old data record represents deletion, and the data version field value corresponding to the old data record represents the update count, thereby obtaining the LSM tree data insertion information.

8. The apparatus according to claim 6, characterized in that, The device further includes: The query unit is used to query data records in the LSM tree data table by using the data record identifier carried in the data record query request, the deletion identifier field value indicating non-deletion, and the data version field value corresponding to the data record identifier indicating the latest update number as query conditions when querying data records in the LSM tree data table.

9. The apparatus according to claim 6, characterized in that, The device further includes: An add unit is used to add a timestamp column to the LSM tree data table. The timestamp field value corresponding to the timestamp column is used to record the update time corresponding to the B+ tree data update information. The second obtaining unit is used to read data records in the B+ tree data table after the data in the B+ tree data table is updated, and to obtain the first statistical data and reading time corresponding to each field column in the B+ tree data table; The third obtaining unit is used to read data records in the LSM tree data table after a preset time interval from the reading time, using each data record identifier, the deletion identifier field value indicating non-deletion, the data version field value corresponding to the data record identifier indicating the latest update count, and the timestamp field value indicating a time earlier than the reading time as reading conditions, to obtain the second statistical data corresponding to each field column in the LSM tree data table. The prompting unit is used to prompt that if the second statistical data is inconsistent with the first statistical data, the data between the LSM tree data table and the B+ tree data table is inconsistent.

10. The apparatus according to claim 6, characterized in that, The apparatus further includes the current B+ tree data update information, which includes the current B+ tree data insertion information, and the first obtaining unit is configured to: Obtain the new data record and update count corresponding to the current B+ tree data insertion information; Configure the inserted data, the deletion identifier field value and the data version field value corresponding to the inserted data, so that the new data record, the deletion identifier field value corresponding to the new data record indicates non-deletion, and the data version field value corresponding to the new data record indicates the update number, to obtain the LSM tree data insertion information.

11. A synchronization device for data updating, characterized in that, It includes a memory and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors to perform the data update synchronization method as described in any one of claims 1 to 5.

12. A machine-readable medium having instructions stored thereon that, when executed by one or more processors, cause a device to perform a synchronization method for data updates as described in any one of claims 1 to 5.

Images