Data synchronization method and apparatus, data storage system

Abstract

This specification provides a data synchronization method, apparatus, and data storage system. The data synchronization method includes: receiving a data synchronization request, wherein the data synchronization request includes a data identifier and a target time; obtaining data change information corresponding to the data identifier and current data from a source database, wherein the data change information includes the time of data change and the amount of change; filtering a target amount of change from the data change information based on the target time and the time of data change; determining data to be synchronized based on the target amount of change and the current data, and synchronizing the data to be synchronized to a target database. This allows for the acquisition of data to be synchronized corresponding to any target time, improving the efficiency of synchronizing data for different target times, i.e., improving synchronization efficiency; it avoids determining the data to be synchronized in the source database, reducing the data processing load of the source database, thereby improving the service efficiency of the source database.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of computer technology, and in particular to a data synchronization method. Background Technology

[0002] With the rapid development of computer and internet technologies, various industries have entered a period of rapid growth, generating an ever-increasing amount of data. Data synchronization is necessary for data management and usage. In data synchronization scenarios, it is often required to synchronize source data with different structures to a unified data storage medium, and simultaneously obtain data change details and the final state of the data. However, directly obtaining source data, especially with large datasets, increases the processing pressure on the source database, hindering its ability to provide stable service. Therefore, an effective solution is urgently needed to address these issues. Summary of the Invention

[0003] In view of this, embodiments of this specification provide a data synchronization method. One or more embodiments of this specification also relate to a data synchronization device, a data storage system, a computing device, a computer-readable storage medium, and a computer program, to address the technical deficiencies existing in the prior art.

[0004] According to a first aspect of the embodiments of this specification, a data synchronization method is provided, comprising:

[0005] Receive a data synchronization request, wherein the data synchronization request includes a data identifier and a target time;

[0006] Obtain the data change information and current data corresponding to the data identifier from the source database, wherein the data change information includes the time and amount of data change;

[0007] Based on the target time and the time of data change, the target change amount is selected from the data change information;

[0008] Based on the target change and the current data, determine the data to be synchronized and synchronize the data to be synchronized to the target database.

[0009] According to a second aspect of the embodiments of this specification, a data synchronization apparatus is provided, comprising:

[0010] The receiving module is configured to receive a data synchronization request, wherein the data synchronization request includes a data identifier and a target time;

[0011] The acquisition module is configured to acquire data change information and current data corresponding to the data identifier from the source database, wherein the data change information includes the time of data change and the amount of change;

[0012] The filtering module is configured to filter out the target change amount from the data change information based on the target time and the time of data change;

[0013] The synchronization module is configured to determine the data to be synchronized based on the target change and the current data, and to synchronize the data to be synchronized to the target database.

[0014] According to a third aspect of the embodiments of this specification, a data storage system is provided, comprising:

[0015] Data synchronization processor, source database, and target database;

[0016] The data synchronization processor is configured to receive a data synchronization request, wherein the data synchronization request includes a data identifier and a target time; obtain data change information corresponding to the data identifier and current data from the source database, wherein the data change information includes the data change time and the change amount; filter out the target change amount from the data change information according to the target time and the data change time; determine the data to be synchronized according to the target change amount and the current data, and send the data to be synchronized to the target database;

[0017] The source database is configured to send the data change information corresponding to the data identifier and the current data to the data synchronization processor;

[0018] The target database is configured to receive and store the data to be synchronized.

[0019] According to a fourth aspect of the embodiments of this specification, a computing device is provided, comprising:

[0020] Memory and processor;

[0021] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the above-described data synchronization method.

[0022] According to a fifth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the data synchronization method described above.

[0023] According to a sixth aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described data synchronization method.

[0024] This specification provides a data synchronization method that receives a data synchronization request, wherein the data synchronization request includes a data identifier and a target time; obtains data change information corresponding to the data identifier and current data from a source database, wherein the data change information includes the time of data change and the amount of change; filters out a target amount of change from the data change information based on the target time and the time of data change; determines the data to be synchronized based on the target amount of change and the current data, and synchronizes the data to be synchronized to the target database. By processing the data change information, the data to be synchronized corresponding to any target time can be obtained, thereby improving the efficiency of synchronizing data for different target times, i.e., improving synchronization efficiency; and by using data change information to determine the data to be synchronized, the method avoids determining the data to be synchronized in the source database, reducing the data processing volume of the source database, thereby improving the service efficiency of the source database. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating a data synchronization system according to one embodiment of this specification.

[0026] Figure 2 This is a schematic diagram of a system architecture applied in a data integration scenario, provided by one embodiment of this specification;

[0027] Figure 3 This is a flowchart illustrating a data synchronization method provided in one embodiment of this specification;

[0028] Figure 4 This is a flowchart of another data synchronization method provided in one embodiment of this specification;

[0029] Figure 5 This is a system block diagram of a data synchronization method provided in one embodiment of this specification;

[0030] Figure 6 This is a flowchart illustrating the processing procedure of a data synchronization method provided in one embodiment of this specification;

[0031] Figure 7 This is a schematic diagram of the structure of a data synchronization device provided in one embodiment of this specification;

[0032] Figure 8 This is a schematic diagram of the structure of a data storage system provided in one embodiment of this specification;

[0033] Figure 9 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0034] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0035] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0036] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0037] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0038] Data synchronization refers to migrating data from one storage space to another.

[0039] Data snapshot: refers to the final state of data at a certain point in time.

[0040] Big data processing: Data processing at the TB (1000GB) level.

[0041] Storage medium: refers to the medium through which data is stored.

[0042] Heterogeneous data sources: Data storage media with different structures, such as relational database management systems (MySQL), remote dictionary service storage systems (Redis), and distributed storage systems (HBase).

[0043] This specification provides a data synchronization method, and also relates to a data synchronization device, a data storage system, a computing device, and a computer-readable storage medium, which will be described in detail in the following embodiments.

[0044] With the rapid development of computer and internet technologies, various industries have also entered a period of rapid growth, resulting in an ever-increasing volume of data. Due to the needs of data management and usage, data synchronization is required. In data synchronization scenarios, it is often necessary to synchronize source data with different structures to a unified data storage medium, and simultaneously obtain data change details and the final state of the data. However, directly obtaining the source data and writing it to a unified storage medium, especially with large data volumes, will increase the processing pressure on the source database, hindering its ability to provide stable service.

[0045] Furthermore, during the big data synchronization process, the data volume is increasing, the data structure is becoming more complex, and projects are placing increasing demands on big data. Due to regulatory and data security requirements, it is necessary to obtain the data change paths at various points in time. However, synchronizing data according to change records requires the source database to store not only data snapshots but also data change records, consuming more storage space and resulting in high data storage costs. On the other hand, synchronizing data on a scheduled basis leads to low data transmission efficiency.

[0046] Therefore, to better meet project requirements, this specification provides a data synchronization method. The method involves receiving a data synchronization request, whereby the request includes a data identifier and a target time; retrieving data change information corresponding to the data identifier and current data from the source database, whereby the data change information includes the time of data change and the amount of change; filtering out the target amount of change from the data change information based on the target time and the time of data change; determining the data to be synchronized based on the target amount of change and the current data; and synchronizing the data to be synchronized to the target database. By processing the data change information, the data to be synchronized corresponding to any target time can be obtained, thereby improving the efficiency of synchronizing data for different target times, i.e., improving synchronization efficiency. Furthermore, using data change information to determine the data to be synchronized avoids determining the data to be synchronized in the source database, reducing the data processing load of the source database and thus improving the service efficiency of the source database.

[0047] See Figure 1 , Figure 1 A data synchronization flowchart of a data synchronization system provided in one embodiment of this specification is shown. Figure 1 As shown, the data synchronization system includes a server and a client.

[0048] The client is configured to send a data synchronization request to the server, wherein the data synchronization request includes a data identifier and a target time;

[0049] The server is configured to receive data synchronization requests; obtain data change information and current data corresponding to the data identifier from the source database, wherein the data change information includes the data change time and the change amount; filter out the target change amount from the data change information according to the target time and the data change time; determine the data to be synchronized according to the target change amount and the current data, and synchronize the data to be synchronized to the target database.

[0050] In practical applications, the server is further configured to feed back the processing results of synchronizing the data to be synchronized to the client.

[0051] This specification provides a data synchronization method that receives a data synchronization request, wherein the data synchronization request includes a data identifier and a target time; retrieves data change information corresponding to the data identifier and current data from a source database, wherein the data change information includes the time of data change and the amount of change; filters out a target amount of change from the data change information based on the target time and the time of data change; determines the data to be synchronized based on the target amount of change and the current data, and synchronizes the data to be synchronized to the target database. By processing the data change information, the data to be synchronized corresponding to any target time can be obtained, thereby improving the efficiency of synchronizing data for different target times, i.e., improving synchronization efficiency; and by using data change information to determine the data to be synchronized, the method avoids determining the data to be synchronized in the source database, reducing the data processing volume of the source database, thereby improving the service efficiency of the source database.

[0052] The embodiments provided in this specification can be applied to various data reading scenarios, such as data migration, data retrieval, data usage, and data integration. In a practical application, it is particularly suitable for data synchronization scenarios within a certain period of time.

[0053] See Figure 2 , Figure 2 This specification illustrates a system architecture diagram for a data integration scenario based on an embodiment of the present invention. The system may include a server 202 and multiple clients 204. The multiple clients 204 can establish communication connections through the server 202. In the data integration scenario, the server 202 is used to provide data synchronization services among the multiple clients 204. Each client 204 can act as a sender or receiver, and real-time processing is achieved through the server 202.

[0054] Users can interact with server 202 through client 204 to receive data sent by other clients 204, or send data to other clients 204, etc. In a data integration scenario, a user can send a data synchronization request to server 202 through client 204. Based on the data synchronization request, server 204 generates a data synchronization processing result and pushes the processing result to other clients that have established communication.

[0055] In this setup, client 204 and server 202 establish a connection via a network. The network provides the medium for the communication link between the client and the server. The network can include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0056] The client 204 can be a browser, an app (application), a web application such as an H5 (HyperText Markup Language 5) application, a lightweight application (also known as a mini-program), or a cloud application. The client 204 can be developed based on the corresponding service SDK (Software Development Kit) provided by the server, such as based on the RTC (Real-time Communications) SDK. The client 204 can be deployed on electronic devices, requiring the device to run or certain apps on the device to function. Electronic devices may have displays and support information browsing, such as personal mobile terminals like smartphones, tablets, and personal computers. Various other types of applications can also be configured on electronic devices, such as human-computer interaction applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, and social media platforms.

[0057] Server 202 may include servers that provide various services, such as servers that provide data synchronization services for multiple clients, servers that support the models used on the clients for background training, and servers that process data synchronization requests sent by clients.

[0058] It should be noted that a 202 server-side configuration can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. The server can also be a server in a distributed system, or a server integrated with blockchain. Furthermore, the server can be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0059] It is worth noting that the data synchronization method provided in the embodiments of this specification is generally executed by the server. However, in other embodiments of this specification, the client may also have similar functions to the server, thereby executing the data synchronization method provided in the embodiments of this specification. In other embodiments, the data synchronization method provided in the embodiments of this specification may also be executed jointly by the client and the server.

[0060] See Figure 3 , Figure 3 A flowchart of a data synchronization method provided in one embodiment of this specification is shown, which specifically includes the following steps.

[0061] Step 302: Receive a data synchronization request, wherein the data synchronization request includes a data identifier and a target time.

[0062] Specifically, a data synchronization request is used to instruct data synchronization processing and can be a data synchronization command; a data identifier refers to the identifier of the data to be synchronized, which can be a data type, data name, etc., such as the number of registered users, the number of visitors, the number of purchases, the purchase amount, etc.; the target time represents the data to be synchronized at a certain point in time or over a period of time. For example, if the data to be synchronized is between June 1, 2022 and June 30, 2022, then June 1, 2022 to June 30, 2022 is the target time.

[0063] In practical applications, the client can generate a data synchronization request based on the user-input data identifier and target time, and send it to the executing entity. The user can input the data identifier and target time on the data synchronization interface. After confirmation, the executing entity receives the data synchronization request carrying the data identifier and target time. For example, if a user sets the data identifier and target time through the client's data synchronization interface, and then inputs "8:00-20:00" and "Number of Purchases" and clicks the "Confirm" or "Save" button, the executing entity receives a data synchronization request carrying "8:00-20:00" and "Number of Purchases". The client can also extract the data identifier and target time to generate a data synchronization request based on the user's needs, and send it to the executing entity. For example, if a user calculates their daily webpage views in July on the client, the executing entity will receive a data synchronization request carrying "July 1st to July 31st" and "Webpage Views". Furthermore, users can also generate data synchronization requests by writing code, which is not limited in this specification.

[0064] Step 304: Obtain the data change information and current data corresponding to the data identifier from the source database, wherein the data change information includes the time of data change and the amount of change.

[0065] Upon receiving a data synchronization request, the system further retrieves the content corresponding to the data identifier from the source database based on the data identifier in the data synchronization request.

[0066] Specifically, the source database refers to a self-sufficient database that directly provides original data or specific data, that is, a database that provides data to be synchronized; the data change moment refers to the point in time when the data changes; the change amount refers to the increment of data, which is a general term for the method and specific value of data changes within a period of time; the data change information refers to the change amount and the data change moment, that is, the data details. For example, if the number of viewers in the live broadcast room increases by 20 at 10:20, then the data change information is 10:20 (data change moment) and +20 (change amount).

[0067] In practical applications, upon receiving a data synchronization request, the system responds by parsing the request to obtain the data identifier and target time. Then, it searches the source database for the data change information and current data corresponding to the data identifier. The data identifier can be matched with each data change information and each current data in the source database. The successfully matched data change information and current data are the data change information and current data corresponding to the data identifier.

[0068] For example, there are four data identifiers: number of registered users, number of visitors, number of purchasers, and purchase amount. The source database stores data change information A and current data a for the number of registered users, data change information B and current data b for the number of visitors, data change information C and current data c for the number of purchasers, and data change information D and current data d for the purchase amount. Assuming the data synchronization request includes the data identifier for the number of purchasers, then data change information C and current data c are retrieved from the source database.

[0069] In one or more optional embodiments of this specification, there may be only one source database. In this case, the executing entity can directly obtain the data change information and current data corresponding to the data identifier from the source database.

[0070] In one or more optional embodiments of this specification, there may be multiple source databases. When performing data synchronization, to ensure accuracy and efficiency, the data synchronization request will indicate the source database that needs to be synchronized. That is, the data synchronization request also includes a source database identifier; correspondingly, the data change information and current data corresponding to the data identifier are obtained from the source database. The specific implementation process can be as follows:

[0071] Retrieve the data change information and current data corresponding to the data identifier from the source database corresponding to the source database identifier.

[0072] Specifically, the source database identifier refers to the identifier that represents the source database, which can be the name, label, address, etc. of the source database.

[0073] In practical applications, when the data synchronization request also carries a source database identifier, the source database requiring data synchronization is matched from multiple source databases based on the source database identifier, i.e., the source database corresponding to the source database identifier. Then, the data change information and current data corresponding to the data identifier are obtained from this source database. The process of obtaining the data change information and current data can refer to any of the methods mentioned above, and will not be elaborated here.

[0074] For example, there are 5 source databases: databases s1-s5, where the source database identifiers for databases s1-s5 are s1-s5 respectively. When a data synchronization request carries a source database identifier of s3, the data change information and current data corresponding to that identifier are retrieved from database s3.

[0075] Thus, when the data synchronization request carries the source database identifier, the data change information and current data are obtained from the source database corresponding to the source database identifier. This avoids the execution entity having to search for the data change information and current data corresponding to the data identifier in each source database. This not only reduces the amount of data processing and improves the data synchronization efficiency, but also avoids errors in the obtained data change information and current data due to errors in the source (source database), thus ensuring the correctness of the data change information and current data.

[0076] It's important to note that the source database only stores detailed data (data change information) and current data. That is, when data changes, the source database records the change information and updates the current data based on the transformed data. By processing the stored detailed data, the current data can be used according to any time slice. Furthermore, storing only one copy of the detailed data in the source database avoids duplicate storage, reducing the data storage size and thus improving the service efficiency of the source database.

[0077] Step 306: Based on the target time and the time of data change, filter out the target change amount from the data change information.

[0078] Based on the data change information and current data, the target change amount is further determined according to the target time and the time of data change in the data change information.

[0079] Specifically, the target change refers to the amount of change used to determine the data to be synchronized, such as the change from the target time to the current time.

[0080] In one or more optional embodiments of this specification, the data change time and the target change amount appear in pairs. Furthermore, a time filtering range can be determined based on the target time and the current time, whereby the time filtering range is used to filter data change times. When the target time is a point in time, the time filtering range is defined with the target time as the start time and the current time as the end time; when the target time is a time period, the time filtering range is defined with the start time of the target time as the start time and the current time as the end time. Then, the change amount corresponding to the data change time within the time filtering range in the data change information is determined as the target change amount. This improves the efficiency and completeness of obtaining the target change amount.

[0081] For example, if the target time is 8:15 and the current time is 8:30, then the time filtering range is [8:15-8:30]. Suppose there are three data change times: 7:20, 8:20, and 8:23, and 7:20, 8:20, and 8:23 correspond to +5, -6, and +20 respectively. Then 8:20 and 8:23 are the target data change times, and -6 and +20 are the target change amounts.

[0082] For example, if the target time is the time period [8:05-8:15], and the current time is 8:30, then the time filtering range is [8:05-8:30]. Suppose there are four data change times: 7:20, 8:05, 8:20, and 8:23. These times correspond to +5, +9, -6, and +20 respectively. Therefore, 8:05, 8:20, and 8:23 are the target data change times, and +9, -6, and +20 are the target change amounts.

[0083] In one or more optional embodiments of this specification, there is a correspondence between the time of data change and the amount of change. In this case, the target time and the time of data change can be compared and matched, and the amount of change corresponding to the time of data change within the target time and after the target time can be determined as the target amount of change. That is, based on the target time and the time of data change, the target amount of change is filtered from the data change information. The specific implementation process can be as follows:

[0084] The target time is matched with each data change time;

[0085] The change corresponding to the moment when the matched data changes is determined as the target change.

[0086] In practical applications, when the target time is a specific point in time, the changes in each data point are matched with the target time to check whether each data change point is at or after the target time. If so, the change amount corresponding to the data change point that meets the condition is determined as the target change amount. When the target time is a time period, the changes in each data point are matched with the target time to check whether each data change point belongs to or is after the target time. If so, the change amount corresponding to the data change point that meets the condition is determined as the target change amount.

[0087] For example, if the target time is 8:15, and there are three data change points, namely 7:20, 8:15, and 8:23, and 7:20, 8:15, and 8:23 correspond to +5, -6, and +20 respectively, then 8:15 is the target time, and 8:23 is after the target time. Therefore, the changes of -6 and +20 corresponding to 8:15 and 8:23 are determined as the target changes.

[0088] For example, if the target time is a time period [8:05-8:15], and there are 3 data change times, namely 7:20, 8:10 and 8:20, and 7:20, 8:10 and 8:20 correspond to +5, +9 and -6 respectively, it can be seen that 8:10 belongs to the target time and 8:20 is after the target time. Therefore, the changes of +9 and -6 corresponding to 8:10 and 8:20 are determined as the target changes.

[0089] Step 308: Based on the target change amount and the current data, determine the data to be synchronized and synchronize the data to be synchronized to the target database.

[0090] Based on the target time and the data change time in the data change information, and after determining the target change amount, further, based on the target change amount and the current data, determine and synchronize the data to be synchronized.

[0091] Specifically, the data to be synchronized refers to the data that needs to be stored, that is, the data to be migrated; the target database refers to the database that stores the data to be synchronized, that is, the storage space for data migration.

[0092] In practical applications, the data to be synchronized can be calculated and integrated based on the current data and the target changes. Then, the data to be synchronized is stored in the database or synchronized to the target database, thus completing the data synchronization.

[0093] In one or more optional embodiments of this specification, to ensure the accuracy of the data to be synchronized, the reverse change amount can be determined first, and then the data to be synchronized can be determined based on the reverse change amount and the current data. That is, the specific implementation process of determining the data to be synchronized based on the target change amount and the current data can be as follows:

[0094] A reverse change is determined based on the target change, wherein the reverse change is opposite in direction and the same in degree as the target change;

[0095] The current data is restored based on the inverse change amount to obtain the data to be synchronized.

[0096] Specifically, the direction of change refers to the direction of change, such as data increase or decrease; the degree of change refers to the specific numerical value of the change; and the reverse change refers to the change that is opposite to the target change.

[0097] In practical applications, the data can be processed based on the target change, for example, by finding its opposite value to obtain the inverse change. Then, based on this inverse change, the current data can be reconstructed to obtain the data to be synchronized. In this way, reconstructing the current data based on the inverse change can improve the efficiency and accuracy of acquiring the data to be synchronized.

[0098] For example, if the target change is -5, then the reverse change is +5; if the target change is ×10, then the reverse change is ÷10.

[0099] In one or more optional embodiments of this specification, if there is one target change, there is also one reverse change. The current data is then restored based on the reverse change to obtain the data to be synchronized. For example, if the reverse change is +6 and the current data is 20, then the data to be synchronized is 20 + 6 = 26.

[0100] In one or more optional embodiments of this specification, there may be multiple target changes, that is, multiple reverse changes. In this case, it is necessary to gradually restore the current data according to the order of data change times from back to front, so as to obtain the data to be synchronized. That is, when there are at least two reverse changes, and there is a correspondence between the reverse changes and the data change times, the specific implementation process of restoring the current data according to the reverse changes to obtain the data to be synchronized can be as follows:

[0101] According to the order of the data change times from back to front, starting from the reverse change amount corresponding to the first data change time, the current data is restored based on the current reverse change amount to obtain the initial data to be synchronized. The initial data to be synchronized is used as the current data, and the step of restoring the current data based on the current reverse change amount is continued until the reverse change amount corresponding to the last data change time is reached.

[0102] The data to be synchronized is determined based on the initial data to be synchronized.

[0103] Specifically, the initial data to be synchronized refers to the data initially restored based on the current data and the reverse change.

[0104] In practical applications, we can start with the inverse change value corresponding to the last data change at the last time point. Based on this inverse change value, we can reconstruct the current data to obtain the initial data to be synchronized. Then, using this initial data to be synchronized as the current data, we reconstruct the current data based on the inverse change value corresponding to the second-to-last data change at the last time point, and so on, until we reconstruct the current data based on the inverse change value corresponding to the earliest data change at the last time point, thus obtaining the initial data to be synchronized. Finally, based on these initial data to be synchronized, we determine the data to be synchronized.

[0105] For example, there are three inverse change quantities: -5, +6, and ÷2, with the corresponding data change times being 7:00, 9:00, and 8:00, respectively. Assuming the current data is 10, first, 10 is restored using +6 to obtain the initial data to be synchronized, 16; then, 16 is restored using ÷2 to obtain the initial data to be synchronized, 8; next, 8 is restored using -5 to obtain the data to be synchronized, 3; finally, based on 3, 8, and 16, the data to be synchronized is determined.

[0106] Optionally, the process of determining the data to be synchronized based on the obtained initial data to be synchronized can be implemented as follows:

[0107] Based on the target time and the corresponding data change time of each initial data to be synchronized, the initial data to be synchronized is filtered to obtain the data to be synchronized.

[0108] Specifically, the data change time corresponding to the initial data to be synchronized is the data change time corresponding to the reverse change amount used to obtain the initial data to be synchronized. Continuing with the previous example, the data change time corresponding to the data to be synchronized 3 is 7:00, the data change time corresponding to the data to be synchronized 8 is 8:00, and the data change time corresponding to the data to be synchronized 16 is 9:00.

[0109] In practical applications, since the data changes corresponding to the target change may not all occur within the time specified in the target time, some of the initial data to be synchronized may not be the final data to be synchronized. Therefore, it is necessary to filter the data to be synchronized based on the target time and the data change times corresponding to each data point, selecting the initial data whose data change times belong to the target time as the data to be synchronized. This can effectively improve the accuracy of the data to be synchronized, thereby improving data synchronization efficiency.

[0110] Continuing with the previous example, if the target time is a specific point in time, assuming the target time is 7:30, the initial data to be synchronized at 7:00 is 3, and the initial data to be synchronized at 8:00 is 8. Therefore, within the time period [7:30, 8:00), the data is 3, and thus 3 is the data to be synchronized. If the target time is a specific end in time, assuming the target time is [7:30, 8:30], the initial data to be synchronized at 7:00 is 3, the initial data to be synchronized at 8:00 is 8, and the initial data to be synchronized at 9:00 is 16. Therefore, within the time period [7:30, 8:00), the data is 3, and within the time period [8:00, 8:30], the data is 8. Thus, 3 and 8 are the data to be synchronized.

[0111] In one or more optional embodiments of this specification, after the data to be synchronized is determined, the data to be synchronized can be directly stored in the target database; alternatively, the data to be synchronized can be stored in the target database in the order of synchronization time, which can improve the storage rate.

[0112] In one or more optional embodiments of this specification, to facilitate the use and retrieval efficiency of the data to be synchronized, the data to be synchronized can be partitioned and stored according to the group identifier carried by each piece of data in the data to be synchronized. That is, when each piece of data in the data to be synchronized carries a group identifier, the specific implementation process of synchronizing the data to be synchronized to the target database can be as follows:

[0113] Based on the group identifier, the data to be synchronized is divided into at least one group of data;

[0114] Based on the group identifier corresponding to each group of data, each group of data is synchronized to the storage area corresponding to the group identifier in the target database.

[0115] Specifically, the group identifier can be the time period corresponding to the data, the source database identifier, etc.

[0116] In practical applications, data in the data to be synchronized can be grouped according to a group identifier: when the group identifier is a time period, data within the same time period is grouped together. The same time period can be the same day, the same hour, the same week, the same month, etc., which is not limited in this specification; then, each group of data is stored in the storage area corresponding to its corresponding time period in the target database. When the group identifier is a source database identifier, data corresponding to the same source database identifier is grouped together; further, each group of data is stored in the storage area corresponding to its corresponding source database identifier in the target database. Alternatively, a hash algorithm can be used to group data in the data to be synchronized.

[0117] For example, if the group identifier is a time period, and there are 5 data items to be synchronized: the first data item corresponds to time period 1-6, the second data item to time period 7-12, the third data item to time period 1-6, the fourth data item to time period 1-6, and the fifth data item to time period 7-12, then the first, third, and fourth data items are grouped into the first group, with the corresponding group identifier 1-6; the second and fifth data items are grouped into the second group, with the corresponding group identifier 7-12. The first group of data is synchronized to the storage area corresponding to 1-6 in the target database; the second group of data is synchronized to the storage area corresponding to 7-12 in the target database.

[0118] For example, if the group identifier is the source database identifier, and there are 5 data items to be synchronized, with the first data item corresponding to the source database identifier L1, the second data item corresponding to the time period L2, the third data item to the time period L3, the fourth data item to the time period L2, and the fifth data item to the time period L1, then the first and fifth data items are grouped into the first group, with the corresponding group identifier L1; the second and fourth data items are grouped into the second group, with the corresponding group identifier L2; and the third data item is grouped into the third group, with the corresponding group identifier L3. The first group of data is synchronized to the storage area corresponding to L1 in the target database; the second group of data is synchronized to the storage area corresponding to L2 in the target database; and the third group of data is synchronized to the storage area corresponding to L3 in the target database.

[0119] In one or more optional embodiments of this specification, to further improve data synchronization efficiency, the storage area is marked with a group label. In this case, the group identifier of each group of data can be compared with the group label to determine the target storage area for each group of data, and each group of data can be stored in its corresponding target storage area. That is, the specific implementation process of synchronizing each group of data to the storage area corresponding to the group identifier in the target database based on the group identifier can be as follows:

[0120] For each set of data, the group identifier corresponding to that set of data is matched with the group label of each storage area in the target database;

[0121] The successfully matched storage area is taken as the target storage area, and the group of data is stored in the target storage area.

[0122] Specifically, a group label refers to a label for a storage area. It can be the source database identifier of the data that the storage area can store, or it can be the time period in which the data that the storage area can store is located. Group labels and group identifiers belong to the same category. If the group identifier is the source database identifier, then the group label is also the source database identifier; if the group identifier is time, then the group label is also a time period.

[0123] In practical applications, for each set of data, the group identifier used for that set of data is matched with the group label of each storage area. The storage area that matches successfully is determined as the target storage area, and the set of data is stored in the target storage area.

[0124] For example, given a group identifier M1 for the first set of data, M2 for the second set, and L3 for the third set; and five target storage areas: the first storage area is labeled M3, the second M4, the third M2, the fourth M5, and the fifth M1. Then, the first set of data will be stored in the fifth storage area, the second in the third, and the third in the first.

[0125] In addition, if the group identifier of a certain group of data does not match the group labels of each storage area, the group identifier can be used as the group label to create a new storage area in the target database and the group of data can be stored in the newly created storage area.

[0126] In one or more optional embodiments of this specification, the source database and the target database can be heterogeneous data sources, meaning that the data storage structures of the source database and the target database are different. To ensure the effectiveness of data synchronization, the data to be synchronized can be formatted before synchronization. Specifically, the process of synchronizing the data to be synchronized to the target database can be as follows:

[0127] The data to be synchronized is formatted.

[0128] The formatted data to be synchronized is then synchronized to the target database.

[0129] Specifically, formatting refers to the process of processing data according to a certain preset format.

[0130] In practical applications, the data to be synchronized can first be formatted according to the data structure of the target database to ensure consistency. Then, the formatted data can be synchronized to the target database. The synchronization process can follow any of the methods described above for synchronizing data to the target database, and will not be elaborated further here. Formatting the data improves synchronization efficiency. Furthermore, after synchronization, if the synchronized data is used in the target database, its readability and usability are guaranteed.

[0131] For example, if the target database is a Redis database with a key-value structure, the data to be synchronized will be formatted according to the key-value structure, and then the key-value data to be synchronized will be synchronized to the target database.

[0132] In one or more optional embodiments of this specification, there may be only one target database, in which case the executing entity can directly synchronize the data to be synchronized to the target database.

[0133] In one or more optional embodiments of this specification, there may be multiple target databases. When performing data synchronization, to ensure accuracy and efficiency, the data synchronization request will indicate the target database to be synchronized. That is, the data synchronization request also carries a target database identifier. Accordingly, the specific implementation process of synchronizing the data to be synchronized to the target database can be as follows:

[0134] The data to be synchronized is synchronized to the target database corresponding to the target database identifier.

[0135] Specifically, the source database identifier refers to the identifier that represents the source database, which can be the name, label, address, etc. of the source database.

[0136] In practical applications, when the data synchronization request also carries a target database identifier, the target database to be synchronized is matched from multiple target databases based on the target database identifier, i.e., the target database corresponding to the target database identifier. The data to be synchronized is then synchronized to the target database corresponding to the target database identifier. The process of synchronizing to the target database can refer to any of the methods described above, and will not be elaborated further here.

[0137] For example, there are three target databases: databases n1-n3, where the target database identifiers for databases n1-n3 are n1-n3 respectively. If the data synchronization request includes a target database identifier of n2, the data to be synchronized will be synchronized to target database n2.

[0138] Thus, when the data synchronization request carries a target database identifier, the data to be synchronized is synchronized to the target database corresponding to the target database identifier, avoiding the execution entity synchronizing the data to be synchronized to the wrong target database and ensuring the correctness of data synchronization.

[0139] See Figure 4 , Figure 4This is a flowchart illustrating another data synchronization method provided in one embodiment of this specification. First, data is generated: data is generated during project startup, project updates, etc. Then, the data is entered into a source database: data change information (time and amount of change) is stored, and updated data overwrites the current data; for example, data is entered into an RDB database. Next, data is parsed, i.e., upon receiving a data synchronization request, the data change information corresponding to the data identifier and the current data are obtained from the source database, and processed accordingly to obtain the data to be synchronized. Then, data is stored, i.e., the data to be synchronized is synchronized to the target database. Finally, data processing is performed, i.e., the stored data to be synchronized is retrieved from the target database and put into use.

[0140] See Figure 5 , Figure 5 This is a system block diagram of a data synchronization method provided in one embodiment of this specification: it includes a heterogeneous source database, a data synchronization processor, and a target database. The heterogeneous source database includes RDB, HBase, and Redis, meaning it includes multiple different source databases. The data synchronization processor can be a data synchronization component. The target database is used to store data and contains multiple storage areas. When the data synchronization processor receives a data synchronization request, it obtains data change information and current data from the source database through a Reader, and parses the data change information and current data to obtain the data to be synchronized. Then, it performs hash processing on the data to be synchronized, i.e., grouping, dividing the data into multiple groups (e.g., group 1, group 2, and group 3). The data synchronization processor then stores the data in each group into the corresponding storage area in the target database. Furthermore, the data synchronization processor can respond to multiple data synchronization requests simultaneously.

[0141] This specification provides a data synchronization method that receives a data synchronization request, wherein the data synchronization request includes a data identifier and a target time; obtains data change information and current data corresponding to the data identifier from a source database, wherein the data change information includes the time of data change and the amount of change; filters out a target amount of change from the data change information based on the target time and the time of data change; determines the data to be synchronized based on the target amount of change and the current data, and synchronizes the data to be synchronized to the target database. By processing the data change information, the data to be synchronized corresponding to any target time can be obtained, thereby improving the efficiency of synchronizing data for different target times, i.e., improving synchronization efficiency; and using data change information to determine the data to be synchronized avoids determining the data to be synchronized in the source database, reducing the data processing load of the source database, thereby improving the service efficiency of the source database. In addition, formatting the data to be synchronized before synchronization allows the target database to uniformly store data of heterogeneous data elements, eliminating the need to set up corresponding target databases for each heterogeneous source database, reducing costs. Data change information and current data from heterogeneous data sources can be obtained at any time, i.e., data change records and data snapshots.

[0142] The following is in conjunction with the appendix Figure 6 Taking the application of the data synchronization method provided in this specification in a real-world scenario as an example, the data synchronization method will be further explained. Among other things, Figure 6 The present specification illustrates a flowchart of a data synchronization method according to an embodiment, which includes the following steps.

[0143] Step 602: Receive a data synchronization request, wherein the data synchronization request includes a source database identifier, a target database identifier, a data identifier, and a target time.

[0144] Step 604: Obtain the data change information and current data corresponding to the data identifier from the source database corresponding to the source database identifier. The data change information includes the time and amount of data change.

[0145] Step 606: Based on the target time and the time of data change, filter out the target change amount from the data change information.

[0146] Optionally, there is a correspondence between the timing of data changes and the amount of change;

[0147] Based on the target time and the time of data change, the target change amount is filtered out from the data change information, including:

[0148] Match the target time with each data change time;

[0149] The change corresponding to the moment when the matched data changes is determined as the target change.

[0150] Step 608: Determine the reverse change amount based on the target change amount, wherein the reverse change amount changes in the opposite direction to the target change amount and changes to the same degree.

[0151] Step 610: Following the order of data change times from back to front, starting from the reverse change amount corresponding to the first data change time, restore the current data based on the current reverse change amount to obtain the initial data to be synchronized. Use the initial data to be synchronized as the current data, and continue to execute the step of restoring the current data based on the current reverse change amount until the reverse change amount corresponding to the last data change time. There is a corresponding relationship between the reverse change amount and the data change time.

[0152] Step 612: Determine the data to be synchronized based on the obtained initial data to be synchronized.

[0153] Step 614: Divide the data to be synchronized into at least one group based on the group identifier carried by each data in the data to be synchronized.

[0154] Step 616: For each set of data, match the group identifier corresponding to the set of data with the group label of each storage area in the target database corresponding to the target database identifier.

[0155] Step 618: Use the successfully matched storage area as the target storage area, and format the data and store it in the target storage area.

[0156] This specification provides a data synchronization method that, by processing data change information, can obtain the data to be synchronized corresponding to any target time, thereby improving the efficiency of synchronizing data for different target times, i.e., improving synchronization efficiency; and by using data change information to determine the data to be synchronized, it avoids determining the data to be synchronized in the source database, reducing the data processing volume of the source database, thereby improving the service efficiency of the source database.

[0157] Corresponding to the above method embodiments, this specification also provides embodiments of a data synchronization device. Figure 7 A schematic diagram of a data synchronization device according to one embodiment of this specification is shown. Figure 7 As shown, the device includes:

[0158] The receiving module 702 is configured to receive a data synchronization request, wherein the data synchronization request includes a data identifier and a target time;

[0159] The acquisition module 704 is configured to acquire data change information and current data corresponding to the data identifier from the source database, wherein the data change information includes the time of data change and the amount of change;

[0160] The filtering module 706 is configured to filter out the target change amount from the data change information based on the target time and the data change time.

[0161] The synchronization module 708 is configured to determine the data to be synchronized based on the target change and the current data, and synchronize the data to be synchronized to the target database.

[0162] Optionally, the data synchronization request may also include a source database identifier;

[0163] The acquisition module 704 is further configured to:

[0164] Retrieve the data change information and current data corresponding to the data identifier from the source database corresponding to the source database identifier.

[0165] Optionally, there is a corresponding relationship between the time of data change and the amount of change;

[0166] The filtering module 706 is further configured to:

[0167] The target time is matched with each data change time;

[0168] The change corresponding to the moment when the matched data changes is determined as the target change.

[0169] Optionally, the synchronization module 708 is further configured to:

[0170] A reverse change is determined based on the target change, wherein the reverse change is opposite in direction and the same in degree as the target change;

[0171] The current data is restored based on the inverse change amount to obtain the data to be synchronized.

[0172] Optionally, the number of the reverse change amounts is at least two, and the reverse change amounts correspond to the time of the data change;

[0173] The synchronization module 708 is also configured to:

[0174] According to the order of the data change times from back to front, starting from the reverse change amount corresponding to the first data change time, the current data is restored based on the current reverse change amount to obtain the initial data to be synchronized. The initial data to be synchronized is used as the current data, and the step of restoring the current data based on the current reverse change amount is continued until the reverse change amount corresponding to the last data change time is reached.

[0175] The data to be synchronized is determined based on the initial data to be synchronized.

[0176] Optionally, each piece of data in the data to be synchronized carries a group identifier;

[0177] The synchronization module 708 is also configured to:

[0178] Based on the group identifier, the data to be synchronized is divided into at least one group of data;

[0179] Based on the group identifier corresponding to each group of data, each group of data is synchronized to the storage area corresponding to the group identifier in the target database.

[0180] Optionally, the storage area is labeled with a grouping tag;

[0181] The synchronization module 708 is also configured to:

[0182] For each set of data, the group identifier corresponding to that set of data is matched with the group label of each storage area in the target database;

[0183] The successfully matched storage area is taken as the target storage area, and the group of data is stored in the target storage area.

[0184] Optionally, the synchronization module 708 is further configured to:

[0185] The data to be synchronized is formatted.

[0186] The formatted data to be synchronized is then synchronized to the target database.

[0187] Optionally, the data synchronization request also carries a target database identifier;

[0188] The synchronization module 708 is also configured to:

[0189] The data to be synchronized is synchronized to the target database corresponding to the target database identifier.

[0190] This specification provides a data synchronization device that, by processing data change information, can obtain the data to be synchronized corresponding to any target time, thereby improving the efficiency of synchronizing data for different target times, i.e., improving synchronization efficiency; and by using data change information to determine the data to be synchronized, it avoids determining the data to be synchronized in the source database, reducing the data processing volume of the source database, thereby improving the service efficiency of the source database.

[0191] The above is an illustrative scheme of a data synchronization device according to this embodiment. It should be noted that the technical solution of this data synchronization device and the technical solution of the data synchronization method described above belong to the same concept. For details not described in detail in the technical solution of the data synchronization device, please refer to the description of the technical solution of the data synchronization method described above.

[0192] Corresponding to the above method embodiments, this specification also provides data storage system embodiments. Figure 8 A schematic diagram of a data storage system according to one embodiment of this specification is shown. Figure 8 As shown, the system includes:

[0193] Data synchronization processor 802, source database 804, and target database 806;

[0194] The data synchronization processor 802 is configured to receive a data synchronization request, wherein the data synchronization request includes a data identifier and a target time; obtain data change information corresponding to the data identifier and current data from the source database 804, wherein the data change information includes the data change time and the change amount; filter out the target change amount from the data change information according to the target time and the data change time; determine the data to be synchronized according to the target change amount and the current data, and send the data to be synchronized to the target database 806;

[0195] The source database 804 is configured to send the data change information corresponding to the data identifier and the current data to the data synchronization processor;

[0196] The target database 806 is configured to receive and store the data to be synchronized.

[0197] Optionally, the data synchronization request also carries a source database 804 identifier;

[0198] The data synchronization processor 802 is further configured to:

[0199] Obtain the data change information and current data corresponding to the data identifier from the source database 804 corresponding to the source database identifier.

[0200] Optionally, there is a corresponding relationship between the time of data change and the amount of change;

[0201] The data synchronization processor 802 is further configured to:

[0202] The target time is matched with each data change time;

[0203] The change corresponding to the moment when the matched data changes is determined as the target change.

[0204] The data synchronization processor 802 is further configured to:

[0205] A reverse change is determined based on the target change, wherein the reverse change is opposite in direction and the same in degree as the target change;

[0206] The current data is restored based on the inverse change amount to obtain the data to be synchronized.

[0207] Optionally, the number of the reverse change amounts is at least two, and the reverse change amounts correspond to the time of the data change;

[0208] The data synchronization processor 802 is further configured to:

[0209] According to the order of the data change times from back to front, starting from the reverse change amount corresponding to the first data change time, the current data is restored based on the current reverse change amount to obtain the initial data to be synchronized. The initial data to be synchronized is used as the current data, and the step of restoring the current data based on the current reverse change amount is continued until the reverse change amount corresponding to the last data change time is reached.

[0210] The data to be synchronized is determined based on the initial data to be synchronized.

[0211] Optionally, each piece of data in the data to be synchronized carries a group identifier;

[0212] The data synchronization processor 802 is also configured to:

[0213] Based on the group identifier, the data to be synchronized is divided into at least one group of data;

[0214] Based on the group identifier corresponding to each group of data, each group of data is synchronized to the storage area corresponding to the group identifier in the target database 806.

[0215] Optionally, the storage area is labeled with a grouping tag;

[0216] The data synchronization processor 802 is further configured to:

[0217] For each set of data, the group identifier corresponding to the set of data is matched with the group label of each storage area in the target database 806;

[0218] The successfully matched storage area is taken as the target storage area, and the group of data is stored in the target storage area.

[0219] Optionally, the data synchronization processor 802 is further configured to:

[0220] The data to be synchronized is formatted.

[0221] The formatted data to be synchronized is then synchronized to the target database 806.

[0222] Optionally, the data synchronization request also carries a target database 806 identifier;

[0223] The data synchronization processor 802 is further configured to:

[0224] The data to be synchronized is synchronized to the target database 806 corresponding to the target database 806 identifier.

[0225] This specification provides a data storage system that, by processing data change information, can obtain the data to be synchronized corresponding to any target time, thereby improving the efficiency of synchronizing data for different target times, i.e., improving synchronization efficiency; and by using data change information in the data synchronization processor to determine the data to be synchronized, it avoids determining the data to be synchronized in the source database, reducing the data processing volume of the source database, and thus improving the service efficiency of the source database.

[0226] The above is an illustrative scheme of a data storage system according to this embodiment. It should be noted that the technical solution of this data storage system and the technical solution of the data synchronization method described above belong to the same concept. For details not described in detail in the technical solution of the data storage system, please refer to the description of the technical solution of the data synchronization method described above.

[0227] Figure 9 A structural block diagram of a computing device according to one embodiment of this specification is shown. The components of the computing device 900 include, but are not limited to, a memory 910 and a processor 920. The processor 920 is connected to the memory 910 via a bus 930, and a database 950 is used to store data.

[0228] The computing device 900 also includes an access device 940, which enables the computing device 900 to communicate via one or more networks 960. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 940 may include one or more of any type of wired or wireless network interface (e.g., a Network Interface Controller (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0229] In one embodiment of this specification, the above-described components of the computing device 900 and Figure 9 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 9 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0230] The computing device 900 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 900 can also be a mobile or stationary server.

[0231] The processor 920 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-described data synchronization method.

[0232] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the data synchronization method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the data synchronization method described above.

[0233] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described data synchronization method.

[0234] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the data synchronization method described above belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the data synchronization method described above.

[0235] An embodiment of this specification also provides a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described data synchronization method.

[0236] The above is an illustrative scheme of a computer program according to this embodiment. It should be noted that the technical solution of this computer program and the technical solution of the data synchronization method described above belong to the same concept. For details not described in detail in the technical solution of the computer program, please refer to the description of the technical solution of the data synchronization method described above.

[0237] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0238] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0239] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0240] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0241] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data synchronization method, comprising: receiving a data synchronization request, wherein the data synchronization request comprises a data identifier and a target time; obtaining data change information and current data corresponding to the data identifier from a heterogeneous source database, wherein the data change information comprises a data change time and a change amount; filtering a target change amount from the data change information according to the target time and the data change time, comprising: determining a time filtering range according to the target time and a current time, wherein the time filtering range is used to filter the data change time; and determining the change amount corresponding to the data change time within the time filtering range in the data change information as the target change amount; determining to-be-synchronized data according to the target change amount and the current data, and synchronizing the to-be-synchronized data to a target database. 2.The method of claim 1, wherein the data synchronization request further comprises a source database identifier; and the obtaining data change information and current data corresponding to the data identifier from a heterogeneous source database comprises: obtaining data change information and current data corresponding to the data identifier from a source database corresponding to the source database identifier. 3.The method of claim 1, wherein the data change time and the change amount have a corresponding relationship; and the filtering a target change amount from the data change information according to the target time and the data change time comprises: matching the target time with each data change time respectively; and determining the change amount corresponding to the data change time that passes the matching as the target change amount. 4.The method of claim 1, wherein the determining to-be-synchronized data according to the target change amount and the current data comprises: determining a reverse change amount according to the target change amount, wherein the reverse change amount is opposite to the target change amount in change direction and has the same change degree; and restoring the current data according to the reverse change amount to obtain the to-be-synchronized data. 5.The method of claim 4, wherein the number of the reverse change amounts is at least two, and the reverse change amounts have a corresponding relationship with the data change times; and the restoring the current data according to the reverse change amount to obtain the to-be-synchronized data comprises: starting from a first data change time, restoring the current data according to a current reverse change amount to obtain initial to-be-synchronized data, taking the initial to-be-synchronized data as the current data, and continuing to perform the step of restoring the current data according to the current reverse change amount until a last data change time, wherein the order is from the last data change time to the first data change time; and determining to-be-synchronized data according to the obtained initial to-be-synchronized data. 6.The method of claim 1, wherein each data in the to-be-synchronized data carries a group identifier; and the synchronizing the to-be-synchronized data to a target database comprises: dividing the to-be-synchronized data into at least one group of data according to the group identifier. ​ ​ ​ ​ ​ ​ ​ According to the group identifier corresponding to each group of data, each group of data is synchronized to the storage area corresponding to the group identifier in the target database.

7. The method of claim 6, wherein the storage area is marked with a group tag; The step of synchronizing each group of data to the storage area corresponding to the group identifier in the target database comprises: For each group of data, matching the group identifier corresponding to the group of data with the group tag of each storage area in the target database; The storage area that matches successfully is taken as a target storage area, and the group of data is stored in the target storage area.

8. The method of claim 1, wherein the step of synchronizing the data to be synchronized to the target database comprises: performing format processing on the data to be synchronized; synchronizing the data to be synchronized after format processing to the target database.

9. The method of claim 1, wherein the data synchronization request further carries a target database identifier; The step of synchronizing the data to be synchronized to the target database comprises: synchronizing the data to be synchronized to the target database corresponding to the target database identifier.

10. A data synchronization apparatus, comprising: a receiving module configured to receive a data synchronization request, wherein the data synchronization request comprises a data identifier and a target time; an obtaining module configured to obtain data change information and current data corresponding to the data identifier from a heterogeneous source database, wherein the data change information comprises a data change time and a change amount; a screening module configured to screen a target change amount from the data change information according to the target time and the data change time, comprising: determining a time screening range according to the target time and a current time, wherein the time screening range is used to screen data change times; and determining the change amount corresponding to the data change times belonging to the time screening range in the data change information as the target change amount; a synchronization module configured to determine data to be synchronized according to the target change amount and the current data, and synchronize the data to be synchronized to a target database.

11. A data storage system, comprising: a data synchronization processor, a heterogeneous source database and a target database; the data synchronization processor is configured to receive a data synchronization request, wherein the data synchronization request comprises a data identifier and a target time; obtain data change information and current data corresponding to the data identifier from a heterogeneous source database, wherein the data change information comprises a data change time and a change amount; screen a target change amount from the data change information according to the target time and the data change time, comprising: determining a time screening range according to the target time and a current time, wherein the time screening range is used to screen data change times; determining the change amount corresponding to the data change times belonging to the time screening range in the data change information as the target change amount; determining data to be synchronized according to the target change amount and the current data, and sending the data to be synchronized to the target database. The source database is configured to send the data identification corresponding data change information and current data to a data synchronization processor; The target database is configured to receive the data to be synchronized and store.

12. A computing device comprising: a memory and a processor; the memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions, and the computer executable instructions, when executed by the processor, implement the steps of the data synchronization method of any one of claims 1 to 9.

13. A computer readable storage medium storing computer executable instructions, and the computer executable instructions, when executed by a processor, implement the steps of the data synchronization method of any one of claims 1 to 9.

Images