A data synchronization method and related apparatus

Abstract

The embodiment of the application discloses a data synchronization method and related device, wherein the method is applied to a target data center under a multi-active data center architecture, the target data center is any one of a plurality of data centers under the architecture, and the plurality of data centers have respective corresponding databases. The method comprises the following steps: receiving to-be-synchronized operation data sent by other data centers, the to-be-synchronized operation data being configured with version information, the version information being determined according to generation time of the to-be-synchronized operation data; determining whether the to-be-synchronized operation data is valid according to the version information of the to-be-synchronized operation data and reference operation data; if it is determined that the to-be-synchronized operation data is valid, performing target data-based change processing in the corresponding database according to the to-be-synchronized operation data; and if it is determined that the to-be-synchronized operation data is invalid, ignoring the to-be-synchronized operation data. The method can solve the problem of data miscoverage caused by data conflicts and ensure complete synchronization of data between the data centers.

Description

Technical Field

[0001] This application relates to the field of database technology, and in particular to a data synchronization method and related apparatus. Background Technology

[0002] A multi-active data center architecture is a network architecture that includes multiple data centers. In a multi-active data center architecture, multiple data centers deployed in different physical regions can provide services to the outside world simultaneously.

[0003] In a multi-active data center architecture, data is synchronized between different data centers. Figure 1 This is a schematic diagram of an exemplary multi-active data center architecture; such as Figure 1 As shown, this architecture includes databases in three data centers (A, B, and C) located in different physical regions. Each data center writes data to its database, and simultaneously, the written data is synchronized to the other data centers via a data synchronization link, ensuring that each data center has the complete dataset. Under this data synchronization mechanism, when a catastrophic failure occurs in one of the data centers, its traffic can be diverted to other data centers, thereby achieving rapid business migration and ensuring normal business operation.

[0004] The most critical issue in the aforementioned data synchronization mechanism is preventing data conflicts. A data conflict occurs when multiple data centers simultaneously send data to be synchronized. If a data conflict exists, the synchronization order may be disrupted, causing later-generated data to overwrite earlier-generated data, resulting in unexpected outcomes. In many cases, even minor data conflicts can lead to severe consequences, such as disrupting the multi-active data center architecture and affecting the normal operation of services within that architecture. Summary of the Invention

[0005] This application provides a data synchronization method and related apparatus, which can effectively solve the problem of data overwriting caused by data conflicts and ensure complete data synchronization between data centers in a multi-active data center architecture.

[0006] In view of this, the first aspect of this application provides a data synchronization method applied to a target data center in a multi-active data center architecture, wherein the multi-active data center architecture includes multiple data centers, the target data center is any one of the multiple data centers, and each of the multiple data centers has its own corresponding database; the method includes:

[0007] Receive synchronization operation data sent by other data centers under the multi-active data center architecture; the synchronization operation data is used to indicate change processing based on target data, and the synchronization operation data is configured with version information, which is determined according to the generation time of the synchronization operation data;

[0008] Based on the version information of the operation data to be synchronized and the version information of the reference operation data, it is determined whether the operation data to be synchronized is valid; the reference operation data includes at least one of the operation data that has been executed by the target data center and the operation data that has been received by the target data center but not executed.

[0009] If the data to be synchronized is determined to be valid, then a change process based on the target data is performed in the database corresponding to the target data center; if the data to be synchronized is determined to be invalid, then the data to be synchronized is ignored.

[0010] A second aspect of this application provides a data synchronization device deployed in a target data center under a multi-active data center architecture, wherein the multi-active data center architecture includes multiple data centers, the target data center is any one of the multiple data centers, and each of the multiple data centers has its own corresponding database; the device includes:

[0011] The data receiving module is used to receive operation data to be synchronized sent by other data centers under the multi-active data center architecture; the operation data to be synchronized is used to indicate change processing based on target data, and the operation data to be synchronized is configured with version information, which is determined according to the generation time of the operation data to be synchronized.

[0012] The data detection module is used to determine whether the operation data to be synchronized is valid based on the version information of the operation data to be synchronized and the version information of the reference operation data; the reference operation data includes at least one of the operation data that has been executed by the target data center and the operation data that has been received by the target data center but not executed.

[0013] The data modification module is used to perform modification processing based on the target data in the database corresponding to the target data center if the data to be synchronized is determined to be valid; otherwise, the data to be synchronized is ignored.

[0014] A third aspect of this application provides a computer device, the device comprising a processor and a memory:

[0015] The memory is used to store computer programs;

[0016] The processor is configured to perform the steps of the data synchronization method as described in the first aspect above, according to the computer program.

[0017] A fourth aspect of this application provides a computer-readable storage medium for storing a computer program for performing the steps of the data synchronization method described in the first aspect.

[0018] A fifth aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of the data synchronization method described in the first aspect.

[0019] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0020] This application provides a data synchronization method applied to a target data center in a multi-active data center architecture. The multi-active data center architecture includes multiple data centers, and the target data can be any one of these data centers. Each data center in the multi-active data center architecture has its own corresponding database. The target data center can receive operation data to be synchronized from other data centers in the multi-active data center architecture. This operation data to be synchronized is used to indicate changes based on the target data, and the operation data to be synchronized is configured with version information, which is determined based on the generation time of the operation data to be synchronized. Then, the target data center can determine whether the operation data to be synchronized is valid based on the version information of the operation data to be synchronized and the version information of reference operation data. Here, the reference operation version data can include at least one of operation data already executed by the target data center and operation data received but not executed by the target data center. If the operation data to be synchronized is determined to be valid, changes based on the target data can be performed in the database corresponding to the target data center. If the operation data to be synchronized is determined to be invalid, it can be ignored. In this way, corresponding version information is configured for each operation data according to the generation time of each operation data, and data synchronization is performed based on the corresponding version information of each operation data. This ensures that each data center in the multi-active data center architecture performs data synchronization operation based on the latest version of the operation data. In this case, even if data conflicts occur, data consistency between data centers can be guaranteed because each data center performs data synchronization based on the latest version of the operation data. Attached Figure Description

[0021] Figure 1This is a schematic diagram of an exemplary multi-active data center architecture;

[0022] Figure 2 This is an illustration of an exemplary scenario where operational data is mistakenly written to another data center.

[0023] Figure 3 This is a schematic diagram illustrating the working principle of the multi-active data center architecture provided in the embodiments of this application;

[0024] Figure 4 A flowchart illustrating the data synchronization method provided in this application embodiment;

[0025] Figure 5 This is a schematic diagram illustrating data synchronization under one scenario provided in an embodiment of this application.

[0026] Figure 6 A schematic diagram illustrating the synchronization principle of a multi-active data center architecture provided in this application embodiment;

[0027] Figure 7 This is a schematic diagram of the structure of the data synchronization device provided in the embodiments of this application;

[0028] Figure 8 This is a schematic diagram of the server structure provided in an embodiment of this application. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0030] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] Cloud technology is a collective term for network technologies, information technologies, integration technologies, management platform technologies, and application technologies applied based on the cloud computing business model. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will all require robust system support, which can only be achieved through cloud computing.

[0032] A database, simply put, can be viewed as an electronic filing cabinet—a place to store electronic files, where users can perform operations such as adding, querying, updating, and deleting data. A "database" is a collection of data stored together in a certain way, capable of being shared by multiple users, with minimal redundancy, and independent of application programs.

[0033] A Database Management System (DBMS) is a computer software system designed to manage databases, generally possessing basic functions such as storage, retrieval, security, and backup. DBMSs can be classified according to the database model they support, such as relational or XML (Extensible Markup Language); or according to the type of computer they support, such as server clusters or mobile devices; or according to the query language used, such as SQL (Structured Query Language) or XQuery; or according to performance priorities, such as maximum scale or maximum operating speed; or other classification methods. Regardless of the classification method used, some DBMSs can cross categories, for example, simultaneously supporting multiple query languages.

[0034] The data synchronization method provided in this application involves database technology in cloud technology.

[0035] For a multi-active data center architecture, the most critical issue when data synchronization is performed between data centers based on a data synchronization mechanism is to prevent data conflicts. Once a data conflict occurs, it is very likely to cause serious consequences and affect the normal operation of the entire multi-active data center architecture.

[0036] In related technologies, there are two main ways to avoid data conflicts: One is to set up corresponding data record tables for different data centers, writing data related to that data center (such as operation data that modifies data stored in that data center) into the corresponding data record table. In a multi-active data center architecture, each data center synchronizes data based on its respective data record table. The other method is to assign different key-value ranges to different data centers within a single data record table. For example, key-value ranges 1, 4, 7, ... are assigned to data center A, 2, 5, 8, ... to data center B, and 3, 6, 9, ... to data center C. For each data center, data related to that data center is written into the corresponding key-value range. In a multi-active data center architecture, each data center synchronizes data based on its respective key-value range within the data record table.

[0037] However, the aforementioned methods for avoiding data conflicts pose significant challenges to the use and operation of the entire multi-active data center architecture. Businesses accessing different data centers need to be aware of which data center they are using and which table or key-value range to write data to in order to avoid data write errors. This introduces numerous inconveniences into business design and deployment, increasing the complexity of business database usage and operation. Furthermore, in extreme cases, if a business deployment error occurs, it may lead to chaos in the entire multi-active data center architecture, potentially requiring manual intervention to restore the entire system. Because the link configuration in a multi-active data center architecture is inherently complex, restoring normal data is also very complicated, resulting in a lengthy business recovery time.

[0038] Figure 2This is an illustration of an exemplary scenario where operational data is mistakenly written to another data center. Specifically, suppose the business side triggers two update operations on data stored in data center A: the first modifies (1, a) to (1, b), and the second modifies (1, b) to (1, c). In this scenario, the first update operation (i.e., (1, a) → (1, b)) is mistakenly sent to data center B. As a result, data center A only receives the operation data of the second update operation and directly executes the second update, modifying its stored data a to data c. After receiving the operation data that modifies (1, a) to (1, b), data center B will also synchronize this operation data to data center A. Since data center A has already modified its stored data a to data c based on the operation data of the second update operation, when data center A receives the synchronized operation data from data center B that modifies (1, a) to (1, b), it will mistakenly modify the already modified data c back to data b, causing data b to overwrite data c. This is inconsistent with the data result expected by the business side, resulting in data corruption in a multi-active data center architecture.

[0039] To address the problems existing in the aforementioned related technologies and effectively resolve data conflict issues in multi-active data center architectures, this application provides a data synchronization method. This method is applied to a target data center in a multi-active data center architecture, which includes multiple data centers. The target data can be any one of these multiple data centers, and each data center in the multi-active data center architecture has its own corresponding database.

[0040] In this data synchronization method, the target data center can receive operation data to be synchronized from other data centers in a multi-active data center architecture. This operation data is used to indicate changes to target data, and it is configured with version information determined based on the generation time of the operation data. Then, the target data center can determine whether the operation data to be synchronized is valid based on the version information of the operation data to be synchronized and the version information of reference operation data. Here, the reference operation version data may include at least one of operation data already executed by the target data center and operation data received but not executed by the target data center. If the operation data to be synchronized is determined to be valid, changes to target data can be performed in the database corresponding to the target data center based on the operation data to be synchronized; if the operation data to be synchronized is determined to be invalid, it can be ignored.

[0041] The aforementioned data synchronization method configures corresponding version information for each operation data based on its generation time, and performs data synchronization based on the respective version information of each operation data. This ensures that each data center in the multi-active data center architecture performs data synchronization operations based on the latest version of the operation data. In this case, even if data conflicts occur, data consistency between data centers can be guaranteed because each data center performs data synchronization based on the latest version of the operation data. In other words, even if data conflicts occur in a multi-active data center architecture, each data center in this architecture can automatically resolve the data conflict according to a pre-configured strategy, thereby improving the robustness of the entire multi-active data center architecture. Furthermore, compared to related technologies that set up corresponding data record tables for different data centers or allocate different key-value ranges in the data record tables for different data centers, the solution provided in this application embodiment does not increase the complexity of database use and operation and maintenance on the business side.

[0042] It should be noted that, in this embodiment, each data center in the multi-active data center architecture can specifically be a server. This server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Each data center in the multi-active data center architecture can interface with a terminal, providing the terminal with a corresponding server. This terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0043] To facilitate understanding of the data synchronization method provided in the embodiments of this application, the application scenarios applicable to the data synchronization method provided in the embodiments of this application will be described below by way of example.

[0044] See Figure 3 , Figure 3 This is a schematic diagram illustrating the working principle of the multi-active data center architecture 300 provided in the embodiments of this application, and also a schematic diagram illustrating the application scenario of the data synchronization method provided in the embodiments of this application. Figure 3As shown, the multi-active data center architecture 300 includes multiple data centers, namely data center 310, data center 320, data center 330, ..., data center 3n0. Each data center has its corresponding database, such as data center 310 corresponding to database 311, data center 320 corresponding to database 321, data center 330 corresponding to database 331, ..., data center 3n0 corresponding to database 3n1. It should be understood that the database corresponding to the data center can be a storage device independent of the data center, or it can be a storage component integrated inside the data center.

[0045] This embodiment takes data center 310 as the target data center as an example to introduce the data synchronization method provided in this application embodiment in a scenario-based example.

[0046] like Figure 3 As shown, data center 310 can receive data to be synchronized from data center 320 via a data synchronization link. This data to be synchronized is used to instruct changes to target data. For example, it can instruct the insertion of the target data into database 311, or it can instruct the deletion of the target data originally stored in database 311, or it can instruct the updating of the target data originally stored in database 311 to the updated target data, and so on. Furthermore, the data to be synchronized is configured with version information, which is determined based on the generation time of the data. It should be understood that the later the generation time of the data to be synchronized, the higher the version number corresponding to the version information.

[0047] Furthermore, data center 310 can determine the validity of the operation data to be synchronized based on the version information of the operation data to be synchronized and the version information of the reference operation data. The reference operation data here can include at least one of operation data already executed by data center 310 and operation data received by data center 310 but not yet executed. Operation data already executed by data center 310 can be understood as data in database 311 that has been modified based on the operation data; operation data received by data center 310 but not yet executed can be understood as data in database 311 that has not yet been modified based on the operation data. This operation data can come from any data center in a multi-active data center architecture. Specifically, when determining the validity of the operation data to be synchronized based on the version information of the operation data to be synchronized and the version information of the reference operation data, data center 310 can determine whether the version information of the operation data to be synchronized is higher than the version information of the reference operation data. If so, it indicates that the generation time of the operation data to be synchronized is relatively late, and the operation data to be synchronized is valid; if not, it indicates that the generation time of the operation data to be synchronized is relatively early, and the operation data to be synchronized is invalid.

[0048] If data center 310 determines that the data to be synchronized is valid through the above operations, it can perform modification processing based on the target data in database 311, such as inserting, updating, or deleting the target data. Conversely, if data center 310 determines that the data to be synchronized is invalid through the above operations, it can ignore the data to be synchronized, that is, it will not perform the operation indicating the data to be synchronized in database 311.

[0049] It should be understood that Figure 3 The application scenarios shown are merely examples. In practical applications, the data synchronization method provided in this application can also be applied to other scenarios, and this application does not impose any limitations on it.

[0050] The data synchronization method provided in this application will be described in detail below through method embodiments.

[0051] See Figure 4 , Figure 4 This is a flowchart illustrating the data synchronization method provided in an embodiment of this application. This data synchronization method is applied to a target data center in a multi-active data center architecture, such as... Figure 4 As shown, this data synchronization method includes the following steps:

[0052] Step 401: Receive the operation data to be synchronized sent by other data centers under the multi-active data center architecture; the operation data to be synchronized is used to indicate the change processing based on the target data, the operation data to be synchronized is configured with version information, and the version information is determined according to the generation time of the operation data to be synchronized.

[0053] In a multi-active data center architecture, in order to ensure data synchronization between the data centers, the data centers need to synchronize data through a data synchronization link so that each data center's corresponding database stores all the data under the multi-active data center architecture.

[0054] For a target data center in a multi-active data center architecture (which can be any data center in the multi-active data center architecture), it can receive data to be synchronized from other data centers in the multi-active data center architecture through a data synchronization link. It should be understood that the target data center can receive data to be synchronized from multiple data centers at the same time, or it can receive data to be synchronized from only one data center. This application does not limit the number of data to be synchronized received.

[0055] It should be noted that, in this embodiment, the data to be synchronized is used to instruct changes to the target data. For example, the data to be synchronized may be used to instruct the insertion of the target data into the database corresponding to the target data center; alternatively, it may be used to instruct the deletion of the target data originally stored in the database corresponding to the target data center; or it may be used to instruct the updating of the target data originally stored in the database corresponding to the target data center to the updated target data, and so on. Specifically, the aforementioned data to be synchronized may be represented as Data Manipulation Language (DML) within the SQL language set, which is a set of instructions responsible for accessing database objects.

[0056] Furthermore, in this embodiment, the data to be synchronized is configured with version information, which is determined based on the generation time of the data. It should be understood that the higher the generation time of the data, the higher the version information should be. For example, the version information can be represented as a timestamp; the later the generation time of the data, the later the timestamp indicates. Alternatively, the version information can also be represented as a version number; the later the generation time of the data, the larger the version number. This application does not impose any limitations on the form of the version information of the data to be synchronized.

[0057] Furthermore, in this embodiment, the data to be synchronized can also be configured with a source data center identifier, which is the data center that initially generated the data to be synchronized. For example, corresponding data center identifiers can be pre-configured for each data center in a multi-active data center architecture. When a data center sends data to be synchronized to a target data center, in addition to configuring version information for the data to be synchronized, it can also configure a source data center identifier, which indicates the original data center that initially generated the data to be synchronized.

[0058] It should be understood that, in the embodiments of this application, each operational data generated in the multi-active data center architecture can be configured with corresponding version information and source data center identifier. That is, each operational data generated in the multi-active data center architecture belongs to operational data that needs to be synchronized to other data centers, i.e., the operational data to be synchronized mentioned above. Accordingly, each operational data generated in the multi-active data center architecture needs to be configured with corresponding version information and source data center identifier.

[0059] It should be noted that the method provided in this application embodiment can be applied to a multi-active data center architecture that records operation data based on a data record table. The data record table here is the data record table in the related technologies mentioned above; the data record table can be configured separately for different data centers to record operation data related to that data center; or the data record table can be uniformly configured for each data center in a multi-active data center architecture, including the key-value range corresponding to each data center, and recording operation data related to that data center within its corresponding key-value range for each data center.

[0060] When the method provided in this application embodiment is applied to a multi-active data center architecture that records operation data based on a data record table, two columns can be added to the data record table: one column is used to carry the version information of the operation data, and the other column is used to carry the source data center identifier of the operation data.

[0061] Step 402: Determine whether the operation data to be synchronized is valid based on the version information of the operation data to be synchronized and the version information of the reference operation data; the reference operation data includes at least one of the operation data that has been executed by the target data center and the operation data that has been received by the target data center but not executed.

[0062] After receiving the data to be synchronized from other data centers, the target data center can determine whether the data to be synchronized is valid based on the version information of the data to be synchronized and the version information of the reference data.

[0063] It should be noted that the reference operation data may include at least one of the following: operation data already executed by the target data center, and operation data received by the target data center but not yet executed. For operation data already executed by the target data center, this can be understood as the target data center having modified the data stored in its corresponding database based on the operation data; for example, the target data center has performed data insertion, data deletion, or data update operations based on the operation data. For operation data received by the target data center but not yet executed, this can be understood as the target data center having received the operation data but not yet modifying the data stored in its corresponding database based on the operation data.

[0064] It should be understood that the reference operation data is also configured with version information, which is determined based on the initial generation time of the reference operation data. In other words, the reference operation data is essentially the operation data obtained by the target data center before receiving the operation data to be synchronized. This type of operation data can be directly written to the target data center by the business side, or it can be transmitted to the target data center by other data centers in a multi-active data center architecture. In the embodiments of this application, each operation data generated under the multi-active data center architecture is configured with corresponding version information.

[0065] In this embodiment, the later the generation time of the operation data, the higher the version information of the operation data. Based on this, when the target data center specifically determines whether the operation data to be synchronized is valid, it can determine whether the version information of the operation data to be synchronized is higher than the version information of the reference operation data. If the version information of the operation data to be synchronized is higher than the version information of the reference operation data, the operation data to be synchronized is determined to be valid; if the version information of the operation data to be synchronized is lower than the version information of the reference operation data, the operation data to be synchronized is determined to be invalid.

[0066] Specifically, if the version information of the data to be synchronized is higher than that of the reference data, it means that the data to be synchronized was generated later than the reference data, and the data to be synchronized is still valid, thus confirming its validity. Conversely, if the version information of the data to be synchronized is lower than that of the reference data, it means that the data to be synchronized was generated earlier than the reference data, and its validity has been superseded by the reference data, thus confirming its invalidity.

[0067] As described in step 401 above, in this embodiment of the application, in addition to configuring version information for the operation data to be synchronized, a source data center identifier can also be configured for the operation data to be synchronized. In some possible cases, the target data center may not be able to determine whether the operation data to be synchronized is valid based on the version information of the operation data to be synchronized and the version information of the reference operation data. In this case, the validity of the operation data to be synchronized can be determined based on the source data center identifier of the operation data to be synchronized and the source data center identifier of the reference operation data.

[0068] In practical applications, a large amount of operational data may be generated across various data centers during the operation of a multi-active data center architecture. This can easily lead to multiple sets of operational data being generated simultaneously at the same time. Since these multiple sets of operational data were generated at the same time, their corresponding version information will also be identical. When the version information of the reference operational data is the same as the version information of the operational data to be synchronized, the target data center cannot simply rely on the version information of the reference operational data to determine the validity of the operational data to be synchronized.

[0069] At this point, the target data center can further use the source data center identifier of the data to be synchronized and the source data center identifier of the reference data to determine the validity of the data to be synchronized.

[0070] Specifically, in a multi-active data center architecture, each data center is configured with a corresponding priority, and the priorities of each data center have a total order relationship, meaning the order of priorities among the data centers is fixed and comparable. Based on this, the target data center can determine the data center corresponding to the source data center identifier of the data to be synchronized, designating it as the first data center, and the data center corresponding to the source data center identifier of the reference data, designating it as the second data center. Then, the target data center can determine whether the priority of the first data center is higher than the priority of the second data center. If so, the data to be synchronized is valid; otherwise, it is invalid.

[0071] In other words, in this embodiment, the priority relationships between each data center in a multi-active data center architecture can be pre-specified. These priority relationships must be unique and comparable throughout the entire synchronization link (i.e., the link composed of the data centers in the multi-active data center architecture). Based on this, if the target data center cannot determine the validity of the data to be synchronized based solely on the version information of the data to be synchronized and the version information of the reference data, it further determines the validity of the data to be synchronized by using the priority of the data center that generated the data to be synchronized and the priority of the data center that generated the reference data. If the priority of the data center that generated the data to be synchronized is higher than the priority of the data center that generated the reference data, the data to be synchronized is determined to be valid; if the priority of the data center that generated the data to be synchronized is lower than the priority of the data center that generated the reference data, the data to be synchronized is determined to be invalid.

[0072] Thus, by using version information and the priority of the data center corresponding to the source data center identifier as reference information, the validity of the received data to be synchronized can be determined, ensuring that the validity of the data to be synchronized is uniquely and reliably determined.

[0073] Step 403: If the data to be synchronized is determined to be valid, then change processing based on the target data is performed in the database corresponding to the target data center; if the data to be synchronized is determined to be invalid, then the data to be synchronized is ignored.

[0074] Through the above operations, if the target data center determines that the data to be synchronized is valid, it can perform change processing based on the target data in the database corresponding to the target data center. Conversely, if the target data center determines that the data to be synchronized is invalid, it can ignore the data to be synchronized, that is, it will not perform the change processing indicated by the data to be synchronized in the database corresponding to the target data center.

[0075] Data operations in DML primarily include INSERT, UPDATE, and DELETE statements. Typically, data synchronization for DML is achieved using a binary log. For INSERT statements, the binary log only contains the back image (the data after insertion); for UPDATE statements, the binary log contains both the front and back images (data before and after the update); for DELETE statements, the binary log only contains the front image, not the back image, because the data has been deleted.

[0076] In this embodiment, the version information of the operation data depends on the generation time of the operation data, so it must exist in the back image. However, the DELETE statement does not have a back image, so the corresponding version information cannot be configured for the DELETE statement in traditional DML, which will also affect the implementation of the method provided in this embodiment. Figure 5 This is an exemplary schematic diagram corresponding to this situation provided in the embodiments of this application.

[0077] like Figure 5 As shown, this includes data center A and data center M. Figure 5 In both data centers A and M, the timelines are from bottom to top. `insert 1A` indicates that data center A inserts data with version number 1 into the corresponding database; `insert 2A` indicates that data center A inserts data with version number 2 into the corresponding database; and `delete 3M` indicates that data center M performs a deletion operation on the corresponding database. Through observation... Figure 5 The timelines below data centers A and M show that data center A eventually had no data, while the final data version on data center M was 2A.

[0078] The reason why the data in data centers A and M are ultimately out of sync is that delete3M is a DELETE operation, and DELETE operations cannot carry a back image, and therefore cannot carry the corresponding version information. Consequently, when insert 1A is synchronized to data center M, data center M will mistakenly believe that there is no higher version operation data operation, and will execute the data change processing indicated by insert 1A. Furthermore, when insert 2A is synchronized to data center M, since its version information is higher than insert 1A, it will also be executed.

[0079] To avoid the aforementioned situation, this application embodiment modifies the original DELETE statement so that the operation data corresponding to the DELETE statement can carry corresponding version information. Specifically, the operation data corresponding to the modified DELETE statement may include the previous image (i.e., the data to be deleted as indicated by the DELETE statement), a deletion identifier (used to identify that the previous image is the deleted data), as well as version information and a source data center identifier. This modified operation data corresponding to the DELETE statement can be understood as another form of operation data corresponding to an UPDATE statement, that is, based on the operation data corresponding to the original UPDATE statement, the subsequent image is adjusted to the deletion identifier.

[0080] If the data to be synchronized is valid, and the data indicates that the target data should be deleted (i.e., the data corresponds to a DELETE statement that indicates the deletion of the target data), the target data center can mark the target data in its corresponding database with a deletion flag, along with the version information of the data to be synchronized and the synchronization operation itself. This deletion flag indicates that the target data in the database is currently invalid. Subsequently, when a preset time condition is met, the target data center can delete the target data marked with the deletion flag.

[0081] Specifically, in addition to adjusting the operation data corresponding to the DELETE statement, this application embodiment also makes certain adjustments to the operation method corresponding to the DELETE statement, so that the data deleted by the DELETE statement can still be reflected after the DELETE statement is executed, which makes it convenient to mark the execution time of the DELETE statement, and thus facilitates comparison with the subsequently received operation data to be synchronized, so as to determine the validity of the subsequently received operation data to be synchronized.

[0082] In other words, when the data to be synchronized corresponds to the data indicated by a DELETE statement that signals the deletion of target data, the target data center, when executing the change processing indicated by the data to be synchronized, can mark the target data in its corresponding database with a deletion flag. By marking the deletion flag, it indicates that the target data in the database is invalid and no further processing of the target data is supported. At the same time, marking the deletion flag also reflects the DELETE statement, thus facilitating the marking of the version information and source data center identifier of the data to be synchronized. This method of reflecting the DELETE statement by marking the deletion flag can also be understood as a soft deletion operation.

[0083] When preset time conditions are met, such as after a preset time interval since the soft delete operation, the target data center can perform a true deletion operation on the target data, that is, delete the target data marked with the deletion identifier. It should be understood that after the above preset time conditions are met, there will usually be no more data to be synchronized with similar version information. That is, there is no need to use the version information of the target data and the source data center identifier to detect the validity of subsequently received data to be synchronized. Therefore, a true deletion operation can be performed to delete the target data.

[0084] When the data to be synchronized is valid, and the data to be synchronized indicates that target data should be inserted, that is, when the data to be synchronized is the data corresponding to an INSERT statement (which indicates that target data should be inserted into the database), the target data center can insert the target data into its corresponding database. For the inserted target data, the version information of the data to be synchronized and the source data center identifier of the data to be synchronized are marked.

[0085] When the target data center executes the insertion operation indicated by the INSERT statement for the operation data, it is similar to the method of executing the insertion operation indicated by the traditional INSERT statement. The only difference is that, in this embodiment, after the target data center executes the insertion operation indicated by the operation data to be synchronized and inserts the target data into the corresponding database, it will also configure the version information and source data center identifier of the operation data to be synchronized for the target data. This makes it convenient to use the target data as reference operation data to compare with the operation data to be synchronized subsequently received by the target data center and detect the validity of the operation data to be synchronized subsequently received.

[0086] For the INSERT operation, the underlying code should be rewritten as follows:

[0087] INSERT IGNORE INTO table1 VALUES(id_src,c1_src,ts_src,node_name_src)

[0088] UPDATE table1 SET c1=c1_src,ts=ts_src,node_name=node_name_srcWHERE id=id_src AND(ts,node_name)<=(ts_src,node_name_src)

[0089] When the data to be synchronized is valid, and the data to be synchronized indicates that the target data should be updated to the target updated data, that is, when the data to be synchronized is the data corresponding to the UPDATE statement (which indicates that the target data originally stored in the database should be updated to the target updated data), the target data center can update the target data in its corresponding database to the target updated data; and for the target updated data, mark the version information of the data to be synchronized, and mark the source data center identifier of the data to be synchronized.

[0090] When the target data center executes the update operation indicated by the UPDATE statement, it is similar to the way the traditional UPDATE statement indicates the update operation. The only difference is that, in this embodiment, after the target data center executes the update operation indicated by the operation data to be synchronized and updates the target data to the target updated data in the corresponding database, it will also configure the version information and source data center identifier of the operation data to be synchronized for the target updated data. This makes it convenient to compare the target updated data as reference operation data with the operation data to be synchronized received by the target data center later, and to detect the validity of the operation data to be synchronized received later.

[0091] For the UPDATE operation, the underlying operation code is rewritten as follows:

[0092] INSERT IGNORE INTO table1 VALUES(id_src_new,c1_src_new,ts_src_new,node_name_src_new)UPDATE table1 SET c1=c1_src_new,ts=ts_src_new,node_name=node_name_src_new WHERE id=id_src_new AND(ts,node_name)<=(ts_src_new,node_name_src_new)

[0093] A schematic diagram illustrating the synchronization principle of a multi-active data center architecture based on the data synchronization method provided in this application is shown below. Figure 6 As shown, when data is synchronized between data centers A, B, and C, it is based on condition coverage. Condition coverage means that the synchronization of the data to be synchronized is determined based on the version information of the reference operation data and the operation data to be synchronized, as well as the source data center identifier.

[0094] The aforementioned data synchronization method configures corresponding version information for each operational data based on its generation time, and performs data synchronization based on the respective version information of each operational data. This ensures that each data center in the multi-active data center architecture performs data synchronization operations based on the latest version of the operational data. In this case, even if data conflicts occur, data consistency between data centers can be guaranteed because each data center synchronizes based on the latest version of the operational data. In other words, even if data conflicts occur in a multi-active data center architecture, each data center in this architecture can automatically resolve the data conflict according to pre-configured strategies, thereby improving the robustness of the entire multi-active data center architecture.

[0095] In response to the data synchronization method described above, this application also provides a corresponding data synchronization device so that the above data synchronization method can be applied and implemented in practice.

[0096] See Figure 7 , Figure 7 This is consistent with the above text Figure 4 The diagram illustrates the structure of a data synchronization device 700 corresponding to the data synchronization method shown. This data synchronization device is deployed in a target data center within a multi-active data center architecture. The multi-active data center architecture includes multiple data centers, and the target data center is any one of these multiple data centers. Each of the multiple data centers has its own corresponding database. Figure 7 As shown, the data synchronization device 700 includes:

[0097] The data receiving module 701 is used to receive operation data to be synchronized sent by other data centers under the multi-active data center architecture; the operation data to be synchronized is used to indicate change processing based on target data, and the operation data to be synchronized is configured with version information, which is determined according to the generation time of the operation data to be synchronized.

[0098] The data detection module 702 is used to determine whether the operation data to be synchronized is valid based on the version information of the operation data to be synchronized and the version information of the reference operation data; the reference operation data includes at least one of the operation data that has been executed by the target data center and the operation data that has been received by the target data center but not executed.

[0099] The data modification module 703 is used to perform modification processing based on the target data in the database corresponding to the target data center if the data to be synchronized is determined to be valid; otherwise, the data to be synchronized is ignored.

[0100] Optionally, the later the operation data is generated, the higher the version information of the operation data; then the data detection module 702 is specifically used for:

[0101] If the version information of the operation data to be synchronized is higher than the version information of the reference operation data, then the operation data to be synchronized is determined to be valid; if the version information of the operation data to be synchronized is lower than the version information of the reference operation data, then the operation data to be synchronized is determined to be invalid.

[0102] Optionally, the data to be synchronized is further configured with a source data center identifier, the data center corresponding to which the source data center identifier is the data center that initially generated the data to be synchronized; the data detection module 702 is further configured to:

[0103] If the validity of the operation data to be synchronized cannot be determined based on the version information of the operation data to be synchronized and the version information of the reference operation data, the validity of the operation data to be synchronized is determined based on the source data center identifier of the operation data to be synchronized and the source data center identifier of the reference operation data.

[0104] Optionally, each data center in the multi-active data center architecture is configured with its own corresponding priority; the data detection module 702 is specifically used for:

[0105] Determine the first data center corresponding to the source data center identifier of the operation data to be synchronized, and determine the second data center corresponding to the source data center identifier of the reference operation data;

[0106] If the priority of the first data center is higher than that of the second data center, the data to be synchronized is determined to be valid; if the priority of the first data center is lower than that of the second data center, the data to be synchronized is determined to be invalid.

[0107] Optionally, when the data to be synchronized indicates that the target data should be deleted, the data modification module 703 is specifically used for:

[0108] For the target data in the database corresponding to the target data center, a deletion identifier is marked, the version information of the data to be synchronized is marked, and the source data center identifier of the data to be synchronized is marked; the deletion identifier is used to indicate that the target data in the database is currently invalid;

[0109] When a preset time condition is met, the target data marked with the deletion identifier is deleted.

[0110] Optionally, when the data to be synchronized indicates the insertion of the target data, the data modification module 703 is specifically used for:

[0111] Insert the target data into the database corresponding to the target data center;

[0112] For the target data, mark the version information of the data to be synchronized and mark the source data center identifier of the data to be synchronized.

[0113] Optionally, when the data to be synchronized indicates that the target data should be updated to the target updated data, the data change module 703 is specifically used for:

[0114] Update the target data in the database corresponding to the target data center to the target updated data;

[0115] For the target update data, mark the version information of the data to be synchronized and mark the source data center identifier of the data to be synchronized.

[0116] The aforementioned data synchronization device configures corresponding version information for each operational data based on its generation time, and performs data synchronization based on the respective version information of each operational data. This ensures that each data center in the multi-active data center architecture performs data synchronization operations based on the latest version of the operational data. In this case, even if data conflicts occur, data consistency between data centers can be guaranteed because each data center synchronizes based on the latest version of the operational data. In other words, even if data conflicts occur in a multi-active data center architecture, each data center in this architecture can automatically resolve the data conflict according to a pre-configured strategy, thereby improving the robustness of the entire multi-active data center architecture.

[0117] This application also provides a computer device for implementing data synchronization. Specifically, the device may be a server. The server provided in this application will be described below from the perspective of hardware implementation.

[0118] See Figure 8 , Figure 8 This is a schematic diagram of the structure of a server 800 provided in an embodiment of this application. The server 800 can vary significantly due to different configurations or performance, and may include one or more central processing units (CPUs) 822 (e.g., one or more processors) and memory 832, and one or more storage media 830 (e.g., one or more mass storage devices) for storing application programs 842 or data 844. The memory 832 and storage media 830 can be temporary or persistent storage. The program stored in the storage media 830 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server. Furthermore, the CPU 822 may be configured to communicate with the storage media 830 and execute the series of instruction operations in the storage media 830 on the server 800.

[0119] Server 800 may also include one or more power supplies 826, one or more wired or wireless network interfaces 850, one or more input / output interfaces 858, and / or one or more operating systems, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0120] The steps performed by the server in the above embodiments can be based on this Figure 8 The server structure shown.

[0121] The CPU 822 is used to perform the following steps:

[0122] Receive synchronization operation data sent by other data centers under the multi-active data center architecture; the synchronization operation data is used to indicate change processing based on target data, and the synchronization operation data is configured with version information, which is determined according to the generation time of the synchronization operation data;

[0123] Based on the version information of the operation data to be synchronized and the version information of the reference operation data, it is determined whether the operation data to be synchronized is valid; the reference operation data includes at least one of the operation data that has been executed by the target data center and the operation data that has been received by the target data center but not executed.

[0124] If the data to be synchronized is determined to be valid, then a change process based on the target data is performed in the database corresponding to the target data center; if the data to be synchronized is determined to be invalid, then the data to be synchronized is ignored.

[0125] Optionally, the CPU 822 can also be used to execute any of the steps of the data synchronization method provided in the embodiments of this application.

[0126] This application also provides a computer-readable storage medium for storing a computer program that executes any one of the data synchronization methods described in the foregoing embodiments.

[0127] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the implementation methods of the data synchronization method described in the foregoing embodiments.

[0128] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0129] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0130] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0131] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0132] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing computer programs.

[0133] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0134] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A data synchronization method, characterized in that, The method is applied to a target data center in a multi-active data center architecture, wherein the multi-active data center architecture includes multiple data centers, the target data center is any one of the multiple data centers, and each of the multiple data centers has its own corresponding database; the method includes: Receive operation data to be synchronized from other data centers under the multi-active data center architecture; the operation data to be synchronized is used to indicate change processing based on target data, and the operation data to be synchronized is configured with version information, which is determined according to the generation time of the operation data to be synchronized. Based on the version information of the operation data to be synchronized and the version information of the reference operation data, it is determined whether the operation data to be synchronized is valid; the reference operation data includes at least one of the operation data that has been executed by the target data center and the operation data that has been received by the target data center but not executed. If the data to be synchronized is determined to be valid, then a change process based on the target data is performed in the database corresponding to the target data center; if the data to be synchronized is determined to be invalid, then the data to be synchronized is ignored. The data to be synchronized is further configured with a source data center identifier, the data center corresponding to which the source data center identifier is the data center that initially generated the data to be synchronized. In the multi-active data center architecture, each data center is configured with its own corresponding priority. The method further includes: If, based on the version information of the data to be synchronized and the version information of the reference data, it cannot be determined whether the data to be synchronized is valid, Determine the first data center corresponding to the source data center identifier of the operation data to be synchronized, and determine the second data center corresponding to the source data center identifier of the reference operation data; If the priority of the first data center is higher than that of the second data center, the data to be synchronized is determined to be valid; if the priority of the first data center is lower than that of the second data center, the data to be synchronized is determined to be invalid.

2. The method according to claim 1, characterized in that, The later the generation time of the operation data, the higher the version information of the operation data; therefore, determining whether the operation data to be synchronized is valid based on the version information of the operation data to be synchronized and the version information of the reference operation data includes: If the version information of the operation data to be synchronized is higher than the version information of the reference operation data, then the operation data to be synchronized is determined to be valid; if the version information of the operation data to be synchronized is lower than the version information of the reference operation data, then the operation data to be synchronized is determined to be invalid.

3. The method according to claim 1, characterized in that, When the operation data to be synchronized indicates the deletion of the target data, the step of performing change processing based on the target data in the database corresponding to the target data center, according to the operation data to be synchronized, includes: For the target data in the database corresponding to the target data center, a deletion identifier is marked, the version information of the data to be synchronized is marked, and the source data center identifier of the data to be synchronized is marked; the deletion identifier is used to indicate that the target data in the database is currently invalid; When a preset time condition is met, the target data marked with the deletion identifier is deleted.

4. The method according to claim 1, characterized in that, When the data to be synchronized indicates the insertion of the target data, the step of performing a change process based on the target data in the database corresponding to the target data center, according to the data to be synchronized, includes: Insert the target data into the database corresponding to the target data center; For the target data, mark the version information of the data to be synchronized and mark the source data center identifier of the data to be synchronized.

5. The method according to claim 1, characterized in that, When the operation data to be synchronized indicates that the target data should be updated to the target updated data, the step of performing a change processing based on the target data in the database corresponding to the target data center, according to the operation data to be synchronized, includes: Update the target data in the database corresponding to the target data center to the target updated data; For the target update data, mark the version information of the data to be synchronized and mark the source data center identifier of the data to be synchronized.

6. A data synchronization device, characterized in that, A target data center deployed in a multi-active data center architecture, wherein the multi-active data center architecture includes multiple data centers, and the target data center is any one of the multiple data centers, each of which has its own corresponding database; the device includes: The data receiving module is used to receive operation data to be synchronized sent by other data centers under the multi-active data center architecture; the operation data to be synchronized is used to indicate change processing based on target data, and the operation data to be synchronized is configured with version information, which is determined according to the generation time of the operation data to be synchronized. The data detection module is used to determine whether the operation data to be synchronized is valid based on the version information of the operation data to be synchronized and the version information of the reference operation data; the reference operation data includes at least one of the operation data that has been executed by the target data center and the operation data that has been received by the target data center but not executed. The data modification module is used to perform modification processing based on the target data in the database corresponding to the target data center if the data to be synchronized is determined to be valid; otherwise, the data to be synchronized is ignored. The data to be synchronized is further configured with a source data center identifier, the data center corresponding to which the source data center identifier is the data center that initially generated the data to be synchronized. In the multi-active data center architecture, each data center is configured with its own corresponding priority. The data detection module is also used for: If, based on the version information of the data to be synchronized and the version information of the reference data, it cannot be determined whether the data to be synchronized is valid, Determine the first data center corresponding to the source data center identifier of the operation data to be synchronized, and determine the second data center corresponding to the source data center identifier of the reference operation data; If the priority of the first data center is higher than that of the second data center, the data to be synchronized is determined to be valid; if the priority of the first data center is lower than that of the second data center, the data to be synchronized is determined to be invalid.

7. The apparatus according to claim 6, characterized in that, The later the time of the operation data generation, the higher the version information of the operation data; therefore, the data detection module is specifically used for: If the version information of the operation data to be synchronized is higher than the version information of the reference operation data, then the operation data to be synchronized is determined to be valid; if the version information of the operation data to be synchronized is lower than the version information of the reference operation data, then the operation data to be synchronized is determined to be invalid.

8. The apparatus according to claim 6, characterized in that, When the data to be synchronized indicates that the target data should be deleted, the data modification module is specifically used for: For the target data in the database corresponding to the target data center, mark the deletion identifier, mark the version information of the data to be synchronized, and mark the source data center identifier of the data to be synchronized; The deletion identifier is used to indicate that the target data in the database is currently invalid; When a preset time condition is met, the target data marked with the deletion identifier is deleted.

9. The apparatus according to claim 6, characterized in that, When the data to be synchronized indicates the insertion of the target data, the data modification module is specifically used for: Insert the target data into the database corresponding to the target data center; For the target data, mark the version information of the data to be synchronized and mark the source data center identifier of the data to be synchronized.

10. The apparatus according to claim 6, characterized in that, When the data to be synchronized indicates that the target data should be updated to the target updated data, the data change module is specifically used for: Update the target data in the database corresponding to the target data center to the target updated data; For the target update data, mark the version information of the data to be synchronized and mark the source data center identifier of the data to be synchronized.

11. A computer device, characterized in that, The device includes a processor and a memory; The memory is used to store computer programs; The processor is configured to execute the data synchronization method according to any one of claims 1 to 5 according to the computer program.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program for performing the data synchronization method according to any one of claims 1 to 5.

13. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or the instructions are executed by the processor, the data synchronization method according to any one of claims 1 to 5 is implemented.

Images