Data synchronization method, data access method, data synchronization system, computing device, readable storage medium and program product

Abstract

Embodiments of the present specification provide a data synchronization method, a data access method, a data synchronization system, a computing device, a readable storage medium and a program product, the data synchronization method comprising: acquiring data to be synchronized and first source information of the data to be synchronized, the first source information comprising a first source library identifier, the first source library identifier represents a source database of a to-be-synchronized data source; based on the first source database identifier, identifying whether a target database to which the to-be-synchronized data is to be synchronized is a source database; and if not, synchronizing the to-be-synchronized data to the target database. When data synchronization is carried out, based on the first source information of the to-be-synchronized data, whether the target database to which the to-be-synchronized data is synchronized is identified, and if not, data synchronization is carried out, so that the phenomenon that the to-be-synchronized data is synchronized to the corresponding source database to cause data repetition of the data in the source database is avoided; and the data synchronization efficiency is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present specification relate to the technical field of data processing, in particular to a data synchronization method, a data access method, a data synchronization system, a computing device, a readable storage medium and a program product. BACKGROUND

[0002] With the continuous advancement of data informatization, data has become one of the core assets, and its importance is increasingly prominent. Data synchronization, as an important means of data management, plays a crucial role in this regard. Data synchronization refers to real-time or periodic updating and matching of data in two or more different locations or systems to maintain data consistency between databases.

[0003] However, when synchronizing data between databases, data loops may occur, resulting in the target database receiving synchronization data that is the same as the data it already has, and the repeated data needs to be deleted, making the data synchronization efficiency low. Therefore, there is an urgent need for a method to improve data synchronization efficiency. SUMMARY

[0004] Therefore, the embodiments of the present specification provide a data synchronization method. One or more embodiments of the present specification also provide a data access method, a data synchronization device, a data access device, a computing device, a computer readable storage medium and a computer program product to solve the technical defects in the prior art.

[0005] According to a first aspect of the embodiments of the present specification, a data synchronization method is provided, comprising:

[0006] obtaining to-be-synchronized data and first source information of the to-be-synchronized data, wherein the first source information includes a first source database identifier, and the first source database identifier represents a source database from which the to-be-synchronized data comes;

[0007] based on the first source database identifier, identifying whether a target database to which the to-be-synchronized data is to be synchronized is the source database;

[0008] if not, synchronizing the to-be-synchronized data to the target database.

[0009] According to a second aspect of the embodiments of the present specification, a data access method is provided, comprising:

[0010] receiving a data access request for a current database, wherein the data access request carries data information of to-be-accessed data;

[0011] In the case of the current database access failure, the data to be accessed is read from a backup database of the current database based on the data information, wherein the backup database synchronizes data from the current database in advance based on a data synchronization method.

[0012] According to a third aspect of the embodiments of the present specification, a data synchronization apparatus is provided, comprising:

[0013] The obtaining module is configured to obtain data to be synchronized and first source information of the data to be synchronized, wherein the first source information comprises a first source database identifier, and the first source database identifier represents a source database from which the data to be synchronized originates.

[0014] The identification module is configured to identify, based on the first source database identifier, whether a target database to which the data to be synchronized is to be synchronized is the source database.

[0015] The synchronization module is configured to synchronize the data to be synchronized to the target database if the answer is no.

[0016] According to a fourth aspect of the embodiments of the present specification, a data access apparatus is provided, comprising:

[0017] The receiving module is configured to receive a data access request for a current database, wherein the data access request carries data information of data to be accessed.

[0018] The reading module is configured to read, in the case of current database access failure, the data to be accessed from a backup database of the current database based on the data information, wherein the backup database synchronizes data from the current database in advance based on a data synchronization method.

[0019] According to a fifth aspect of the embodiments of the present specification, a computing device is provided, comprising:

[0020] a memory and a processor;

[0021] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which realize the steps of the above-mentioned data synchronization method and data access method when executed by the processor.

[0022] According to a sixth aspect of the embodiments of the present specification, a computer readable storage medium is provided, which stores computer programs / instructions, which realize the steps of the above-mentioned data synchronization method and data access method when executed by the processor.

[0023] According to a seventh aspect of the embodiments of the present specification, a computer program product is provided, comprising computer programs / instructions which, when executed by a processor, implement the steps of the above-mentioned data synchronization method and data access method.

[0024] One embodiment of the present specification obtains to-be-synchronized data and first source information of the to-be-synchronized data, wherein the first source information comprises a first source library identifier, and the first source library identifier represents a source database from which the to-be-synchronized data is derived; based on the first source library identifier, it is identified whether a target database to which the to-be-synchronized data is to be synchronized is the source database; if not, the to-be-synchronized data is synchronized to the target database. When performing data synchronization, based on the first source information of the to-be-synchronized data, it is identified whether the target database to which the to-be-synchronized data is to be synchronized is the source database, and if not, data synchronization is performed again, thereby avoiding synchronization of the to-be-synchronized data to the corresponding source database, so that the phenomenon of data duplication in the source database is avoided, and the efficiency of data synchronization is improved. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 is a schematic diagram of an interaction process under a data synchronization system architecture provided by one embodiment of the present specification;

[0026] Figure 2 is a framework diagram of a data synchronization system provided by one embodiment of the present specification;

[0027] Figure 3 is a flowchart of a data synchronization method provided by one embodiment of the present specification;

[0028] Figure 4 is a flowchart of a data access method provided by one embodiment of the present specification;

[0029] Figure 5a is a flowchart of a processing process of a data synchronization method provided by one embodiment of the present specification;

[0030] Figure 5b is a schematic diagram of data synchronization between databases in a data synchronization method provided by one embodiment of the present specification;

[0031] Figure 6a is a flowchart of a processing process of another data synchronization method provided by one embodiment of the present specification;

[0032] Figure 6b is a schematic diagram of data synchronization between databases in another data synchronization method provided by one embodiment of the present specification;

[0033] Figure 7 is a structural schematic diagram of a data synchronization device provided by one embodiment of the present specification;

[0034] Figure 8 is a structural diagram of a data access device provided by an embodiment of the present specification;

[0035] Figure 9 is a structural block diagram of a computing device provided by an embodiment of the present specification. DETAILED DESCRIPTION

[0036] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present specification. However, the present specification can be practiced without the specific details, other than in the examples described herein. Those skilled in the art, in light of the present specification, can appreciate that many changes can be made in the specific details, and other implementations can be used instead without departing from the scope of the present specification.

[0037] The terminology used in this description of one or more embodiments of the present specification is for the purpose of describing specific embodiments only and is not intended to be limiting of one or more embodiments of the present specification. As used in this description of one or more embodiments of the present specification and the accompanying claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0038] It will be understood that, although the terms first, second, etc. can be used herein to describe various information, these terms are not intended to denote a temporal or chronological order. Rather, these terms are used solely to distinguish one from another only. For example, without departing from the scope of one or more embodiments of the present specification, first can be termed second, and similarly, second can be termed first. The word "if' as used herein means "when" or "upon" or "in response to the determination" depending on the context.

[0039] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in one or more embodiments of the present specification are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0040] First, the terms involved in one or more embodiments of the present specification are explained.

[0041] Synchronization object: library table information synchronized by data synchronization.

[0042] Initialization object: The synchronization object specified when establishing a forward synchronization link.

[0043] Bin log event: A bin log event is a binary log event in MySQL that records database change operations. It contains detailed information about insert, update, delete, DDL, and other operations on the database, which can be used for data recovery, backup, replication, synchronization, and other application scenarios. Data Definition Language (DDL, Data Definition Language) is a language in the SQL language set that is responsible for data structure definition and database object definition. Structured Query Language (SQL, Structured Query Language) is a standard language for managing relational databases, designed to store, manipulate, and retrieve data. MySQL is a relational database management system that saves data in different tables instead of putting all data in one large warehouse, which increases speed and improves flexibility.

[0044] Bidirectional data synchronization: refers to the implementation of bidirectional data transmission and update between two or more databases to ensure the consistency and availability of data between two or more databases. Bidirectional data synchronization is required in many scenarios, such as multi-active and standby disaster recovery. Multi-active refers to multiple writes and reads in multiple regions, requiring bidirectional data synchronization to ensure data consistency. Standby disaster recovery refers to the backup database taking over immediately when the main database fails to ensure system reliability and continuity.

[0045] With the continuous advancement of data informatization, data has become one of the core assets, and its importance has become increasingly prominent. Data synchronization, as an important means of data management, plays a crucial role in this regard. Data synchronization refers to real-time or periodic updating and matching of data in two or more different locations or systems to maintain data consistency between databases.

[0046] However, when synchronizing data between databases, data loops may occur, resulting in the target database receiving synchronization data that it already possesses, requiring the removal of duplicate data to improve data synchronization efficiency. Therefore, there is an urgent need for a method to improve data synchronization efficiency.

[0047] In this specification, a data synchronization method is provided. One or more embodiments of this specification also provide a data access method, a data synchronization device, a data access device, a computing device, a computer-readable storage medium, and a computer program product, which are described in detail in the following embodiments.

[0048] Referring toFigure 1 , Figure 1 An interaction flow diagram under a data synchronization system architecture provided by one embodiment of the present specification is shown, as shown in the figure, the system includes a server 100 and at least two databases 200, the at least two databases 200 include a current database storing data to be synchronized and a target database to which the data to be synchronized is synchronized; Figure 1

[0049] The server 100 is used to obtain data to be synchronized and first source information of the data to be synchronized, wherein the first source information includes a first source database identifier, and the first source database identifier represents a source database of the data to be synchronized; based on the first source database identifier, it is identified whether the target database to which the data to be synchronized is synchronized is the source database; if not, the data to be synchronized is synchronized to the target database.

[0050] Optionally, the data synchronization system further includes a client, and the client is used to send a data synchronization request to the server 100; the server 100 is further used to obtain data to be synchronized and first source information of the data to be synchronized based on the data synchronization request.

[0051] Optionally, the server 100 can include a data transmission service execution unit (DTS, Data Transmission Service) for data transmission, which is connected to each database 200 respectively, can read data in each database 200, and drives data synchronization transmission between each database 200.

[0052] In an optional implementation of the present specification, the server 100 is further used to determine second source information of the data to be synchronized based on the database identifier of the current database, wherein the second source information is used to write a data change log of the target database.

[0053] After the data to be synchronized is synchronized to the target database, the second source information of the data to be synchronized in the target database is determined based on the database identifier of the current database, so that when the target database synchronizes data to other databases, it can be identified based on the second source information of the data to be synchronized whether the data to be synchronized is synchronized to other databases, thereby avoiding that the data synchronization causes the databases to contain repeated data, and improving the efficiency of data synchronization.

[0054] Referring to Figure 2 , Figure 2 ​A framework diagram of a data synchronization system provided by one embodiment of the present specification is shown, which can include a server 100 and a plurality of clients 300. The plurality of clients 300 can establish a communication connection through the server 100, and in a data synchronization scenario, the server 100 is used to provide data synchronization services between the plurality of clients 300. The plurality of clients 300 can respectively act as a sending end or a receiving end to realize communication through the server 100.

[0055] A user can interact with the server 100 through the client 300 to receive data sent by other clients 300, or send data to other clients 300, etc. In a data synchronization scenario, the user can publish a data synchronization request to the server 100 through the client 300, and the server 100 generates a data synchronization result according to the data synchronization request and pushes the data synchronization result to other clients 300 that establish a communication connection.

[0056] Among them, the client 300 and the server 100 establish a connection through a network. The network provides a medium for a communication link between the client 300 and the server 100. The network can include various connection types, such as wired, wireless communication links, or optical fiber cables, etc. The data transmitted by the client 300 can need to be processed through encoding, transcoding, compression, etc. before being published to the server 100.

[0057] The client 300 can be a browser, an application (APP), or a web application such as a Hyper Text Markup Language 5 (H5) application, or a light application (also known as a small program, a lightweight application), or a cloud application, etc. The client 300 can be developed based on the software development kit (SDK) of the corresponding service provided by the server, such as based on the real-time communication (RTC) SDK, etc. The client 300 can be deployed in an electronic device and needs to rely on the device or some APP in the device to run, etc. The electronic device can have a display screen and support information browsing, etc., such as a personal mobile terminal such as a mobile phone, a tablet computer, a personal computer, etc. Various other types of applications can also be configured in the electronic device, such as human-computer dialogue applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0058] The server 100 can include a server providing various services, for example, a server providing a communication service for a plurality of clients, a server for background training providing support for a model used on a client, a server processing data sent by a client, and the like. It should be noted that the server 100 can be implemented as a distributed server cluster composed of multiple servers, or as a single server. The server can also be a server of a distributed system, or a server combined with a blockchain. The server can also be a cloud server of a cloud service, a cloud database, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms, and the like basic cloud computing services, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0059] It should be noted that the data synchronization method provided in the embodiments of the present specification is generally executed by the server 100, but in other embodiments of the present specification, the client 300 can also have similar functions as the server, so as to execute the data synchronization method provided in the embodiments of the present specification. In other embodiments, the data synchronization method provided in the embodiments of the present specification can also be executed by the client 300 and the server 100 together.

[0060] Referring to Figure 3 , Figure 3 A flowchart of a data synchronization method provided by one embodiment of the present specification is shown, which specifically includes the following steps.

[0061] Step 302: obtaining to-be-synchronized data and first source information of the to-be-synchronized data, wherein the first source information includes a first source library identifier, and the first source library identifier represents a source database of the to-be-synchronized data.

[0062] One or more embodiments of the present specification can include an execution unit in each database, which is used to determine whether to synchronize the to-be-synchronized data included in itself to the database to be synchronized; or the server can include a DTS for data transmission, which is connected to each database respectively, can read data in each database, and drive data synchronization transmission between each database.

[0063] Specifically, the data to be synchronized refers to data to be synchronized from a current database to a target database, and the data to be synchronized can be an initialization object, that is, a synchronization object specified when a data synchronization link is established, and the data to be synchronized can be library table information in the database or pure data without library table format in the library table information. The first source information refers to a first source library identifier of a source database including a source of the data to be synchronized, and the source of the data to be synchronized can be determined through the first source information. The first source library identifier is stored in the current database where the data to be synchronized is currently located. The first source information can also include tool information of a data synchronization tool, and the data synchronization tool is a synchronization tool for synchronizing the data to be synchronized from the source database to the current database, for example, the tool information is a thread number. The source database refers to a database for synchronizing the data to be synchronized to the current database, that is, the source database of the data to be synchronized compared with the current database, for example, data 1 is synchronized from a database A to a database B, and when there is a data synchronization demand for the data 1 in the database B, the database A can be determined as the source database of the data 1, and the database B can be determined as the current database of the data 1; for another example, data 2 is written from the outside to a database C, and when there is a data synchronization demand for the data 2 in the database C, the database C can be determined as both the source database and the current database of the data 2; for another example, there is a data synchronization demand for inventory data in a database D, and the database D can be determined as both the source database and the current database of the inventory data.

[0064] The scheme is based on an open-source AliSQL, supports a function of customizing a connection_id, and the connection_id is a thread Id in a Bin log corresponding to the MySQL, wherein the AliSQL is a deeply customized independent MySQL branch, and in addition to all functions of a community edition, the AliSQL provides many functions similar to an enterprise edition of the MySQL, such as enterprise-level backup and recovery, a thread pool, parallel query and the like.

[0065] The thread identifier is used for marking a change transaction included in the Bin log, each change transaction corresponds to a change event, and the thread identifier is used for uniquely identifying each thread on hardware and software, and is linearly increased.

[0066] According to actual needs, when there is a demand for synchronizing data in the database, the data identifier can be read from the Bin log, and the data to be synchronized is determined based on the data identifier, so as to synchronize the data to be synchronized.

[0067] The incremental synchronization module performs the data writing task by a transmission thread, and the transmission thread corresponds to a thread identifier, which can be represented by thread Id, and the range of thread Id is 0xffff0000-0xffffffff, wherein 0x is the prefix of hexadecimal data, and the high four bits ffff are inherent form data. Taking thread Id=0xffffXXYY as an example, XX represents a unique database identifier (DBNo, Data Base No) of a database, which can distinguish 256 databases, and YY represents the thread number of the transmission thread when the incremental synchronization module performs data writing by the transmission thread, and the range of the thread number is 0-255. The data writing task performed by the incremental synchronization module refers to writing data from the first database to the second database by the transmission thread.

[0068] The MySQL automatically generates the thread identifier of the transmission thread when writing the project writing data, which can be represented by thread Id, and the range of thread Id is 0x00000000-0xffff0000, wherein 0x is the prefix of hexadecimal data, and the low four bits 0000 are inherent form data. The project writing data refers to the data written from the outside to the database, rather than the data synchronized from other databases, for example, the project writing data is the data written into the database by the client through the write request.

[0069] Optionally, the module for performing the project writing data can be a project data writing module.

[0070] The source information includes a source library identifier, which can be extracted from the thread identifier, for example, the thread identifier 1 is represented by thread Id, and the thread Id is within the data range of 0xffff0000-0xffffffff, it is determined that the data corresponding to the thread identifier 1 is the data written by the data synchronization module, and the third and fourth bits are determined as the source library identifier, that is, the DBNo in the thread Id is determined as the source library identifier in the source information; or it can be inferred based on the thread identifier, for example, the thread identifier 2 is represented by thread Id, and the thread Id is within the data range of 0x00000000-0xffff0000, it is determined that the incremental data corresponding to the thread identifier 2 is the project writing data, and the library identifier of the database where the project writing data is currently located is determined as the source library identifier.

[0071] Optionally, the source information can also include tool information of a data synchronization tool, and it is determined that the source information includes the source library identifier and the tool information, and correspondingly, the tool information can also be extracted from the thread identifier, for example, the thread identifier is represented by thread Id, and the thread number in the thread Id is the tool information.

[0072] Optionally, the thread identifier can also be directly determined as the source information, and the source database identifier is read from the thread identifier when there is a data synchronization requirement, to determine whether to synchronize the to-be-synchronized data to the target database.

[0073] Optionally, there are many implementation manners for triggering the acquisition of the to-be-synchronized data and the first source information of the to-be-synchronized data, which are determined according to actual conditions, and the present specification does not limit here.

[0074] In a possible implementation manner of the present specification, the implementation manner for triggering the acquisition of the to-be-synchronized data and the first source information of the to-be-synchronized data can be receiving a data synchronization request input by the front end, acquiring the to-be-synchronized data in response to the data synchronization request, and acquiring the first source information of the to-be-synchronized data.

[0075] The data synchronization request can be a request generated by the front end based on a data synchronization page and a data selection operation for the to-be-synchronized data.

[0076] In another possible implementation manner of the present specification, triggering the acquisition of the to-be-synchronized data and the first source information of the to-be-synchronized data can also be receiving a timing data synchronization item set in advance by the front end, and triggering the acquisition of the to-be-synchronized data and the first source information of the to-be-synchronized data when it is detected that the timing condition is reached.

[0077] The timing can be timing for time, or timing for change detection in the database, and a change operation for the database is detected as the timing condition being reached.

[0078] In yet another possible implementation manner of the present specification, the data synchronization request sent by the front end can be received, the data synchronization request carries a data synchronization type, and the server acquires the to-be-synchronized data and the first source information of the to-be-synchronized data based on the data synchronization type.

[0079] The data synchronization type can include full data synchronization and incremental data synchronization, and the full data synchronization is divided into inventory data synchronization and incremental data synchronization based on the synchronization manner, that is, based on the requirement of full data synchronization, the inventory data is synchronized first, and then the incremental data is synchronized, to complete the synchronization of full data.

[0080] By acquiring the to-be-synchronized data and the first source information of the to-be-synchronized data, it can be determined whether to synchronize the to-be-synchronized data to the target database based on the first source information, to improve the efficiency of data synchronization.

[0081] Step 304: Based on the first source database identifier, it is determined whether the target database to which the to-be-synchronized data is to be synchronized is the source database.

[0082] The first source database identifier characterizes a source database of the database to be synchronized. Through the first source database identifier, the source database corresponding to the data to be synchronized can be determined.

[0083] Based on the first source database identifier, whether the target database to which the data to be synchronized is synchronized is the implementation mode of the source database. The target database can be determined by comparing the first source database identifier and the target database identifier.

[0084] The comparison between the first source database identifier and the target database identifier can be the calculation of the text similarity between them, and can also be the calculation of the semantic feature similarity between them.

[0085] By comparing the source database and the target database, it is determined whether the data to be synchronized is synchronized to the source database. By identifying and determining whether to synchronize data based on the identification result, it is ensured that the data synchronization result will not cause the data contained in the source database to be repeated, the data space occupation in the source database is reduced, and the data storage space is increased.

[0086] Step 306: If not, synchronize the data to be synchronized to the target database.

[0087] Based on the first source information, it is identified whether the target database to which the data to be synchronized is synchronized is the source database. If not, it is determined that the target database and the source database are different databases, and the data to be synchronized can be synchronized to the target database. If yes, it is determined that the target database and the source database are the same database. In the case that the source database already contains the data to be synchronized, the data to be synchronized is prohibited to be synchronized to the target database.

[0088] The implementation mode of synchronizing the data to be synchronized to the target database can be to start the data synchronization tool, call the data synchronization tool, and synchronize the data to be synchronized to the target database.

[0089] Optionally, there are many implementation modes of starting the data synchronization tool, which are not limited in the specification, and are determined according to the actual situation.

[0090] In a possible implementation mode of the specification, starting the data synchronization tool can be determined based on the tool information of the data synchronization tool contained in the first source information and started.

[0091] Exemplarily, the tool information of the data synchronization tool contained in the first source information is a thread number. Based on the thread number, the data synchronization tool corresponding to the thread number is determined as the first transmission thread, and the first transmission thread is started.

[0092] In another possible implementation of the present specification, the starting of the data synchronization tool can also be based on the data amount of the to-be-synchronized data and the synchronization unit of the data synchronization tool corresponding tool to determine and start.

[0093] Exemplarily, the data amount of the to-be-synchronized data is 10, the data amount of a data synchronization tool, that is, a thread, that can be transmitted at a time is 10, the data synchronization tool required for data transmission of the to-be-synchronized data is determined to be 10, and then 10 data synchronization tools are started.

[0094] In another possible implementation of the present specification, the starting of the data synchronization tool can also be based on the data synchronization rule between the current database and the target database to determine and start.

[0095] Exemplarily, the data synchronization tool for data synchronization between the current database and the target database is the second transmission thread, and then the second transmission thread is started as the data synchronization tool.

[0096] Optionally, in the implementation of synchronizing the to-be-synchronized data to the target database, the tool information included in the first source information can be acquired, the data synchronization tool is determined based on the tool information, the data synchronization tool is called to synchronize the to-be-synchronized data to the target database, and then the second source information is generated based on the current database after the to-be-synchronized data is synchronized to the target database; or the data synchronization tool is newly started, the second source information is determined for the data synchronization tool, the data synchronization tool is called to synchronize the to-be-synchronized data to the target database, and then the data change log of the target database is updated based on the write operation of the target database when the to-be-synchronized data is written.

[0097] In an optional embodiment of the present specification, the above step of acquiring the to-be-synchronized data and the first source information of the to-be-synchronized data includes the following steps:

[0098] The data synchronization request is received, wherein the data synchronization request carries a data synchronization type;

[0099] The to-be-synchronized data and the first source information of the to-be-synchronized data are acquired based on the data synchronization type.

[0100] Specifically, the data synchronization request refers to a data synchronization request initiated by the front end based on the data synchronization demand between databases, and the data synchronization request can carry a data synchronization type. The data synchronization type refers to the type of data synchronization, and the data synchronization type is related to the data amount of data synchronization. For example, the data synchronization is to synchronize all data in the database, and the corresponding data synchronization type is full-data synchronization. The data synchronization is to synchronize the data written after a certain time in the database, and the corresponding data synchronization type is incremental data synchronization.

[0101] The received data synchronization request can be a data synchronization request initiated by a front end. The front end can initiate the data synchronization request in the following ways: in a data synchronization page, for a selected operation of a data synchronization type, and for a selection operation of a current database and a target database. Based on the characteristics that the current database is a master database and the target database is a backup database, a request for data synchronization from the source database to the target database is initiated.

[0102] Based on the data synchronization type, the implementation of obtaining the to-be-synchronized data and the first source information of the to-be-synchronized data is different based on different data synchronization types.

[0103] If the data synchronization type is full data synchronization, the originally stored to-be-synchronized data is obtained from the current database; the library identifier of the current database is taken as the first source library identifier, and based on the first source library identifier, the first source information of the to-be-synchronized data is determined, and after the stock data synchronization ends, incremental data synchronization is performed in the manner of the data synchronization type being an incremental data synchronization type.

[0104] If the data synchronization type is incremental data synchronization, the data change log of the current database is parsed, wherein the data change log records the data change information of the current database, and the data change information includes the data identifier and the source information of the changed data; in the case that there is incremental data in the current database, the incremental data is obtained from the current database as the to-be-synchronized data based on the data identifier of the incremental data; and the first source information of the to-be-synchronized data recorded in the data change log is read.

[0105] The data synchronization type carried in the received data synchronization request makes it possible to obtain the corresponding to-be-synchronized data and the first source information of the to-be-synchronized data based on the different data corresponding to the data synchronization type, ensuring that the data synchronization method can handle a wide range of data synchronization types and ensuring the accuracy of data synchronization.

[0106] In an optional embodiment of the present specification, the above step of obtaining the to-be-synchronized data and the first source information of the to-be-synchronized data includes the following steps:

[0107] The originally stored to-be-synchronized data is obtained from the current database;

[0108] The library identifier of the current database is taken as the first source library identifier, and based on the first source library identifier, the first source information of the to-be-synchronized data is determined.

[0109] Specifically, the current database refers to a database in which the to-be-synchronized data currently exists. The originally stored to-be-synchronized data refers to data at a certain point in time, which represents the amount of data held by the database at the point in time. For example, data written into the database from 23:08:48.340 on January 1 to 23:08:48.340 on January 10 is determined as the amount of data, and data written into the database after 23:08:48.340 on January 10 is determined as incremental data.

[0110] The implementation manner of obtaining the originally stored to-be-synchronized data from the current database can be identifying data in the current database and determining the originally stored data as the to-be-synchronized data.

[0111] In the case where the to-be-synchronized data is the originally stored data, the source of the to-be-synchronized data is determined as the current database when the to-be-synchronized data is synchronized, the library identifier of the current database is determined as the first source library identifier, and the first source information of the to-be-synchronized data is determined based on the first source library identifier.

[0112] Optionally, in the case where the to-be-synchronized data is the originally stored data in the current database, the source database of the to-be-synchronized data is determined to be different from the target database to which the to-be-synchronized data is to be synchronized, it is determined that the to-be-synchronized data is to be synchronized to the target database, and therefore the data synchronization tool is started. The first source information of the to-be-synchronized data is determined based on the tool information of the data synchronization tool and the first source library identifier.

[0113] The source information is automatically generated by the server and is a function of Jave in other programming languages. Jave is a general term for object-oriented programming languages and Java platforms.

[0114] The originally stored to-be-synchronized data is obtained from the current database, the library identifier of the current database is determined as the first source library identifier, and the first source information of the to-be-synchronized data is determined based on the first source library identifier. In the case where the to-be-synchronized data is the originally stored data in the current database, the source database of the to-be-synchronized data is determined to be non-existent, the current database is determined as the source database of the to-be-synchronized data, that is, the library identifier of the current database is determined as the first source library identifier, and the first source information is determined. When the to-be-synchronized data is synchronized, the corresponding first source information exists, the data synchronization is ensured to be performed, and the efficiency of the data synchronization is improved.

[0115] In an optional embodiment of the present specification, the first source information further includes tool information of a data synchronization tool; and before the step of determining the first source information of the to-be-synchronized data based on the first source library identifier, the following step is further included:

[0116] Starting the data synchronization tool, obtaining tool information of the data synchronization tool;

[0117] Based on the first source library identifier, determine the first source information of the data to be synchronized, including:

[0118] Based on the first source library identifier and the tool information, determine the first source information of the data to be synchronized;

[0119] Synchronize the data to be synchronized to the target database, including:

[0120] Based on the tool information, call the data synchronization tool to synchronize the data to be synchronized to the target database.

[0121] Specifically, the data synchronization tool refers to a tool used for data synchronization between different databases. The data synchronization tool can be a transmission thread, and the transmission thread can perform data synchronization in a single-threaded or multi-threaded parallel manner. The tool information is used to identify the number of data synchronization tools. For example, the tool information can be a thread number. If the thread number is 10, then the corresponding data synchronization tool has 10 threads in parallel.

[0122] The implementation of starting the data synchronization tool and obtaining the tool information of the data synchronization tool can be based on the data synchronization rule of writing the data to be synchronized into the current database to determine the data synchronization tool and start the data synchronization tool, based on the thread number of the data synchronization tool to determine the tool information of the data synchronization tool. It can also be based on the data volume of the data to be synchronized to determine the number of data synchronization tools and start the corresponding number of data synchronization tools, and determine the number as the tool information of the data synchronization tool.

[0123] Combining the first source library identifier and the tool information to determine the first source information of the data to be synchronized, wherein the combination can be filling the first source library identifier and the tool information into the identification template according to the preset source information identification template to obtain the first source information of the data to be synchronized. For example, the first source library identifier is 01 and the tool information is 02. The corresponding first source information can be 0102.

[0124] Based on the tool information, the implementation of calling the data synchronization tool to synchronize the data to be synchronized to the target database can be based on the tool information to determine the data synchronization tool, call the data synchronization tool, and send the data to be synchronized to the target database.

[0125] Based on the tool information, the implementation of determining the data synchronization tool can be based on the tool information to construct the data synchronization tool corresponding to the tool information.

[0126] For example, the tool information is 6, and based on the tool information 6, a transmission thread containing the tool information 6 is constructed.

[0127] The data information of the data synchronization tool is also included in the first source information. When synchronizing the to-be-synchronized data to the target database, the data synchronization tool is called based on the tool information in the first source information to synchronize the to-be-synchronized data to the target database, so that the data synchronization is performed based on the data synchronization tool corresponding to the tool information in the first source information, and the accuracy of the correspondence between the data synchronization and the first source information is ensured.

[0128] In an optional embodiment of the present specification, the step of obtaining the to-be-synchronized data and the first source information of the to-be-synchronized data includes the following steps:

[0129] parsing a data change log of the current database, wherein the data change log records data change information of the current database, and the data change information includes data identifiers and source information of changed data;

[0130] In a case where it is parsed that there is incremental data in the current database, the incremental data is obtained from the current database as the to-be-synchronized data based on the data identifiers of the incremental data.

[0131] reading the first source information of the to-be-synchronized data recorded in the data change log.

[0132] Specifically, the data change log refers to a log recording a change operation of a database, and the data change log records data change information of the database, and the data change information includes data identifiers and source information of changed data.

[0133] parsing a data change log of the current database, determining whether there is incremental data, and in a case where it is parsed that there is incremental data in the current database, obtaining the incremental data from the current database as the to-be-synchronized data based on the data identifiers of the incremental data.

[0134] parsing a data change log of the current database, obtaining an incremental change transaction, extracting data identifiers and first source information in the incremental change transaction, and obtaining data corresponding to the data identifiers from the current database based on the data identifiers, and taking the obtained data as the to-be-synchronized data.

[0135] By parsing the data change log of the current database, the incremental data is obtained, and the first source information corresponding to the incremental data is read in the data change log, which facilitates subsequent identification and judgment of data synchronization based on the incremental data and the first source information of the incremental data, so as to realize data synchronization of the incremental data as the to-be-synchronized data.

[0136] In an optional embodiment of the present specification, the first source information further includes tool information of a data synchronization tool; and the step of synchronizing the to-be-synchronized data to the target database includes the following steps:

[0137] Based on the tool information, a data synchronization tool is called to synchronize the to-be-synchronized data to the target database.

[0138] Based on the tool information, a data synchronization tool is called to synchronize the to-be-synchronized data to the target database. The implementation manner can be that based on the tool information, a data synchronization tool is determined, and the data synchronization tool is taken as an execution unit of data synchronization, and the to-be-synchronized data is sent to the target database.

[0139] Exemplarily, the tool information is a thread number, based on the thread number, a transmission thread for synchronizing the to-be-synchronized data is determined, and the to-be-synchronized data is synchronized to the target database by using the transmission thread.

[0140] Based on the tool information, a data synchronization tool is called to synchronize the to-be-synchronized data to the target database, based on the data synchronization tool, the accuracy of data synchronization is ensured, and the accuracy of synchronizing the to-be-synchronized data to the target database is ensured.

[0141] In an optional embodiment of the present specification, after the step of synchronizing the to-be-synchronized data to the target database, the following steps are further included:

[0142] Based on the library identifier of the current database, second source information of the to-be-synchronized data is determined, wherein the second source information is used for writing into a data change log of the target database.

[0143] Based on the library identifier of the current database, the implementation manner of determining the second source information of the to-be-synchronized data can be that the library identifier of the current database is determined as a second source library identifier, based on the second source library identifier, the second source information of the to-be-synchronized data is determined; or the library identifier of the current data is read from the data synchronization tool, and the library identifier is determined as the second source library identifier, based on the second source library identifier, the second source information of the to-be-synchronized data is determined.

[0144] Exemplarily, the data synchronization tool is a transmission thread, the transmission thread at least includes the library identifier of the current database, after the to-be-synchronized data is synchronized to the target database by using the transmission thread, the library identifier of the current database is read from the transmission thread, the library identifier of the current database is determined as the second source library identifier, and based on the second source library identifier, the second source information of the to-be-synchronized data is determined.

[0145] Based on the second source library identifier included in the second source information, the source database of the to-be-synchronized data relative to the target database can be determined.

[0146] Optionally, based on the second source library identifier, the implementation manner of determining the second source information of the to-be-synchronized data can be that tool information used for synchronizing the to-be-synchronized data to the target database is obtained, based on the tool information and the second source library identifier, the second source information of the to-be-synchronized data is determined.

[0147] Optionally, the data change log can also include a correspondence relationship between the synchronization tool identifier and the data identifier, such as writing data with a data identifier of yyy by a transmission thread with a thread identifier of xxx, and recording the correspondence relationship between the thread identifier xxx and the data identifier yyy in the data change log. When there is a need for data synchronization, the source information is read from the thread identifier recorded in the data change log to obtain the source library identifier, and the source library identifier is identified to determine whether to perform data synchronization.

[0148] When synchronizing the to-be-synchronized data to the target database, the data change log of the target database will automatically write data change information based on the synchronization operation, where the writing of the data change information can be writing the data identifier of the to-be-synchronized data and the second source information, and when writing, the correspondence relationship between the data identifier of the to-be-synchronized data and the second source information is implicitly established. And subsequently, the second source information is read from the data change log based on the data identifier of the to-be-synchronized data, and the read second source information is determined as the source information of the source database that determines the source of the to-be-synchronized data.

[0149] Wherein, the data change log of the target database based on the synchronization operation will automatically write data change information based on the synchronization operation, which is called Bin log event.

[0150] In an optional embodiment of the present specification, after the above step of identifying whether the target database to which the to-be-synchronized data is to be synchronized is the source database based on the first source library identifier, the following steps are further included:

[0151] If yes, the to-be-synchronized data is prohibited from being synchronized to the target database.

[0152] In the case where it is determined that the target database is the same as the source database, it is determined that the to-be-synchronized data exists in the target database, and the to-be-synchronized data is prohibited from being synchronized to the target database.

[0153] Optionally, the implementation of prohibiting the to-be-synchronized data from being synchronized to the target database can be discarding the synchronization change transaction of synchronizing the to-be-synchronized data to the target database.

[0154] In the case where it is determined that the target database to which the to-be-synchronized data is to be synchronized is the same as the source database, after data synchronization, the target database has duplicate to-be-synchronized data, and the to-be-synchronized data is prohibited from being synchronized to the target database, so that the source database does not contain duplicate data due to data synchronization.

[0155] One or more embodiments of the present specification not only support bidirectional data synchronization, but also support star-shaped data synchronization, ring-shaped data synchronization and the like.

[0156] The target database can be a backup database of the current database, or the target database and the current database can be backup databases of each other. The data to be synchronized in the target database can also be synchronized to the current database by the data synchronization method described above, that is, the data synchronization method is used to make the data in the current database consistent with the data in the target database.

[0157] Referring to Figure 4 , Figure 4 A flowchart of a data access method provided by an embodiment of the present specification is shown, and the method specifically includes the following steps.

[0158] Step 402: receiving a data access request for a current database, wherein the data access request carries data information of data to be accessed.

[0159] When there is a demand for data access in the front end, a data access request is sent to the server, so that the server receives the data access request and performs data access based on the data access request.

[0160] Specifically, the data information refers to the data information of the data to be accessed, and the data information is used to identify the data to be accessed. The data information can be a data identifier, a data index address, etc.

[0161] The implementation of receiving a data access request for a current database can be receiving operation information of a selected operation, a data identifier input operation, etc. for data to be accessed in the current database.

[0162] Step 404: in the case of a current database access failure, reading the data to be accessed from a backup database of the current database based on the data information, wherein the backup database synchronizes data from the current database in advance based on a data synchronization method.

[0163] After receiving the data access request, the current database is accessed based on the data access request. In the case of determining that the current database has an access failure, a backup database of the current database is found, and the data to be accessed is read from the backup database based on the data information.

[0164] After reading the data to be accessed, the data to be accessed is returned to the front end.

[0165] Optionally, in the case of returning the read data to be accessed to the front end, a prompt information that the current database has an access failure and the data to be accessed is read from the backup database can be displayed on the front end.

[0166] With the scheme of the embodiments of the present specification, when a data access request is received, in a case where it is determined that the current database access fails, the data to be accessed is read from the backup database based on the data information of the data to be accessed carried in the data access request to implement data access, which improves the efficiency of data access, that is, the data to be synchronized is efficiently synchronized to the target database in a data synchronization manner, so that when there is a data access request for the current database that fails, the data to be accessed can be read from the backup database of the current database, and the efficiency of data access is improved.

[0167] The following describes the embodiments of the present specification in conjunction with the accompanying Figure 5a and Figure 5b The data synchronization method provided by the present specification is further described taking the application of the data synchronization method in inventory data synchronization as an example. Among them, Figure 5a Fig. 1 shows a processing process flow diagram of a data synchronization method provided by an embodiment of the present specification, Figure 5b Fig. 1 shows a data synchronization schematic diagram between databases in a data synchronization method provided by an embodiment of the present specification, which specifically includes the following steps.

[0168] The embodiments of the present specification are applied to a DTS for data transmission, and the execution unit is connected to database A, database B and database C respectively, can read the data in each database, and drive the data synchronization transmission between each database.

[0169] The star type data synchronization is used for data transmission between the following embodiments.

[0170] The inventory data 1 is pre-stored in the database A, the inventory data 2 is pre-stored in the database B, and the inventory data 3 is pre-stored in the database C, wherein the inventory data is the data stored before the target time point, for example, in the present embodiment, the current time is 23:59:48.101 milliseconds on March 15, and the target time point is 23:59:48.101 milliseconds on March 10, and the data written into the database before 23:59:48.101 milliseconds on March 10 is the inventory data.

[0171] Step 502: Obtain the original inventory data c1 stored in the database A and the source information s1 of the inventory data c1, determine that the data source of the inventory data c1 is the database A based on the source information s1, identify that the database B to which the inventory data c1 is to be synchronized is different from the source database A, determine the transmission thread t1, and set the thread identification d1 for the transmission thread t1, wherein the thread identification d1 includes the source information s1.

[0172] For example, the data identifier of the inventory data c1 is id=1000, the source information s1 is 01, and 01 is the database identifier of database A; the thread identifier d1 can be represented by thread Id, for example, thread Id=0xffff0101, where 0x is the prefix of hexadecimal, 01 in the third and fourth positions is the database identifier of database A, 01 in the first and second positions is the thread number of the transmission thread t1 for data transmission, and the high four positions ffff are inherent form data, and the source information s1 is the database identifier 01 of database A.

[0173] The implementation of obtaining the inventory data c1 originally stored in the database A and the source information s1 of the inventory data c1 can be obtaining the inventory data c1 originally stored in the database A, and determining the database identifier of the database A as the source database identifier, and determining the source information s1 based on the source database identifier.

[0174] Step 504: Based on the transmission thread t1, the inventory data c1 in the database A is synchronized to the database B, the database B stores the inventory data c1, and the Bin log of the database B is automatically updated based on the write operation of the inventory data c1. The Bin log after updating records at least the correspondence between the data identifier of the inventory data c1 and the source information s1.

[0175] Optionally, the Bin log in the database B can also record the correspondence between the data identifier of the inventory data c1 and the thread identifier d1. When the inventory data c1 in the database B is synchronized as the to-be-synchronized data, the thread identifier d1 can be parsed from the Bin log, the data range where the data in the thread identifier d1 is located is identified, the inventory data c1 is determined as the incremental data based on DTS writing, the source database identifier is read from the third and fourth positions of the thread identifier, and the source database identifier is determined as the source information s1.

[0176] The above steps 502 to 504 refer to the data synchronization process of synchronizing the inventory data in the database A to the database B in T1 in Figure 5b

[0177] Step 506: Obtain the inventory data c2 originally stored in the database B and the source information s2 of the inventory data c2, determine that the data source of the inventory data c2 is the database B based on the source information s2, identify that the database C to which the inventory data c2 is to be synchronized is different from the source database B, determine the transmission thread t2, and set the thread identifier d4 for the transmission thread t2, where the thread identifier d4 includes the source information s2.

[0178] ​For example, the data identifier of the inventory data c2 is id=2000, the source information s2 is 02, and the thread identifier d4 can be represented by thread Id, for example, thread Id=0xffff0201, where 0x is the prefix of hexadecimal, 02 is the library identifier of the database B, 01 is the thread number of the data transmission performed by the transmission thread t2, and the high four bits ffff are inherent form data, where the source information s2 is the library identifier 02 of the database B.

[0179] Step 508: Based on the transmission thread t2, the inventory data c2 in the database B is synchronized to the database C, the database C stores the inventory data c2, and the Bin log of the database C is automatically updated based on the write operation of the inventory data c2. The updated Bin long at least records the correspondence between the data identifier of the inventory data c2 and the source information s2.

[0180] The above steps 506 to 508 refer to the data synchronization process of synchronizing the inventory data in the database B to the database C in T2 in Figure 5b .

[0181] Step 510: Obtain the inventory data c3 originally stored in the database C and the source information s3 of the inventory data c3, determine that the data source of the inventory data c3 is the database C based on the source information s3, identify that the database B to which the inventory data c3 is to be synchronized is different from the source database C, determine the transmission thread t3, and set the thread identifier d3 for the transmission thread t3, where the thread identifier d3 includes the source information s3.

[0182] For example, the data identifier of the inventory data c3 is id=3000, the source information s3 is 03, and the thread identifier d3 can be represented by thread Id, for example, thread Id=0xffff0301, 0x is the prefix of hexadecimal, 03 is the library identifier of the database C, 01 is the thread number of the data transmission performed by the transmission thread t3, and the high four bits ffff are inherent form data, where the source information s3 is the library identifier 03 of the database C.

[0183] Step 512: Based on the transmission thread t3, the inventory data c3 in the database C is synchronized to the database B, the database B stores the inventory data c3, and the Bin log of the database B is automatically updated based on the write operation of the inventory data c3. The updated Bin long at least records the correspondence between the data identifier of the inventory data c3 and the source information s3.

[0184] The above steps 510 to 512 refer to the data synchronization process of synchronizing the inventory data in the database C to the database B in T3 in Figure 5b .

[0185] Step 514: Obtain the original stored data c2 and source information s2 of stored data c2 in database B. Based on the source information s2, determine that the data source of stored data c2 is database B. Identify that the database A to which stored data c2 is to be synchronized is different from the source database B. Determine the transmission thread t4 and set the thread identifier d4 for transmission thread t4. The thread identifier d4 includes the source information s2.

[0186] For example, source information s2 is the database identifier 02 of database B, and thread identifier d4 can be represented by thread Id, for example, thread Id = 0xffff0201, where 0x is the hexadecimal prefix, 02 is the database identifier of database B, 01 is the thread number of the transmission thread t4 for data transmission, and the high four bits ffff are inherent data. Source information s2 is the database identifier 02 of database B.

[0187] Step 516: Based on the transmission thread t4, synchronize the existing data c2 in database B to database A. Database A stores the existing data c2. The Bin log of database A is automatically updated based on the write operation of the existing data c2. The updated Bin long records at least the correspondence between the data identifier of the existing data c2 and the source information s2.

[0188] For steps 514 to 516 above, please refer to... Figure 5b T4 in the database synchronizes existing data in database B to database A.

[0189] When there is a need to synchronize all data in the database, based on steps 502 to 516 above, the existing data in each database is synchronized. Then, based on the incremental data synchronization method in steps 602 to 614 in Figure 6 below, the data written in each database after the target time point is synchronized to complete the synchronization of all data.

[0190] The following is in conjunction with the appendix Figure 6a and Figure 6b Taking the data synchronization method provided in this specification in the application of incremental data synchronization as an example, the data synchronization method will be further explained. Among other things, Figure 6a This specification illustrates a flowchart of another data synchronization method provided in one embodiment. Figure 6b This diagram illustrates data synchronization between databases in another data synchronization method provided in one embodiment of this specification, specifically including the following steps.

[0191] The following embodiments use star-based data synchronization for data synchronization.

[0192] The database D pre-stores the incremental data i1 (id=100), the database E pre-stores the incremental data i2 (id=200), and the database F pre-stores the incremental data i3 (id=300). The incremental data is data written after a target time point. In this embodiment, the target time point is 23:59:48.101 on April 15, 2024. The data written in the database after 23:59:48.101 on April 15, 2024 is determined as the incremental data. The incremental data i1, the incremental data i2, and the incremental data i3 are all project write data written in the database after 23:59:48.101 on April 15, 2024. The data identifier of the incremental data and the corresponding source information are recorded in the Bin log of the corresponding database.

[0193] The thread identifier can be represented by a thread Id. If the thread Id is within the data range of 0x00000000-ffff0000, it can be determined that the incremental data corresponding to the thread Id is project write data, and the database currently storing the incremental data is determined as the source database, that is, the data source (source information). For example, the thread identifier thread Id corresponding to the incremental data DD is 0x22330000. By identifying the range of the thread Id, it is determined that the incremental data DD is project write data. The database currently storing the incremental data DD is determined as the source database, that is, the current database is determined as the data source.

[0194] If the thread Id is within the data range of 0xffff0000-0xffffffff, it can be determined that the incremental data corresponding to the thread Id is data synchronized from another database. The third and fourth bits in the low four bits ffff are read to obtain the source database identifier, and the source database is determined through the source database identifier. The source database is the data source (source information). For example, the thread identifier thread Id corresponding to the incremental data EE is 0xffff0813. By identifying the range of the thread Id, it is determined that the incremental data EE is data synchronized from the database 08. The source database of the incremental data EE is determined as the database corresponding to 08, that is, the database corresponding to the library identifier 08 is determined as the data source.

[0195] In step 602, the Bin log in the database D is parsed to determine the incremental data i1 and the source information s4 of the incremental data i1. Based on the source information s4, it is determined that the data source of the incremental data i1 is the database D. It is identified that the database E to which the incremental data i1 is to be synchronized is different from the source database D. The transmission thread t5 is determined, and the thread identifier d5 is set for the transmission thread t5. The thread identifier d5 includes the source information s4.

[0196] For example, the data identifier of the incremental data i 1 is id = 100, the source information s4 is the database identifier 04 of the database D, and the thread identifier d5 can be represented by thread Id, for example, thread Id = 0xffff0401, 0x is the prefix of hexadecimal, 04 is the database identifier of the database D, 01 is the thread number of the transmission thread t5 for data transmission, the high four bits ffff are inherent form data, and the source information s5 is 04 in the thread identifier d5, that is, the database identifier 04 of the database D.

[0197] The implementation of the incremental data i 1 and the source information s4 of the incremental data i 1 can be based on the target time point, determining whether there is data written after the target time point in the Bin log of the database D, determining that there is a data with the data identifier of the incremental data i 1, and then determining the incremental data i 1, and reading the source information corresponding to the data identifier of the incremental data i 1 from the Bin log to obtain the source information s4.

[0198] Optionally, the Ein long in the database D can also store the data identifier of the incremental data i 1 and the thread identifier corresponding to the source information s4 of the incremental data i 1. The thread identifier corresponding to the source information s4 obtained from the Bin log is read to obtain the source information s4. For example, the thread identifier corresponding to the source information s4 is thread Id = 0x04040000. It is found through reading that the thread identifier is in the range of 0x00000000-0xffff0000, and then it can be determined that the incremental data i 1 is project write data, the database where the project write data is currently located is determined as the source database, and the database identifier of the source database is determined as the source information s4.

[0199] Step 604: Based on the transmission thread 5, the incremental data i 1 in the database D is synchronized to the database E, the database E stores the incremental data i 1, the Bin log of the database E is automatically updated based on the write operation of the incremental data i 1, and the updated Bin long at least records the corresponding relationship between the data identifier of the incremental data i 1 and the source information s4.

[0200] The above steps 602 to 604 refer to the data synchronization process of synchronizing the incremental data in the database D to the database E in S1 in Figure 6b

[0201] ​Step 606: parsing the Bin log in the database E, determining the incremental data i 1 and the incremental data i 2, and the source information s 4 and the source information s 5 corresponding to the incremental data i 1 and the incremental data i 2 respectively, determining that the data source of the incremental data i 1 is the database D and the data source of the incremental data i 2 is the database E based on the source information s 4 and the source information s 5, identifying that the database F to which the incremental data i 1 is to be synchronized is different from the source database D and the database F to which the incremental data i 2 is to be synchronized is different from the source database E, determining a transmission thread 6, and setting a thread identifier 6 for the transmission thread 6, wherein the thread identifier 6 includes the source information s 5.

[0202] For example, the data identifier of the incremental data i 2 is id=200, the source information s 10 is the database identifier 05 of the database E, and the thread identifier 6 can be represented by thread Id, for example, thread Id=0xffff0503, 0x is the prefix of hexadecimal, 05 is the database identifier of the database E, 03 is the thread number of the transmission thread 6 for data transmission, and the low four bits 0000 are inherent form data. The source information s 10 is the database identifier 05 of the database E in the thread identifier 6.

[0203] Step 608: based on the transmission thread 6, synchronizing the incremental data i 1 and the incremental data i 2 to the database F, the database F storing the incremental data i 1 and the incremental data i 2, the Bin log of the database F being automatically updated based on the write operation of the incremental data i 1 and the incremental data i 2, and the updated Bin log at least recording the correspondence between the data identifier of the incremental data i 1 and the incremental data i 2 and the source information s 5.

[0204] The above steps 606 to 608 refer to the data synchronization process of synchronizing the incremental data in the database E to the database F in S2 in Figure 6b

[0205] Step 610: parsing the Bin log in the database F, determining the incremental data i 1, the incremental data i 2 and the incremental data i 3, and the source information s 5 of the incremental data i 1 and the incremental data i 2 and the source information s 6 of the incremental data i 3, determining that the data source of the incremental data i 1 and the incremental data i 2 is the database E and the data source of the incremental data i 3 is the database F based on the source information s 5 and the source information s 6, identifying that the database E to which the incremental data i 1 and the incremental data i 2 are to be synchronized is the same as the source database E and the database E to which the incremental data i 3 is to be synchronized is different from the source database F, determining a transmission thread 7, and setting a thread identifier 7 for the transmission thread 7, wherein the thread identifier 7 includes the source information s 6.

[0206] ​For example, the data identifier of the incremental data i3 is id=300, the source information s6 is the database identifier 06 of the database F, and the thread identifier 7 can be represented by thread Id, for example, thread Id=0xffff0602, 0x is the prefix of hexadecimal, 06 is the database identifier of the database F, 02 is the thread number of the transmission thread 7 for data transmission, and the low four bits 0000 are inherent form data. The source information s6 is the database identifier 06 of the database F in the thread identifier 7.

[0207] Step 612: Based on the transmission thread 8, the incremental data i3 in the database F is synchronized to the database E, the database E stores the incremental data i3, and the Bin log of the database E is automatically updated based on the write operation of the incremental data i3. The updated Bin log records at least the correspondence between the data identifier of the incremental data i3 and the source information s6.

[0208] The above steps 610 to 612 refer to the data synchronization process of synchronizing the incremental data in the database F to the database E in S3 in Figure 6b

[0209] Step 614: Analyzing the Bin log in the database E, determining the incremental data i1, the incremental data i2, and the incremental data i3, and the source information s4, the source information s5, and the source information s6 corresponding to the incremental data i1, the incremental data i2, and the incremental data i3, respectively, based on the source information s4, the source information s5, and the source information s6, determining that the data source of the incremental data i1 is the database D, the data source of the incremental data i2 is the database E, and the data source of the incremental data i3 is the database F, identifying that the database D to which the incremental data i1 is to be synchronized is the same as the source database D, the database D to which the incremental data i2 is to be synchronized is different from the source database E, and the database D to which the incremental data i3 is to be synchronized is different from the source database F, determining the transmission thread 8, and setting the thread identifier 8 for the transmission thread 8, wherein the thread identifier 8 includes the source information s5.

[0210] For example, the thread identifier 8 can be represented by thread Id, for example, thread Id=0xffff0602, 0x is the prefix of hexadecimal, 06 is the database identifier of the database F, 02 is the thread number of the transmission thread 7 for data transmission, and the low four bits 0000 are inherent form data. The source information s6 is the database identifier 06 of the database F in the thread identifier 7.

[0211] ​Step 616: based on the transmission thread 8, the incremental data i2 and the incremental data i3 are synchronized to the database D, the database D stores the incremental data i2 and the incremental data i3, the Bin log of the database D is automatically updated based on the writing operation of the incremental data i2 and the incremental data i3, and the updated Bin log at least records the correspondence between the data identifier of the incremental data i2 and the incremental data i3 and the source information s5.

[0212] The steps 614 to 616 are described in S4 of the above Figure 6b , which is a data synchronization process of synchronizing the incremental data in the database E to the database D.

[0213] By applying the scheme of the embodiments of the present specification, when data synchronization is performed, based on the first source information of the to-be-synchronized data, it is identified whether the target database to which the to-be-synchronized data is synchronized is the source database, and if not, the data synchronization is performed again, so as to avoid the to-be-synchronized data being synchronized to the corresponding source database, so that the phenomenon of data duplication in the source database is avoided, and the efficiency of data synchronization is improved.

[0214] Corresponding to the above method embodiments, the present specification also provides data synchronization device embodiments, Figure 7 which show the structure schematic diagram of a data synchronization device provided by one embodiment of the present specification. As Figure 7 shown, the device comprises:

[0215] The acquisition module 702 is configured to acquire to-be-synchronized data and first source information of the to-be-synchronized data, wherein the first source information comprises a first source database identifier, and the first source database identifier represents a source database from which the to-be-synchronized data comes;

[0216] The identification module 704 is configured to identify, based on the first source database identifier, whether a target database to which the to-be-synchronized data is synchronized is the source database;

[0217] The synchronization module 706 is configured to synchronize the to-be-synchronized data to the target database if not.

[0218] Optionally, the acquisition module 702 is further configured to receive a data synchronization request, wherein the data synchronization request carries a data synchronization type; based on the data synchronization type, the to-be-synchronized data and the first source information of the to-be-synchronized data are acquired.

[0219] Optionally, the acquisition module 702 is further configured to acquire the to-be-synchronized data stored originally from a current database; the database identifier of the current database is taken as the first source database identifier, and based on the first source database identifier, the first source information of the to-be-synchronized data is determined.

[0220] Optionally, the first source information further comprises tool information of the data synchronization tool; the data synchronization apparatus further comprises a starting module configured to start the data synchronization tool, and acquire the tool information of the data synchronization tool; the acquiring module 702 is further configured to determine the first source information of the to-be-synchronized data based on the first source database identifier and the tool information; and the synchronizing module 706 is further configured to invoke the data synchronization tool to synchronize the to-be-synchronized data to the target database based on the tool information.

[0221] Optionally, the acquiring module 702 is further configured to parse a data change log of the current database, wherein the data change log records data change information of the current database, and the data change information comprises a data identifier and source information of changed data; in a case where it is parsed that there is incremental data in the current database, the incremental data is acquired from the current database as the to-be-synchronized data based on a data identifier of the incremental data; and the first source information of the to-be-synchronized data is read from the data change log.

[0222] Optionally, the first source information further comprises tool information of the data synchronization tool; the synchronizing module 706 is further configured to invoke the data synchronization tool to synchronize the to-be-synchronized data to the target database based on the tool information.

[0223] Optionally, the data synchronization apparatus further comprises an updating module configured to determine second source information of the to-be-synchronized data based on a database identifier of the current database, wherein the second source information is used to be written into a data change log of the target database.

[0224] Optionally, the data synchronization apparatus further comprises a prohibiting module configured to prohibit the to-be-synchronized data from being synchronized to the target database if the first source information of the to-be-synchronized data is the same as the second source information of the to-be-synchronized data.

[0225] By applying the scheme of the embodiments of the present specification, when data is synchronized, whether the target database to which the to-be-synchronized data is synchronized is the source database is identified based on the first source information of the to-be-synchronized data, and if not, the data synchronization is performed again, so that the to-be-synchronized data is prevented from being synchronized to the corresponding source database, and the phenomenon of data duplication in the source database is avoided, and the efficiency of data synchronization is improved.

[0226] The above is a schematic scheme of a data synchronization apparatus of the present embodiment. It should be noted that the technical scheme of the data synchronization apparatus belongs to the same concept as the technical scheme of the data synchronization method described above, and the details of the technical scheme of the data synchronization apparatus that are not described in detail can be referred to the description of the technical scheme of the data synchronization method.

[0227] Corresponding to the method embodiments described above, the present specification also provides a data access apparatus embodiment, Figure 8 FIG. 1 shows a structural schematic diagram of a data access apparatus provided by an embodiment of the present specification. As shown in FIG. 1, the data access apparatus comprises a data access module 1001 and a data synchronization module 1002. Figure 8As shown, the apparatus comprises:

[0228] The receiving module 802 is configured to receive a data access request for a current database, wherein the data access request carries data information of data to be accessed;

[0229] The reading module 804 is configured to read, in the case of a current database access failure, the data to be accessed from a backup database of the current database based on the data information, wherein the backup database is based on a data synchronization method to synchronize data from the current database in advance.

[0230] The scheme of the embodiment of the present specification is applied, when the data access request is received, in the case of determining the current database access failure, the data to be accessed is read from the backup database based on the data information of the data to be accessed in the data access request to realize data access, which improves the efficiency of data access, that is, the data to be synchronized is efficiently synchronized to the target database through the data synchronization method, so that when there is a data access request for the current database which fails, the data to be accessed can be read from the backup database of the current database, which improves the efficiency of data access.

[0231] The above is a schematic scheme of a data access device of the embodiment. It should be noted that the technical scheme of the data access device belongs to the same concept as the technical scheme of the data access method described above, and the details of the technical scheme of the data access device which are not described in detail can be referred to the description of the technical scheme of the data access method.

[0232] Figure 9 A structural block diagram of a computing device provided by an embodiment of the present specification is shown. The components of the computing device 900 include but are not limited to a memory 910 and a processor 920. The processor 920 is connected with the memory 910 through a bus 930, and a database 950 is used to save data.

[0233] The computing device 900 also includes an access device 940 that enables the computing device 900 to communicate via one or more networks 960. Examples of such networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of such networks, such as the Internet. The access device 940 can include one or more of any type of network interface (e.g., a network interface controller (NIC)) such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC).

[0234] In one embodiment of the present specification, the above-mentioned components of the computing device 900 and other components not shown in the Figure 9 may be connected to each other, such as through a bus. It should be understood that Figure 9 The computing device structure diagram shown is merely for the purpose of example, and is not a limitation on the scope of the present specification. Those skilled in the art can add or replace other components as needed.

[0235] The computing device 900 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 900 can also be a mobile or stationary server.

[0236] The processor 920 is configured to execute computer program / instructions that implement the steps of the above-mentioned data synchronization method and data access method when the computer program / instructions are executed by the processor.

[0237] The above is a schematic scheme of the computing device of the embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the data synchronization method and the data access method described above belong to the same concept, and the details of the technical scheme of the computing device that are not described in detail can be seen from the description of the technical scheme of the data synchronization method and the data access method.

[0238] An embodiment of the present specification also provides a computer readable storage medium storing computer programs / instructions, which, when executed by a processor, implement the steps of the data synchronization method and the data access method described above.

[0239] The above is a schematic scheme of the computer readable storage medium of the embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the data synchronization method and the data access method described above belong to the same concept, and the details of the technical scheme of the storage medium that are not described in detail can be seen from the description of the technical scheme of the data synchronization method and the data access method.

[0240] An embodiment of the present specification also provides a computer program product comprising computer programs / instructions, which, when executed by a processor, implement the steps of the data synchronization method and the data access method described above.

[0241] The above is a schematic scheme of the computer program product of the embodiment. It should be noted that the technical scheme of the computer program product and the technical scheme of the data synchronization method and the data access method described above belong to the same concept, and the details of the technical scheme of the computer program product that are not described in detail can be seen from the description of the technical scheme of the data synchronization method and the data access method.

[0242] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims can be performed in a different order than the order described in the embodiments and still achieve the desired result. In addition, the processes depicted in the figures do not necessarily require the particular order shown or sequential order to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous or possible.

[0243] The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, software distribution medium, etc. It should be noted that the computer readable medium can include appropriate additions or deletions according to the requirements of patent practice, for example, according to the patent practice in some regions, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0244] It should be noted that, for the foregoing method embodiments, in order to facilitate description, they are all expressed as a combination of a series of actions, but those skilled in the art should know that the embodiments of the present specification are not limited by the order of the described actions, because according to the embodiments of the present specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of the present specification.

[0245] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0246] The preferred embodiments of the present specification disclosed above are only used to help explain the present specification. The alternative embodiments do not describe all the details and limit the invention to the specific embodiments described. Obviously, according to the content of the embodiments of the present specification, many modifications and changes can be made. The present specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of the present specification, so that those skilled in the art can well understand and use the present specification. The present specification is limited by the claims and their entire scope and equivalents.

Claims

1. A data synchronization method, comprising: obtaining to-be-synchronized data and first source information of the to-be-synchronized data, wherein the first source information comprises a first source database identifier, and the first source database identifier represents a source database from which the to-be-synchronized data is derived; identifying, based on the first source database identifier, whether a target database to which the to-be-synchronized data is to be synchronized is the source database; if not, synchronizing the to-be-synchronized data to the target database. 2.The method of claim 1, wherein the obtaining to-be-synchronized data and first source information of the to-be-synchronized data comprises: receiving a data synchronization request, wherein, the data synchronization request carrying a data synchronization type; obtaining to-be-synchronized data and first source information of the to-be-synchronized data based on the data synchronization type. 3.The method of claim 1, wherein the obtaining to-be-synchronized data and first source information of the to-be-synchronized data comprises: obtaining to-be-synchronized data originally stored in a current database; determining the first source information of the to-be-synchronized data based on a database identifier of the current database as the first source database identifier. 4.The method of claim 3, wherein the first source information further comprises tool information of a data synchronization tool; before the determining the first source information of the to-be-synchronized data based on the first source database identifier, the method further comprises: starting the data synchronization tool to obtain the tool information of the data synchronization tool; the determining the first source information of the to-be-synchronized data based on the first source database identifier comprises: determining the first source information of the to-be-synchronized data based on the first source database identifier and the tool information; the synchronizing the to-be-synchronized data to the target database comprises: calling the data synchronization tool to synchronize the to-be-synchronized data to the target database based on the tool information. 5.The method of claim 1, wherein the obtaining to-be-synchronized data and first source information of the to-be-synchronized data comprises: parsing a data change log of a current database, wherein the data change log records data change information of the current database, and the data change information comprises data identifiers of changed data and source information; in a case where there is incremental data in the current database, obtaining the incremental data from the current database as to-be-synchronized data based on the data identifiers of the incremental data; reading the first source information of the to-be-synchronized data recorded in the data change log. 6.The method of claim 5, wherein the first source information further comprises tool information of a data synchronization tool; the synchronizing the to-be-synchronized data to the target database comprises: calling the data synchronization tool to synchronize the to-be-synchronized data to the target database based on the tool information. 7.The method of any one of claims 3-6, after the synchronizing the to-be-synchronized data to the target database, further comprising: determining second source information of the to-be-synchronized data based on a database identifier of the current database, wherein the second source information is used to be written into a data change log of the target database.

8. The method of any one of claims 1-6, after the identifying, based on the first source database identifier, whether the target database to which the data to be synchronized is to be synchronized is the source database, further comprising: if yes, then prohibiting the data to be synchronized from being synchronized to the target database.

9. A data access method, comprising: receiving a data access request for a current database, wherein the data access request carries data information of data to be accessed; in a case where the current database is accessed fails, reading the data to be accessed from a backup database of the current database based on the data information, wherein the backup database is pre-synchronized with the current database based on the data synchronization method of any one of claims 1-7.

10. A data synchronization system, comprising a server and at least two databases, the at least two databases comprising a current database storing data to be synchronized and a target database to which the data to be synchronized is to be synchronized; The server is configured to acquire the to-be-synchronized data and first source information of the to-be-synchronized data from the current database, wherein the first source information comprises a first source database identifier, the first source database identifier representing a source database from which the data to be synchronized is sourced; based on the first source database identifier, identifying whether the target database is the source database; if no, then synchronizing the data to be synchronized to the target database. 11.The system of claim 10, wherein the server is further configured to determine, based on a library identification of the current database, second source information of the data to be synchronized. the second source information is used to write a data change log of the target database.

12. A computing device, comprising: a memory and a processor; the memory is configured to store computer programs / instructions, and the processor is configured to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method of any one of claims 1-9.

13. A computer-readable storage medium storing computer programs / instructions, which, when executed by a processor, implement the steps of the method of any one of claims 1-9.

14. A computer program product comprising computer programs / instructions, which, when executed by a processor, implement the steps of the method of any one of claims 1-9.

Images