Data synchronization method and terminal of internet data center

Abstract

This invention discloses a data synchronization method and terminal for Internet data centers. After the system generates data to be written, regardless of whether the IDC corresponding to the system is a primary IDC or a secondary IDC, the data to be written is directly written to that IDC. If the IDC corresponding to the system is a secondary IDC, the data to be written is written to the primary IDC, and a synchronization operation is triggered to synchronize the data from the primary IDC to all secondary IDCs. By prioritizing the writing of generated data to the local IDC, there is no need to wait for the primary IDC to synchronize the data to the secondary IDC corresponding to the system, as in the prior art. After the data to the secondary IDC corresponding to the system is written, the data can be queried at any time, avoiding the data synchronization delay problem and improving the response time of the data access interface, thereby improving the reliability of data synchronization and the efficiency of data access.

Description

Technical Field

[0001] This invention relates to the field of Internet data center technology, and in particular to a data synchronization method and terminal for Internet data centers. Background Technology

[0002] In modern large-scale systems, the rapid increase in user volume and user behavior places higher demands on server-side systems, requiring them to meet various performance requirements such as high concurrency and high throughput. To meet these performance requirements, developers typically adopt a distributed architecture, deploying the system across multiple IDCs (Internet Data Centers). When a user accesses the system, they will preferentially access the system in the nearest IDC.

[0003] To reduce data conflicts during data synchronization between different data centers (IDCs), a distributed architecture typically employs a master IDC and slave IDCs for the others. The master IDC primarily handles data write operations for various systems and synchronizes data to the other slave IDCs, ultimately achieving data consistency across all IDCs. However, due to inherent latency in data synchronization between different IDCs, delays in synchronizing data from the master IDC to the slave IDCs can lead to data access errors when a system subsequently accesses a slave IDC, ultimately resulting in system errors. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a data synchronization method and terminal for Internet data centers, which can improve the reliability of data synchronization and improve data access efficiency.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0006] A data synchronization method for an Internet data center includes the following steps:

[0007] Retrieve the data to be written generated by the system;

[0008] Write the data to be written to the IDC corresponding to the system;

[0009] If the IDC corresponding to the system is a slave IDC, then the data to be written is written to the master IDC and a synchronization operation is triggered. The synchronization operation synchronizes the data of the master IDC to all slave IDCs.

[0010] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is as follows:

[0011] A data synchronization terminal for an Internet data center includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the following steps:

[0012] Retrieve the data to be written generated by the system;

[0013] Write the data to be written to the IDC corresponding to the system;

[0014] If the IDC corresponding to the system is a slave IDC, then the data to be written is written to the master IDC and a synchronization operation is triggered. The synchronization operation synchronizes the data of the master IDC to all slave IDCs.

[0015] The beneficial effects of this invention are as follows: After the system generates data to be written, regardless of whether the IDC corresponding to the system is a primary IDC or a secondary IDC, the data to be written is directly written to that IDC. If the IDC corresponding to the system is a secondary IDC, the data to be written is written to the primary IDC, and a synchronization operation is triggered to synchronize the data of the primary IDC to all secondary IDCs. By prioritizing the writing of the generated data to the local IDC, there is no need to wait for the primary IDC to synchronize the data to the secondary IDC corresponding to the system, as in the prior art. After the data corresponding to the system is written, the data can be queried at any time, avoiding the data synchronization delay problem and improving the response time of the data access interface, thereby improving the reliability of data synchronization and improving data access efficiency. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the steps of a data synchronization method for an Internet data center according to an embodiment of the present invention.

[0017] Figure 2 This is a schematic diagram of the structure of a data synchronization terminal for an Internet data center according to an embodiment of the present invention. Detailed Implementation

[0018] To explain in detail the technical content, objectives, and effects of the present invention, the following description is provided in conjunction with the embodiments and accompanying drawings.

[0019] Please refer to Figure 1 A data synchronization method for an Internet data center, comprising the following steps:

[0020] Retrieve the data to be written generated by the system;

[0021] Write the data to be written to the IDC corresponding to the system;

[0022] If the IDC corresponding to the system is a slave IDC, then the data to be written is written to the master IDC and a synchronization operation is triggered. The synchronization operation synchronizes the data of the master IDC to all slave IDCs.

[0023] As can be seen from the above description, the beneficial effects of the present invention are as follows: After the system generates data to be written, regardless of whether the IDC corresponding to the system is a primary IDC or a secondary IDC, the data to be written is directly written to that IDC. If the IDC corresponding to the system is a secondary IDC, the data to be written is written to the primary IDC, and a synchronization operation is triggered to synchronize the data of the primary IDC to all secondary IDCs. By prioritizing the writing of the generated data to the local IDC, there is no need to wait for the primary IDC to synchronize the data to the secondary IDC corresponding to the system, as in the prior art. After the data corresponding to the system is written, the data can be queried at any time, avoiding the data synchronization delay problem and improving the response time of the data access interface, thereby improving the reliability of data synchronization and improving data access efficiency.

[0024] Furthermore, writing the data to be written to the IDC corresponding to the system includes:

[0025] Determine the number range of the IDC corresponding to the system;

[0026] Generate the primary key ID of the data to be written based on the number segment;

[0027] Write the primary key ID and the data to be written to the IDC corresponding to the system.

[0028] As described above, different data centers have different number segments. Distinguishing data primary keys by number segment is beneficial for directly determining whether data already exists based on the primary key during data synchronization, which is simple and effective.

[0029] Further, writing the data to be written to the main IDC includes:

[0030] The system generates an asynchronous message from the corresponding IDC based on the primary key ID and the data to be written, and sends the asynchronous message to the consumption queue.

[0031] The primary key ID and the data to be written in the asynchronous message are written to the main IDC through the consumption queue.

[0032] As described above, by generating asynchronous messages from the corresponding IDC based on the primary key ID and the data to be written, and sending the asynchronous messages to the consumption queue, the primary key ID and the data to be written are written to the main IDC by consuming the messages through the consumption queue, thus ensuring processing efficiency.

[0033] Furthermore, the triggering of the synchronization operation, which synchronizes the primary IDC data to all secondary IDCs, includes:

[0034] The primary key ID and the data to be written are sent to all slave IDCs through the primary IDC;

[0035] The slave IDC determines whether the data to be written exists based on the primary key ID. If it does, the data to be written is not written to the slave IDC. If not, the primary key ID and the data to be written are written to the slave IDC.

[0036] As described above, after receiving the data synchronized from the primary IDC, the system determines whether the corresponding data already exists based on the primary key ID. If it does, no data synchronization is required; otherwise, data synchronization is performed, thus improving data synchronization efficiency.

[0037] Furthermore, after writing the data to be written to the IDC corresponding to the system, the process further includes:

[0038] Receive a query request sent by the system, the query request including data to be queried;

[0039] The query request is used to retrieve the data to be queried from the IDC corresponding to the system.

[0040] As described above, after the data to be written is written to the IDC corresponding to the system, the data can be retrieved from the IDC corresponding to the system according to the query request, without waiting for the data synchronization to be completed, thus improving the data access efficiency.

[0041] Please refer to Figure 2 A data synchronization terminal for an Internet data center includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the following steps:

[0042] Retrieve the data to be written generated by the system;

[0043] Write the data to be written to the IDC corresponding to the system;

[0044] If the IDC corresponding to the system is a slave IDC, then the data to be written is written to the master IDC and a synchronization operation is triggered. The synchronization operation synchronizes the data of the master IDC to all slave IDCs.

[0045] As can be seen from the above description, the beneficial effects of the present invention are as follows: After the system generates data to be written, regardless of whether the IDC corresponding to the system is a primary IDC or a secondary IDC, the data to be written is directly written to that IDC. If the IDC corresponding to the system is a secondary IDC, the data to be written is written to the primary IDC, and a synchronization operation is triggered to synchronize the data of the primary IDC to all secondary IDCs. By prioritizing the writing of the generated data to the local IDC, there is no need to wait for the primary IDC to synchronize the data to the secondary IDC corresponding to the system, as in the prior art. After the data corresponding to the system is written, the data can be queried at any time, avoiding the data synchronization delay problem and improving the response time of the data access interface, thereby improving the reliability of data synchronization and improving data access efficiency.

[0046] Furthermore, writing the data to be written to the IDC corresponding to the system includes:

[0047] Determine the number range of the IDC corresponding to the system;

[0048] Generate the primary key ID of the data to be written based on the number segment;

[0049] Write the primary key ID and the data to be written to the IDC corresponding to the system.

[0050] As described above, different data centers have different number segments. Distinguishing data primary keys by number segment is beneficial for directly determining whether data already exists based on the primary key during data synchronization, which is simple and effective.

[0051] Further, writing the data to be written to the main IDC includes:

[0052] The system generates an asynchronous message from the corresponding IDC based on the primary key ID and the data to be written, and sends the asynchronous message to the consumption queue.

[0053] The primary key ID and the data to be written in the asynchronous message are written to the main IDC through the consumption queue.

[0054] As described above, by generating asynchronous messages from the corresponding IDC based on the primary key ID and the data to be written, and sending the asynchronous messages to the consumption queue, the primary key ID and the data to be written are written to the main IDC by consuming the messages through the consumption queue, thus ensuring processing efficiency.

[0055] Furthermore, the triggering of the synchronization operation, which synchronizes the primary IDC data to all secondary IDCs, includes:

[0056] The primary key ID and the data to be written are sent to all slave IDCs through the primary IDC;

[0057] The slave IDC determines whether the data to be written exists based on the primary key ID. If it does, the data to be written is not written to the slave IDC. If not, the primary key ID and the data to be written are written to the slave IDC.

[0058] As described above, after receiving the data synchronized from the primary IDC, the system determines whether the corresponding data already exists based on the primary key ID. If it does, no data synchronization is required; otherwise, data synchronization is performed, thus improving data synchronization efficiency.

[0059] Furthermore, after writing the data to be written to the IDC corresponding to the system, the process further includes:

[0060] Receive a query request sent by the system, the query request including data to be queried;

[0061] The query request is used to retrieve the data to be queried from the IDC corresponding to the system.

[0062] As described above, after the data to be written is written to the IDC corresponding to the system, the data can be retrieved from the IDC corresponding to the system according to the query request, without waiting for the data synchronization to be completed, thus improving the data access efficiency.

[0063] The data synchronization method and terminal for Internet Data Centers described above are applicable to scenarios where there are at least two Internet Data Centers (IDCs), with at least one IDC being the master IDC and the others being slave IDCs. Furthermore, the system employs a distributed deployment approach, with each IDC containing an independent application instance. The following detailed implementation methods illustrate this:

[0064] Please refer to Figure 1 Embodiment 1 of the present invention is as follows:

[0065] A data synchronization method for an Internet data center includes the following steps:

[0066] S1. Obtain the data to be written generated by the system.

[0067] S2. Write the data to be written to the IDC corresponding to the system, specifically including S21-S23:

[0068] S21. Determine the number range of the IDC corresponding to the system.

[0069] S22. Generate the primary key ID of the data to be written based on the number segment.

[0070] S23. Write the primary key ID and the data to be written to the IDC corresponding to the system.

[0071] S3. If the IDC corresponding to the system is a slave IDC, then the data to be written is written to the master IDC, and a synchronization operation is triggered. The synchronization operation synchronizes the data from the master IDC to all slave IDCs, specifically including S31-S34:

[0072] S31. If the IDC corresponding to the system is a slave IDC, then an asynchronous message is generated by the slave IDC corresponding to the system based on the primary key ID and the data to be written, and the asynchronous message is sent to the consumption queue.

[0073] S32. Write the primary key ID and the data to be written in the asynchronous message to the main IDC through the consumption queue.

[0074] S33. Send the primary key ID and the data to be written to all slave IDCs through the primary IDC.

[0075] S34. The slave IDC determines whether the data to be written exists based on the primary key ID. If yes, the data to be written is not written to the slave IDC. If no, the primary key ID and the data to be written are written to the slave IDC.

[0076] S4. If the IDC corresponding to the system is the primary IDC, then a synchronization operation is triggered, which synchronizes the primary IDC data to all secondary IDCs.

[0077] In one alternative implementation, it further includes:

[0078] S5. Receive a query request sent by the system, the query request including data to be queried;

[0079] S6. Query the data to be queried from the IDC corresponding to the system according to the query request.

[0080] S5-S6 can be executed synchronously with S3.

[0081] An IDC refers to a server room or data center containing at least one database cluster. This cluster has at least one master node and multiple slave nodes. In other words, an IDC contains a master-slave cluster, and multiple IDCs refer to multiple database clusters, each containing a master-slave database node. In this invention, different IDCs have their own data clusters, and applications in different IDCs can write data to the master node of their own cluster, achieving master-slave data synchronization within their own cluster, as well as data synchronization between different clusters. The method described in this invention eliminates the need to wait for data from the master IDC to be synchronized to its own IDC, effectively avoiding potential occasional data anomalies and improving system interface response time, system performance, data synchronization reliability, and data access efficiency.

[0082] Please refer to Figure 2 Embodiment two of the present invention is as follows:

[0083] A data synchronization terminal for an Internet data center includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the various steps of the data synchronization method for the Internet data center in Embodiment 1.

[0084] In summary, the data synchronization method and terminal for Internet data centers provided by this invention allow the system to directly write data to the corresponding IDC (Internet Data Center) after generating data to be written, regardless of whether the IDC is the primary or secondary IDC. If the IDC is a secondary IDC, the data is written to the primary IDC, triggering a synchronization operation to synchronize the primary IDC data to all secondary IDCs. By prioritizing the writing of generated data to the local IDC, unlike existing technologies that wait for the primary IDC to synchronize the data to the corresponding secondary IDC, data can be queried at any time after the data is written to the corresponding secondary IDC, avoiding data synchronization delays and improving the response time of the data access interface. This enhances the reliability of data synchronization and improves data access efficiency. Furthermore, different IDCs have different number segments, which distinguish the primary key of the data. This allows for direct determination of data existence based on the primary key during data synchronization, which is simple and effective. Simultaneously, after writing the data to be written to the corresponding IDC, data can be queried from the corresponding IDC based on a query request without waiting for data synchronization to complete, further improving data access efficiency.

[0085] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent modifications made based on the content of the present invention specification and drawings, or direct or indirect applications in related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for data synchronization of an internet data center, the method comprising: The method comprises the steps of: obtaining to-be-written data generated by a system; writing the to-be-written data to an IDC corresponding to the system, comprising: determining a number segment of the IDC corresponding to the system; generating a primary key ID of the to-be-written data according to the number segment; writing the primary key ID and the to-be-written data to the IDC corresponding to the system; if the IDC corresponding to the system is a slave IDC, writing the to-be-written data to a master IDC and triggering a synchronization operation, the synchronization operation synchronizing the data of the master IDC to all slave IDCs, comprising: generating an asynchronous message according to the primary key ID and the to-be-written data through a slave IDC corresponding to the system, and sending the asynchronous message to a consumption queue; writing the primary key ID and the to-be-written data in the asynchronous message to the master IDC through the consumption queue; sending the primary key ID and the to-be-written data to all slave IDCs through the master IDC; judging whether the to-be-written data exists according to the primary key ID through the slave IDC, if yes, the to-be-written data is not written to the slave IDC, and if no, the primary key ID and the to-be-written data are written to the slave IDC.

2. The method of claim 1, wherein, After the to-be-written data is written to the IDC corresponding to the system, the method further comprises the steps of: receiving a query request sent by the system, the query request comprising to-be-queried data; querying the to-be-queried data from the IDC corresponding to the system according to the query request.

3. A data synchronization terminal of an Internet data center, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, The processor implements the following steps when executing the computer program: obtaining to-be-written data generated by a system; writing the to-be-written data to an IDC corresponding to the system, comprising: determining a number segment of the IDC corresponding to the system; generating a primary key ID of the to-be-written data according to the number segment; writing the primary key ID and the to-be-written data to the IDC corresponding to the system; if the IDC corresponding to the system is a slave IDC, writing the to-be-written data to a master IDC and triggering a synchronization operation, the synchronization operation synchronizing the data of the master IDC to all slave IDCs, comprising: generating an asynchronous message according to the primary key ID and the to-be-written data through a slave IDC corresponding to the system, and sending the asynchronous message to a consumption queue; writing the primary key ID and the to-be-written data in the asynchronous message to the master IDC through the consumption queue; sending the primary key ID and the to-be-written data to all slave IDCs through the master IDC; judging whether the to-be-written data exists according to the primary key ID through the slave IDC, if yes, the to-be-written data is not written to the slave IDC, and if no, the primary key ID and the to-be-written data are written to the slave IDC.

4. The data synchronization terminal of an Internet data center according to claim 3, wherein, After the to-be-written data is written to the IDC corresponding to the system, the method further comprises the steps of: receiving a query request sent by the system, the query request comprising to-be-queried data; querying the to-be-queried data from the IDC corresponding to the system according to the query request.

Images