Data synchronization method and terminal of Internet data center

By directly writing the data to be written to the main IDC in the Internet data center and triggering the synchronization operation, the data synchronization delay problem is solved, data access efficiency and system response time are improved, and data synchronization reliability is ensured.

CN118916422AActive Publication Date: 2024-11-08FUJIAN TQ ONLINE INTERACTIVE INC
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202410958271.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-17
Publication Date
2024-11-08
Estimated Expiration
2044-07-17

AI Technical Summary

Technical Problem

In distributed Internet data centers, there are delay problems in data synchronization, resulting in data exceptions and system errors that may occur when accessing from IDC.

Method used

When the system generates data to be written, it is directly written to the IDC corresponding to the system. If it is a slave IDC, the data is written to the master IDC and a synchronization operation is triggered to synchronize the master IDC data to all slave IDCs, thereby avoiding the Delay to wait for data synchronization to complete.

Benefits of technology

It improves the reliability and access efficiency of data synchronization, reduces data access latency, and improves the system's response time.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916422A_ABST
    Figure CN118916422A_ABST
Patent Text Reader

Abstract

After a system generates data to be written, no matter whether an IDC corresponding to the system is a master IDC or a slave IDC, the data to be written is directly written into the IDC, if the IDC corresponding to the system is the slave IDC, the data to be written is written into the master IDC, a synchronization operation is triggered, the data of the master IDC is synchronized to all the slave IDCs, and the data to be written is synchronized to all the slave IDCs. The generated data are written into the local IDC preferentially, the situation that in the prior art, the data can be inquired at any time after the master IDC synchronizes the data to the slave IDC corresponding to the system and the slave IDC corresponding to the system completes data writing is not needed, the delay problem of data synchronization is avoided, the response time of a data access interface is shortened, and therefore the reliability of data synchronization is improved, and the data synchronization efficiency is improved. And the data access efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of Internet data centers, and in particular to a data synchronization method and a terminal for an Internet data center. Background Art

[0002] In modern large-scale systems, with the rapid increase in the number of users and user behaviors, higher requirements are placed on the server system, among which the server system needs to meet various performance requirements such as high concurrency and high throughput. In order to meet these performance requirements, developers usually adopt a distributed architecture and deploy the system in multiple IDCs (Internet Data Centers). When users access the system, they will give priority to accessing the system in the nearest IDC.

[0003] In order to reduce data conflicts during data synchronization between different IDCs, there will be a master IDC among multiple IDCs in the distributed architecture, and the other IDCs are slave IDCs. The master IDC mainly provides data writing operations for each system, and synchronizes data to other slave IDCs through the master IDC, ultimately achieving data consistency among all IDCs. However, because there is a certain delay in data synchronization between different IDCs, when there is a delay in synchronizing data from the master IDC to other slave IDCs, when the system accesses the slave IDC again, it will cause data access exceptions, and ultimately cause system exceptions and errors. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a data synchronization method and a terminal for an Internet data center, which can improve the reliability of data synchronization and improve the efficiency of data access.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A data synchronization method for an Internet data center comprises the following steps:

[0007] Get the data to be written generated by the system;

[0008] Writing the data to be written into the IDC corresponding to the system;

[0009] If the IDC corresponding to the system is a slave IDC, the data to be written is written to the master IDC, and a synchronization operation is triggered. The synchronization operation synchronizes the master IDC data to all slave IDCs.

[0010] In order to solve the above technical problems, another technical solution adopted by the present invention is:

[0011] A data synchronization terminal for an Internet data center includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0012] Get the data to be written generated by the system;

[0013] Writing the data to be written into the IDC corresponding to the system;

[0014] If the IDC corresponding to the system is a slave IDC, the data to be written is written to the master IDC, and a synchronization operation is triggered. The synchronization operation synchronizes the master IDC data to all slave IDCs.

[0015] The beneficial effect of the present invention is that after the system generates data to be written, no matter whether the IDC corresponding to the system is a master IDC or a slave IDC, the data to be written is directly written to the IDC; if the IDC corresponding to the system is a slave IDC, the data to be written is written to the master IDC, and a synchronization operation is triggered to synchronize the master IDC data to all slave IDCs. By preferentially writing the generated data to the local IDC, there is no need to wait for the master IDC to synchronize the data to the slave IDC corresponding to the system as in the prior art. After the data of the slave IDC corresponding to the system is written, the data can be queried at any time, thus avoiding the delay problem of data synchronization and improving the response time of the data access interface, thereby improving the reliability of data synchronization and improving data access efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flowchart of a method for synchronizing data in an Internet data center according to an embodiment of the present invention;

[0017] Figure 2 The present invention is a schematic diagram of the structure of a data synchronization terminal of an Internet data center according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to explain the technical content, achieved objectives and effects of the present invention in detail, the following is an explanation in combination with the implementation modes and the accompanying drawings.

[0019] Please refer to Figure 1 , a data synchronization method for an Internet data center, comprising the steps of:

[0020] Get the data to be written generated by the system;

[0021] Writing the data to be written into the IDC corresponding to the system;

[0022] If the IDC corresponding to the system is a slave IDC, the data to be written is written to the master IDC, and a synchronization operation is triggered. The synchronization operation synchronizes the master IDC data to all slave IDCs.

[0023] From the above description, it can be seen that the beneficial effect of the present invention is that after the system generates data to be written, regardless of whether the IDC corresponding to the system is the master IDC or the slave IDC, the data to be written is directly written to the IDC; if the IDC corresponding to the system is a slave IDC, the data to be written is written to the master IDC, and a synchronization operation is triggered to synchronize the master IDC data to all slave IDCs. By preferentially writing the generated data to the local IDC, there is no need to wait for the master IDC to synchronize the data to the slave IDC corresponding to the system as in the prior art. After the data of the slave IDC corresponding to the system is written, the data can be queried at any time, avoiding the problem of data synchronization delay and improving the response time of the data access interface, thereby improving the reliability of data synchronization and improving data access efficiency.

[0024] Further, writing the data to be written into the IDC corresponding to the system includes:

[0025] Determine the number segment of the IDC corresponding to the system;

[0026] Generate a primary key ID of the data to be written according to the number segment;

[0027] The primary key ID and the data to be written are written into the IDC corresponding to the system.

[0028] From the above description, we can see that the number segments of different IDCs are different. Distinguishing data primary keys by number segments is conducive to directly judging whether data already exists based on the primary key during data synchronization, which is simple and effective.

[0029] Further, writing the data to be written into the main IDC includes:

[0030] Generate an asynchronous message according to the primary key ID and the data to be written through the slave IDC corresponding to the system, and send the asynchronous message to the consumption queue;

[0031] The primary key ID and the data to be written in the asynchronous message are written into the primary IDC through the consumption queue.

[0032] From the above description, it can be seen that the slave IDC corresponding to the system generates an asynchronous message according to the primary key ID and the data to be written, and sends the asynchronous message to the consumer queue. The primary key ID and the data to be written are written to the master IDC by consuming the message through the consumer queue, thereby ensuring processing efficiency.

[0033] Further, the triggering synchronization operation, wherein the synchronization operation synchronizes the master IDC data to all slave IDCs, includes:

[0034] Sending the master key ID and the data to be written to all slave IDCs through the master IDC;

[0035] The slave IDC determines whether the data to be written exists according to the master key ID. If so, the data to be written is not written to the slave IDC. If not, the master key ID and the data to be written are written to the slave IDC.

[0036] From the above description, it can be seen that after receiving the data synchronized by the primary IDC, the slave IDC determines whether the corresponding data already exists based on the primary key ID. If so, there is no need to synchronize the data. Otherwise, data synchronization is performed, which improves the efficiency of data synchronization.

[0037] Furthermore, after writing the data to be written into the IDC corresponding to the system, the method further includes:

[0038] Receiving a query request sent by the system, wherein the query request includes data to be queried;

[0039] The to-be-queried data is queried from an IDC corresponding to the system according to the query request.

[0040] From the above description, it can be known that after the data to be written is written to the IDC corresponding to the system, the data can be queried from the IDC corresponding to the system according to the query request, without waiting for the data synchronization to be completed, thereby improving the data access efficiency.

[0041] Please refer to Figure 2 , a data synchronization terminal of an Internet data center, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0042] Get the data to be written generated by the system;

[0043] Writing the data to be written into the IDC corresponding to the system;

[0044] If the IDC corresponding to the system is a slave IDC, the data to be written is written to the master IDC, and a synchronization operation is triggered. The synchronization operation synchronizes the master IDC data to all slave IDCs.

[0045] From the above description, it can be seen that the beneficial effect of the present invention is that after the system generates data to be written, regardless of whether the IDC corresponding to the system is the master IDC or the slave IDC, the data to be written is directly written to the IDC; if the IDC corresponding to the system is a slave IDC, the data to be written is written to the master IDC, and a synchronization operation is triggered to synchronize the master IDC data to all slave IDCs. By preferentially writing the generated data to the local IDC, there is no need to wait for the master IDC to synchronize the data to the slave IDC corresponding to the system as in the prior art. After the data of the slave IDC corresponding to the system is written, the data can be queried at any time, avoiding the problem of data synchronization delay and improving the response time of the data access interface, thereby improving the reliability of data synchronization and improving data access efficiency.

[0046] Further, writing the data to be written into the IDC corresponding to the system includes:

[0047] Determine the number segment of the IDC corresponding to the system;

[0048] Generate a primary key ID of the data to be written according to the number segment;

[0049] The primary key ID and the data to be written are written into the IDC corresponding to the system.

[0050] From the above description, we can see that the number segments of different IDCs are different. Distinguishing data primary keys by number segments is conducive to directly judging whether data already exists based on the primary key during data synchronization, which is simple and effective.

[0051] Further, writing the data to be written into the main IDC includes:

[0052] Generate an asynchronous message according to the primary key ID and the data to be written through the slave IDC corresponding to the system, and send the asynchronous message to the consumption queue;

[0053] The primary key ID and the data to be written in the asynchronous message are written into the primary IDC through the consumption queue.

[0054] From the above description, it can be seen that the slave IDC corresponding to the system generates an asynchronous message according to the primary key ID and the data to be written, and sends the asynchronous message to the consumer queue. The primary key ID and the data to be written are written to the master IDC by consuming the message through the consumer queue, thereby ensuring processing efficiency.

[0055] Further, the triggering synchronization operation, wherein the synchronization operation synchronizes the master IDC data to all slave IDCs, includes:

[0056] Sending the master key ID and the data to be written to all slave IDCs through the master IDC;

[0057] The slave IDC determines whether the data to be written exists according to the master key ID. If so, the data to be written is not written to the slave IDC. If not, the master key ID and the data to be written are written to the slave IDC.

[0058] From the above description, it can be seen that after receiving the data synchronized by the primary IDC, the slave IDC determines whether the corresponding data already exists based on the primary key ID. If so, there is no need to synchronize the data. Otherwise, data synchronization is performed, which improves the efficiency of data synchronization.

[0059] Furthermore, after writing the data to be written into the IDC corresponding to the system, the method further includes:

[0060] Receiving a query request sent by the system, wherein the query request includes data to be queried;

[0061] The to-be-queried data is queried from an IDC corresponding to the system according to the query request.

[0062] From the above description, it can be known that after the data to be written is written to the IDC corresponding to the system, the data can be queried from the IDC corresponding to the system according to the query request, without waiting for the data synchronization to be completed, thereby improving the data access efficiency.

[0063] The data synchronization method and terminal of the Internet data center of the present invention can be applicable to the scenario where there are at least two IDCs, at least one of which is the master IDC and the others are slave IDCs. At the same time, the system adopts a distributed deployment mode, and there is an independent application instance in each IDC. The following is an explanation through a specific implementation method:

[0064] Please refer to Figure 1 , Embodiment 1 of the present invention is:

[0065] A data synchronization method for an Internet data center comprises the following steps:

[0066] S1. Obtain the data to be written generated by the system.

[0067] S2, writing the data to be written into the IDC corresponding to the system, specifically including S21-S23:

[0068] S21. Determine the number segment of the IDC corresponding to the system.

[0069] S22. Generate a primary key ID of the data to be written according to the number segment.

[0070] S23. Write the primary key ID and the data to be written into the IDC corresponding to the system.

[0071] S3. If the IDC corresponding to the system is a slave IDC, the data to be written is written to the master IDC, and a synchronization operation is triggered. The synchronization operation synchronizes the master IDC data to all slave IDCs, specifically including S31-S34:

[0072] S31. If the IDC corresponding to the system is a slave IDC, an asynchronous message is generated according to the primary key ID and the data to be written by the slave IDC corresponding to the system, and the asynchronous message is sent to a consumption queue.

[0073] S32. Write the primary key ID and the data to be written in the asynchronous message into the primary IDC through the consumption queue.

[0074] S33. Send the primary key ID and the data to be written to all slave IDCs through the primary IDC.

[0075] S34. Determine whether the data to be written exists according to the master key ID through the slave IDC. If yes, the data to be written is not written to the slave IDC. If no, the master key ID and the data to be written are written to the slave IDC.

[0076] S4. If the IDC corresponding to the system is a master IDC, a synchronization operation is triggered, and the synchronization operation synchronizes the master IDC data to all slave IDCs.

[0077] In an optional implementation, it also includes:

[0078] S5. receiving a query request sent by the system, wherein the query request includes data to be queried;

[0079] S6. Query the to-be-queried data from the IDC corresponding to the system according to the query request.

[0080] S5-S6 can be executed synchronously with S3.

[0081] An IDC refers to a computer room or a data center. There is at least one database cluster in the data center. There is at least one master node and multiple slave nodes in the cluster, that is, there is one set of one master and multiple slave clusters in an IDC. Multiple IDCs refer to multiple sets of database clusters, each of which has one master and multiple slave database nodes. In the present invention, different IDCs have their own data clusters, and applications of different IDCs can write data to the master node of their own clusters, and realize master-slave data synchronization of their own clusters, as well as data synchronization between different clusters. Through the above method of the present invention, there is no need to wait for the master IDC data to be synchronized to its own IDC, which effectively avoids the problem that may cause occasional data anomalies in the system, and can also effectively improve the response time of the system interface, improve system performance, thereby improving the reliability of data synchronization, and improve data access efficiency.

[0082] Please refer to Figure 2 , Embodiment 2 of the present invention is:

[0083] A data synchronization terminal for an Internet data center comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, each step of the data synchronization method for an Internet data center in Embodiment 1 is implemented.

[0084] In summary, the present invention provides a data synchronization method and terminal for an Internet data center. After the system generates data to be written, no matter whether the IDC corresponding to the system is a master IDC or a slave IDC, the data to be written is directly written to the IDC. If the IDC corresponding to the system is a slave IDC, the data to be written is written to the master IDC, and a synchronization operation is triggered to synchronize the master IDC data to all slave IDCs. By preferentially writing the generated data to the local IDC, there is no need to wait for the master IDC to synchronize the data to the slave IDC corresponding to the system as in the prior art. After the data of the slave IDC corresponding to the system is written, the data can be queried at any time, thereby avoiding the delay problem of data synchronization and improving the response time of the data access interface, thereby improving the reliability of data synchronization and improving the data access efficiency. In addition, the number segments of different IDCs are different, and the data primary key is distinguished by the number segment, which is conducive to directly judging whether the data already exists according to the primary key during data synchronization, which is simple and effective. At the same time, after the data to be written is written to the IDC corresponding to the system, the data can be queried from the IDC corresponding to the system according to the query request, without waiting for the data synchronization to be completed, thereby improving the data access efficiency.

[0085] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A data synchronization method for an Internet data center, characterized in that: Includes steps: Get the data to be written generated by the system; Writing the data to be written into the IDC corresponding to the system; If the IDC corresponding to the system is a slave IDC, the data to be written is written to the master IDC, and a synchronization operation is triggered. The synchronization operation synchronizes the master IDC data to all slave IDCs.

2. The data synchronization method of an Internet data center according to claim 1, characterized in that: Writing the data to be written into the IDC corresponding to the system comprises: Determine the number segment of the IDC corresponding to the system; Generate a primary key ID of the data to be written according to the number segment; The primary key ID and the data to be written are written into the IDC corresponding to the system.

3. The data synchronization method of an Internet data center according to claim 2, characterized in that: Writing the data to be written into the main IDC includes: Generate an asynchronous message according to the primary key ID and the data to be written through the slave IDC corresponding to the system, and send the asynchronous message to the consumption queue; The primary key ID and the data to be written in the asynchronous message are written into the primary IDC through the consumption queue.

4. The data synchronization method of an Internet data center according to claim 3, characterized in that: The triggering synchronization operation, wherein the synchronization operation synchronizes the master IDC data to all slave IDCs, includes: Sending the master key ID and the data to be written to all slave IDCs through the master IDC; The slave IDC determines whether the data to be written exists according to the master key ID. If so, the data to be written is not written to the slave IDC. If not, the master key ID and the data to be written are written to the slave IDC.

5. The data synchronization method of an Internet data center according to claim 1, characterized in that: After writing the data to be written into the IDC corresponding to the system, the method further includes: Receiving a query request sent by the system, wherein the query request includes data to be queried; The to-be-queried data is queried from an IDC corresponding to the system according to the query request.

6. A data synchronization terminal for an Internet data center, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the following steps are implemented: Get the data to be written generated by the system; Writing the data to be written into the IDC corresponding to the system; If the IDC corresponding to the system is a slave IDC, the data to be written is written to the master IDC, and a synchronization operation is triggered. The synchronization operation synchronizes the master IDC data to all slave IDCs.

7. The data synchronization terminal of an Internet data center according to claim 6, characterized in that: Writing the data to be written into the IDC corresponding to the system comprises: Determine the number segment of the IDC corresponding to the system; Generate a primary key ID of the data to be written according to the number segment; The primary key ID and the data to be written are written into the IDC corresponding to the system.

8. The data synchronization terminal of an Internet data center according to claim 7, characterized in that: Writing the data to be written into the main IDC includes: Generate an asynchronous message according to the primary key ID and the data to be written through the slave IDC corresponding to the system, and send the asynchronous message to the consumption queue; The primary key ID and the data to be written in the asynchronous message are written into the primary IDC through the consumption queue.

9. The data synchronization terminal of an Internet data center according to claim 8, characterized in that: The triggering synchronization operation, wherein the synchronization operation synchronizes the master IDC data to all slave IDCs, includes: Sending the master key ID and the data to be written to all slave IDCs through the master IDC; The slave IDC determines whether the data to be written exists according to the master key ID. If so, the data to be written is not written to the slave IDC. If not, the master key ID and the data to be written are written to the slave IDC.

10. The data synchronization terminal of an Internet data center according to claim 6, characterized in that: After writing the data to be written into the IDC corresponding to the system, the method further includes: Receiving a query request sent by the system, wherein the query request includes data to be queried; The to-be-queried data is queried from an IDC corresponding to the system according to the query request.

Citation Information

Patent Citations

  • Trans-IDC (internet data center) data sharing caching method

    CN104519130A

  • Method of integrated synchronization of loosely coupled data based on enterprise-level application scenario

    CN107480302A

  • A data zero delay processing method and a terminal

    CN109213766A

  • Multi-IDC distributed system data synchronization method based on SQL command level

    CN114595288A

  • Data storage method and server applicable to distributed server cluster

    US20180367610A1