Data synchronization processing method and device for multiple data centers, equipment and medium
By employing aspect ratio technology and message queues to achieve data synchronization across multiple data centers, combined with periodic comparison and verification, the problem of data synchronization across multiple data centers is solved, enabling real-time data backup and improved security, while reducing costs.
Patent Information
- Application Number
- CN202510781881.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-12
- Publication Date
- 2025-10-31
AI Technical Summary
The problem of data synchronization between multiple data centers urgently needs to be solved to prevent system unavailability due to a disaster in one of the data centers, and existing technologies have failed to effectively achieve data backup and reduce implementation costs.
The aspect ratio technique is used to send the primary key of the changed data item to the target application in this data center, convert it into data information in the target string format, and push it to the message queues of multiple other data centers. Data synchronization is achieved through the message queues, and data comparison and verification are performed at a predetermined time to ensure synchronization.
It enables real-time data synchronization and mutual backup between multiple data centers, improving data security and reducing the implementation cost of data backup.
Smart Images
Figure CN120873079A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data synchronization technology, and in particular to a data synchronization processing method, apparatus, equipment and medium for multiple data centers. Background Technology
[0002] The construction of financial information systems is not only an inherent requirement for the financial industry itself to improve its competitiveness, reduce operating costs, and improve service quality in order to cope with increasingly fierce market competition, but also the foundation for the informatization of the national economy, such as taxation, customs, trade, and e-commerce.
[0003] For critical financial information systems, it is necessary to deploy multiple data centers in different locations to prevent the entire system from becoming unavailable due to a disaster in one of the data center locations, which could cause losses to the company and its customers. However, the problem of data synchronization between multiple data centers urgently needs to be solved. Summary of the Invention
[0004] Therefore, it is necessary to provide a data synchronization processing method, apparatus, equipment, and medium for data centers to address the aforementioned technical problems. This method or apparatus can at least enable data synchronization and mutual backup across multiple data centers, thereby improving data security and reducing implementation costs.
[0005] Firstly, this application provides a data synchronization processing method for multiple data centers, the method comprising:
[0006] The target application in this data center uses aspect ratioing to send the primary key of the changed data item to the database in this data center;
[0007] Based on the primary key of the data item, the data information of the data item is determined, and the data information of the data item is converted into data information in the target string format. The data information in the target string format is then pushed to message queues of multiple other data centers, so that multiple other data centers can complete data synchronization with the local data center based on the data information in the message queues.
[0008] In one embodiment, multiple other data centers synchronize data with the local data center based on data information in the message queue, including:
[0009] Control and monitor multiple target applications in multiple other data centers and obtain target string information from message queues in multiple other data centers;
[0010] The target application is controlled to convert the target string information into a target object;
[0011] The target application is controlled to update or add data to multiple databases in multiple other data centers based on the target object.
[0012] In one embodiment, the method further includes:
[0013] The system controls multiple data centers to compare and verify the updated data from the previous day at a predetermined time each day, ensuring that the data information in each data center is synchronized with the data information in other data centers.
[0014] In one embodiment, multiple data centers are controlled to compare and verify the updated data from the previous day at a predetermined time each day, including:
[0015] Control each data center to retrieve multiple updated data from the previous day based on the update time field;
[0016] Perform a national cryptographic SM3 hash operation on multiple updated data to obtain the hash value of each updated data;
[0017] The hash values of multiple updated data are stored in the form of files and uploaded to the data platform, and multiple data centers are controlled to obtain multiple updated data in the form of files from the data platform;
[0018] Control each data center to compare and verify multiple updated data in file format.
[0019] In one embodiment, each data center performs a comparison and verification based on multiple updated data in file format, including:
[0020] Based on the primary key of the data item in the data center and the data hash value in multiple updated data in file form, determine whether the data center needs to be updated;
[0021] If the data center needs to be updated, the data in the data center will be updated based on the data update time of the primary key of the data item in the data center and the data update time of the data hash value in multiple updated data in file format.
[0022] In one embodiment, the target string format is a JSON string format, the message queue is a RabbitMQ message queue, and the target object is a JSON object.
[0023] Secondly, this application also provides a data synchronization processing device for multiple data centers, including a data sending module and a data synchronization module. The data sending module is used to control the target application in this data center to send the primary key of the changed data item to the database of this data center through aspect ratio technology.
[0024] The data synchronization module is used to determine the data information of the data item based on the primary key of the data item, convert the data information of the data item into data information in a target string format, and push the data information in the target string format to message queues of multiple other data centers, so that multiple other data centers can complete data synchronization with the local data center based on the data information in the message queues.
[0025] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:
[0026] The target application in this data center uses aspect ratioing to send the primary key of the changed data item to the database in this data center;
[0027] Based on the primary key of the data item, the data information of the data item is determined, and the data information of the data item is converted into data information in the target string format. The data information in the target string format is then pushed to message queues of multiple other data centers, so that multiple other data centers can complete data synchronization with the local data center based on the data information in the message queues.
[0028] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:
[0029] The target application in this data center uses aspect ratioing to send the primary key of the changed data item to the database in this data center;
[0030] Based on the primary key of the data item, the data information of the data item is determined, and the data information of the data item is converted into data information in the target string format. The data information in the target string format is then pushed to message queues of multiple other data centers, so that multiple other data centers can complete data synchronization with the local data center based on the data information in the message queues.
[0031] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:
[0032] The target application in this data center uses aspect ratioing to send the primary key of the changed data item to the database in this data center;
[0033] Based on the primary key of the data item, the data information of the data item is determined, and the data information of the data item is converted into data information in the target string format. The data information in the target string format is then pushed to message queues of multiple other data centers, so that multiple other data centers can complete data synchronization with the local data center based on the data information in the message queues.
[0034] The aforementioned data synchronization processing methods, devices, equipment, and media for data centers can at least achieve data synchronization and mutual backup across multiple data centers, thereby improving data security and reducing implementation costs. Attached Figure Description
[0035] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0036] Figure 1 This is a flowchart illustrating a data synchronization processing method for multiple data centers in one embodiment;
[0037] Figure 2 This is a flowchart illustrating a data synchronization processing method for multiple data centers in another embodiment;
[0038] Figure 3 This is a flowchart illustrating a data synchronization processing method for multiple data centers in yet another embodiment;
[0039] Figure 4 This is a structural block diagram of a data synchronization processing device for multiple data centers in one embodiment;
[0040] Figure 5 This is an internal structural diagram of a computer device in one embodiment.
[0041] Figure labels and descriptions:
[0042] 20. Data sending module; 30. Data synchronization module. Detailed Implementation
[0043] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0044] The multi-data center data synchronization processing method provided in this application can be applied to various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc.
[0045] Please refer to Figure 1 In one exemplary embodiment, a data synchronization processing method for multiple data centers is provided, including the following steps S101 to S102. Wherein:
[0046] Step S101: Control the target application in this data center to send the primary key of the changed data item to the database of this data center through aspect ratio technology.
[0047] Specifically, when adding an aspect to an interface that modifies data, if the interface call is successful, the aspect obtains the request parameters and retrieves the primary key of the data item based on the request parameters; if the interface call fails or an exception occurs, the aspect does not perform any processing.
[0048] Step S102: Determine the data information of the data item based on the primary key of the data item, convert the data information of the data item into data information in the target string format, and push the data information in the target string format to the message queues of multiple other data centers, so that the multiple other data centers can complete the data synchronization with the local data center based on the data information in the message queues.
[0049] Specifically, multiple other data centers synchronize data with the local data center based on data information in message queues, including: controlling multiple target applications in multiple other data centers to monitor and obtain target string information in message queues of multiple other data centers; controlling the target applications to convert the target string information into target objects; and controlling the target applications to update or add data to multiple databases in multiple other data centers based on the target objects.
[0050] For example, please refer to Figure 2When data in application A in data center A is changed, application A needs to send the primary key of the changed data item to database A in data center A through aspect-oriented programming. At the same time, it also needs to determine the data information of the data item based on the primary key of the data item, convert the data information of the data item into JSON format data information, and push the JSON format data information to the message queue (RabbitMQ) of other data center B.
[0051] It should be noted that multiple data centers also include data center C, data center D, data center E, etc. While pushing JSON format data information to the message queue (RabbitMQ) of other data center B, it is also necessary to push JSON format data information to data center C, data center D, and data center E to ensure real-time data synchronization between multiple data centers.
[0052] Meanwhile, through data synchronization across multiple data centers, which serve as backups for each other, data recovery can be achieved as long as the data in one data center remains intact, even if one or more data centers experience an anomaly due to a sudden accident, thus improving data security.
[0053] Furthermore, by using multiple data centers to back each other up, there is no need to establish additional data backup centers, thus reducing the implementation cost of data backup.
[0054] In one optional embodiment, the method further includes: controlling multiple data centers to compare and verify the updated data of the previous day at a predetermined time each day, so as to ensure that the data information of each data center is synchronized with the data information of other data centers.
[0055] Specifically, the system controls multiple data centers to compare and verify the updated data from the previous day at a predetermined time each day. This includes: controlling each data center to obtain multiple updated data from the previous day based on the update time field; performing a national cryptographic SM3 hash operation on the multiple updated data to obtain the hash value of each updated data; storing the hash values of the multiple updated data in file format and uploading them to the data platform, and controlling the multiple data centers to obtain the multiple updated data in file format from the data platform; and controlling each data center to perform comparison and verification based on the multiple updated data in file format.
[0056] Each data center performs a comparison and verification based on multiple updated data in file format, including: determining whether the data center needs to update its data based on the primary key of the data item in the data center and the data hash value in the multiple updated data in file format; if the data center needs to update its data, then the data in the data center is updated based on the data update time of the primary key of the data item in the data center and the data update time of the data hash value in the multiple updated data in file format.
[0057] Wherein, the target string format is JSON string format, the message queue is RabbitMQ message queue, and the target object is JSON object.
[0058] For example, please refer to Figure 3 In one exemplary embodiment, yet another method for data synchronization processing across multiple data centers is provided.
[0059] like Figure 3 As shown, at a predetermined time each day, the updated data from the previous day needs to be compared and verified. Specifically, at the predetermined time each day, applications A and B in data centers A and B need to push files A and B to the data platform. Applications A and B retrieve files from other centers for comparison to determine if there are any erroneous data that need to be corrected. If erroneous data that needs to be corrected is determined, then applications A and B will correct the data in databases A and B.
[0060] The data in files A and B are files formed by hash values of multiple updated data in data centers A and B.
[0061] In this way, each data center performs a daily scheduled task to compare and verify the updated data from the previous day. It queries all data whose update time was the previous day through the "update time" field, performs a national cryptographic SM3 hash on each data entry to obtain the corresponding hash value, stores the retrieved data in file form, and uploads it to the data platform. Each data center retrieves files from other data centers from the data platform and compares the hash values of the data based on the data primary key. If they are different, they compare the "update time" field. The data with the latest update time is ultimately used. Through the above automatic data verification function, when the system malfunctions, the data can still be automatically repaired through the daily scheduled task without manual intervention, greatly reducing the workload.
[0062] It should be noted that multiple data centers also include data center C, data center D, data center E, etc. When comparing files, it is necessary to obtain files from all other data centers. Specifically, the verification process based on files from data centers C, D, E, etc. is the same as the data verification process for data center B, and will not be elaborated here.
[0063] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0064] Based on the same inventive concept, this application also provides a data synchronization processing apparatus for implementing the data synchronization processing method for multiple data centers as described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the data synchronization processing apparatus for multiple data centers provided below can be found in the limitations of the data synchronization processing method for multiple data centers described above, and will not be repeated here.
[0065] Please refer to Figure 4 In one exemplary embodiment, a multi-data center data synchronization processing apparatus is provided, comprising: a data sending module 20 and a data synchronization module 30. The data sending module is used to control a target application in the local data center to send the primary key of a changed data item to the database of the local data center through aspect ratioing. The data synchronization module is used to determine the data information of the data item based on the primary key of the data item, convert the data information of the data item into data information in a target string format, and push the data information in the target string format to message queues of multiple other data centers, so that multiple other data centers can complete data synchronization with the local data center based on the data information in the message queues.
[0066] In one embodiment, the data synchronization module is specifically used to control the monitoring of multiple target applications in multiple other data centers and to obtain target string information in the message queues of multiple other data centers;
[0067] The target application is controlled to convert the target string information into a target object;
[0068] The target application is controlled to update or add data to multiple databases in multiple other data centers based on the target object.
[0069] In one embodiment, the apparatus further includes a data synchronization module, which controls multiple data centers to compare and verify the updated data of the previous day at a predetermined time each day, so as to ensure that the data information of each data center is synchronized with the data information of other data centers.
[0070] In one embodiment, the data synchronization module is specifically used to control each data center to obtain multiple updated data from the previous day based on the update time field;
[0071] Perform a national cryptographic SM3 hash operation on multiple updated data to obtain the hash value of each updated data;
[0072] The hash values of multiple updated data are stored in the form of files and uploaded to the data platform, and multiple data centers are controlled to obtain multiple updated data in the form of files from the data platform;
[0073] Control each data center to compare and verify multiple updated data in file format.
[0074] In one embodiment, the data synchronization module is further configured to determine whether the data center needs to perform a data update based on the primary key of the data item in the data center and the data hash value in multiple update data in file form;
[0075] If the data center needs to be updated, the data in the data center will be updated based on the data update time of the primary key of the data item in the data center and the data update time of the data hash value in multiple updated data in file format.
[0076] Wherein, the target string format is JSON string format, the message queue is RabbitMQ message queue, and the target object is JSON object.
[0077] Each module in the aforementioned multi-data center data synchronization processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.
[0078] In an exemplary embodiment, a computer device is provided, which may be a terminal. The computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface of the computer device is used for exchanging information between the processor and external devices. The communication interface of the computer device is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a multi-data center data synchronization processing method. The display unit of the computer device is used to form a visually visible image and may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.
[0079] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0080] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:
[0081] The target application in this data center uses aspect ratioing to send the primary key of the changed data item to the database in this data center;
[0082] Based on the primary key of the data item, the data information of the data item is determined, and the data information of the data item is converted into data information in the target string format. The data information in the target string format is then pushed to message queues of multiple other data centers, so that multiple other data centers can complete data synchronization with the local data center based on the data information in the message queues.
[0083] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:
[0084] The target application in this data center uses aspect ratioing to send the primary key of the changed data item to the database in this data center;
[0085] Based on the primary key of the data item, the data information of the data item is determined, and the data information of the data item is converted into data information in the target string format. The data information in the target string format is then pushed to message queues of multiple other data centers, so that multiple other data centers can complete data synchronization with the local data center based on the data information in the message queues.
[0086] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:
[0087] The target application in this data center uses aspect ratioing to send the primary key of the changed data item to the database in this data center;
[0088] Based on the primary key of the data item, the data information of the data item is determined, and the data information of the data item is converted into data information in the target string format. The data information in the target string format is then pushed to message queues of multiple other data centers, so that multiple other data centers can complete data synchronization with the local data center based on the data information in the message queues.
[0089] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.
[0090] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0091] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A data synchronization processing method for multiple data centers, characterized in that, The method includes: The target application in this data center uses aspect ratioing to send the primary key of the changed data item to the database in this data center; Based on the primary key of the data item, the data information of the data item is determined, and the data information of the data item is converted into data information in the target string format. The data information in the target string format is then pushed to message queues of multiple other data centers, so that multiple other data centers can complete data synchronization with the local data center based on the data information in the message queues.
2. The data synchronization processing method for multiple data centers according to claim 1, characterized in that, Multiple other data centers synchronize data with this data center based on data information in the message queue, including: Control and monitor multiple target applications in multiple other data centers and obtain target string information from message queues in multiple other data centers; The target application is controlled to convert the target string information into a target object; The target application is controlled to update or add data to multiple databases in multiple other data centers based on the target object.
3. The data synchronization processing method for multiple data centers according to claim 1, characterized in that, The method further includes: The system controls multiple data centers to compare and verify the updated data from the previous day at a predetermined time each day, ensuring that the data information in each data center is synchronized with the data information in other data centers.
4. The data synchronization processing method for multiple data centers according to claim 3, characterized in that, Control multiple data centers to compare and verify the updated data from the previous day at a predetermined time each day, including: Control each data center to retrieve multiple updated data from the previous day based on the update time field; Perform a national cryptographic SM3 hash operation on multiple updated data to obtain the hash value of each updated data; The hash values of multiple updated data are stored in the form of files and uploaded to the data platform, and multiple data centers are controlled to obtain multiple updated data in the form of files from the data platform; Control each data center to compare and verify multiple updated data in file format.
5. The data synchronization processing method for multiple data centers according to claim 4, characterized in that, Each data center performs comparison and verification based on multiple updated data in file format, including: Based on the primary key of the data item in the data center and the data hash value in multiple updated data in file form, determine whether the data center needs to be updated; If the data center needs to be updated, the data in the data center will be updated based on the data update time of the primary key of the data item in the data center and the data update time of the data hash value in multiple updated data in file format.
6. The data synchronization processing method for multiple data centers according to claim 2, characterized in that, The target string is in JSON format, the message queue is a RabbitMQ message queue, and the target object is a JSON object.
7. A data synchronization processing device for multiple data centers, characterized in that, The device includes: The data sending module is used to control the target application in this data center to send the primary key of the changed data item to the database of this data center through aspect ratio technology; The data synchronization module is used to determine the data information of the data item based on the primary key of the data item, convert the data information of the data item into data information in a target string format, and push the data information in the target string format to message queues of multiple other data centers, so that multiple other data centers can complete data synchronization with the local data center based on the data information in the message queues.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.