Multi-site data synchronization method, system, device and medium

Abstract

The present application discloses a multi-site data synchronization method, system, device and medium, which are applied to the field of data processing. The multi-site data synchronization method provided by the present application determines whether there is any site missing data in each site by comparing the data information of each site, and when it is confirmed that there is a site missing data, the missing data is obtained from other sites and written to the site where the missing data is missing. This method can not only achieve the redundancy of multi-site data to achieve the purpose of backup, but also restore data through other sites after the site data is lost, ensuring the data synchronization between the faulty site and other sites, thereby making the multi-site data synchronization function more powerful and improving the market competitiveness of distributed object products. The beneficial effects of the multi-site data synchronization system, device and medium provided by the present application are the same as above.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a multi-site data synchronization method, system, device and medium. Background Art

[0002] With the explosive growth of massive amounts of unstructured data, distributed object storage has become the storage cornerstone for users building data center architectures. More and more critical businesses are integrating with distributed object storage. In this scenario, distributed object storage products must meet customers' storage needs for massive amounts of data while also providing disaster recovery and backup for critical data.

[0003] Based on the above requirements, object storage multi-site is realizing data synchronization between multiple sites. The current multi-site is to synchronize data to another site after writing data at one site to achieve backup. When a site fails, data access is switched to another site to ensure normal access.

[0004] However, current data synchronization methods can only synchronize data from the primary site with the backup site, serving as a backup. When the primary site fails, accessing the backup site to obtain data cannot restore data from the failed primary site. Summary of the Invention

[0005] The purpose of this application is to provide a multi-site data synchronization method, system, device and medium to solve the problem of being unable to recover data after a failure of one or some sites in a multi-site situation, thereby ensuring data synchronization between the failed site and other sites and improving the market competitiveness of distributed object products.

[0006] To solve the above technical problems, this application provides a multi-site data synchronization method, including:

[0007] Obtain data information of each site;

[0008] By comparing the data information of each site, it is determined whether each site is missing data;

[0009] If so, the missing data is obtained through other sites and written into the site where the missing data is located;

[0010] If not, return to the step of obtaining data information of each site.

[0011] Preferably, the site includes a first site and a second site;

[0012] Accordingly, the data information obtained from each site includes:

[0013] First data information of a first site and second data information of a second site are obtained.

[0014] Preferably, the first data information includes a first bucket list, a first bucket quota, and a first number of objects, and the second data information includes a second bucket list, a second bucket quota, and a second number of objects.

[0015] Preferably, determining whether each site is missing data by comparing data information of each site includes:

[0016] Determine whether the first bucket list is consistent with the second bucket list;

[0017] If the first bucket list is inconsistent with the second bucket list, it is determined that the first site or the second site is missing data;

[0018] If the first bucket list is consistent with the second bucket list, determine whether the first bucket quota is consistent with the second bucket quota;

[0019] If the first bucket quota is inconsistent with the second bucket quota, it is determined that the first site or the second site is missing data;

[0020] If the first bucket quota is consistent with the second bucket quota, it is determined that there is no missing data at the first site or the second site.

[0021] Preferably, when the first bucket list is inconsistent with the second bucket list, the site that obtains the missing data through other sites and writes the missing data includes:

[0022] Create a first bucket shard that is consistent with the other site in the site where the data is missing based on the first bucket list and the second bucket list and synchronize the bucket information;

[0023] Get a list of objects on a bucket shard at another site;

[0024] Write the objects on the other site to the first bucket shard based on the object list.

[0025] Preferably, when the first bucket quota is inconsistent with the second bucket quota, the site that obtains the missing data through other sites and writes the missing data includes:

[0026] Compare the number of first objects on the bucket shard at the first site and the number of second objects on the bucket shard at the second site to see if they are consistent;

[0027] If they are consistent, continue to compare whether the number of the first object on the bucket shard of the first site is consistent with the number of the second object on the bucket shard of the second site;

[0028] If they are inconsistent, the missing objects of the site where the data is missing are obtained according to the first object list of the first site and the second object list of the second site, and the objects are written to the site where the data is missing.

[0029] Preferably, when the number of first objects on the bucket shards of the first site is inconsistent with the number of second objects on the bucket shards of the second site, obtaining the missing objects of the site where the data is missing according to the first object list of the first site and the second object list of the second site, and writing the objects to the site where the data is missing further includes:

[0030] Return to the step of obtaining the first data information of the first site and the second data information of the second site.

[0031] To solve the above technical problems, the present application also provides a multi-site data synchronization system, including:

[0032] The first acquisition module is used to obtain data information of each site;

[0033] A judgment module is used to judge whether each site is missing data by comparing the data information of each site;

[0034] The second acquisition module is used to, when data is missing at each site, acquire the missing data through other sites and write the missing data into the site;

[0035] The return module is used to return to the step of obtaining data information of each site when there is no missing data for each site.

[0036] To solve the above technical problems, the present application also provides a multi-site data synchronization device, comprising a memory for storing a computer program;

[0037] A processor is configured to implement the steps of a multi-site data synchronization method when executing a computer program.

[0038] In order to solve the above technical problems, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the multi-site data synchronization method are implemented.

[0039] The multi-site data synchronization method provided in the present application determines whether there is any site missing data in each site by comparing the data information of each site, and when it is confirmed that there is a site missing data, the missing data is obtained from other sites and written to the site where the missing data is missing. This not only realizes the redundancy of multi-site data to achieve the purpose of backup, but also can restore data through other sites after site data is lost, ensuring data synchronization between the faulty site and other sites, thereby making the multi-site data synchronization function more powerful and improving the market competitiveness of distributed object products.

[0040] The beneficial effects of the multi-site data synchronization system, device, and medium provided in this application are the same as above. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0042] Figure 1 A flowchart of a multi-site data synchronization method provided in an embodiment of the present application;

[0043] Figure 2 A mapping relationship diagram of bucket information, bucket shards, and objects provided in an embodiment of the present application;

[0044] Figure 3 A flowchart of a multi-site data synchronization method provided in an embodiment of the present application;

[0045] Figure 4 A schematic diagram of a multi-site data synchronization system provided in an embodiment of the present application;

[0046] Figure 5 This is a structural diagram of a multi-site data synchronization device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0048] The core of this application is to provide a multi-site data synchronization method, system, device and medium to solve the problem of being unable to recover data after a failure of one or some sites in a multi-site situation, thereby ensuring data synchronization between the failed site and other sites and improving the market competitiveness of distributed object products.

[0049] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0050] With the explosive growth of massive amounts of unstructured data, distributed object storage has become the storage cornerstone for users building data center architectures. More and more critical businesses are integrating with distributed object storage. In this scenario, distributed object storage products must meet customers' storage needs for massive amounts of data while also providing disaster recovery and backup for critical data.

[0051] Disaster recovery typically requires the establishment of two data centers: one data center, responsible for daily production operations; the other, a disaster recovery center, responsible for application system operations after a disaster. Disaster recovery options include those where the data center and disaster recovery center are located in the same city or within 200 kilometers of each other, and those located in different locations, where the data center and disaster recovery center are located more than 200 kilometers apart. In-city disaster recovery is generally used to protect against disasters caused by fire, building damage, power failures, computer systems, and human sabotage. In-city disaster recovery protects against not only significant risks like fire and building damage, but also risks such as war, earthquakes, and floods. Because both in-city and off-city disaster recovery have their own strengths, to achieve optimal disaster prevention, a backup center is typically established in both the same city and a different location. The purpose of multi-site object storage is to achieve data synchronization between multiple sites. However, the current multi-site synchronization is to synchronize data to the backup site after writing data to the primary site to achieve a backup effect. When the primary site fails, data access is switched to the backup site to ensure normal access. The original data of the primary site cannot be restored to continue to maintain data synchronization, and it only serves a backup purpose.

[0052] Based on the above requirements, the present invention provides a multi-site data synchronization method. Figure 1 As shown, Figure 1 This is a flowchart of a multi-site data synchronization method provided in an embodiment of the present application, such as Figure 1 Shown, including:

[0053] S10: Obtain data information of each site.

[0054] Specifically, the multi-site data synchronization method provided in the embodiment of the present application starts a thread through a wireless access gateway (RGW) to periodically obtain data information of each site of the distributed object storage system. The embodiment of the present application does not limit the period, and is usually designed according to demand or adopts a default period. It should also be noted that each site refers to multiple sites. Since the multi-site data synchronization method provided in the present application can synchronize the data recovery of the failed site with other sites, two sites are usually preferred. However, it should be noted that the embodiment of the present application does not limit the specific number of sites, and the number is often determined according to demand. In addition, the embodiment of the present application does not limit the specific content of the data information. The data information usually includes bucket lists, bucket quotas, the number of objects, etc.

[0055] S11: Determine whether each site has missing data by comparing the data information of each site.

[0056] Specifically, after the RGW starts the thread to obtain data information, it compares the data information of each site one by one. When the data information is different, it indicates that there is a site with missing data. More specifically, when the data information included in other sites is not included in one site, it indicates that there is data missing at this site. When the data information included in one site is not included in other sites, it indicates that the other sites are missing data. By comparing all the data information, the site with missing data and the missing data can be found. It should be noted that when the data information includes bucket lists, bucket quotas, and the number of objects, the information lists are compared first, then the bucket quotas, and finally the number of objects, until the site with missing data and the missing data information are found.

[0057] S12: If yes, obtain the missing data through other sites and write the missing data to the site.

[0058] Specifically, when it is confirmed that there is missing data in each site, since the data missing is obtained by comparing data information, it means that at least one of the other sites includes the data missing from the site with missing data. Therefore, the missing data is obtained by other sites, and then the missing data is written into the site with missing data to keep the data between multiple sites synchronized.

[0059] S13: If not, return to the step of obtaining data information of each site.

[0060] Specifically, when it is confirmed that there is no site data missing in each site, the data information of each site is continuously acquired, thereby continuing to determine whether there is site data missing in each site, so as to ensure that the data between the sites remains synchronized.

[0061] The multi-site data synchronization method provided in the embodiment of the present application determines whether there is any site missing data in each site by comparing the data information of each site, and when it is confirmed that there is a site missing data, the missing data is obtained from other sites and written to the site where the missing data is missing. This not only realizes the redundancy of multi-site data to achieve the purpose of backup, but also can restore data through other sites after site data is lost, ensuring data synchronization between the faulty site and other sites, thereby making the multi-site data synchronization function more powerful and improving the market competitiveness of distributed object products.

[0062] Based on the above embodiment, the embodiment of the present application is a preferred embodiment, wherein the site includes a first site and a second site;

[0063] Accordingly, the data information obtained from each site includes:

[0064] First data information of a first site and second data information of a second site are obtained.

[0065] Specifically, since the multi-site data synchronization method provided in the embodiment of the present application can write the missing data by obtaining data from other sites after data loss occurs at a site, in order to reduce the cost of the distributed storage system, as a preferred embodiment, the number of sites is limited to two, namely the first site and the second site. Accordingly, when obtaining data information of each site, the first data information of the first site and the second data information of the second site are obtained, so as to determine whether there is data loss in the first site and the second site by comparing the first data information and the second data information, thereby ensuring that the data of the first site and the second site remain synchronized.

[0066] It should be noted that the first site and the second site are only used to represent two sites. There is no distinction between the primary site and the secondary site between the first site and the second site. The two sites can be enabled according to usage.

[0067] The multi-site data synchronization method provided in the embodiment of the present application limits the sites to include two sites, a first site and a second site, which reduces the cost required to establish the site while also achieving data redundancy for the purpose of backup. In addition, after data is lost at a site, data can be restored through other sites to ensure data synchronization between the failed site and other sites, thereby making the multi-site data synchronization function more powerful and improving the market competitiveness of distributed object products.

[0068] Based on the above embodiment, as a preferred embodiment of the present application, the first data information includes a first bucket list, a first bucket quota, and a first number of objects, and the second data information includes a second bucket list, a second bucket quota, and a second number of objects.

[0069] Specifically, the multi-site data synchronization method provided in the embodiment of the present application further defines the data information, and the defined data signals include bucket list, bucket quota, and object number. Each storage cluster consists of countless buckets, each bucket consists of 1024 shards, and each bucket shard stores many objects. The number of objects on the bucket shard is recorded on the bucket shard, and the total number of objects on all shards of the bucket is recorded on the bucket information. The total number of all objects in the bucket is the bucket quota, such as Figure 2 As shown, Figure 2 A mapping diagram of bucket information, bucket shards, and objects provided in an embodiment of the present application. Therefore, the data information is limited to include a bucket list, a bucket quota, and a number of objects. The first data information corresponds to the first bucket list, the first bucket quota, and the first number of objects. The second data information corresponds to the second bucket list, the second bucket quota, and the second number of objects. This facilitates comparing the first bucket list with the second bucket list, the first bucket quota with the second bucket quota, and the first number of objects with the second number of objects to determine whether data is missing from the first site or the second site.

[0070] The multi-site data synchronization method provided in the embodiment of the present application limits the first data information to include a first bucket list, a first bucket quota, and a first number of objects, and the second data information to include a second bucket list, a second bucket quota, and a second number of objects, as a basis for determining whether there is site missing data in the first site and the second site. In addition, after data is lost at a site, data can be restored through another site to ensure data synchronization between the failed site and the other sites.

[0071] Based on the above embodiment, as a preferred embodiment, the embodiment of the present application further limits the determination of whether data is missing at each site by comparing the data information of each site to include:

[0072] Determine whether the first bucket list is consistent with the second bucket list;

[0073] If the first bucket list is inconsistent with the second bucket list, it is determined that the first site or the second site is missing data;

[0074] If the first bucket list is consistent with the second bucket list, determine whether the first bucket quota is consistent with the second bucket quota;

[0075] If the first bucket quota is inconsistent with the second bucket quota, it is determined that the first site or the second site is missing data;

[0076] If the first bucket quota is consistent with the second bucket quota, it is determined that there is no missing data at the first site or the second site.

[0077] Specifically, after obtaining the first bucket list and the second bucket list, determine whether the two bucket lists are consistent. If the two bucket lists are inconsistent, that is, there are differences in the bucket shards, it means that at least one of the first site and the second site has missing data; if the two bucket lists are consistent, it is necessary to further compare whether the first bucket quota is consistent with the second bucket quota; when the first bucket quota is inconsistent with the second bucket quota, it means that at least one of the first site and the second site has missing data; when the first bucket quota is inconsistent with the second bucket quota, it means that at least one of the first site and the second site has missing data; when the first bucket quota is consistent with the second bucket quota, it means that there is no data missing between the first site and the second site.

[0078] The multi-site data synchronization method provided in the embodiment of the present application determines whether there is a basis for site missing data in the first site and the second site by gradually comparing the first bucket list with the second bucket list, and the first bucket quota with the second bucket quota. After data is lost at a site, data can be restored through another site to ensure data synchronization between the faulty site and other sites.

[0079] Based on the above embodiment, as a preferred embodiment of the present application, when the first bucket list and the second bucket list are inconsistent, the site that obtains the missing data through other sites and writes the missing data includes:

[0080] Create a first bucket shard that is consistent with the other site in the site where the data is missing based on the first bucket list and the second bucket list and synchronize the bucket information;

[0081] Get a list of objects on a bucket shard at another site;

[0082] Write the objects on the other site to the first bucket shard based on the object list.

[0083] Specifically, taking the case where data is missing at the first site as an example, a first bucket shard is created at the first site to synchronize bucket information with the second site. After obtaining a list of objects on the bucket shard at the second site, objects that are more on the second site than on the first site are written to the first bucket shard based on the obtained object list, thereby completing data synchronization between the first site and the second site.

[0084] It should be noted that the embodiment of the present application only takes the case where data is missing at the first site as an example and does not constitute a limitation. The processing method when data is missing at the second site is the same as that when data is missing at the first site, and will not be repeated here.

[0085] The multi-site data synchronization method provided in the embodiment of the present application, when the first bucket list and the second bucket list are inconsistent, creates a new bucket shard in the site where the data is missing to write the missing data, obtains the missing object through the object column of the bucket shard on the normal site, and writes the missing object to the site where the data is missing, so as to achieve the purpose of data recovery at the site where the data is missing, thereby ensuring data synchronization between the faulty site and other sites.

[0086] Based on the above embodiment, as a preferred embodiment, the embodiment of the present application, when the first bucket quota is inconsistent with the second bucket quota, is limited to obtaining the missing data through other sites and writing the missing data to the site including:

[0087] Compare the number of first objects on the bucket shard at the first site and the number of second objects on the bucket shard at the second site to see if they are consistent;

[0088] If they are consistent, continue to compare whether the number of the first object on the bucket shard of the first site is consistent with the number of the second object on the bucket shard of the second site;

[0089] If they are inconsistent, the missing objects of the site where the data is missing are obtained according to the first object list of the first site and the second object list of the second site, and the objects are written to the site where the data is missing.

[0090] Specifically, when the first bucket list and the second bucket list are consistent, it is necessary to continue to compare whether the first bucket quota is consistent with the second bucket quota to determine whether there is data missing in any site between the first site and the second site. After confirming that the first bucket quota is inconsistent with the second bucket quota, the first object number of each bucket shard in the first site is compared with the second object number of each bucket shard in the second site. If the object number is consistent, continue to compare the next bucket shard until the object number is inconsistent. When the first object number is inconsistent with the second object number, the missing object of the site with missing data is obtained through the first object list of the first site and the second object list of the second site to write to the site of the missing object. For example, when the first object number is less than the second object number, it means that the first site is missing data. The missing data is obtained through the comparison result of the first object list and the second object list, and the missing data is written to the first site.

[0091] The multi-site data synchronization method provided in the embodiment of the present application finds the missing object by determining whether the first object number is consistent with the second object number when the first bucket quota is inconsistent with the second bucket quota, and writes the missing object to the site where the data is missing, so as to achieve the purpose of data recovery at the site where the data is missing, thereby ensuring data synchronization between the faulty site and other sites.

[0092] Based on the above embodiment, as a preferred embodiment of the present application, when the number of first objects on the bucket shard of the first site is inconsistent with the number of second objects on the bucket shard of the second site, obtaining the missing objects of the site with missing data according to the first object list of the first site and the second object list of the second site, and writing the objects to the site with missing data further includes:

[0093] Return to the step of obtaining the first data information of the first site and the second data information of the second site.

[0094] Specifically, when the number of first objects on the bucket shard of the first site is inconsistent with the number of second objects on the bucket shard of the second site, the missing objects of the site with missing data are obtained according to the first object list of the first site and the second object list of the second site, and after writing the objects to the site with missing data, the process returns to the step of obtaining the first data information of the first site and the second data information of the second site, thereby continuing to compare whether the data information of the first site and the second site are consistent, so as to ensure data synchronization between the two sites by writing the missing data when the data information of the two sites is inconsistent.

[0095] The multi-site data synchronization method provided in the embodiment of the present application reacquires data information of the two sites after writing missing data to the corresponding site where the data is missing, thereby ensuring data synchronization between the two sites.

[0096] The present application provides an embodiment of a multi-site data synchronization method application, wherein the sites include a first site and a second site. Figure 3 This is a flowchart of a multi-site data synchronization method application provided in an embodiment of the present application, such as Figure 3 Shown, including:

[0097] S20: Get the bucket lists of the two sites;

[0098] S21: Compare the bucket lists of the two sites to see if they are consistent;

[0099] If the bucket lists of the two sites are inconsistent, then proceed to step S22: synchronize the missing bucket information of the site with missing data and create bucket shards, then proceed to step S221;

[0100] S221: Get the object list on the bucket shard of the normal site;

[0101] S222: Obtain the missing objects of the site with missing data from the normal site and write them to the site with missing data;

[0102] After executing S212 , the process returns to the step of obtaining the bucket lists of the two sites;

[0103] If the two bucket lists are consistent, proceed to step S23: compare the bucket quotas of the two sites to see if they are consistent;

[0104] If the wild quotas of the two sites are the same, return to the step of comparing the quotas of the two sites.

[0105] If the wild quotas of the two sites are consistent, step S24 is executed: comparing the number of objects on the bucket shards of the two sites to see if they are consistent;

[0106] If so, return to the step of comparing the quotas of the two sites to see if they are consistent;

[0107] If not, proceed to step S25: compare the object lists on the bucket shards of the two sites;

[0108] S26: Obtain the missing objects of the site with missing data from the normal site and write them to the site with missing data;

[0109] Return to the step of obtaining the two site bucket lists.

[0110] Specifically, the first bucket list of the first site and the second bucket list of the second site are first obtained and compared to see if they are consistent. If the first bucket list and the second bucket list are inconsistent, the site with missing data obtains the missing communication information compared to the normal site, creates bucket information and bucket shards at the site with missing data, obtains the object list of the normal site, and writes the data of the normal site to the site with missing data according to the object list. If the first bucket list and the second bucket list are consistent, the first bucket quota of the first site is obtained and compared with the second bucket quota. If the first bucket quota and the second bucket quota are inconsistent, the first object count of the first site and the second object count of the second site are obtained and compared to see if they are consistent. If the object counts are inconsistent, the object lists of the first site and the second site are obtained and compared to obtain the missing data and the corresponding object list of the site with missing data. Finally, the missing objects are obtained according to the object list and written to the site with missing data.

[0111] The multi-site data synchronization method provided in the embodiment of the present application determines whether there is any site missing data by comparing the data information of the two sites. When it is confirmed that there is a site missing data, the missing data is obtained from other sites and written to the site where the missing data is found. This method can not only achieve the redundancy of multi-site data for the purpose of backup, but also restore data through other sites after site data is lost, ensuring data synchronization between the faulty site and other sites, thereby making the multi-site data synchronization function more powerful and improving the market competitiveness of distributed object products.

[0112] In the above embodiments, a multi-site data synchronization method is described in detail. This application also provides corresponding embodiments of a multi-site data synchronization apparatus. It should be noted that this application describes the embodiments of the apparatus from two perspectives: one is based on the functional module perspective, and the other is based on the hardware perspective.

[0113] Based on the perspective of functional modules, this application provides a multi-site data synchronization system. Figure 4 A schematic diagram of a multi-site data synchronization system provided in an embodiment of the present application is shown in FIG. Figure 4 Shown, including:

[0114] The first acquisition module 10 is used to obtain data information of each site;

[0115] A judgment module 11 is used to judge whether each site is missing data by comparing the data information of each site;

[0116] The second acquisition module 12 is used to, when data is missing at each site, acquire the missing data through other sites and write the missing data into the site;

[0117] The return module 13 is used to return to the step of obtaining data information of each site when there is no missing data for each site.

[0118] Since the embodiments of the system part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the system part, and will not be repeated here.

[0119] From the perspective of hardware, the present invention provides a multi-site data synchronization device. Figure 5 This is a structural diagram of a multi-site data synchronization device provided in another embodiment of the present application, such as Figure 5 As shown, the multi-site data synchronization device includes: a memory 20 for storing computer programs;

[0120] The processor 21 is configured to implement the steps of the multi-site data synchronization method mentioned in the above embodiment when executing a computer program.

[0121] The multi-site data synchronization device provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer, or a desktop computer.

[0122] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of a digital signal processor (DSP), a field programmable gate array (FPGA), and a programmable logic array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0123] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201, wherein, after the computer program is loaded and executed by the processor 21, it can implement the relevant steps of the multi-site data synchronization method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 202 may include Windows, Unix, Linux, etc. The data 203 may include but is not limited to data information, etc.

[0124] In some embodiments, the multi-site data synchronization device may further include a display screen 22 , an input / output interface 23 , a communication interface 24 , a power supply 25 , and a communication bus 26 .

[0125] Those skilled in the art will understand that Figure 5 The structure shown in the figure does not constitute a limitation to the multi-site data synchronization device, and may include more or fewer components than shown in the figure.

[0126] The multi-site data synchronization device provided in an embodiment of the present application includes a memory and a processor. When the processor executes a program stored in the memory, it can implement the following method: a multi-site data synchronization method.

[0127] Finally, the present application also provides an embodiment corresponding to a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps described in the above method embodiment.

[0128] It is understandable that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and executes all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0129] The above is a detailed introduction to the multi-site data synchronization method, system, device and medium provided by the present application. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

[0130] It should also be noted that, in this specification, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

Claims

1. A multi-site data synchronization method, characterized in that: include: Obtain data information of each site; Determining whether each site is missing data by comparing the data information of each site; If so, obtaining the missing data through the other sites and writing the missing data to the site; If not, return to the step of obtaining data information of each site; The sites include a first site and a second site; Accordingly, the acquisition of data information of each site includes: Acquire first data information of the first site and second data information of the second site; The first data information includes a first bucket list, a first bucket quota, and a first number of objects; the second data information includes a second bucket list, a second bucket quota, and a second number of objects; The determining whether each site is missing data by comparing the data information of each site includes: Determining whether the first bucket list is consistent with the second bucket list; If the first bucket list is inconsistent with the second bucket list, determining that the first site or the second site is missing data; If the first bucket list is consistent with the second bucket list, determining whether the first bucket quota is consistent with the second bucket quota; If the first bucket quota is inconsistent with the second bucket quota, determining that the first site or the second site is missing data; If the first bucket quota is consistent with the second bucket quota, it is determined that the first site or the second site has no missing data; When the first bucket quota is inconsistent with the second bucket quota, the site acquiring the missing data through the other site and writing the missing data includes: Comparing whether the number of first objects on the bucket shards of the first site is consistent with the number of second objects on the bucket shards of the second site; If they are consistent, continue to compare whether the first number of objects on the bucket shard of the first site is consistent with the second number of objects on the bucket shard of the second site; If they are inconsistent, obtaining the missing objects of the site where the data is missing according to the first object list of the first site and the second object list of the second site, and writing the objects to the site where the data is missing; When the number of first objects on the bucket shards of the first site is inconsistent with the number of second objects on the bucket shards of the second site, after acquiring the missing objects of the site with missing data according to the first object list of the first site and the second object list of the second site and writing the objects to the site with missing data further includes: Return to the step of obtaining the first data information of the first site and the second data information of the second site; Among them, the multi-site data synchronization method starts a thread through a wireless access gateway to periodically obtain data information of each site of the distributed object storage system, so that after the wireless access gateway starts the thread to obtain the data information, the data information of each site is compared one by one. When the data information is different, the site with missing data is determined.

2. The multi-site data synchronization method according to claim 1, characterized in that: When the first bucket list is inconsistent with the second bucket list, the site acquiring the missing data through the other site and writing the missing data includes: Creating a first bucket shard that is consistent with the other site in the site where the data is missing according to the first bucket list and the second bucket list and synchronizing the bucket information; Obtain a list of objects on the bucket shard of the other site; The object on the other site is written into the first bucket shard according to the object list.

3. A multi-site data synchronization system, characterized in that: include: The first acquisition module is used to obtain data information of each site; A judgment module, configured to judge whether each site is missing data by comparing the data information of each site; A second acquisition module is configured to, when each site is missing data, acquire the missing data through other sites and write the missing data into the site; A return module, configured to return to the step of obtaining data information of each site when there is no missing data for each site; The sites include a first site and a second site; Accordingly, the acquisition of data information of each site includes: Acquire first data information of the first site and second data information of the second site; The first data information includes a first bucket list, a first bucket quota, and a first number of objects; the second data information includes a second bucket list, a second bucket quota, and a second number of objects; The determining whether each site is missing data by comparing the data information of each site includes: Determining whether the first bucket list is consistent with the second bucket list; If the first bucket list is inconsistent with the second bucket list, determining that the first site or the second site is missing data; If the first bucket list is consistent with the second bucket list, determining whether the first bucket quota is consistent with the second bucket quota; If the first bucket quota is inconsistent with the second bucket quota, determining that the first site or the second site is missing data; If the first bucket quota is consistent with the second bucket quota, it is determined that the first site or the second site has no missing data; When the first bucket quota is inconsistent with the second bucket quota, the site acquiring the missing data through the other site and writing the missing data includes: Comparing whether the number of first objects on the bucket shards of the first site is consistent with the number of second objects on the bucket shards of the second site; If they are consistent, continue to compare whether the first number of objects on the bucket shard of the first site is consistent with the second number of objects on the bucket shard of the second site; If they are inconsistent, obtaining the missing objects of the site where the data is missing according to the first object list of the first site and the second object list of the second site, and writing the objects to the site where the data is missing; When the number of first objects on the bucket shards of the first site is inconsistent with the number of second objects on the bucket shards of the second site, after acquiring the missing objects of the site with missing data according to the first object list of the first site and the second object list of the second site and writing the objects to the site with missing data further includes: Return to the step of obtaining the first data information of the first site and the second data information of the second site; Among them, the multi-site data synchronization method starts a thread through a wireless access gateway to periodically obtain data information of each site of the distributed object storage system, so that after the wireless access gateway starts the thread to obtain the data information, the data information of each site is compared one by one. When the data information is different, the site with missing data is determined.

4. A multi-site data synchronization device, characterized in that: including a memory for storing a computer program; A processor is configured to implement the steps of the multi-site data synchronization method as claimed in claim 1 or 2 when executing the computer program.

5. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the multi-site data synchronization method according to claim 1 or 2.

Images