Data query method, device and readable storage medium

Abstract

The present application discloses a data query method, device, and readable storage medium, relating to the field of electronic information. The data query method provided by the present application is applied to a source site, obtains a synchronization mark in a log shard from a target site, thereby obtaining a corresponding bucket, and sends the bucket name to the corresponding target site, thereby obtaining the quota of the corresponding bucket in synchronization at the target site and the quota of the corresponding bucket at the source site. Based on the ratio of the two, the corresponding synchronization progress can be obtained. The synchronization speed can also be obtained by contacting the cycle time. Compared with the previous data query only through software, since the synchronization progress and speed can be directly obtained, the problem of the inability to accurately query the data synchronization progress and speed between the source site and the destination site in the current data synchronization function is solved.

Description

Technical Field

[0001] The present application relates to the field of electronic information, and in particular to a data query method, device, and readable storage medium. Background Art

[0002] With the recent growth of data transmission, data query has become a crucial feature in distributed object storage systems. It's currently being applied to numerous market projects across diverse industries, providing an indispensable boost to the stable development of businesses. For example, consider sites A and B. Customers connect their front-end services to site A, writing data to site A. This data is then synchronized to site B for backup. Data writing can support up to 512 concurrent threads, while data queries typically only execute on a single thread.

[0003] In the existing management logic, current data queries provide a data query status query function, which includes a total of three synchronization states: initialization, synchronizing, and synchronization completed. When the data query status is synchronizing, the number of remaining shards and the specific remaining shards can also be viewed. The current distributed object storage data query function does not yet provide an effective and intuitive method for users and operations personnel to quickly and easily understand the progress and speed of data queries. Business site data is continuously written without clear start and end times, and it is impossible to predict the total amount of data to be synchronized. In addition, data queries are performed at all times and at uneven speeds, making it impossible to intuitively query the corresponding synchronization progress.

[0004] In view of the above technologies, finding a more intuitive data query method is an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this application is to provide a data query method, device and readable storage medium to solve the problem that data of business sites are continuously written without clear start and end times, and it is impossible to predict how much data to synchronize in total. In addition, data queries are also carried out at all times and at uneven speeds, making it impossible to intuitively query the corresponding synchronization progress.

[0006] To solve the above technical problems, the present application provides a data query method, which is applied to a source site. The method includes:

[0007] Read the synchronization markers of all log shards stored in the storage pool of the target site;

[0008] Obtaining the names of all buckets requiring data synchronization according to the synchronization mark and sending them to the target site, so that the target site can save the buckets and record the quota of the buckets in the target site;

[0009] The quota of the bucket in the source site is obtained and sent to the target site, so that the target site can obtain the progress of data synchronization by comparing the quota of the bucket in the target site with the quota of the bucket in the source site.

[0010] Preferably, obtaining the names of all buckets requiring synchronization data according to the synchronization mark and sending the names to the target site includes:

[0011] Parse the log shard to be queried, and then, based on the synchronization flag carried in the request, send a request to the current storage pool to list the entries of the log shard, and start listing from the specified synchronization flag until all entries on the log shard are listed;

[0012] Parse each of the listed entries in turn, obtain the bucket shard corresponding to the log of the entry, and determine the name of the bucket corresponding to the bucket shard based on the bucket shard name;

[0013] The parsed names of all the buckets that need to synchronize data are sent to the target site.

[0014] Preferably, the synchronization mark of reading all log shards stored in the storage pool of the target site includes:

[0015] Obtain the http request sent by the synchronization mark of the log segment, and parse the http request to obtain the corresponding synchronization mark.

[0016] To solve the above problem, the present application also provides a data query method, which is applied to a target site, and the method includes:

[0017] Get the synchronization markers of all log shards stored in the storage pool and send them to the source site;

[0018] Obtaining the names of all buckets requiring synchronization data from the source site according to the synchronization mark;

[0019] Saving the bucket and recording the quota of the bucket in the target site;

[0020] The quota of the bucket in the source site is obtained, and the progress of data synchronization is obtained by comparing the quota of the bucket in the target site with the quota of the bucket in the source site.

[0021] Preferably, after saving the bucket and recording the quota of the bucket in the target site, the method further includes:

[0022] Send the information of the bucket to the management software for display so that users can view it;

[0023] After obtaining the progress of data synchronization by comparing the quota of the bucket in the target site with the quota of the bucket in the source site, the method further includes:

[0024] The progress of the data synchronization is sent to the management software for display so that the user can view it.

[0025] Preferably, after obtaining the progress of data synchronization by comparing the quota of the bucket in the target site with the quota of the bucket in the source site, the method further includes:

[0026] The total quota of the destination site of the bucket being synchronized is compared with the total quota of the destination site saved in the previous query cycle, and divided by the query cycle to obtain the overall synchronization speed of the cluster.

[0027] To solve the above problems, the present application further provides a data query device, which is applied to a source site and includes:

[0028] The reading module is used to read the synchronization markers of all log shards stored in the storage pool of the target site;

[0029] a sending module, configured to obtain the names of all buckets requiring synchronization data according to the synchronization mark and send the names to the target site, so that the target site can save the buckets and record the quota of the buckets in the target site;

[0030] The quota module is used to obtain the quota of the bucket in the source site and send it to the target site, so that the target site can obtain the progress of data synchronization by comparing the quota of the bucket in the target site with the quota of the bucket in the source site.

[0031] To solve the above problems, the present application provides a data query device, which is applied to a target site and includes:

[0032] The acquisition module is used to obtain the synchronization marks of all log shards stored in the storage pool and send them to the source site;

[0033] A synchronization module, configured to obtain the names of all buckets requiring synchronization data from the source site according to the synchronization mark;

[0034] a saving module, configured to save the bucket and record the quota of the bucket in the target site;

[0035] The comparison module is used to obtain the quota of the bucket in the source site, and obtain the progress of data synchronization by comparing the quota of the bucket in the target site with the quota of the bucket in the source site.

[0036] To solve the above problems, the present application further provides a data query device, comprising a memory for storing a computer program;

[0037] A processor is used to implement the steps of the above-mentioned data query method when executing the computer program.

[0038] To solve the above problems, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the data query method as described above are implemented.

[0039] The data query method provided in the present application is applied to the source site, and obtains the corresponding bucket by obtaining the synchronization mark in the log shard from the target site, and sends the bucket name to the corresponding target site, so as to obtain the quota of the corresponding bucket in the synchronization in the target site and the quota of the corresponding bucket in the source site. According to the ratio of the two, the corresponding synchronization progress can be obtained, and the synchronization speed can also be obtained through the contact cycle time. Compared with the previous data query only through software, since the synchronization progress and speed can be directly obtained, the problem of the inability to accurately query the data synchronization progress and speed between the source site and the destination site in the current data synchronization function is solved, the data synchronization function is optimized, and a better user experience is brought to users, so that when data synchronization is abnormal, timely adjustments can be made.

[0040] The data query method, data query device, and computer-readable storage medium provided in this application for a target site correspond to the above-mentioned data query detection method, and have the same beneficial effects as above. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0042] Figure 1 A flow chart of a data query method provided in an embodiment of the present application;

[0043] Figure 2 A flowchart of a data query method provided in another embodiment of the present application;

[0044] Figure 3 A schematic diagram of a data query device provided in an embodiment of the present application;

[0045] Figure 4 A schematic diagram of another data query device provided in an embodiment of the present application;

[0046] Figure 5 A schematic diagram of a data query device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0048] The core of this application is to provide a data query method, device and readable storage medium to solve the problem that data at business sites are continuously written without clear start and end times, and it is impossible to predict how much data to synchronize in total. In addition, data queries are also carried out at all times, and the speed is uneven, making it impossible to intuitively query the corresponding synchronization progress.

[0049] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0050] Figure 1 A flow chart of a data query method provided in an embodiment of the present application is shown as follows: Figure 1 As shown, the method includes:

[0051] S10: Read the synchronization markers of all log shards stored in the storage pool of the target site;

[0052] It should be noted that the data synchronization function in the distributed object storage module synchronizes data in units of bucket shards, and records the data that needs to be synchronized in the form of two-level logs. The first-level log has a total of 128 shards, and each shard stores the bucket shard where the data that needs to be synchronized is located. When the synchronization status on the current monitoring interface is in synchronization, the remaining shards displayed are the first-level log shards. Each bucket shard stores a second-level log, and each second-level log represents an object that needs to be synchronized on the bucket shard. In this embodiment, there is no limitation on the specific content of the log and the method of calling it.

[0053] S11: Obtain the names of all buckets that need to synchronize data based on the synchronization mark and send them to the target site;

[0054] The above steps are for the target site to save the bucket and record the quota of the bucket in the target site. It should be noted that the bucket is a data structure. The purpose of the data structure is to count data in a special way, so that we can quickly modify and query the part of the data we want. But generally when we want to count a set of data, users are more concerned about what these data are. For example, when counting a series, users are more concerned about what numbers are in the series, rather than how many times these numbers appear. The bucket breaks this status quo. As a "special" data structure, it counts how many times each data appears in the data set. In this embodiment, the specific content of the allocation of buckets and quotas is not specifically defined. Quotas are easy to understand. In this embodiment and the corresponding target site embodiment, it generally refers to

[0055] S12: Obtain the quota of the bucket in the source site and send it to the target site.

[0056] The above steps are for the target site to obtain the progress of data synchronization by comparing the quota of the bucket in the target site with the quota of the bucket in the source site.

[0057] This embodiment proposes a log-based method for querying data synchronization progress. Based on the synchronization flags of the first-level log shards at the destination site, the method searches for buckets on the corresponding first-level log shards at the source site that require synchronization. The method then filters the buckets requiring synchronization and calculates the quota difference between the source and destination sites for each bucket, ultimately determining the overall synchronization progress. By recording all buckets requiring synchronization at the destination site and the quota for each bucket, the method compares the data with the last query and divides it by the query interval to determine the overall synchronization speed between the master and slave sites.

[0058] The data query method provided in this embodiment is applied to the source site, and obtains the corresponding bucket by obtaining the synchronization mark in the log shard from the target site, and sends the bucket name to the corresponding target site, thereby obtaining the quota of the corresponding bucket in the synchronization in the target site and the quota of the corresponding bucket in the source site. According to the ratio of the two, the corresponding synchronization progress can be obtained, and the synchronization speed can also be obtained through the contact cycle time. Compared with the previous data query only through software, since the synchronization progress and speed can be directly obtained, the problem of the inability to accurately query the data synchronization progress and speed between the source site and the destination site in the current data synchronization function is solved, the data synchronization function is optimized, and a better user experience is brought to users, so that when data synchronization is abnormal, timely adjustments can be made.

[0059] The above embodiment does not limit the method for obtaining the bucket. Here, a preferred solution is provided. The names of all buckets that need to synchronize data are obtained according to the synchronization mark and sent to the target site, including:

[0060] Parse the log shard to be queried, and then send a request to the current storage pool to list the entries in the log shard based on the synchronization mark carried in the request. The request starts from the specified synchronization mark and continues until all entries in the log shard are listed.

[0061] Parse each listed entry in turn, obtain the bucket shard corresponding to the entry's log, and determine the name of the bucket corresponding to the bucket shard based on the bucket shard name;

[0062] Send the parsed names of all buckets that need to synchronize data to the target site.

[0063] The above embodiment does not limit the method for obtaining the synchronization mark. Here, a preferred solution is provided. Reading the synchronization marks of all log shards stored in the storage pool of the target site includes:

[0064] Get the HTTP request sent by the synchronization mark of the log shard, and parse the HTTP request to obtain the corresponding synchronization mark.

[0065] It should be noted that, in this embodiment, it is limited that the synchronization mark is transmitted to the corresponding site according to the http request, thereby ensuring the timeliness and speed of signal transmission.

[0066] Figure 2 A flowchart of a data query method provided in another embodiment of the present application is applied to a target site, such as Figure 2 As shown, the method includes:

[0067] S20: Obtain the synchronization markers of all log shards stored in the storage pool and send them to the source site;

[0068] S21: Obtain the names of all buckets that need to synchronize data based on the synchronization mark at the source site;

[0069] S22: Save the bucket and record the quota of the bucket in the target site;

[0070] S23: Obtain the quota of the bucket in the source site, and obtain the progress of data synchronization by comparing the quota of the bucket in the target site with the quota of the bucket in the source site.

[0071] The data query method provided in this embodiment is applied to the target site. The synchronization mark in the log shard is obtained from the target site, thereby obtaining the corresponding bucket, and the name of the bucket is sent to the corresponding target site, thereby obtaining the quota of the corresponding bucket in the synchronization in the target site and the quota of the corresponding bucket in the source site. According to the ratio of the two, the corresponding synchronization progress can be obtained. The synchronization speed can also be obtained through the contact cycle time. Compared with the previous data query only through software, since the synchronization progress and speed can be directly obtained, the problem of the inability to accurately query the data synchronization progress and speed between the source site and the destination site in the current data synchronization function is solved, the data synchronization function is optimized, and a better user experience is brought to users. When data synchronization is abnormal, timely adjustments can be made.

[0072] Considering that it is convenient for users to view and adjust the synchronization tasks in time, a preferred solution is provided here. After saving the bucket and recording the quota of the bucket in the target site, it also includes:

[0073] Send the bucket information to the management software for display so that users can view it;

[0074] After comparing the bucket quotas on the target site with those on the source site to determine the progress of data synchronization, the following steps are also performed:

[0075] The progress of data synchronization is sent to the management software for display so that users can view it.

[0076] It should be noted that in this embodiment, there is no limitation on the specific method of data synchronization and the content of the corresponding management software. It can be understood that for the management software, in order to facilitate user viewing, there needs to be a carrier that is easy to view, so it can be a mobile phone, calculator, etc., which is not specifically limited in this embodiment.

[0077] Considering the speed test, a preferred solution is provided here. After obtaining the data synchronization progress by comparing the bucket quotas in the target site with the bucket quotas in the source site, the following is also included:

[0078] Compare the total quota of the destination site of the bucket being synchronized with the total quota of the destination site saved in the previous query cycle, and divide it by the query cycle to get the overall synchronization speed of the cluster.

[0079] It should be noted that the quota here is the same as the above quota, so the corresponding speed can be obtained based on historical reference. That is, you only need to compare the current synchronization progress with the previous synchronization progress, so that you can get the current completed number based on the progress, and divide it by the cycle to get the corresponding synchronization speed. Compared with the previous method of obtaining synchronization speed through software, this method is more stable and does not consume the computer's own computing power, thereby increasing operating efficiency.

[0080] In the above embodiments, the data query method is described in detail. This application also provides corresponding embodiments of the data query device. It should be noted that this application describes the embodiments of the device from two perspectives: one is based on the functional module perspective, and the other is based on the hardware perspective.

[0081] Figure 3 A schematic diagram of a data query device provided in an embodiment of the present application is applied to a source site, such as Figure 3 As shown, the device includes:

[0082] Reading module 1, used to read the synchronization marks of all log shards stored in the storage pool of the target site;

[0083] Sending module 2, used to obtain the names of all buckets that need to synchronize data according to the synchronization mark and send them to the target site, so that the target site can save the buckets and record the quota of the buckets in the target site;

[0084] The quota module 3 is used to obtain the quota of the bucket in the source site and send it to the target site, so that the target site can obtain the progress of data synchronization by comparing the quota of the bucket in the target site with the quota of the bucket in the source site.

[0085] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, and will not be repeated here.

[0086] Figure 4 A schematic diagram of another data query device provided in an embodiment of the present application is applied to a target site, such as Figure 4 As shown, the device includes:

[0087] An acquisition module 10 is configured to acquire synchronization marks of all log shards stored in the storage pool and send them to the source site;

[0088] Synchronization module 11, used to obtain the names of all buckets that need to synchronize data based on the synchronization mark at the source site;

[0089] A saving module 12 is used to save the bucket and record the quota of the bucket in the target site;

[0090] The comparison module 13 is used to obtain the quota of the bucket in the source site, and obtain the progress of data synchronization by comparing the quota of the bucket in the target site with the quota of the bucket in the source site.

[0091] The data query device provided in this embodiment is applied to the target site or the source site, and obtains the corresponding bucket by obtaining the synchronization mark in the log shard from the target site, and sends the bucket name to the corresponding target site, thereby obtaining the quota of the corresponding bucket in the synchronization in the target site and the quota of the corresponding bucket in the source site. According to the ratio of the two, the corresponding synchronization progress can be obtained, and the synchronization speed can also be obtained through the contact cycle time. Compared with the previous data query only through software, since the synchronization progress and speed can be directly obtained, the problem of the inability to accurately query the data synchronization progress and speed between the source site and the destination site in the current data synchronization function is solved, the data synchronization function is optimized, and a better user experience is brought to users, so that when data synchronization is abnormal, adjustments can be made in time.

[0092] Figure 5 A schematic diagram of a data query device provided in another embodiment of the present application is shown as follows: Figure 5 As shown, the data query device includes: a memory 20 for storing computer programs;

[0093] The processor 21 is configured to implement the steps of the data query method mentioned in the above embodiment when executing a computer program.

[0094] The data query device provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer, or a desktop computer.

[0095] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of a digital signal processor (DSP), a field programmable gate array (FPGA), and a programmable logic array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0096] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201, wherein, after the computer program is loaded and executed by the processor 21, it can implement the relevant steps of the data query method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 202 may include Windows, Unix, Linux, etc. The data 203 may include but is not limited to the data involved in the data query method, etc.

[0097] In some embodiments, the data query device may further include a display screen 22 , an input / output interface 23 , a communication interface 24 , a power supply 25 , and a communication bus 26 .

[0098] Those skilled in the art will understand that Figure 5 The structure shown in the figure does not constitute a limitation on the data query device, and may include more or fewer components than shown in the figure.

[0099] The data query device provided in the embodiment of the present application includes a memory and a processor. When the processor executes the program stored in the memory, it can implement the following method: the data query method involved in the above embodiment.

[0100] Since the embodiments of the apparatus part correspond to the embodiments of the method part, the embodiments of the apparatus part and their corresponding beneficial effects can be found in the description of the embodiments of the method part, which will not be repeated here.

[0101] The data query device provided in this embodiment includes a memory and a processor. When the storage content in the memory is read by the processor, it can be achieved by obtaining the synchronization mark in the log shard from the target site, thereby obtaining the corresponding bucket, and sending the bucket name to the corresponding target site, thereby obtaining the quota of the corresponding bucket in the synchronization in the target site and the quota of the corresponding bucket in the source site. According to the ratio of the two, the corresponding synchronization progress can be obtained, and the synchronization speed can also be obtained through the contact cycle time. Compared with the previous data query only through software, since the synchronization progress and speed can be directly obtained, the problem of the inability to accurately query the data synchronization progress and speed between the source site and the destination site in the current data synchronization function is solved, the data synchronization function is optimized, and a better user experience is brought to users, so that when data synchronization is abnormal, adjustments can be made in time.

[0102] Finally, the present application also provides an embodiment corresponding to a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps described in the above method embodiment.

[0103] It is understandable that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of the method of each embodiment of the present application. The aforementioned storage medium includes: various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0104] Since the embodiments of the readable storage medium part correspond to the embodiments of the method part, the embodiments of the device part and their corresponding beneficial effects can be found in the description of the embodiments of the method part, which will not be repeated here.

[0105] The computer-readable storage medium provided in this embodiment can obtain the synchronization mark in the log shard from the target site when the storage content is read, thereby obtaining the corresponding bucket, and sending the bucket name to the corresponding target site, thereby obtaining the quota of the corresponding bucket in the synchronization in the target site and the quota of the corresponding bucket in the source site. According to the ratio of the two, the corresponding synchronization progress can be obtained, and the synchronization speed can also be obtained through the contact cycle time. Compared with the previous data query only through software, since the synchronization progress and speed can be directly obtained, the problem of the inability to accurately query the data synchronization progress and speed between the source site and the destination site in the current data synchronization function is solved, the data synchronization function is optimized, and a better user experience is brought to users, so that when data synchronization is abnormal, adjustments can be made in time.

[0106] The above is a detailed introduction to a data query method, device and computer-readable storage medium provided by the present application. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

[0107] It should also be noted that, in this specification, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the aforementioned elements.

Claims

1. A data query method, characterized in that: Applied to the source site, the method includes: Reading synchronization marks of all log shards stored in the storage pool of the target site; wherein the log shards are first-level log shards; Obtaining the names of all buckets requiring data synchronization according to the synchronization mark and sending them to the target site, so that the target site can save the buckets and record the quota of the buckets in the target site; Obtaining the quota of the bucket in the source site and sending it to the target site, so that the target site can obtain the progress of data synchronization by comparing the quota of the bucket in the target site with the quota of the bucket in the source site; The acquiring, according to the synchronization mark, the names of all buckets requiring synchronization data and sending the names to the target site includes: Parsing the log shard to be queried, and then sending a request to the current storage pool to list the entries of the log shard according to the synchronization mark carried in the request, and starting from the specified synchronization mark until all the entries on the log shard are listed; Parse each of the listed entries in turn, obtain the bucket shard corresponding to the log of the entry, and determine the name of the bucket corresponding to the bucket shard based on the bucket shard name; The parsed names of all the buckets requiring data synchronization are sent to the target site.

2. The data query method according to claim 1, characterized in that: The synchronization mark of reading all log shards stored in the storage pool of the target site includes: Obtain the http request sent by the synchronization mark of the log segment, and parse the http request to obtain the corresponding synchronization mark.

3. A data query method, characterized in that: Applied to a target site, the method includes: Get the synchronization markers of all log shards stored in the storage pool and send them to the source site; Obtaining the names of all buckets requiring synchronization data from the source site according to the synchronization mark; Saving the bucket and recording the quota of the bucket in the target site; Obtaining the quota of the bucket in the source site, and obtaining the progress of data synchronization by comparing the quota of the bucket in the target site with the quota of the bucket in the source site; The obtaining, by the source site, the names of all buckets requiring synchronization of data according to the synchronization mark includes: Parsing the log shard to be queried through the source site, and then sending a request to the current storage pool to list the entries of the log shard according to the synchronization mark carried in the request, and starting the listing from the specified synchronization mark until all the entries on the log shard are listed; The source site parses each of the listed entries in turn, obtains the bucket shard corresponding to the log of the entry, determines the name of the bucket corresponding to the bucket shard based on the bucket shard name, and sends the bucket name to the target site.

4. The data query method according to claim 3, characterized in that: After saving the bucket and recording the quota of the bucket in the target site, the method further includes: Send the information of the bucket to the management software for display so that users can view it; After obtaining the progress of data synchronization by comparing the quota of the bucket in the target site with the quota of the bucket in the source site, the method further includes: The progress of the data synchronization is sent to the management software for display so that the user can view it.

5. The data query method according to claim 4, characterized in that: After obtaining the progress of data synchronization by comparing the quota of the bucket in the target site with the quota of the bucket in the source site, the method further includes: The total quota of the destination site of the bucket being synchronized is compared with the total quota of the destination site saved in the previous query cycle, and divided by the query cycle to obtain the overall synchronization speed of the cluster.

6. A data query device, characterized in that: Applied to the source site, including: A reading module, configured to read synchronization marks of all log shards stored in the storage pool of the target site; wherein the log shards are first-level log shards; a sending module, configured to obtain the names of all buckets requiring synchronization data according to the synchronization mark and send the names to the target site, so that the target site can save the buckets and record the quota of the buckets in the target site; A quota module, configured to obtain the quota of the bucket in the source site and send it to the target site, so that the target site can obtain the progress of data synchronization by comparing the quota of the bucket in the target site with the quota of the bucket in the source site; The sending module is specifically used to parse the log shard to be queried, and then send a request to the current storage pool to list the entries of the log shard according to the synchronization mark carried in the request, and start listing from the specified synchronization mark until all the entries on the log shard are listed; parse each listed entry in turn, obtain the bucket shard corresponding to the log of the entry, and determine the name of the bucket corresponding to the bucket shard according to the bucket shard name; send the parsed names of all the buckets that need to synchronize data to the target site.

7. A data query device, characterized in that: Applied to target sites, including: The acquisition module is used to obtain the synchronization marks of all log shards stored in the storage pool and send them to the source site; A synchronization module, configured to obtain the names of all buckets requiring synchronization data from the source site according to the synchronization mark; a saving module, configured to save the bucket and record the quota of the bucket in the target site; a comparison module, configured to obtain the quota of the bucket in the source site, and obtain the progress of data synchronization by comparing the quota of the bucket in the target site with the quota of the bucket in the source site; The synchronization module is specifically used to parse the log shard to be queried through the source site, and then send a request to the current storage pool to list the entries of the log shard based on the synchronization mark carried in the request, and start listing from the specified synchronization mark until all the entries on the log shard are listed; parse each listed entry in turn through the source site, obtain the bucket shard corresponding to the log of the entry, determine the name of the bucket corresponding to the bucket shard based on the bucket shard name, and send the bucket name to the target site.

8. A data query device, characterized in that: including a memory for storing a computer program; A processor, configured to implement the steps of the data query method according to any one of claims 1 to 5 when executing the computer program.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data query method according to any one of claims 1 to 5 are implemented.

Images