ES synchronization ceph object-based synchronization consistency verification method, system and device, and storage medium
By providing a synchronization consistency verification method between Ceph and ES, using regular matching search and automated comparison, the problem of failing synchronization data cannot be quickly located in the prior art, and an efficient and reliable data synchronization and comparison process is achieved.
Patent Information
- Application Number
- CN202510121572.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-26
- Publication Date
- 2025-05-23
AI Technical Summary
In the process of synchronizing Ceph objects to ES, the data that failed synchronization cannot be quickly and intuitively located, and manual comparison is required and the comparison results are prone to errors.
Provide a synchronization consistency verification method based on ES synchronization ceph objects. By obtaining the batch files to be uploaded and the total upload amount, using boto3 to upload files to the cephrgw object cluster, query the synchronization status of rgw zone and es zone, use ES regular matching search to obtain the total number of files that meet the search conditions, and determine whether the total number of files is consistent with the total number of uploads. If it is inconsistent, search out all data of regular matching, extract valid fields, sort and compare the original uploaded file name to locate the data that failed synchronization.
Through regular matching search and automated comparison, it can quickly and intuitively locate synchronized data, reduce manual operations, and improve efficiency and reliability of results.
Smart Images

Figure CN120029984A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the technical field of data storage, and in particular to a synchronization consistency verification method, system, device and storage medium based on ES synchronization ceph object. Background Art
[0002] In today's digital age, the development of data storage and processing technology is crucial. As an advanced distributed storage system, Ceph occupies an important position in the field of data storage. It includes multiple storage methods such as file storage, block storage and object storage. Among them, rgw, as Ceph's object storage gateway, provides a way to access rgw object storage resources through the s3 interface. Users can easily upload data to Cephrgw for storage with the help of front-end APIs and other methods, and can also download the data stored in it to the local computer through the same front-end API method, which provides certain convenience for user data management.
[0003] Elasticsearch is a highly anticipated open source distributed search and analysis engine that has been widely used in big data analysis scenarios such as real-time data search and full-text search. It is known for its high performance, excellent scalability, and powerful query capabilities. It can meet the needs of large-scale data storage and processing and has become an important tool for many companies and institutions in data processing.
[0004] At present, Ceph has the function of synchronizing the metadata information of the main zone to elasticsearchzone (es zone for short) in the same distributed storage cluster, and then using the ES search engine for search. This search method has changed the previous mode of relying solely on a single s3 api search. Because the traditional s3 api search has many limitations, it cannot implement advanced search functions such as full-text search. When facing massive data, it is easy to encounter performance bottlenecks, resulting in low search efficiency. Moreover, when using the s3 api search, it is also impossible to sort the search results, which greatly limits the user's in-depth analysis and use of the data. In contrast, when rgw synchronizes metadata to es zone, users can use the powerful search function of elasticsearch to use a variety of flexible methods such as full-text search, fuzzy search, and paging query for fast query, and can sort and analyze the search results, providing users with a more efficient and comprehensive data retrieval experience.
[0005] However, in the process of synchronizing Ceph objects to ES, there are still some problems that need to be solved in the existing technology:
[0006] 1.) Elasticsearch's search tools include curl, kaizen, etc. The curl tool searches through the command line. Other search methods require manual input of the http request method, request body, etc. The manual input method is cumbersome, prone to errors, and inefficient;
[0007] 2) Curl, kaizen command line or visual tools are suitable for single search, and the search conditions cannot be reused;
[0008] 3) The es search syntax is difficult, and you need to construct the search request body yourself for different test scenarios;
[0009] 4) The search results of search tools such as Curl and Kaizen are nested structures of multiple jsons and contain multiple fields, such as _index, _type, _id, _source, etc. It is difficult to view without quick viewing;
[0010] 5) The search results of search tools such as Curl and Kaizen cannot quickly and intuitively locate the data that failed to be synchronized from the rgw zone to the eszone. Manual comparison is required one by one, and the comparison results are prone to errors.
[0011] In summary, the existing technologies have many shortcomings, and a new method is urgently needed to solve these problems in order to improve the efficiency and quality of data synchronization and search. Summary of the invention
[0012] To this end, an embodiment of the present invention provides a synchronization consistency verification method, system, device and storage medium based on ES synchronization ceph object, so as to solve the technical problems in the prior art that the ES search tool cannot quickly and intuitively locate the data that fails to synchronize, needs to be compared and the comparison results are prone to high error rate.
[0013] In order to achieve the above purpose, the embodiment of the present invention provides the following technical solutions:
[0014] According to a first aspect of an embodiment of the present invention, a synchronization consistency verification method based on ES synchronization ceph object is provided, the method comprising:
[0015] Get the batch files to be uploaded and the total upload amount, and use boto3 to upload the batch files to the cephrgw object cluster;
[0016] Check whether the synchronization status between the rgw zone and the es zone is complete. If it is complete, use ES regular matching search to obtain the total number of files that meet the search criteria.
[0017] Determine whether the total amount of files is consistent with the total amount of uploads. If they are consistent, the synchronization is successful and ES outputs all search data;
[0018] If they are inconsistent, the synchronization between cephrgw zone and es zone fails. Search for all data matching the regular expression and extract valid fields.
[0019] Sorting the numeric suffixes of the valid fields, storing the numbers of all the fields in order into a list and grouping the data in the list into sorted data groups;
[0020] Compare the data in the sorted data group with the original uploaded file name. If the names on both sides are the same, the synchronization is successful, and all the successfully synchronized data are output;
[0021] If the names on both sides are different, the synchronization fails and all the data that failed to be synchronized is output.
[0022] Furthermore, check whether the synchronization status of the rgw zone and the es zone is complete, including:
[0023] Use radosgw-admin zone sync status to query the synchronization status of rgwzone and es zone in the ceph cluster;
[0024] If the synchronization is not completed, wait for the synchronization to complete before searching for data;
[0025] If the synchronization is complete, data search will be performed directly.
[0026] Furthermore, after the synchronization is completed, the ES database fields now include bucket, bucket_name, bucket_owner, meta, name, and permission fields.
[0027] Furthermore, the total number of files that meet the search criteria is obtained through ES regular matching search, including:
[0028] Search by bucket name, object name, or bucket_owner, use regexp for regular matching, automatically expand compound search conditions according to the number of search conditions entered, construct a search body for search, and obtain files that meet the search conditions.
[0029] Furthermore, the total number of files that meet the search criteria is obtained through ES regular matching search, which also includes:
[0030] Get all files that meet the search criteria, and use the encapsulated component get_count to count the total number of files that meet the search criteria.
[0031] According to a second aspect of an embodiment of the present invention, a synchronization consistency verification system based on ES synchronization ceph object is provided, the system comprising:
[0032] The upload module is used to obtain the batch files to be uploaded and the total upload amount, and upload the batch files to the cephrgw object cluster using boto3;
[0033] The query module is used to query whether the synchronization status of rgwzone and es zone is complete. If the synchronization is complete, the total number of files that meet the search criteria is obtained through ES regular matching search;
[0034] The synchronization consistency verification module is used to determine whether the total number of files is consistent with the total number of uploads. If they are consistent, the synchronization is successful and ES outputs all search data. If they are inconsistent, the synchronization between the ceph rgw zone and the es zone fails, and all data matching the regular expression is searched and valid fields are extracted.
[0035] A sorting module, used for sorting the numeric suffixes of the valid fields, storing the numbers of all the fields in a list in order and grouping the data in the list into sorted data groups;
[0036] The comparison module is used to compare the data in the sorted data group with the original uploaded file name. If the names on both sides are the same, the synchronization is successful and all the successfully synchronized data is output; if the names on both sides are different, the synchronization fails and all the failed synchronization data is output.
[0037] According to a third aspect of an embodiment of the present invention, a synchronization consistency verification device based on ES synchronization ceph object is provided, the device comprising: a processor and a memory;
[0038] The memory is used to store one or more program instructions;
[0039] The processor is used to run one or more program instructions to execute the steps of a synchronization consistency verification method based on ES synchronization ceph objects as described in any of the above items.
[0040] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of a synchronization consistency verification method based on ES synchronization of ceph objects as described in any of the above items are implemented.
[0041] The embodiments of the present invention have the following advantages:
[0042] 1) Reduced the difficulty of ES search syntax, adapted to diversified search scenarios through regular matching, got rid of the single character matching limitation, and improved search flexibility and ease of use;
[0043] 2) Solved the problem of search method reuse, modularized various ES search methods, defined common method classes, and expanded conditions by simply passing in parameters during search, improving code reusability and development efficiency;
[0044] 3) Reduce the burden of viewing search results, filter the returned result fields, and only retain key information such as _source to make the results more concise and clear;
[0045] 4) Effectively locate the data that failed to synchronize, automatically compare the uploaded files with the number, number and other information of the search results, quickly and accurately locate them, reduce manual operations, and improve efficiency and result reliability. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] In order to more clearly illustrate the implementation of the present invention or the technical solution in the prior art, the following briefly introduces the drawings required for the implementation or the prior art description. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.
[0047] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.
[0048] Figure 1 A schematic diagram of the logical structure of a synchronization consistency verification system based on ES synchronization ceph objects provided by an embodiment of the present invention;
[0049] Figure 2 A flowchart of a synchronization consistency verification method based on ES synchronization ceph object provided by an embodiment of the present invention;
[0050] Figure 3 A schematic diagram of a comparison flow of a synchronization consistency verification method based on ES synchronization ceph object provided by an embodiment of the present invention;
[0051] Figure 4 A schematic diagram of the output results of a synchronization consistency verification method based on ES synchronization ceph objects provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0052] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0053] At present, Ceph has the function of synchronizing the metadata information of the main zone to the elasticsearch zone (es zone for short) in the same distributed storage cluster, and then using the ES search engine to search. This search method has changed the previous mode of relying solely on a single s3 api search. Because the traditional s3 api search has many limitations, it cannot implement advanced search functions such as full-text search. When facing massive data, it is easy to encounter performance bottlenecks, resulting in low search efficiency. Moreover, when using the s3 api search, it is also impossible to sort the search results, which greatly limits the user's in-depth analysis and use of the data. In contrast, when rgw synchronizes metadata to es zone, users can use the powerful search function of elasticsearch to use a variety of flexible methods such as full-text search, fuzzy search, and paging query for fast query, and can sort and analyze the search results, providing users with a more efficient and comprehensive data retrieval experience.
[0054] However, in the process of synchronizing Ceph objects to ES, there are still some problems that need to be solved in the existing technology:
[0055] 1.) Elasticsearch's search tools include curl, kaizen, etc. The curl tool searches through the command line. Other search methods require manual input of the http request method, request body, etc. The manual input method is cumbersome, prone to errors, and inefficient;
[0056] 2) Curl, kaizen command line or visual tools are suitable for single search, and the search conditions cannot be reused;
[0057] 3) The es search syntax is difficult, and you need to construct the search request body yourself for different test scenarios;
[0058] 4) The search results of search tools such as Curl and Kaizen are nested structures of multiple jsons and contain multiple fields, such as _index, _type, _id, _source, etc. It is difficult to view without quick viewing;
[0059] 5) The search results of search tools such as Curl and Kaizen cannot quickly and intuitively locate the data that failed to be synchronized from the rgw zone to the eszone. Manual comparison is required one by one, and the comparison results are prone to errors.
[0060] In order to solve the technical problem that the above ES search tool cannot quickly and intuitively locate the data that failed to be synchronized, a comparison is required and the comparison results are prone to high error rates.
[0061] refer to Figure 1 The embodiment of the present invention discloses a synchronization consistency verification system based on ES synchronization ceph object, the system includes: an upload module 1; a query module 2; a synchronization consistency verification module 3; a sorting module 4; and a comparison module 5.
[0062] Corresponding to the above-disclosed synchronization consistency verification system based on ES synchronization ceph object, the embodiment of the present invention also discloses a synchronization consistency verification method based on ES synchronization ceph object. The following describes in detail a synchronization consistency verification method based on ES synchronization ceph object disclosed in the embodiment of the present invention in combination with the above-described synchronization consistency verification system based on ES synchronization ceph object.
[0063] The embodiment of the present invention is a test method for synchronizing ceph rgw zone to es zone. Based on the python Elasticsearch library, the es.search method is encapsulated according to different test conditions, and the search conditions can be automatically expanded and the request body can be constructed according to the length of the input parameters. A modular and reused search method is adopted and called according to different test scenarios.
[0064] The embodiment of the present invention implements a unified search condition for elasticsearch field search and can automatically locate data that fails to be synchronized between the rgw zone and the es zone.
[0065] refer to Figures 2 to 3The present invention discloses a synchronization consistency verification method based on ES synchronization ceph object, the method comprising: obtaining batch files to be uploaded and the total upload amount, uploading the batch files to the cephrgw object cluster by using boto3; querying whether the synchronization status of rgwzone and es zone is completed synchronization, if the synchronization is completed, obtaining the total file amount that meets the search condition by ES regular matching search; judging whether the total file amount is consistent with the total upload amount, if they are consistent, the synchronization is successful, and ES outputs all search data; if they are inconsistent, the synchronization of ceph rgw zone and es zone fails, searching for all regular matching data and extracting valid fields; sorting the numeric suffixes of the valid fields, storing the numbers of all fields in a list in order, and grouping the data in the list to form a sorted data group; comparing the data in the sorted data group with the original uploaded file name, if the names on both sides are the same, the synchronization is successful, and all the synchronized data are output; if the names on both sides are different, the synchronization fails, and all the synchronized data are output.
[0066] Use boto3 to batch upload files aaa1-aaa10000 to the cephrgw object cluster, and check the synchronization status of rgw zone and es zone in the Ceph cluster. Use radosgw-admin zone sync status to query synchronization. If synchronization is not completed, wait for synchronization. If synchronization is completed, perform data search.
[0067] Furthermore, query whether the synchronization status of the rgw zone and the es zone is completed, including: using radosgw-admin zone sync status to query the synchronization status of the rgw zone and the es zone in the ceph cluster; if the synchronization is not completed, wait for the synchronization to complete before searching for data; if the synchronization is completed, directly search for data.
[0068] Furthermore, after the synchronization is completed, the ES database fields now include bucket, bucket_name, bucket_owner, meta, name, and permission fields.
[0069] After synchronization is completed, the es database fields include bucket, bucket_name, bucket_owner, meta, name, permission and other fields. For the uploaded objects, you can search based on bucket name, object name or bucket_owner. Due to the complexity of es search syntax rules, this solution chooses a general regular expression method to use regexp in es for regular matching. According to the number of input search conditions, the compound search conditions can be automatically expanded to construct the search body.
[0070] Furthermore, the total number of files that meet the search criteria is obtained through ES regular matching search, including: searching according to bucket name bucket, object name name or bucket_owner, using regexp for regular matching, automatically expanding compound search conditions according to the number of input search conditions, constructing a search body for search, and obtaining files that meet the search criteria.
[0071] Use the get_count method encapsulated by python Elasticsearch to count the total number. If the total number is inconsistent with the total number of uploads, it is determined that the synchronization between cephrgw zone and es zone has failed.
[0072] Furthermore, obtaining the total number of files that meet the search criteria through ES regular matching search also includes: obtaining all files that meet the search criteria, and using the encapsulation component get_count to count the total number of files that meet the search criteria.
[0073] Use the get_total_data method encapsulated by python Elasticsearch to search for all data matched by the regular expression. Since the search results return multiple fields and the search results are in disorder, the valid fields of the search results are taken out, and the numeric suffixes of the valid fields are sorted by a quick sort algorithm. All the numbers of the valid data are stored in the list total_list in order.
[0074] Use the grouping method to group the data in total_list, and then compare it with the original uploaded file name. If the names on both sides are the same, it is judged that the synchronization is successful and all the data that is successfully synchronized is output. If the names on both sides are different, it is judged that the synchronization fails and all the data that fails to synchronize is output. Please refer to Figure 4 , Figure 4 is the output result.
[0075] Based on the data uploaded to the ceph cluster, we can determine whether the synchronization is successful by correctly searching all the data through the python Elasticsearch encapsulation method. To verify the synchronization failure, we can first compare the total amount of test data and then quickly sort the suffix of the numeric part of the file name of the test data content and then screen it one by one with the suffix of the uploaded continuous data. Based on the comparison results, we can determine whether the ceph rgw zone is correctly synchronized to the es zone and directly output the test results, including the data that is successfully synchronized and the data that fails to synchronize.
[0076] The embodiments of the present invention solve the following problems:
[0077] 1) Solve the complexity of es search syntax: Use regular matching to adapt to different search scenarios, instead of using single characters, starting with or ending with a certain character;
[0078] 2) Solve the problem that search methods cannot be reused: Elasticsearch includes precise search term, fuzzy operation match, range search range, and compound query bool. According to different search methods, you can define a common method class and modularize each search method. When you need to search, you only need to pass in parameters to expand the search conditions;
[0079] 3) Solve the difficulty of viewing search results: filter the returned result fields and only retain information such as _source;
[0080] 4) Solve the problem of locating data that failed to synchronize: Automatically compare the number and number of uploaded files with the number and number of search results to locate data that failed to synchronize. This facilitates quick locating, reduces manual operations, and improves efficiency and reliability of results.
[0081] The embodiments of the present invention have the following advantages:
[0082] 1) Improved the shortcomings of elasticsearch's complex and cumbersome search methods. For the fields synchronized from rgw zone to eszone, a unified regular matching method is used for search. Different search schemes are no longer customized for different search conditions, which reduces the difficulty of search syntax.
[0083] 2) Modular search method: for different test scenarios, you only need to use different encapsulated search methods and pass in the search field to complete the search. You don’t need to manually enter the search format every time you search, which improves the degree of reuse.
[0084] 3) View the synchronization status and test results intuitively and accurately to improve test efficiency.
[0085] In addition, an embodiment of the present invention also provides a synchronization consistency verification device based on ES synchronization ceph objects, the device comprising: a processor and a memory; the memory is used to store one or more program instructions; the processor is used to run one or more program instructions to execute the steps of a synchronization consistency verification method based on ES synchronization ceph objects as described in any of the above items.
[0086] In addition, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a synchronization consistency verification method based on ES synchronization of ceph objects as described in any of the above items are implemented.
[0087] In the embodiment of the present invention, the processor may be an integrated circuit chip having the signal processing capability. The processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0088] The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present invention can be directly embodied as a hardware decoding processor for execution, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The processor reads the information in the storage medium and completes the steps of the above method in combination with its hardware.
[0089] The storage medium may be a memory, which may be, for example, a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memory.
[0090] Among them, the non-volatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory.
[0091] The volatile memory may be a random access memory (RAM) which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDRSDRAM), enhanced synchronous DRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus DRAM (DRRAM).
[0092] The storage media described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memory.
[0093] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in the present invention can be implemented using a combination of hardware and software. When software is used, the corresponding functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. Storage media can be any available media that can be accessed by general or special-purpose computers.
[0094] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.
Claims
1. A synchronization consistency verification method based on ES synchronization ceph object, characterized in that: The method comprises: Get the batch files to be uploaded and the total upload amount, and use boto3 to upload the batch files to the cephrgw object cluster; Check whether the synchronization status between the rgw zone and the es zone is complete. If it is complete, use ES regular matching search to obtain the total number of files that meet the search criteria. Determine whether the total amount of files is consistent with the total amount of uploads. If they are consistent, the synchronization is successful and ES outputs all search data; If they are inconsistent, the synchronization between cephrgw zone and es zone fails. Search for all data matching the regular expression and extract valid fields. Sorting the numeric suffixes of the valid fields, storing the numbers of all the fields in order into a list and grouping the data in the list into sorted data groups; Compare the data in the sorted data group with the original uploaded file name. If the names on both sides are the same, the synchronization is successful, and all the successfully synchronized data are output; If the names on both sides are different, the synchronization fails and all the data that failed to be synchronized is output.
2. A synchronization consistency verification method based on ES synchronization ceph object as described in claim 1, characterized in that: Check whether the synchronization status between the rgw zone and the es zone is complete, including: Use radosgw-admin zone sync status to query the synchronization status of rgwzone and es zone in the ceph cluster; If the synchronization is not completed, wait for the synchronization to complete before searching for data; If the synchronization is complete, data search will be performed directly.
3. A synchronization consistency verification method based on ES synchronization ceph object as described in claim 2, characterized in that: After the synchronization is completed, the ES database fields include bucket, bucket_name, bucket_owner, meta, name, and permission fields.
4. A synchronization consistency verification method based on ES synchronization ceph object as described in claim 3, characterized in that: The total number of files that meet the search criteria is obtained through ES regular matching search, including: Search by bucket name, object name, or bucket_owner, use regexp for regular matching, automatically expand compound search conditions according to the number of search conditions entered, construct a search body for search, and obtain files that meet the search conditions.
5. A synchronization consistency verification method based on ES synchronization ceph object as described in claim 4, characterized in that: The total number of files that meet the search criteria is obtained through ES regular matching search, including: Get all files that meet the search criteria, and use the encapsulated component get_count to count the total number of files that meet the search criteria.
6. A synchronization consistency verification system based on ES synchronization ceph object, characterized in that: The system comprises: The upload module is used to obtain the batch files to be uploaded and the total upload amount, and upload the batch files to the cephrgw object cluster using boto3; The query module is used to query whether the synchronization status of rgwzone and es zone is complete. If the synchronization is complete, the total number of files that meet the search criteria is obtained through ES regular matching search; The synchronization consistency verification module is used to determine whether the total number of files is consistent with the total number of uploads. If they are consistent, the synchronization is successful and ES outputs all search data. If they are inconsistent, the synchronization between the ceph rgw zone and the es zone fails, and all data matching the regular expression is searched and valid fields are extracted. A sorting module, used for sorting the numeric suffixes of the valid fields, storing the numbers of all the fields in a list in order and grouping the data in the list into sorted data groups; The comparison module is used to compare the data in the sorted data group with the original uploaded file name. If the names on both sides are the same, the synchronization is successful and all the successfully synchronized data is output; if the names on both sides are different, the synchronization fails and all the failed synchronization data is output.
7. A synchronization consistency verification device based on ES synchronization ceph object, characterized in that: The device comprises: a processor and a memory; The memory is used to store one or more program instructions; The processor is used to run one or more program instructions to execute the steps of a synchronization consistency verification method based on ES synchronization ceph objects as described in any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of a synchronization consistency verification method based on ES synchronization of ceph objects are implemented as described in any one of claims 1 to 5.