Consistency auditing compensation method and device, equipment, medium and program product

Through the data audit of the operation log after filtering and deduplication, the problem of low compensation efficiency of consistency audit in distributed object storage is solved, and a more efficient data audit and compensation process is achieved.

CN118939669APending Publication Date: 2024-11-12CHINA UNITED NETWORK COMM GRP CO LTD +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410998185.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-24
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In the prior art, the consistency audit compensation method in multi-site disaster recovery for distributed object storage is more cumbersome and it is difficult to ensure the efficiency of data audit compensation.

Method used

By obtaining multiple operation logs of the main site, filtering out valid operation logs, removing invalid or repeated operation records, determining the second operation log, and only performing data audits on the second operation log to generate audit results to indicate the master and slave Consistency of site data.

Benefits of technology

The data audit process is simplified, processing efficiency is improved, the workload and complexity of the consistency audit process is reduced, and the efficiency of data audit is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118939669A_ABST
    Figure CN118939669A_ABST
Patent Text Reader

Abstract

The invention provides a consistency auditing compensation method and device, equipment, a medium and a program product. The method comprises the following steps: acquiring a plurality of first operation logs of a main site, wherein the first operation logs are operation logs corresponding to HTTP requests sent by other equipment to the main site; according to a preset operation type, the operation type of the HTTP request contained in each first operation log, the target version number of the target data, the storage address and the target data identifier, a second operation log is determined from the multiple first operation logs, and the operation type of the second operation log is the preset operation type; the target version number, the storage address and the target data identifier of the target data included in different second operation logs are different; and according to each second operation log, performing data auditing on the master site and the slave site to generate an auditing result. According to the data auditing method, the complexity of data auditing is effectively reduced, and the efficiency of data auditing is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of communication business support technology, and in particular to a consistency audit compensation method, device, equipment, medium and program product. Background Art

[0002] With the diversification of enterprise data storage methods and the increasing importance of data security, more and more enterprises and institutions are using distributed object storage multi-site disaster recovery to perform data redundancy backup to reduce the risk of single point failure. When modifying data at one site, it is necessary to ensure that the corresponding data at other sites can be correctly synchronized to maintain data integrity and consistency. Therefore, consistency audit compensation methods play a key role in distributed object storage multi-site disaster recovery to ensure data consistency and integrity in a complex distributed environment.

[0003] In the prior art, the specific process of the consistency audit compensation method in multi-site disaster recovery of distributed object storage is: first, all data operation log lists are checked one by one to check whether the changed data in all data operation log lists exist in the master and slave sites at the same time, and when it is found that the data in a certain data operation log list is inconsistent between the master and slave sites, a compensation operation is performed to repair the data corresponding to different sites.

[0004] However, the consistency audit compensation method in the distributed object storage multi-site disaster recovery in the existing technology is relatively complicated and difficult to ensure the efficiency of data audit compensation. Summary of the invention

[0005] The present application provides a consistency audit compensation method, device, equipment, medium and program product to solve the technical problem that the audit compensation method in the prior art is relatively cumbersome and it is difficult to ensure the efficiency of data audit compensation.

[0006] In a first aspect, the present application provides a consistency audit compensation method, comprising:

[0007] Acquire multiple first operation logs of the main site, where the first operation logs are operation logs corresponding to HTTP requests sent by other devices to the main site;

[0008] Determine a second operation log from multiple first operation logs according to a preset operation type and an operation type of the HTTP request contained in each first operation log, as well as a target version number, a storage address, and a target data identifier of the target data, wherein the preset operation type is an operation type corresponding to a valid change request, the operation type of the second operation log is the preset operation type, and the target version number, the storage address, and the target data identifier of the target data contained in different second operation logs are all different;

[0009] According to each second operation log, data audit is performed on the master site and the slave site to generate an audit result, the slave site is used to back up the data in the master site, and the audit result is used to indicate whether the data in the master site and the slave site are consistent.

[0010] Optionally, in the method as described above, determining the second operation log from the plurality of first operation logs according to the preset operation type and the operation type of the HTTP request contained in each first operation log and the target version number, storage address and target data identifier of the target data includes:

[0011] According to the preset operation type and the operation type of the HTTP request contained in each first operation log, screening out a plurality of third operation logs whose operation type is the preset operation type from the plurality of first operation logs;

[0012] According to the target version number, storage address and target data identifier of the target data contained in each third operation log, a deduplication operation is performed on the plurality of third operation logs to obtain the second operation log.

[0013] Optionally, in the method as described above, performing data audit on the master site and the slave site according to each second operation log to generate an audit result includes:

[0014] Determine the audit time of the second operation log according to a preset time and the operation time of the target data contained in each second operation log, wherein the preset time is greater than or equal to the data backup time of backing up the data in the primary site to the secondary site;

[0015] When the current time is the audit time of the second operation log, data audit is performed on the master site and the slave site according to the second operation log to generate the audit result.

[0016] Optionally, in the above method, determining the audit time of the second operation log according to the preset duration and the operation time of the target data contained in each second operation log includes:

[0017] The operation time corresponding to the second operation log is accumulated with the preset duration, and the accumulated time is determined as the audit time of the second operation log.

[0018] Optionally, in the method as described above, performing data audit on the master site and the slave site according to the second operation log to generate the audit result includes:

[0019] According to the second operation log, a data audit request is sent to the master site and the slave site respectively, wherein the data audit request includes a target version number, a storage address and a target data identifier of the target data corresponding to the second operation log;

[0020] Respectively obtaining response information returned by the master site and the slave site, wherein the response information is used to indicate whether target data corresponding to the second operation log exists in the master site or the slave site;

[0021] An audit result corresponding to the second operation log is generated according to the response information returned by the master site and the slave site.

[0022] Optionally, in the method as described above, the response information includes a return code and an ETag, the return code is used to indicate whether the storage address stores the to-be-confirmed data with a data identifier being the target data identifier, and the ETag is used to indicate whether the version number of the to-be-confirmed data is the target version number;

[0023] Correspondingly, generating the audit result corresponding to the second operation log according to the response information returned by the master site and the slave site includes:

[0024] If any return code in the response information returned by the master site and the slave site indicates that the data to be confirmed does not exist, an audit result indicating that the data of the master site and the slave site are inconsistent is generated;

[0025] If the return codes in the response information returned by the master site and the slave site both indicate that the data to be confirmed exists, and any ETag indicates that the version number corresponding to the stored data to be confirmed is not the target version number, then an audit result indicating that the data of the master site and the slave site are inconsistent is generated;

[0026] If the return codes in the response information returned by the master site and the slave site both indicate the existence of the data to be confirmed, and the ETag both indicate that the version number corresponding to the stored data to be confirmed is the target version number, an audit result is generated indicating that the data of the master site and the slave site are consistent.

[0027] Optionally, the method as described above further comprises:

[0028] If the audit result of the second operation log indicates that the data of the master site and the slave site are inconsistent, the target data corresponding to the second operation log is synchronized between the master site and the slave site.

[0029] Optionally, in the method described above, the preset operation type includes at least one of the following: DELETE, PUT object, POST to complete multi-part upload.

[0030] Optionally, in the method as described above, before determining the second operation log from the plurality of first operation logs according to the preset operation type and the operation type of the HTTP request contained in each first operation log, and the target version number, storage address and target data identifier of the target data, the method further comprises:

[0031] For a first operation log whose operation type is DELETE, obtaining a target version number of the first operation log according to a request parameter of an HTTP request contained in the first operation log;

[0032] For a first operation log whose operation type is PUT object or POST to complete segmented upload, obtain the target version number of the first operation log according to a response header corresponding to the HTTP request contained in the first operation log.

[0033] In a second aspect, the present application provides a consistency audit compensation device, comprising:

[0034] An acquisition module, used to acquire a plurality of first operation logs of the main site, where the first operation logs are operation logs corresponding to Hypertext Transfer Protocol HTTP requests sent by other devices to the main site;

[0035] a determination module, configured to determine a second operation log from a plurality of first operation logs according to a preset operation type and an operation type of the HTTP request contained in each first operation log, as well as a target version number, a storage address, and a target data identifier of the target data, wherein the preset operation type is an operation type corresponding to a valid change request, the operation type of the second operation log is the preset operation type, and the target version number, the storage address, and the target data identifier of the target data contained in different second operation logs are all different;

[0036] A generation module is used to perform data audit on the master site and the slave site according to each second operation log and generate an audit result. The slave site is used to back up the data in the master site. The audit result is used to indicate whether the data in the master site and the slave site are consistent.

[0037] In a possible implementation, the determining module is specifically configured to:

[0038] According to the preset operation type and the operation type of the HTTP request contained in each first operation log, screening out a plurality of third operation logs whose operation type is the preset operation type from the plurality of first operation logs;

[0039] According to the target version number, storage address and target data identifier of the target data contained in each third operation log, a deduplication operation is performed on the plurality of third operation logs to obtain the second operation log.

[0040] In a possible implementation, the generating module is specifically used to:

[0041] Determine the audit time of the second operation log according to a preset time and the operation time of the target data contained in each second operation log, wherein the preset time is greater than or equal to the data backup time of backing up the data in the primary site to the secondary site;

[0042] When the current time is the audit time of the second operation log, data audit is performed on the master site and the slave site according to the second operation log to generate the audit result.

[0043] In a possible implementation, the generating module is specifically used to:

[0044] The operation time corresponding to the second operation log is accumulated with the preset duration, and the accumulated time is determined as the audit time of the second operation log.

[0045] In a possible implementation, the generating module is specifically used to:

[0046] According to the second operation log, a data audit request is sent to the master site and the slave site respectively, wherein the data audit request includes a target version number, a storage address and a target data identifier of the target data corresponding to the second operation log;

[0047] Respectively obtaining response information returned by the master site and the slave site, wherein the response information is used to indicate whether target data corresponding to the second operation log exists in the master site or the slave site;

[0048] An audit result corresponding to the second operation log is generated according to the response information returned by the master site and the slave site.

[0049] In a possible implementation, the generating module is specifically used to:

[0050] The response information includes a return code and an ETag, wherein the return code is used to indicate whether the storage address stores the to-be-confirmed data whose data identifier is the target data identifier, and the ETag is used to indicate whether the version number of the to-be-confirmed data is the target version number;

[0051] Accordingly, the generation module is specifically used for:

[0052] If any return code in the response information returned by the master site and the slave site indicates that the data to be confirmed does not exist, an audit result indicating that the data of the master site and the slave site are inconsistent is generated;

[0053] If the return codes in the response information returned by the master site and the slave site both indicate that the data to be confirmed exists, and any ETag indicates that the version number corresponding to the stored data to be confirmed is not the target version number, then an audit result indicating that the data of the master site and the slave site are inconsistent is generated;

[0054] If the return codes in the response information returned by the master site and the slave site both indicate the existence of the data to be confirmed, and the ETag both indicate that the version number corresponding to the stored data to be confirmed is the target version number, an audit result is generated indicating that the data of the master site and the slave site are consistent.

[0055] In a possible implementation, the consistency audit compensation device further includes a processing module, which is specifically configured to:

[0056] If the audit result of the second operation log indicates that the data of the master site and the slave site are inconsistent, the target data corresponding to the second operation log is synchronized between the master site and the slave site.

[0057] In a possible implementation, the determining module is specifically configured to:

[0058] The preset operation type includes at least one of the following: delete DELETE, update PUT object, submit POST to complete segment upload.

[0059] In a possible implementation, the determining module is further configured to, before determining the second operation log from the plurality of first operation logs according to the preset operation type and the operation type of the HTTP request contained in each first operation log and the target version number, storage address and target data identifier of the target data,

[0060] For a first operation log whose operation type is DELETE, obtaining a target version number of the first operation log according to a request parameter of an HTTP request contained in the first operation log;

[0061] For a first operation log whose operation type is PUT object or POST to complete segmented upload, obtain the target version number of the first operation log according to a response header corresponding to the HTTP request contained in the first operation log.

[0062] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0063] Memory stores computer-executable instructions;

[0064] The processor executes the computer-executable instructions stored in the memory to implement the consistency audit compensation method as described in the first aspect and various possible implementations of the first aspect.

[0065] In a fourth aspect, the present application provides a computer-readable storage medium, comprising computer execution instructions. When a processor executes the computer execution instructions, it executes the consistency audit compensation method as described in the first aspect and various possible implementations of the first aspect.

[0066] In a fifth aspect, the present application provides a program product, which includes a computer program, and the computer program is executed by a processor to implement the consistency audit compensation method as described in the first aspect and various possible implementation methods of the first aspect.

[0067] The consistency audit compensation method, device, equipment, medium and program product provided by the present application obtain multiple first operation logs of the main site, the first operation log is the operation log corresponding to the Hypertext Transfer Protocol HTTP request sent by other devices to the main site; according to the preset operation type and the operation type of the HTTP request contained in each first operation log and the target version number, storage address and target data identifier of the target data, the second operation log is determined from the multiple first operation logs, the operation type of the second operation log is the preset operation type, and the target version number, storage address and target data identifier of the target data contained in different second operation logs are all different; according to each second operation log, the data of the main site and the slave site are audited to generate an audit result. The method takes advantage of the fact that there are many invalid or repeated operation records in the operation log, deletes the first operation log corresponding to the invalid operation or repeated operation record, determines the valid and non-repeated second operation log from all the first operation logs, and subsequently only performs data audit on the main site and the slave site for the second operation log, effectively reducing redundant processing processes, simplifying the data audit process, and improving processing efficiency. This method can greatly reduce the workload and complexity of the consistency audit process, effectively reduce the complexity of data auditing, and improve the efficiency of data auditing. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0069] Figure 1 A system diagram of a consistency audit compensation method provided in an embodiment of the present application;

[0070] Figure 2 Schematic diagram of the process of the consistency audit compensation method provided in the embodiment of the present application Figure 1 ;

[0071] Figure 3 Schematic diagram of the process of the consistency audit compensation method provided in the embodiment of the present application Figure 2 ;

[0072] Figure 4 A schematic diagram of the structure of a consistency audit compensation device provided in an embodiment of the present application;

[0073] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0074] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0075] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0076] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0077] With the explosive growth of enterprise data, choosing a reasonable data storage method is crucial for enterprises. Distributed object storage is a storage system that stores data on multiple nodes and connects these nodes through a network to provide high availability, scalability and fault tolerance. In distributed object storage, data is stored in the form of objects, and each object contains the data itself and the metadata related to it. The system mainly consists of three parts: data node cluster, metadata cluster and gateway cluster (or client). The data node cluster is responsible for storing object data, the metadata cluster is responsible for storing the metadata of the cluster, and the gateway cluster and client provide a simple access interface to the outside world, and access metadata and data nodes to read and write data internally. On this basis, ensuring the security of enterprise data is also crucial, and multi-site disaster recovery backup can ensure that the system can quickly recover and continue to run in the event of a catastrophic event by deploying another set of storage clusters in different geographical locations. It has data replication and redundancy, that is, deploying multiple storage clusters in different geographical locations, and ensuring the backup and consistency of data between different sites through data replication and redundancy mechanisms. Users can configure replication strategies to meet different disaster recovery and backup requirements, quasi-real-time data replication, that is, data addition, deletion, modification and other operations can be replicated to the target bucket (also known as the data storage address of the slave site) in quasi-real time, data consistency, that is, in the process of copying data from the source bucket (also known as the data storage address of the master site) to the target bucket, the system generally ensures the final consistency of data at both ends, automatic failover and recovery, that is, when a data center or storage node fails, it can automatically perform failover and data recovery, and migrate services to other normally operating nodes or data centers to ensure data availability and system continuity, and object multi-version control, that is, for source and target buckets that are in the version control state at the same time, the final consistency of their object data versions is guaranteed. Therefore, more and more enterprises and institutions use distributed object storage multi-site disaster recovery to perform data redundancy backup to reduce the risk of single point failure. The principle is generally based on multi-site synchronization: the storage address of the master site, the change log of the data (also known as the operation log), the audit compensation device obtains the change log from the master site, performs bucket and data addition, deletion and modification operations based on this log, and maintains the compensation task progress of the change log. When a site modifies data, due to system or network errors, it is difficult to ensure that the corresponding data of other sites can be correctly synchronized, and thus the integrity and consistency of the data cannot be guaranteed. However, by adopting the consistency audit compensation method, the target data of the master and slave sites can be audited and compensated for consistency, ensuring the ultimate consistency of the master and slave sites.

[0078] In the prior art, the specific process of the consistency audit compensation method in multi-site disaster recovery of distributed object storage is: first, by checking the data operation log (also called operation log) list one by one, check whether the changed data in all data operation log lists exist in the master and slave sites at the same time, and when it is found that a certain data is inconsistent between the master and slave sites, perform compensation operations to repair the data corresponding to different sites.

[0079] However, in the prior art, most of the consistency audit compensation methods in multi-site disaster recovery of distributed object storage need to check all operation logs one by one to check whether there are inconsistencies in the data of the master and slave sites. Since a large number of invalid data operation logs in the operation logs are ignored, these operation logs do not need to be fully audited. For example, some data preparation operation logs do not need to perform data collection operations. On the other hand, for repeated submission requests caused by network failures, only one of the operation logs is valid. In actual applications, if all operation logs are audited, it will not only increase the complexity of the audit, but also make it difficult to ensure the efficiency of the audit.

[0080] Based on this, the present application proposes a consistency audit compensation method, device, equipment medium and program product. Considering the characteristics that there are many invalid or abnormal operation records in the operation log, the operation log can be screened and deduplicated first, and then these valid operation logs can be audited and compensated, that is, firstly, the valid operation logs are screened from the first operation log obtained, and then these valid operation logs are deduplicated to determine the second operation log. Further, for the operation logs in the second operation log, the master site and the slave site are audited, and finally the audit results are generated and compensated. This method can greatly reduce the workload and complexity of the consistency audit process by screening and checking the validity of all the acquired operation logs and removing invalid and duplicate operation logs, thereby reducing the overall workload. In addition, the logs after screening and deduplication are more concise, the audit process is more efficient, and the data inconsistencies between the master and slave sites can be identified more quickly, thereby improving the efficiency of data auditing.

[0081] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0082] Figure 1 A system diagram of the consistency audit compensation method provided in the embodiment of the present application. Figure 1 As shown, the system includes a master site, a slave site and an audit compensation device.

[0083] The master site is the main data storage and processing center. It is responsible for the main read and write operations. When the user needs to modify the data, all write operations first occur at the master site, and the corresponding operation log is generated for retention. Then the master site synchronizes the data to the slave site to ensure that the data at the slave site is consistent and complete with the data at the master site. The slave site is the backup storage center for the data. When the data at the master site changes, the slave site receives and stores all data changes pushed by the master site to maintain data consistency. And when the master site fails, the slave site provides data access services as a disaster recovery site to ensure business continuity and improve the overall performance and availability of the system. The audit compensation device is responsible for monitoring and verifying the data consistency between the master site and the slave site, and performing compensation operations when the data at the master and slave sites are inconsistent. The device collects the operation logs of the master site regularly or in real time, then screens and deduplicates the operation logs, filters invalid or repeated operation records, and generates valid operation logs. According to the valid operation logs and sending data audit requests to the master site and the slave site respectively, after obtaining the response information returned by the master site and the slave site, the data consistency is checked and the audit results are generated. The auditing device then performs compensation operations on inconsistent data items based on the auditing results.

[0084] By applying this consistency audit compensation method to distributed object storage multi-site disaster recovery, data consistency between the master and slave sites is ensured. When the master site fails, the slave site can quickly take over and respond, ensuring business continuity and improving the system's processing performance. In addition, the generated audit results are beneficial to subsequent system troubleshooting and system maintenance.

[0085] Figure 2 Schematic diagram of the process of the consistency audit compensation method provided in the embodiment of the present application Figure 1 .like Figure 2 As shown, the method includes:

[0086] S201, obtaining a plurality of first operation logs of a primary site;

[0087] The execution subject of the embodiment of the present application is an electronic device, which can be a terminal device, such as a laptop, a desktop computer, a tablet computer, etc., or a server. In practical applications, whether the electronic device is a terminal device or a server can be determined according to actual conditions, and no specific restrictions are imposed on this. In the embodiment of the present application, the electronic device can be implemented as Figure 1 Audit compensation equipment in.

[0088] The first operation log is an operation log corresponding to a Hypertext Transfer Protocol (HTTP) request sent by other devices to the main site. HTTP is a protocol for transmitting hypertext data between a browser and a server, and is mainly used to obtain resource data.

[0089] Optionally, the audit compensation device can obtain operation logs by sending HTTP requests to the main site at regular intervals. After receiving the HTTP request, the main site pulls the access log list of the interface gateway in the main site through the Secure File Transfer Protocol (SFTP) protocol, and downloads the operation log in the access log list and sends it to the audit compensation device. After obtaining the corresponding operation log, the audit compensation device records the line number information of the last operation log obtained at the current moment, so that when the operation log is obtained at the next regular interval, it starts from this line number to ensure the complete acquisition and audit of the operation log in the system.

[0090] It can be understood that by sending an HTTP request to the main site to obtain the corresponding first operation log, a necessary data basis is provided for subsequent audit processing.

[0091] S202, determining a second operation log from a plurality of first operation logs according to a preset operation type and an operation type of an HTTP request contained in each first operation log, and a target version number, a storage address, and a target data identifier of the target data;

[0092] Among them, the preset operation type is the operation type corresponding to the valid change request, the operation type of the second operation log is the preset operation type, and the target version number, storage address and target data identifier of the target data contained in different second operation logs are all different. A valid change request refers to triggering the modification of the main site data resources through an HTTP request and performing related operations to implement it. The target data refers to the specific data object operated in the operation log; the target version number, storage address and target data identifier refer to the version number corresponding to the target data, the address where the target data is stored and the unique identifier of the target data.

[0093] Among them, in the embodiment of the present application, the preset operation type includes at least one of the following: DELETE, PUT object, POST to complete the multi-part upload.

[0094] Among them, DELETE is used to indicate that the data is deleted at the primary site; the PUT object is used to indicate that the data is updated and modified at the primary site; and POST completes the multi-segment upload and is used to indicate that the new data has been completely uploaded to the primary site.

[0095] It can be understood that by setting a preset operation type and filtering in the first operation log by the preset operation type, operation logs that have substantial operations on the data can be obtained, thereby filtering out some invalid operation logs, reducing the computational complexity of the subsequent audit processing of the operation logs and improving data audit efficiency.

[0096] S203, performing data audit on the master site and the slave site according to each second operation log, and generating an audit result;

[0097] The slave site is used to back up the data in the master site, and the audit result is used to indicate whether the data in the master site and the slave site are consistent. Data auditing refers to checking the consistency of the data in the master site and the slave site based on the operation log to find inconsistent data.

[0098] It is understandable that, according to the target version number, storage address and target data identifier contained in each second operation log, consistency check is performed in the master site and the slave site, and an audit result is generated. In this way, it is more accurate to determine whether the data after the operation exists in the master site and the slave site, and form a corresponding audit result for subsequent system troubleshooting and system maintenance.

[0099] The embodiment of the present application provides a consistency data audit compensation method, by obtaining multiple first operation logs of the main site, the first operation log is the operation log corresponding to the Hypertext Transfer Protocol HTTP request sent by other devices to the main site; according to the preset operation type and the operation type of the HTTP request contained in each first operation log and the target version number, storage address and target data identifier of the target data, the second operation log is determined from the multiple first operation logs, the operation type of the second operation log is the preset operation type, and the target version number, storage address and target data identifier of the target data contained in different second operation logs are all different; according to each second operation log, the data of the main site and the slave site are audited to generate an audit result. This method takes advantage of the fact that there are many invalid or repeated operation records in the operation log, deletes the first operation log corresponding to the invalid operation or repeated operation record, determines the valid and non-repeated second operation log from all the first operation logs, and subsequently only performs data audit on the main site and the slave site for the second operation log, effectively reducing redundant processing processes, simplifying the data audit process, and improving processing efficiency. This method can greatly reduce the workload and complexity of the consistency audit process, thereby reducing the overall workload. In addition, the logs after screening and deduplication are more concise, the audit process is more efficient, and data inconsistencies between master and slave sites can be identified more quickly, improving the efficiency of data auditing.

[0100] In this embodiment of the present application, before S202, it is also necessary to obtain the target version number of the first operation log, which can be specifically achieved by the following steps:

[0101] For the first operation log whose operation type is DELETE, obtain the target version number of the first operation log according to the request parameters of the HTTP request contained in the first operation log; for the first operation log whose operation type is PUT object or POST to complete segmented upload, obtain the target version number of the first operation log according to the response header corresponding to the HTTP request contained in the first operation log.

[0102] The request parameters refer to the parameters that are included in the HTTP request and passed to the server. The target version number of the first operation log can be obtained by searching for a specific target key in the DELETE request parameters. The response header refers to an important part of the HTTP request response, which contains detailed information about the response status, content, etc. By obtaining the response header of the PUT object or POST to complete the multi-segment upload, the target version number of the corresponding operation log can be determined for subsequent data auditing and consistency verification.

[0103] It should be understood that since the purpose, implementation mechanism and information returned by the server for each operation type are different, the first operation logs for different operation types need to obtain the corresponding target version number through different data parameters. For DELETE operations, the request parameters of the HTTP request are extracted from the first operation log and a specific key is searched in the request parameters to obtain the corresponding target version number. For operations such as PUT objects or POST to complete multi-segment uploads, the response header corresponding to the HTTP request in the operation log needs to be parsed and a specific key is searched in the response header to obtain the corresponding target version number. In this way, the target version number of each first operation log can be accurately extracted for use in data auditing and compensation processes.

[0104] In a specific implementation, according to the preset operation type and the operation type of the HTTP request contained in each first operation log, as well as the target version number, storage address and target data identifier of the target data, the specific process of determining the second operation log from multiple first operation logs can be implemented by the following steps:

[0105] First, according to the preset operation type and the operation type of the HTTP request contained in each first operation log, multiple third operation logs with the preset operation type are screened out from multiple first operation logs. Then, according to the target version number, storage address and target data identifier of the target data contained in each third operation log, the multiple third operation logs are deduplicated to obtain the second operation log.

[0106] The third operation log is an operation log that meets a preset operation type among the multiple first operation logs.

[0107] Exemplarily, the rule for filtering out a plurality of third operation logs whose operation types are preset operation types is as shown in Table 1 below:

[0108] Table 1 Screening rules

[0109] HTTP Operation Type Keep How to obtain the version number DELETE yes Request parameter version number field PUT Object yes Response header version number field POST completes multi-part upload yes Response header version number field GET no / HEAD no / OPTIONS no /

[0110] Among them, the GET request is to request to obtain the specified resource from the main site; the HEAD request is used to obtain the metadata of the specified resource (i.e. the response header information) without returning the main content of the resource. The HEAD request is very similar to the GET request, the only difference is that the HEAD request only returns the response header, not the resource body. The pre-check OPTIONS request is used to query the main site for those requested operations that are supported without actually accessing the resource.

[0111] Assume that the first operation log obtained and the corresponding operation types are:

[0112] 1. 2024-07-08T12:00:00POST / bucket1 / object1? version=1(POST);

[0113] 2. 2024-07-08T12:01:00GET / bucket1 / object1? version=1(GET);

[0114] 3. 2024-07-08T12:02:00DELETE / bucket1 / object1? version=1(DELETE);

[0115] 4. 2024-07-08T12:03:00POST / bucket2 / object2? version=1(POST);

[0116] 5. 2024-07-08T12:04:00POST / bucket1 / object1? version=1(POST).

[0117] After matching and screening according to the preset operation type, the obtained third operation log and the target version number, storage address and target data identifier of the target data are respectively:

[0118] 1. 2024-07-08T12:00:00POST / bucket1 / object1? version=1(version=1,bucket1,object1);

[0119] 2. 2024-07-08T12:02:00DELETE / bucket1 / object1version=1(version=1,bucket1,object1);

[0120] 3. 2024-07-08T12:03:00POST / bucket2 / object2? version=1(version=1,bucket2,object2);

[0121] 4. 2024-07-08T12:04:00POST / bucket1 / object1? version=1(version=1, bucket1, object1).

[0122] By comparing the target version number, storage address and target data identifier of the target data, the second operation log obtained after deduplication processing includes:

[0123] 1. 2024-07-08T12:03:00POST / bucket2 / object2? version=1(version=1,bucket2,object2);

[0124] 2. 2024-07-08T12:04:00POST / bucket1 / object1? version=1(version=1, bucket1, object1).

[0125] Optionally, the extracted information can be stored in a Remote Dictionary Server (Redis) cluster to implement deduplication operations on multiple third operation logs based on the target version number, storage address (i.e., the bucket name where the target data is stored at the main site) and target data identifier (i.e., the object name of the target data in the main site) of the target data contained in each third operation log. The Redis cluster is an open source memory data structure storage system, commonly used as a database, cache, and message middleware. The Redis cluster will identify records of operation logs corresponding to duplicate target version numbers, storage addresses, and target data identifiers, delete them, and only retain unique target data. In addition, using the Redis cluster as a data cache layer can reduce database load and improve system operating efficiency.

[0126] It is understandable that, firstly, multiple first operation logs are traversed, the HTTP request operation type of each first operation log is checked, and the logs whose operation type matches the preset operation type are screened out to form a set of third operation logs. Then, the target version number, storage address and target data identifier of the target data are extracted from each third operation log, and it is determined whether each target data has the same target version number, storage address and target data identifier. If there are any two or more target data with the same target version number, storage address and target data identifier, it is deemed that the operation log is repeated, and the redundant target data needs to be deleted, and finally the second operation log is obtained. In this way, the operation logs of the preset operation type can be effectively screened out from a large number of operation logs, and redundant information can be reduced by deduplication operations, which can significantly reduce the workload and complexity of the data audit and compensation process, and improve the audit efficiency of the system.

[0127] Figure 3 Schematic diagram of the process of the consistency audit compensation method provided in the embodiment of the present application Figure 2 The present application embodiment is based on the above embodiment and describes in detail the process of generating the audit result. Figure 3 As shown, the method includes:

[0128] S301, determining the audit time of the second operation log according to the preset duration and the operation time of the target data contained in each second operation log;

[0129] Among them, the preset duration is greater than or equal to the data backup duration of backing up the data in the primary site to the secondary site; the operation time of the target data refers to the time when the preset request operation is performed on the target data at the primary site; the audit time refers to the time when the operation log is audited.

[0130] Optionally, after the audit time of the second operation log is determined, the audit time can be used as the file name of the second operation log, so that the audit time corresponding to the currently auditable second operation log can be quickly identified using the file name, thereby improving audit efficiency.

[0131] It is understandable that the preset time length and the time when the preset request operation is performed on the target data contained in each second operation log are comprehensively considered to determine the time for auditing the second operation log. This method takes into account that after the master site operates on the data, the slave site needs a certain delay time when performing corresponding data synchronization, especially for the case where the amount of data modification is large. If the audit is performed immediately after the second operation log is determined, it may result in the slave site synchronizing the master site data, but the audit result may be misjudged as data being out of sync or lost, increasing the network and computing burden.

[0132] In one achievable manner, a specific manner of determining the audit time of the second operation log may be:

[0133] The operation time corresponding to the second operation log is accumulated with the preset duration, and the accumulated time is determined as the audit time of the second operation log.

[0134] For example, assuming that the preset time is 30 minutes, the second operation log is:

[0135] 1. 2024-07-08T12:03:00POST / bucket2 / object2? version=1;

[0136] 2. 2024-07-08T12:04:00POST / bucket1 / object1? version=1.

[0137] By adding the operation time corresponding to the second operation log to the preset duration, the audit time of the second operation log is obtained as follows:

[0138] 1. 2024-07-08T12:33:00POST / bucket2 / object2? version=1;

[0139] 2. 2024-07-08T12:34:00POST / bucket1 / object1? version=1.

[0140] It is understandable that by comprehensively considering the preset duration and the operation time of the second operation log, determining a reasonable audit time can effectively avoid the problem of misjudgment of audit results due to data synchronization delay. This method not only takes into account the actual situation of data synchronization, but also ensures the accuracy of audit results, which helps to improve the reliability and performance of the system.

[0141] S302: When the current time is the audit time of the second operation log, perform data audit on the master site and the slave site according to the second operation log to generate an audit result.

[0142] It can be understood that when the current moment matches the audit moment of the second operation log, it means that the target data in the second operation log has been synchronized and can be audited. Then, based on the data-related information of the second operation log, the data of the master site and the slave site are respectively audited and verified for consistency, and an audit result is formed.

[0143] Furthermore, when auditing the data of the master site and the slave site according to the second operation log, it is necessary to compare the response information returned by the master site and the slave site to determine whether the data exists in both the master site and the slave site, so as to generate the final audit result. Therefore, it is necessary to obtain the response information of the master site and the slave site before generating the audit result.

[0144] Specifically, it can be achieved through the following embodiments.

[0145] In a specific embodiment, according to the second operation log, data audit is performed on the master site and the slave site, and the specific process of generating the audit result is as follows:

[0146] First, according to the second operation log, a data audit request is sent to the master site and the slave site respectively; then, the response information returned by the master site and the slave site is obtained respectively; finally, according to the response information returned by the master site and the slave site, the audit result corresponding to the second operation log is generated.

[0147] The data audit request includes the target version number, storage address and target data identifier of the target data corresponding to the second operation log; the response information is used to indicate whether the target data corresponding to the second operation log exists in the master site or the slave site.

[0148] It is understandable that the target version number, storage address, target data identifier and other information of the target data are extracted from the second operation log, and data audit requests are sent to the master site and the slave site respectively based on the extracted information. Afterwards, based on the response information returned by the master site and the slave site, it is determined whether the target data corresponding to the second operation log exists in the master site or the slave site, thereby generating an audit result corresponding to the second operation log. In this way, by using the target data information, it is possible to quickly and accurately determine whether the target data exists in the master site or the slave site to ensure the consistency of data in the system.

[0149] Furthermore, in an embodiment of the present application, the response information includes a return code and an ETag.

[0150] The return code is used to indicate whether the storage address stores the to-be-confirmed data whose data identifier is the target data identifier, and the ETag is used to indicate whether the version number of the to-be-confirmed data is the target version number.

[0151] It can be understood that based on the return code and ETag contained in the response information returned by the master site and the slave site, it is confirmed whether there is information to be confirmed in the master site and the slave site, and when it is determined that there is information to be confirmed, it is confirmed whether the version number of the data to be confirmed is the target version number, which further ensures the accuracy of the audit during the data audit process, thereby improving the efficiency of the system audit.

[0152] Accordingly, S302 can be implemented as follows:

[0153] If any return code in the response information returned by the master site and the slave site indicates that there is no data to be confirmed, an audit result is generated indicating that the data on the master site and the slave site are inconsistent; if the return codes in the response information returned by the master site and the slave site both indicate that there is data to be confirmed, and any ETag indicates that the version number corresponding to the stored data to be confirmed is not the target version number, an audit result is generated indicating that the data on the master site and the slave site are inconsistent; if the return codes in the response information returned by the master site and the slave site both indicate that there is data to be confirmed, and both ETags indicate that the version number corresponding to the stored data to be confirmed is the target version number, an audit result is generated indicating that the data on the master site and the slave site are consistent.

[0154] For example, assuming that the return code 200 indicates that the data to be confirmed exists in the site, and the return code 404 indicates that the data to be confirmed does not exist in the site, the response information returned by the master site and the slave site and the corresponding audit results generated are as follows:

[0155] 1. If the return code returned by the master site is 200 and the return code returned by the slave site is 404, the audit result is that there is data to be confirmed on the master site, but not on the slave site. In this case, since the data to be confirmed does not exist on both the master site and the slave site, there is no need to compare the ETag.

[0156] 2. If the return code returned by the master site is 404 and the return code returned by the slave site is 200, the audit result generated is that there is data to be confirmed on the slave site, but there is no data to be confirmed on the master site.

[0157] 3. The return code returned by the master site is 200, and the return code returned by the slave site is 200, but the ETags returned by the master site and the slave site are inconsistent. The audit result generated is that the data to be confirmed exists in both the master site and the slave site, but the version numbers of the data to be confirmed in the master site and the slave site are different;

[0158] It can be understood that by first comparing the return code in the response information to confirm whether there is information to be confirmed, and then using ETag to further confirm the version number of the data to be confirmed based on the confirmation of existence, the system's computing resources and time can be reduced to a certain extent, and the accuracy of the audit during the data audit process can be ensured.

[0159] Furthermore, when an audit result is generated that the data of the master site and the slave site are inconsistent, a compensation operation needs to be performed on the master site or the slave site to maintain data consistency between the master site and the slave site.

[0160] In one achievable manner, if the audit result of the second operation log indicates that the data of the master site and the slave site are inconsistent, the target data corresponding to the second operation log is synchronized between the master site and the slave site.

[0161] Optionally, when the audit result of the second operation log indicates that the data of the master site and the slave site are inconsistent, the compensation task needs to be added to the compensation blocking queue to wait for data synchronization after the compensation task is generated, and the compensation task can be synchronized to the Redis cluster for storage at the same time to prevent the synchronization task from being lost due to system equipment failure, thereby reducing the audit efficiency. Furthermore, after the target data corresponding to the second operation log is synchronized between the master site and the slave site, a corresponding new first operation log can also be generated to facilitate subsequent troubleshooting and system maintenance.

[0162] Continuing with the example in the above embodiment, when processing the compensation blocking queue, for situation 1, it is necessary to download and cache the data to be confirmed of the main site and then upload it to the slave site; for situation 2, it is necessary to delete the data to be confirmed at the slave site; for situation 3, it is still necessary to download and cache the data to be confirmed of the main site and then upload it to the slave site, and ensure that the target version numbers of the data to be confirmed in the main site and the slave site are the same.

[0163] Furthermore, if the return code returned by the main site or the slave site is garbled, it means that the current system or network has a failure and cannot complete the audit task of the second operation log. The second operation log will then be determined as the next new second operation log, and the next audit time will be waited for to re-audit the second operation log.

[0164] The present application provides a consistency audit compensation method, which first determines the audit time of the second operation log by accumulating the preset time length and the operation time of the target data contained in each second operation log; then, when the current time matches the audit time of the second operation log, according to the second operation log, a data audit request is sent to the master site and the slave site respectively, and data audit is performed according to the response information returned by the master site and the slave site to generate corresponding audit results. Through this method, the problem of misjudgment of audit results caused by data synchronization delay can be effectively avoided, and the accuracy of the audit results is ensured, which helps to improve the reliability and performance of the system.

[0165] Figure 4 A schematic diagram of the structure of the consistency audit compensation device provided in the embodiment of the present application; Figure 4 As shown, the consistency audit compensation device 40 includes:

[0166] The acquisition module 401 is used to acquire multiple first operation logs of the main site, where the first operation log is an operation log corresponding to the HTTP request sent by other devices to the main site;

[0167] The determination module 402 is used to determine the second operation log from the plurality of first operation logs according to the preset operation type and the operation type of the HTTP request contained in each first operation log, as well as the target version number, storage address and target data identifier of the target data, wherein the preset operation type is the operation type corresponding to the valid change request, the operation type of the second operation log is the preset operation type, and the target version number, storage address and target data identifier of the target data contained in different second operation logs are all different;

[0168] The generation module 403 is used to perform data audit on the master site and the slave site according to each second operation log and generate audit results. The slave site is used to back up the data in the master site. The audit results are used to indicate whether the data in the master site and the slave site are consistent.

[0169] In a possible implementation, the determination module 402 is specifically configured to:

[0170] According to the preset operation type and the operation type of the HTTP request contained in each first operation log, a plurality of third operation logs having an operation type of the preset operation type are screened out from the plurality of first operation logs;

[0171] According to the target version number, storage address and target data identifier of the target data contained in each third operation log, a deduplication operation is performed on the plurality of third operation logs to obtain the second operation log.

[0172] In a possible implementation, the generating module 403 is specifically configured to:

[0173] Determine the audit time of the second operation log according to the preset time and the operation time of the target data contained in each second operation log, wherein the preset time is greater than or equal to the data backup time of backing up the data in the primary site to the secondary site;

[0174] When the current time is the audit time of the second operation log, data audit is performed on the master site and the slave site according to the second operation log to generate an audit result.

[0175] In a possible implementation, the generating module 403 is specifically configured to:

[0176] The operation time corresponding to the second operation log is accumulated with the preset duration, and the accumulated time is determined as the audit time of the second operation log.

[0177] In a possible implementation, the generating module 403 is specifically configured to:

[0178] According to the second operation log, a data audit request is sent to the master site and the slave site respectively, wherein the data audit request includes a target version number, a storage address and a target data identifier of the target data corresponding to the second operation log;

[0179] Respectively obtain response information returned by the master site and the slave site, where the response information is used to indicate whether target data corresponding to the second operation log exists in the master site or the slave site;

[0180] According to the response information returned by the master site and the slave site, an audit result corresponding to the second operation log is generated.

[0181] In a possible implementation, the generating module 403 is specifically configured to:

[0182] The response information includes a return code and ETag. The return code is used to indicate whether the storage address stores the data to be confirmed whose data identifier is the target data identifier. The ETag is used to indicate whether the version number of the data to be confirmed is the target version number.

[0183] Accordingly, the generating module 403 is specifically used for:

[0184] If any return code in the response information returned by the master site and the slave site indicates that there is no data to be confirmed, an audit result indicating that the data of the master site and the slave site are inconsistent is generated;

[0185] If the return codes in the response information returned by the master site and the slave site both indicate that there is data to be confirmed, and any ETag indicates that the version number corresponding to the stored data to be confirmed is not the target version number, an audit result indicating that the data of the master site and the slave site are inconsistent is generated;

[0186] If the return codes in the response information returned by the master site and the slave site both indicate that there is data to be confirmed, and the ETag both indicate that the version number corresponding to the stored data to be confirmed is the target version number, an audit result is generated indicating that the data of the master site and the slave site are consistent.

[0187] In a possible implementation, the consistency audit compensation device 40 further includes a processing module, which is specifically configured to:

[0188] If the audit result of the second operation log indicates that the data of the master site and the slave site are inconsistent, the target data corresponding to the second operation log is synchronized between the master site and the slave site.

[0189] In a possible implementation, the determination module 402 is specifically configured to:

[0190] The preset operation type includes at least one of the following: DELETE, PUT object, and POST to complete multi-part upload.

[0191] In a possible implementation, the determination module 402, before determining the second operation log from the plurality of first operation logs according to the preset operation type and the operation type of the HTTP request contained in each first operation log, and the target version number, storage address, and target data identifier of the target data, is further configured to:

[0192] For a first operation log whose operation type is DELETE, obtaining a target version number of the first operation log according to a request parameter of an HTTP request included in the first operation log;

[0193] For a first operation log whose operation type is PUT object or POST to complete segmented upload, obtain the target version number of the first operation log according to a response header corresponding to the HTTP request contained in the first operation log.

[0194] The consistency audit compensation device 40 provided in the embodiment of the present application can be used to execute the consistency audit compensation method in any of the above embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0195] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the electronic device 50 provided in the embodiment of the present application includes: at least one processor 501 and a memory 502. The electronic device 50 also includes a communication component 503. The processor 501, the memory 502 and the communication component 503 are connected via a bus 504.

[0196] In the specific implementation process, at least one processor 501 executes the computer execution instructions stored in the memory 502, so that at least one processor 501 executes the above-mentioned consistency audit compensation method.

[0197] The specific implementation process of the processor 501 can be found in the above-mentioned method embodiment, and its implementation principle and technical effect are similar, so the embodiments of the present application will not be repeated here.

[0198] The electronic device provided in the embodiment of the present application can be implemented as Figure 1 The audit compensation device is used to execute the consistency audit compensation method provided by any of the above method embodiments. Its implementation principle and technical effect are similar and will not be repeated here.

[0199] In the above Figure 5In the illustrated embodiment, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present application may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0200] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0201] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0202] The present application also provides a computer program product, including a computer program, which implements the above-mentioned consistency audit compensation method when executed by a processor.

[0203] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above-mentioned consistency audit compensation method is implemented.

[0204] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0205] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0206] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0207] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0208] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0209] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0210] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0211] It should be further noted that, although the various steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order restriction on the execution of these steps, and these steps can be executed according to other orders. Moreover, at least a part of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0212] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0213] In the above embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0214] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0215] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A consistency audit compensation method, It is characterized in that include: Acquire multiple first operation logs of the main site, where the first operation logs are operation logs corresponding to Hypertext Transfer Protocol (HTTP) requests sent by other devices to the main site; Determine a second operation log from multiple first operation logs according to a preset operation type and an operation type of the HTTP request contained in each first operation log, as well as a target version number, a storage address, and a target data identifier of the target data, wherein the preset operation type is an operation type corresponding to a valid change request, the operation type of the second operation log is the preset operation type, and the target version number, the storage address, and the target data identifier of the target data contained in different second operation logs are all different; According to each second operation log, data audit is performed on the master site and the slave site to generate an audit result, the slave site is used to back up the data in the master site, and the audit result is used to indicate whether the data in the master site and the slave site are consistent.

2. The method according to claim 1, It is characterized in that The determining of the second operation log from the plurality of first operation logs according to the preset operation type and the operation type of the HTTP request contained in each first operation log and the target version number, storage address and target data identifier of the target data comprises: According to the preset operation type and the operation type of the HTTP request contained in each first operation log, screening out a plurality of third operation logs whose operation type is the preset operation type from the plurality of first operation logs; According to the target version number, storage address and target data identifier of the target data contained in each third operation log, a deduplication operation is performed on the plurality of third operation logs to obtain the second operation log.

3. The method according to claim 1, It is characterized in that The step of performing data audit on the master site and the slave site according to each second operation log and generating audit results includes: Determine the audit time of the second operation log according to a preset time and the operation time of the target data contained in each second operation log, wherein the preset time is greater than or equal to the data backup time of backing up the data in the primary site to the secondary site; When the current time is the audit time of the second operation log, data audit is performed on the master site and the slave site according to the second operation log to generate the audit result.

4. The method according to claim 3, It is characterized in that The step of determining the audit time of the second operation log according to the preset duration and the operation time of the target data contained in each second operation log includes: The operation time corresponding to the second operation log is accumulated with the preset duration, and the accumulated time is determined as the audit time of the second operation log.

5. The method according to claim 3 or 4, It is characterized in that The step of performing data audit on the master site and the slave site according to the second operation log to generate the audit result includes: According to the second operation log, a data audit request is sent to the master site and the slave site respectively, wherein the data audit request includes a target version number, a storage address and a target data identifier of the target data corresponding to the second operation log; Respectively obtaining response information returned by the master site and the slave site, wherein the response information is used to indicate whether target data corresponding to the second operation log exists in the master site or the slave site; An audit result corresponding to the second operation log is generated according to the response information returned by the master site and the slave site.

6. The method according to claim 5, It is characterized in that The response information includes a return code and an ETag, wherein the return code is used to indicate whether the storage address stores the to-be-confirmed data whose data identifier is the target data identifier, and the ETag is used to indicate whether the version number of the to-be-confirmed data is the target version number; Correspondingly, generating the audit result corresponding to the second operation log according to the response information returned by the master site and the slave site includes: If any return code in the response information returned by the master site and the slave site indicates that the data to be confirmed does not exist, an audit result indicating that the data of the master site and the slave site are inconsistent is generated; If the return codes in the response information returned by the master site and the slave site both indicate that the data to be confirmed exists, and any ETag indicates that the version number corresponding to the stored data to be confirmed is not the target version number, then an audit result indicating that the data of the master site and the slave site are inconsistent is generated; If the return codes in the response information returned by the master site and the slave site both indicate the existence of the data to be confirmed, and the ETag both indicate that the version number corresponding to the stored data to be confirmed is the target version number, an audit result is generated indicating that the data of the master site and the slave site are consistent.

7. The method according to any one of claims 1 to 4, It is characterized in that The method further comprises: If the audit result of the second operation log indicates that the data of the master site and the slave site are inconsistent, the target data corresponding to the second operation log is synchronized between the master site and the slave site.

8. The method according to any one of claims 1 to 4, It is characterized in that The preset operation type includes at least one of the following: delete DELETE, update PUT object, submit POST to complete segment upload.

9. The method according to any one of claims 1 to 4, It is characterized in that Before determining the second operation log from the plurality of first operation logs according to the preset operation type and the operation type of the HTTP request contained in each first operation log, and the target version number, storage address and target data identifier of the target data, the method further includes: For a first operation log whose operation type is DELETE, obtaining a target version number of the first operation log according to a request parameter of an HTTP request contained in the first operation log; For a first operation log whose operation type is PUT object or POST to complete segmented upload, obtain the target version number of the first operation log according to a response header corresponding to the HTTP request contained in the first operation log.

10. A consistency audit compensation device, It is characterized in that include: An acquisition module, used to acquire a plurality of first operation logs of the main site, where the first operation logs are operation logs corresponding to Hypertext Transfer Protocol HTTP requests sent by other devices to the main site; a determination module, configured to determine a second operation log from a plurality of first operation logs according to a preset operation type and an operation type of the HTTP request contained in each first operation log, as well as a target version number, a storage address, and a target data identifier of the target data, wherein the preset operation type is an operation type corresponding to a valid change request, the operation type of the second operation log is the preset operation type, and the target version number, the storage address, and the target data identifier of the target data contained in different second operation logs are all different; A generation module is used to perform data audit on the master site and the slave site according to each second operation log and generate an audit result. The slave site is used to back up the data in the master site. The audit result is used to indicate whether the data in the master site and the slave site are consistent.

11. An electronic device, It is characterized in that include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the consistency audit compensation method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the consistency audit compensation method as described in any one of claims 1 to 9.

13. A computer program product, It is characterized in that It includes a computer program, which, when executed by a processor, is used to implement the consistency audit compensation method as described in any one of claims 1 to 9.

Citation Information

Patent Citations

  • Data synchronization method and system, electronic apparatus, and computer-readable storage medium

    CN109359139A

  • Data synchronization method and device

    CN118132645A