Consistency detection method, system, equipment and medium

By obtaining and comparing the target document elements of local and server files, and automatically adjusting the file list, the problems of low manual comparison efficiency and poor accuracy are solved, and efficient and accurate data consistency detection and synchronization are achieved.

CN118885318APending Publication Date: 2024-11-01STATE GRID INFORMATION & TELECOMM BRANCH +1
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410749172.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-11
Publication Date
2024-11-01

AI Technical Summary

Technical Problem

In the prior art, manual comparison of backup files and server data is inefficient and prone to errors, making it difficult to ensure accuracy, especially when the data volume is large and there are many files.

Method used

Automate comparisons and tweak files to ensure consistency by obtaining target document elements for local file lists and server file lists, including decompressing incremental files, generating full file sets, selecting detection modes (online or offline), and adjusting file lists.

Benefits of technology

Improve the efficiency and accuracy of file comparison, reduce manual intervention, avoid human errors, and ensure data consistency and synchronization accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118885318A_ABST
    Figure CN118885318A_ABST
Patent Text Reader

Abstract

The invention discloses a consistency detection method, system and device and a medium, and the method comprises the steps: obtaining target document elements corresponding to all files in a local file list and target document elements corresponding to all files in a server file list; judging whether the local file list is consistent with the server file list or not, and adjusting files in the local file list based on the files in the server file list in response to a judgment result representing that the local file list is inconsistent with the server file list, and the local file list is consistent with the server file list. When it is detected that the local file list is inconsistent with the server file list, the automation tool can automatically perform necessary adjustments, such as updating, deleting or adding files, so as to ensure the consistency of the two lists. The requirement of manual intervention is reduced, and the synchronization efficiency and accuracy are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a consistency detection method, system, device and medium. Background Art

[0002] In daily system or server management, in order to prevent data loss, a regular backup strategy is usually adopted, especially the incremental backup method. Incremental backup refers to backing up only the data that has changed since the last backup, which can save storage space and improve backup efficiency. These backup files are also saved in a location that is different from the main server (i.e., offline storage) to prevent the risk of data loss due to natural disasters, human errors or other local disasters. In theory, if all incremental backups completely and accurately record every data change, then merging these incremental backup files should be able to completely restore the latest data status on the server. However, in actual operations, due to various management negligence, software defects, network problems and other factors, there may be differences between the backup file collection stored off-site and the actual data on the server, that is, file inconsistency.

[0003] In order to solve the problem of inconsistency between backup files and server data, the traditional approach is to use manual comparison. However, this method becomes extremely cumbersome and error-prone when the amount of data is huge and there are many files. Manual comparison is not only time-consuming and labor-intensive, but also inefficient. In addition, due to the existence of human factors, it is difficult to ensure the accuracy of the comparison results. Especially when dealing with large amounts of data, the possibility of human error increases significantly. Summary of the invention

[0004] Based on the above problems, the present application provides a consistency detection method, system, device and medium to improve the accuracy of comparison results.

[0005] To solve the above problems, the technical solutions provided in the embodiments of the present application are as follows:

[0006] The first aspect of the present application provides a consistency detection method, comprising:

[0007] Obtaining a target document element corresponding to each file in a local file list and a target document element corresponding to each file in a server file list, wherein the local file list includes backup files to be detected stored in a local storage space, and the server file list includes original files to be detected stored in the server;

[0008] Determining whether the local file list is consistent with the server file list according to a target document element, wherein the target document element is identification information used to characterize the status and content of a file;

[0009] In response to the determination result indicating that the local file list is inconsistent with the server file list, the files in the local file list are adjusted based on the files in the server file list until the local file list is consistent with the server file list.

[0010] In a possible implementation, before obtaining the target document element corresponding to each file in the local file list and the target document element corresponding to each file in the server file list, the method further includes:

[0011] Decompress the local incremental file package and merge the decompressed files to obtain the full file set;

[0012] Pull the files to be detected from the full file set to obtain a local file list.

[0013] In a possible implementation, before obtaining the target document element corresponding to each file in the local file list and the target document element corresponding to each file in the server file list, the method further includes:

[0014] Determine a target inspection method based on preset requirements, where the preset requirements are used to characterize restriction requirements for at least one of network status, data volume, data security, and detection real-time performance;

[0015] The target verification method includes an online detection mode, and the obtaining of the target document element corresponding to each file in the local file list and the target document element corresponding to each file in the server file list includes:

[0016] In response to the target verification mode being the online detection mode, a target document element corresponding to the local file list and a target document element corresponding to the server file list are obtained.

[0017] In a possible implementation, the target verification mode includes an offline detection mode, and obtaining the target document element corresponding to each file in the local file list and the target document element corresponding to each file in the server file list includes:

[0018] Get the server file list and prepared document elements;

[0019] Generate a data table based on the server file list and the prepared document elements;

[0020] Serializing the data table to obtain a data table serialization file;

[0021] Migrating the data table serialization file to the local storage space;

[0022] The step of obtaining the target document element corresponding to each file in the local file list and the target document element corresponding to each file in the server file list includes:

[0023] The target document element corresponding to each file in the local file list is obtained, and the target document element corresponding to the data table serialization file is obtained.

[0024] In a possible implementation, in response to the judgment result indicating that the local file list is inconsistent with the server file list, adjusting the files in the local file list based on the files in the server file list until the local file list is consistent with the server file list includes:

[0025] In response to the presence of a first file in the local file list, the first file in the local file list is deleted, where the first file is a backup file that has no corresponding relationship with each file in the server file list.

[0026] In a possible implementation, in response to the judgment result indicating that the local file list is inconsistent with the server file list, adjusting the files in the local file list based on the files in the server file list until the local file list is consistent with the server file list includes:

[0027] In response to the existence of a second file in the local file list, a corresponding third file is downloaded from the server file list based on the second file, and the second file in the local file list is deleted, wherein the second file is a backup file in the server file list that is inconsistent with a target document element of the third file in the server file list, and the third file is an original file in the server file list that has a corresponding relationship with the second file.

[0028] In a possible implementation, in response to the judgment result indicating that the local file list is inconsistent with the server file list, adjusting the files in the local file list based on the files in the server file list until the local file list is consistent with the server file list includes:

[0029] In response to the fourth file not existing in the local file list, the fourth file in the server file list is downloaded to the local file list, wherein the fourth file is an original file existing in the server file list but not in the local file list.

[0030] A second aspect of the present application provides a consistency detection system, including:

[0031] an acquisition unit, configured to acquire a target document element corresponding to each file in a local file list and a target document element corresponding to each file in a server file list, wherein the local file list includes backup files to be detected stored in a local storage space, and the server file list includes original files to be detected stored in the server;

[0032] A judging unit, configured to judge whether the local file list is consistent with the server file list according to a target document element, wherein the target document element is identification information used to characterize the status and content of a file;

[0033] The adjusting unit is used for adjusting the files in the local file list based on the files in the server file list in response to the judgment result indicating that the local file list is inconsistent with the server file list, until the local file list is consistent with the server file list.

[0034] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the consistency detection method described in the first aspect is implemented.

[0035] The fourth aspect of the present application provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes the consistency detection method as described in the first aspect above.

[0036] Compared with the prior art, this application has the following beneficial effects:

[0037] By obtaining the target document elements corresponding to each file in the local file list and the target document elements corresponding to each file in the server file list, it is determined whether the local file list is consistent with the server file list according to the target document elements. In response to the determination result indicating that the local file list is inconsistent with the server file list, the files in the local file list are adjusted based on the files in the server file list until the local file list is consistent with the server file list. The automated solution can quickly traverse and compare a large number of files, greatly improving the efficiency of detection. Thus, human errors, such as misreading file names, ignoring subtle differences, etc., are avoided, and the automated solution can usually provide more accurate comparison results. When it is detected that the local file list is inconsistent with the server file list, the automated tool can automatically make necessary adjustments, such as updating, deleting or adding files, to ensure the consistency of the two lists. The need for manual intervention is reduced, and the efficiency and accuracy of synchronization are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 A flow chart of a consistency detection method provided in an embodiment of the present application;

[0040] Figure 2 A schematic diagram of a consistency detection method provided in an embodiment of the present application;

[0041] Figure 3 A structural diagram of a consistency detection system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0043] To facilitate understanding of the technical solutions provided by the embodiments of the present application, the background technology involved in the embodiments of the present application will be described below.

[0044] As mentioned above, in order to prevent data loss due to natural disasters, human errors or local disasters (such as fire, hardware failure, etc.), backup files are usually stored in a location different from the main server, which is called "offline off-site storage". This strategy ensures that even if the main server fails unrecoverably, the data can be restored from the backup.

[0045] Ideally, if all incremental backups are complete and accurate, then by merging these incremental backup files, the latest data status on the server can be completely restored in theory. This is based on the core principle of incremental backup: each backup records all changes since the last backup. However, in actual operations, due to various factors (such as management negligence, software defects, network problems, etc.), the backup file collection stored off-site may be inconsistent with the actual data on the server. This inconsistency may be caused by a variety of reasons, such as some data changes are not backed up correctly, the backup process is interrupted, or data is corrupted.

[0046] To address the above issues, manual comparison is currently commonly used. However, manual comparison of the differences between the server and the offsite backup files is an extremely tedious and error-prone task, especially when the data volume is huge and the files are numerous. Manual comparison is not only time-consuming and labor-intensive, but also inefficient, and it is difficult to guarantee the accuracy of the comparison results.

[0047] In order to solve this problem, a consistency detection method is provided in an embodiment of the present application. By obtaining the target document elements corresponding to each file in the local file list and the target document elements corresponding to each file in the server file list, according to the target document elements, it is judged whether the local file list is consistent with the server file list, and in response to the judgment result indicating that the local file list is inconsistent with the server file list, the files in the local file list are adjusted based on the files in the server file list until the local file list is consistent with the server file list. The automation provided by the present application can quickly traverse and compare a large number of files, efficiently perform data matching and verification, and greatly improve the efficiency of detection. Compared with manual verification, the automation scheme is not limited by human factors, can work continuously and stably, and will not reduce efficiency due to fatigue, distraction or speed limit. The automation scheme is based on the comparison of precise target document elements, avoiding errors that may be introduced by human factors, such as misreading file names, ignoring subtle differences, etc. The automation tool can accurately identify the status and content differences of files, and provide accurate judgment results, reducing the possibility of false positives and false negatives.

[0048] It should be noted that the consistency detection method, system, device and medium provided in the present application can be applied to the field of computer technology. The above is only an example and does not limit the application field of the consistency detection method, system, device and medium provided in the present application. In addition, the embodiments of the present application may not limit the execution subject of the consistency detection. For example, the consistency detection method of the embodiment of the present application can be applied to data processing devices such as terminal devices or servers. Among them, the terminal device can be an electronic device such as a smart phone, a computer, a personal digital assistant (PDA), a tablet computer, etc. The server can be an independent server, a cloud server or a cluster server composed of multiple servers.

[0049] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0050] The following describes a consistency detection method provided by the present application through an embodiment. Figure 1 ,Should Figure 1 A flow chart of a consistency detection method provided in an embodiment of the present application, the method comprising:

[0051] S101, obtaining a target document element corresponding to each file in a local file list and a target document element corresponding to each file in a server file list.

[0052] The local file list includes backup files to be detected stored in the local storage space, and the server file list includes original files to be detected stored in the server. For each file locally and on the server, a target document element for comparison is determined. The target document element may include a file name, a file size, a modification time, and a hash value of the file content.

[0053] In a possible implementation, the local file list may be determined as follows:

[0054] Traverse the specified backup file directory in the local storage space and generate a list of all files. For each file in the list, extract its name, path, size, modification time and other basic information, and may include the hash value of the file as a content verification identifier, and then generate a local file list.

[0055] In a possible implementation, the server file list may be determined in the following manner:

[0056] Obtain a list of original files to be tested from the server. Also extract the basic information of these files, including name, path, size, modification time, hash value, etc., and then generate a server file list.

[0057] In a possible implementation, a filter condition may be set to determine a corresponding server file list from the server or local storage space. For example, the filter condition is set to select files with a modification time within the past month. Thus, the files that meet the filter condition in the local storage space and the server are traversed to obtain a local file list and a server file list.

[0058] S102: Determine whether the local file list is consistent with the server file list according to the target document element.

[0059] The target document element is identification information used to characterize the status and content of the file. The target document element (such as file name, size, modification time and hash value) is used to compare the local file and the file on the server.

[0060] In a possible implementation, a quick match can be performed by file name and path. If the file name and path match, but the target document elements are different, it means that the file content has been changed, the backup is wrong, or the file is damaged.

[0061] S103: In response to the judgment result indicating that the local file list is inconsistent with the server file list, adjust the files in the local file list based on the files in the server file list until the local file list is consistent with the server file list.

[0062] In actual application scenarios, a difference report can be generated based on the judgment results, listing the differences between the local and server, including which files need to be updated, which files need to be deleted, which files need to be added, etc.

[0063] Based on the judgment result, the corresponding operation is performed to update the local file list. This may include downloading missing or updated files from the server, deleting redundant local files, or merging local changes into server files.

[0064] Also, after the synchronization is complete, recompare the local and server file lists to ensure that they are now consistent. If there are any inconsistencies, further error handling or recovery operations may need to be performed. Throughout the process, log all operations, errors, and warnings. If you encounter problems that cannot be solved, provide clear error messages and possible solutions to the operator.

[0065] The following describes a consistency detection method provided by the present application through an embodiment.

[0066] The method includes:

[0067] S201. The local incremental file packages are decompressed and merged together to form a full file set.

[0068] During the daily maintenance of the server, the server data will be backed up regularly. The backup method is to record only the data that has changed since the last backup, that is, incremental backup.

[0069] Local incremental files are backup files generated within a specific time interval for the data that has changed since the last backup. These files only contain the data that has been added or modified since the last backup, rather than the complete data of the entire system. This backup strategy can significantly reduce the time and storage space required for backup.

[0070] A full file set refers to a complete backup set that contains all the data of a system or server at a certain point in time. It is a full file set that includes all files and directories by decompressing and merging multiple local incremental file packages. This process involves decompressing the incremental files at each point in time one by one in the order in which they were packaged, and then merging these incremental data into the same folder, ultimately forming a full backup set that contains all data and is consistent with the server data status. The purpose of this is to ensure that even in the event of data loss or server failure, the complete data status of the server can be restored by restoring this full file set, thereby ensuring data security and business continuity.

[0071] The local incremental files can be packaged in ZIP or ISO format. ZIP is a commonly used compression and archiving format that is easy to store and transmit; ISO is a CD image format that is often used to completely copy the contents of a CD, including the file structure and system information. Both formats are easy to manage and move large amounts of data.

[0072] In a possible implementation, the incremental backup can correctly reflect the data change process only if it is applied in chronological order. Therefore, before decompression, the incremental files can be sorted in chronological order: First, sort the incremental backup files according to their packaging time.

[0073] Then, decompress these incremental backup files one by one in chronological order. The decompression process will restore the file changes recorded at the time of backup. The decompressed files can be merged into a specified folder, which will eventually constitute a complete set of full files. During the merging process, new data will overwrite or add to existing files to reflect the latest status of the server data.

[0074] After the above steps, all incremental backup contents are integrated together to form a complete set of files that are consistent with the current data status of the server. This process ensures that even if the server data is lost or damaged, data can be quickly restored through these backup files, ensuring business continuity and data security.

[0075] S202. Determine the target inspection method according to preset requirements.

[0076] The target verification method can be an online verification method or an offline verification method. In actual application scenarios, the mode can be selected based on one or more of the actual application scenarios, network conditions, and data security requirements. Different verification methods have different steps. For the consistency detection process, please refer to Figure 2 , Figure 2 A process diagram of a consistency detection method provided in an embodiment of the present application.

[0077] When the preset requirement is determined based on the network status, it includes the following two cases:

[0078] If the target environment has a high-speed network connection that is stable and continuous, online verification is the first choice. This allows instant access to server file information, quickly completes the comparison, and ensures real-time consistency of data. In the face of unreliable network connections or speed-limited scenarios, offline mode is more appropriate. By pre-downloading the server file list and verification data to the local computer, it reduces dependence on the network and improves the feasibility of verification.

[0079] When the preset requirements are determined based on the amount of data, there are two cases:

[0080] Large data volume: When the data set is large, frequent online inspection may cause network congestion and excessive resource consumption. Offline mode reduces network burden and improves efficiency through batch processing and periodic synchronization. Small data volume: When the data volume is not large, it is relatively simple to implement online inspection, which can quickly complete the comparison and has limited impact on network resources. Among them, whether the data volume is large or small can be divided according to the preset data volume threshold.

[0081] When the preset requirements are determined based on the real-time nature of the detection, there are two cases:

[0082] High immediacy requirements: When business operations require immediate feedback or data must always be kept up to date, online verification is a must to ensure that any changes can be captured and processed immediately. Tolerable delays: If a small delay in data consistency is within an acceptable range, such as regular data synchronization tasks, offline mode provides a more economical and efficient solution.

[0083] For example, if the preset requirements indicate that the network status is intermittent connection, the data volume is large, and there are high requirements for data security, but the detection real-time requirements are relatively loose, then offline mode will be the best choice. In this case, the server file list and related verification information will be downloaded to the local computer when the network condition is good, and then a detailed file consistency check will be performed in the local environment to ensure data security while effectively coping with the challenges brought by network instability.

[0084] In the online verification mode, the system or client will directly interact with the server-side interface in real time to verify the consistency of the file set. The advantage of this mode is immediacy, which can quickly obtain the latest data set elements on the server side for comparison to ensure the timeliness and accuracy of the verification results. It is suitable for scenarios with stable network environments and high requirements for data real-time. Through real-time verification, differences can be discovered and processed immediately, reducing the potential risks caused by data inconsistency.

[0085] When it is impossible to connect to the server in real time due to network restrictions, security considerations or other reasons, the offline verification method is used. The offline mode is suitable for scenarios where the network is unstable or data verification needs to be performed in an isolated network environment. Although the real-time verification is not as good as the online mode, it provides flexibility and independence to ensure that necessary data consistency checks and maintenance can be performed even when there is no network connection.

[0086] If the target inspection mode is offline detection mode, perform the following steps:

[0087] Step A1: Obtain a server file set list and a document element corresponding to the server file set list, and generate a data table based on the obtained server file set list and document element.

[0088] This step involves operating on the server side, first retrieving all relevant file information from the server's storage system. Document elements refer to a collection of key information used to uniquely identify and verify file attributes, mainly including the file's name, creation time, last modification time, and the MD5 hash value of the file content. These elements together constitute the digital fingerprint of the file, allowing us to accurately identify and track file changes at different time points or between different systems, and verify their integrity and authenticity.

[0089] Subsequently, one or more items of this information are combined to generate a structured data table, which is a list containing all file metadata, providing basic data for subsequent consistency checks.

[0090] Step A2: Serialize the generated data table into a data table serialization file.

[0091] A data table is essentially a data structure in memory. In order to facilitate storage, transmission and reuse, it needs to be converted into a persistent format. Serialization is the process of converting this data structure into a series of bytes, usually saving it in a file format such as JSON, XML or binary format. In this way, a data table serialization file is obtained, and the original complex object structure is encoded into a form that is easy to store and transmit.

[0092] Step A3: Migrate the data table serialization file to the local computer.

[0093] After serialization is completed, the generated data table file needs to be transferred from the server to the local environment where the verification operation is performed. This can be achieved by downloading from the network, using FTP services, cloud storage synchronization, or other secure data transmission methods. The purpose is to ensure that the local environment has the same copy of the data table as the server, preparing for the subsequent offline consistency check. This process ensures that even without a real-time network connection, the latest file information on the server can be used for file consistency comparison.

[0094] Finally, the data table serialization file that has been transferred to the local is imported into the local preset location. Then, the target information of the local file and the server file is compared to determine whether there is a difference in the file.

[0095] In a possible implementation, a detection unit can be set in a local device or a device connected to the local device to read the information in the data table and compare it with the actual content of the local file set. By comparing the file name, timestamp and MD5 value, the tool can quickly identify which files are matched and which are different, thereby guiding subsequent synchronization or repair operations.

[0096] S203: Obtain a local file set list and a document element corresponding to the local file set list.

[0097] The local file list refers to a list generated by scanning and recording all files in a specific directory or a series of directories on the local storage medium (such as a hard disk, flash drive, etc.) during the document set consistency check. For each listed file, in order to accurately match the file information on the server later, the system can extract the following document elements that represent the key attributes of the file:

[0098] Filename: The base name of the file, without path information.

[0099] Path: The full path of the file in the local storage system, used to locate the file.

[0100] Size: The size of the file in bytes, used to estimate storage usage and data transfer requirements.

[0101] Creation Time: The date and time the file was created on the local system.

[0102] Modification time: The date and time when the file was last modified, used to track the file update status.

[0103] Access Time: The date and time when the file was last accessed. Some systems or applications may require this information.

[0104] MD5 Hash: By calculating the MD5 hash summary of the file content, a fixed-length string is generated as the unique fingerprint of the file content, which is used to efficiently verify whether the file content is consistent.

[0105] Permissions: Access permissions for files or directories, such as read, write, and execute permissions, which are also important in some scenarios.

[0106] The purpose of generating a local file list is to establish a comprehensive and detailed local document snapshot, so that any differences with the server-side document set can be discovered and corrected by comparing this list to ensure that the data stored in the two places are fully synchronized and consistent.

[0107] S204: Check whether the document elements of the local document and the server document in the file collection are consistent.

[0108] Among them, the document element used for comparison and verification can be MD5. In this step, the system compares the MD5 value of each file previously obtained locally with the MD5 value of the corresponding file obtained from the server-side serialized data table. Comparison of MD5 values ​​is an efficient means of verifying the consistency of file content. If the MD5 values ​​of two files are the same, it can be considered that the contents of the two files are completely consistent. This process is automated and accurate, and can quickly identify whether the file has changed during the transmission or backup process.

[0109] The first detection result is that if the MD5 value of a local file matches the MD5 value recorded on the server, it means that no action is required on the file because it is already the latest or the exact same version as the server. Therefore, you can ignore this file and continue to check the next file to save time and resources and improve overall inspection efficiency. This strategy ensures that only files with differences are further processed, such as updating or re-downloading.

[0110] The second test result, if the test result shows inconsistency, it means that there is a difference between the local document and the corresponding document on the server. In this case, the strategy adopted is to regard the document on the server as the latest or correct version. The specific operation is that for those inconsistent documents, the system will re-download the document from the server to ensure that the latest and accurate version is obtained. After the download is complete, this latest document is merged into the local file set to replace the original, inconsistent file. The purpose of this is to ensure that the local backup file set is completely consistent with the document data on the server, to ensure the integrity and reliability of the data, and to facilitate future data recovery or backup verification.

[0111] Specifically, in actual application scenarios, the detection results may appear in the following situations:

[0112] The local file exists but the server does not: In this case, the local file may be a redundant file generated by an erroneous operation or some modifications that are no longer needed have been made locally. The solution is to delete the file directly from the local file set to ensure that the local file set is consistent with the server.

[0113] If the document exists locally, but the MD5 of the document on the server is inconsistent with the local file MD5: This usually means that the file has been modified or updated on the server since the last backup, and the local file has not yet synchronized these changes. At this time, you need to download the file from the server again based on the document information on the server and replace the old local version to ensure that the file content is up to date.

[0114] If the document does not exist locally but exists on the server: This means that a file was not successfully downloaded to the local computer during the backup or synchronization process. To ensure the integrity of the file set, the system needs to download the missing file from the server based on the document information on the server and add it to the local file set.

[0115] In summary, compared with manual detection, the method provided by this application has the following significant advantages:

[0116] Automation and efficiency improvement: The automated file list comparison and adjustment process greatly improves work efficiency and reduces the time and errors of manual intervention. Manual detection is not only time-consuming but also prone to errors, especially when dealing with a large number of files. The automated solution can quickly complete the verification and synchronization of file status, regardless of the number of files.

[0117] Accuracy and consistency: Through precise matching of target document elements, the solution can more accurately identify file differences and ensure the consistency of the file set. Manual inspection may miss details due to negligence or visual fatigue, while the automated solution is not limited by such limitations and provides reliable results.

[0118] Save resources: Automated processes reduce the need for human resources, allowing IT teams to focus on other higher-value tasks. It also avoids the waste of additional storage space or unnecessary repeated file transfers caused by manual operations, thereby making more efficient use of network and storage resources.

[0119] Scalability and flexibility: As data volumes grow or detection requirements change, automated solutions can be more easily adjusted and expanded to accommodate new requirements. In contrast, manual methods are difficult to scale and costly to adjust when dealing with large or complex changes.

[0120] Audit and Compliance: Automated recording and reporting functions facilitate audit and compliance. Each detection and synchronization process has a detailed log, which is convenient for tracing and proving that the consistency of data is maintained in accordance with the specified standards, which is difficult to achieve through manual detection.

[0121] Instant response capability: For detected inconsistencies, the system can immediately adjust local files based on the server file list to ensure instant update and recovery of data, reducing the risk of business interruption caused by data inconsistency.

[0122] The above are some specific implementations of the consistency detection method provided in the embodiment of the present application. Based on this, the present application also provides a corresponding system for consistency detection. The system provided in the embodiment of the present application will be introduced from the perspective of functional modularization. Figure 3 A structural diagram of a consistency detection system provided in an embodiment of the present application.

[0123] The system comprises:

[0124] An acquisition unit 110 is used to acquire a target document element corresponding to each file in a local file list and a target document element corresponding to each file in a server file list, wherein the local file list includes backup files to be detected stored in a local storage space, and the server file list includes original files to be detected stored in the server;

[0125] A judging unit 111, configured to judge whether the local file list is consistent with the server file list according to a target document element, wherein the target document element is identification information used to characterize the status and content of a file;

[0126] The adjusting unit 112 is configured to adjust the files in the local file list based on the files in the server file list in response to the judgment result indicating that the local file list is inconsistent with the server file list, until the local file list is consistent with the server file list.

[0127] The embodiments of the present application also provide corresponding devices and computer storage media for implementing the consistency detection method provided in the embodiments of the present application.

[0128] The device includes a memory and a processor, the memory is used to store instructions or codes, and the processor is used to execute the instructions or codes, so that the device executes the consistency detection method described in any embodiment of the present application.

[0129] The computer storage medium stores codes, and when the codes are executed, the device executing the codes implements the consistency detection method described in any embodiment of the present application.

[0130] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the system or device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0131] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0132] It should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0133] It should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0134] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0135] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0136] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A consistency detection method, It is characterized in that include: Obtaining a target document element corresponding to each file in a local file list and a target document element corresponding to each file in a server file list, wherein the local file list includes backup files to be detected stored in a local storage space, and the server file list includes original files to be detected stored in the server; Determining whether the local file list is consistent with the server file list according to a target document element, wherein the target document element is identification information used to characterize the status and content of a file; In response to the determination result indicating that the local file list is inconsistent with the server file list, the files in the local file list are adjusted based on the files in the server file list until the local file list is consistent with the server file list.

2. The method according to claim 1, It is characterized in that Before obtaining the target document element corresponding to each file in the local file list and the target document element corresponding to each file in the server file list, the method further includes: Decompress the local incremental file package and merge the decompressed files to obtain the full file set; Pull the files to be detected from the full file set to obtain a local file list.

3. The method according to claim 1, It is characterized in that Before obtaining the target document element corresponding to each file in the local file list and the target document element corresponding to each file in the server file list, the method further includes: Determine a target inspection method based on preset requirements, where the preset requirements are used to characterize restriction requirements for at least one of network status, data volume, data security, and detection real-time performance; The target verification method includes an online detection mode, and the obtaining of the target document element corresponding to each file in the local file list and the target document element corresponding to each file in the server file list includes: In response to the target verification mode being the online detection mode, a target document element corresponding to the local file list and a target document element corresponding to the server file list are obtained.

4. The method according to claim 3, It is characterized in that The target verification method includes an offline detection mode, and the obtaining of the target document element corresponding to each file in the local file list and the target document element corresponding to each file in the server file list includes: Get the server file list and prepared document elements; Generate a data table based on the server file list and the prepared document elements; Serializing the data table to obtain a data table serialization file; Migrating the data table serialization file to the local storage space; The step of obtaining the target document element corresponding to each file in the local file list and the target document element corresponding to each file in the server file list includes: The target document element corresponding to each file in the local file list is obtained, and the target document element corresponding to the data table serialization file is obtained.

5. The method according to claim 4, It is characterized in that In response to the judgment result indicating that the local file list is inconsistent with the server file list, adjusting the files in the local file list based on the files in the server file list until the local file list is consistent with the server file list, includes: In response to the presence of a first file in the local file list, the first file in the local file list is deleted, where the first file is a backup file that has no corresponding relationship with each file in the server file list.

6. The method according to claim 4, It is characterized in that In response to the judgment result indicating that the local file list is inconsistent with the server file list, adjusting the files in the local file list based on the files in the server file list until the local file list is consistent with the server file list, includes: In response to the existence of a second file in the local file list, a corresponding third file is downloaded from the server file list based on the second file, and the second file in the local file list is deleted, wherein the second file is a backup file in the server file list that is inconsistent with a target document element of the third file in the server file list, and the third file is an original file in the server file list that has a corresponding relationship with the second file.

7. The method according to claim 4, It is characterized in that In response to the judgment result indicating that the local file list is inconsistent with the server file list, adjusting the files in the local file list based on the files in the server file list until the local file list is consistent with the server file list, includes: In response to the fourth file not existing in the local file list, the fourth file in the server file list is downloaded to the local file list, wherein the fourth file is an original file existing in the server file list but not in the local file list.

8. A consistency detection system, It is characterized in that The system comprises: an acquisition unit, configured to acquire a target document element corresponding to each file in a local file list and a target document element corresponding to each file in a server file list, wherein the local file list includes backup files to be detected stored in a local storage space, and the server file list includes original files to be detected stored in the server; A judging unit, configured to judge whether the local file list is consistent with the server file list according to a target document element, wherein the target document element is identification information used to characterize the status and content of a file; The adjusting unit is used for adjusting the files in the local file list based on the files in the server file list in response to the judgment result indicating that the local file list is inconsistent with the server file list, until the local file list is consistent with the server file list.

9. An electronic device, It is characterized in that include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the consistency detection method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the consistency detection method according to any one of claims 1 to 7.

Citation Information

Cited By

  • Construction waste wagon balance data distributed storage verification method, computer equipment and medium

    CN119988360A