Data synchronization method, client, computer-readable storage medium, and computer program product

Abstract

This application provides a data synchronization method, a client, a computer program product, and a computer-readable storage medium. The method includes: initiating a synchronization request after a synchronization interval; wherein the synchronization interval is the time interval between the completion time of the previous synchronization and the start time of the current synchronization; responding to the synchronization request and obtaining a first hash value of the root folder of the first directory tree of the client and a second hash value of the root folder of the second directory tree of the server; wherein the directory tree includes multiple levels of files and folders; determining whether directory changes have occurred on the client and server during the synchronization interval based on the first hash value and the second hash value; if directory changes have occurred, generating synchronization operation information; wherein the synchronization operation information includes operations such as uploading, downloading, and deleting files or folders; and executing the synchronization operation information to the client and server to complete directory synchronization.

Description

Technical Field

[0001] This application relates to computer technology, and more particularly to a data synchronization method, a client, a computer-readable storage medium, and a computer program product. Background Technology

[0002] With the popularization of cloud computing and cloud storage technologies, users' data storage and access patterns are no longer limited to single terminal devices. To meet the urgent needs of business scenarios such as cross-regional office work and multi-terminal collaboration, achieving efficient and consistent synchronization of user files and folders across multiple clients and servers has become a crucial fundamental capability. Currently, mainstream synchronization methods mainly fall into two categories: first, schemes based on periodic full directory tree comparison, where the client periodically scans its entire local directory structure and compares it with the full directory information retrieved from the server to infer file changes; second, schemes based on monitoring the underlying file system of the operating system, capturing local file change events in real time through kernel drivers or system logs and reporting them to the server. However, existing methods heavily rely on the full data collection and comparison of the local directory trees on the client and server for change detection and comparison, resulting in large amounts of data acquisition, high client resource consumption, and long comparison times. Summary of the Invention

[0003] This application provides a data synchronization method, a client, a computer-readable storage medium, and a computer program product, which can reduce the amount of data transmitted over the network and the consumption of client computing resources, and improve comparison efficiency to reduce time consumption.

[0004] The technical solution of this application embodiment is implemented as follows: This application provides a data synchronization method, the method comprising: A synchronization request is initiated after a synchronization interval; wherein, the synchronization interval is the time interval between the completion time of the previous synchronization and the start time of the current synchronization. In response to the synchronization request, the system obtains the first hash value of the root folder of the first directory tree on the client and the second hash value of the root folder of the second directory tree on the server; wherein the directory tree includes files and folders at multiple levels; Based on the first hash value and the second hash value, determine whether directory changes have occurred on the client and server sides within the synchronization interval; If the directory change occurs, synchronization operation information is generated; wherein, the synchronization operation information includes operations such as uploading, downloading, and deleting files or folders; The synchronization operation information is sent to the client and server to complete the directory synchronization.

[0005] In the above scheme, determining whether a directory change has occurred on the client and server within the synchronization interval based on the first hash value and the second hash value includes: If the first hash value and the second hash value are different, obtain the incremental metadata that has changed since the last synchronization completion time from the server; Based on the first metadata of the first directory tree of the client, the incremental metadata, and the second metadata of the second directory tree of the server, the directory changes that occurred in the client and the server during the synchronization interval are determined; wherein, the second directory tree is the server directory tree stored locally on the client at the time of the last synchronization completion.

[0006] In the above scheme, determining the directory changes that occur on the client and the server within the synchronization interval based on the first metadata of the first directory tree of the client, the incremental metadata, and the second metadata of the second directory tree of the server includes: Based on the incremental metadata and the second metadata, the target metadata of the target directory tree of the server at the start time of this synchronization is obtained; Based on the first metadata and the target metadata, determine the directory changes that occurred on the client and the server within the synchronization interval.

[0007] In the above scheme, determining the directory changes that occur on the client and the server within the synchronization interval based on the first metadata and the target metadata includes: Starting from the root folder of the target directory tree, the hash value of each node in the target directory tree is compared with the hash value of the node under the same path in the first directory tree to obtain a second comparison result; the metadata includes at least the hash value; If the second comparison result indicates that the hash value of the first target folder in the target directory tree is different from the hash value of the first target folder under the same path in the first directory tree, it is determined that the subfolders or files of the first target folder have undergone directory changes; wherein, the hash value of the folder is obtained by concatenating the hash value of the folder name and the hash values ​​of all subfolders and subfiles included in the folder; If the second comparison result indicates that the hash value of the first target file in the target directory tree is different from the hash value of the first target file under the same path in the first directory tree, it is determined that the first target file has undergone a directory change; wherein, the hash value of the file is obtained by concatenating the hash value of the file name and the hash value of the file content.

[0008] In the above scheme, generating synchronization operation information if the directory change occurs includes: Based on the path of a folder or file, the directory changes on the client and the directory changes on the server are matched to obtain different combinations of operations; wherein, the directory change represents the change in the state of files and folders caused by the corresponding operation within the synchronization interval, and the operation includes adding, modifying and deleting files, as well as adding and deleting folders; The synchronization operation information is determined from the target conflict matrix based on the different operation combinations; wherein, the target conflict matrix defines the processing method corresponding to the different operation combinations, and the target conflict matrix includes upload, download, delete, create, or conflict.

[0009] In the above scheme, the conflict handling methods include at least one of the following: overwriting the server's file with the client's file; overwriting the client's file with the server's file; and renaming the client's file.

[0010] In the above scheme, determining the synchronization operation information based on the target change information and the target conflict matrix includes: If the subfiles and subfolders of the second target folder have already been deleted, then delete the second target folder. If the parent folder of the second target file has already been created, then create the second target file. If the parent folder of the newly created third target folder has already been created, then create the third target folder.

[0011] A data synchronization device, the device comprising: A synchronization request unit is used to initiate a synchronization request after a synchronization interval; wherein, the synchronization interval is the time interval between the time when the last synchronization was completed and the time when the current synchronization started. The acquisition unit is used to respond to the synchronization request and acquire the first hash value of the root folder of the first directory tree of the client and the second hash value of the root folder of the second directory tree of the server; wherein, the directory tree includes files and folders at multiple levels; The change determination unit is used to determine, based on the first hash value and the second hash value, the directory changes that occur on the client and server within the synchronization interval; An operation generation unit is used to generate synchronization operation information based on the directory change; wherein, the synchronization operation information includes operations such as uploading, downloading, and deleting files or folders; The synchronization execution unit is used to execute the synchronization operation information to the client and the server to complete the directory synchronization.

[0012] A client, the client comprising: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the data synchronization method provided in the embodiments of this application.

[0013] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the data synchronization method provided in this application when executed by a processor.

[0014] This application provides a computer program product, including a computer program or computer executable instructions, which, when executed by a processor, implements the data synchronization method provided in this application.

[0015] The embodiments of this application have the following beneficial effects: by comparing only the hash values ​​of the root folders of the client and server directory trees, it is possible to quickly determine whether a directory change has occurred within the synchronization interval, instead of requiring the full fetching and comparison of directory tree metadata as in related technologies. This not only reduces the amount of network data transmitted (only incremental metadata needs to be transmitted instead of the full data) and the consumption of client computing resources, but also improves the comparison efficiency and thus reduces the time consumption. At the same time, the hash value-based change detection mechanism can sensitively capture any changes in the directory tree (because the hash value aggregates the information of all lower-level nodes), ensuring the timeliness and accuracy of synchronization. Attached Figure Description

[0016] Figure 1 This is a first flowchart illustrating the data synchronization method provided in this application embodiment; Figure 2 This is a second flowchart illustrating the data synchronization method provided in the embodiments of this application; Figure 3 This is a first flowchart illustrating the data synchronization method provided in this application embodiment; Figure 4 This is a first flowchart illustrating the data synchronization method provided in this application embodiment; Figure 5 This is a first flowchart illustrating the data synchronization method provided in this application embodiment; Figure 6 This is a schematic diagram of the structure of the data synchronization device provided in the embodiments of this application; Figure 7 This is a schematic diagram of the client structure provided in the embodiments of this application.

[0017] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] It should be noted that with the development of cloud computing and cloud storage technologies, users are no longer limited to storing data on a single terminal. Instead, they want to maintain synchronization across multiple terminals to meet the needs of business scenarios involving multi-location work and multi-user collaboration (i.e., multi-client application scenarios). This requires a method to keep user files and folders synchronized across multiple terminals. Several related technologies for multi-terminal file / folder synchronization include: 1. Obtaining file changes by comparing directory tree data before and after. The client periodically scans its local directory tree (client-side directory tree) and pulls the server-side directory tree. By comparing this with the previous directory tree, changes in the local and server directories within the interval are obtained. The local changes are then pushed to the cloud (server-side), and the changes in the cloud are pulled back to the local machine. The characteristic of this solution is that it obtains directory changes on the client and server by comparing full directory information at regular intervals. 2. The client monitors the local file system in real time (by embedding underlying operating system drivers or reading system change logs) to obtain file / folder changes, and then sends the corresponding changes to the server, which broadcasts them to other clients. Other clients then synchronously apply these changes. When a user closes the client and modifies local files / folders after a period of time, upon restarting the client, a directory tree comparison is needed to detect the changes during that period. The key features of this solution are real-time monitoring of directory changes, server-side broadcast synchronization, and reliance on directory comparison to address the issue of the inability to monitor the directory when the client exits.

[0020] However, while the approach of obtaining file / folder changes by comparing directory tree data before and after is relatively simple to implement, with the main logic handled by the client and no need for server-side broadcasting, it becomes very time-consuming and consumes significant client-side computing and disk I / O resources when dealing with a large number of subdirectories or files (tens of thousands or more). This involves fetching directory data from the cloud, scanning local directory data, and comparing the directory data, especially when comparing local files, which is typically based on file hash values. A more efficient approach is to monitor the local file system for file / folder changes. This solution relies on the operating system's mechanisms for monitoring local directories and broadcasting directory changes in real time via the server. It also relies on directory comparison to address the issue of missing directory monitoring when the client exits. Furthermore, this solution depends on the operating system's directory monitoring mechanism, requiring separate adaptation for different operating systems. If the operating system does not support it, the solution cannot be implemented. Additionally, the server needs to broadcast directory changes, necessitating special server-side adaptation.

[0021] Based on this, embodiments of this application provide a data synchronization method, referring to... Figure 1 As shown, it is applied to the client side and is suitable for multi-client application scenarios. Specifically, it can be implemented in the following ways.

[0022] Step 101: Initiate a synchronization request after the synchronization interval.

[0023] The synchronization interval is the time interval between the completion time of the previous synchronization and the start time of the current synchronization.

[0024] In this embodiment of the application, the synchronization interval refers to the time period between the last synchronization completion time (T0) and the current synchronization start time (T1) (see reference). Figure 2 As shown in the figure, this is the time window during which the client does not actively perform synchronization operations; the synchronization request is an instruction sent by the client to the server, requesting the start of the directory synchronization process; it should be noted that if it is the first synchronization, then T0=T1.

[0025] In this embodiment of the application, after the client completes the last synchronization, it will wait for a synchronization interval, and then actively initiate a new synchronization request to the server (business layer) through the network at time T1. This synchronization interval can be a fixed time period or it can be triggered based on a specific event. In this way, the continuous synchronization operation is avoided from consuming system resources.

[0026] Step 102: Respond to the synchronization request and obtain the first hash value of the root folder of the first directory tree on the client and the second hash value of the root folder of the second directory tree on the server.

[0027] The directory tree includes files and folders at multiple levels.

[0028] In this embodiment, a directory tree is a hierarchical data structure used to organize files and folders, containing the complete path relationships of the root folder, subfolders, and files; the first directory tree refers to the directory structure on the client's local machine; the second directory tree refers to the directory structure stored on the server; the root folder of the first directory tree is the top-level folder of the first directory tree, and the root folder of the second directory tree is the top-level folder of the second directory tree, which is the ancestor node of all other files and folders; the hash value is a unique identifier obtained by calculating the file content or folder structure characteristics.

[0029] In this embodiment, the client responds to the synchronization request and obtains the first hash value of the root folder of the first directory tree from the local machine (i.e., the client), and only obtains the hash value of the root folder of the second directory tree (i.e., the second hash value) from the server, instead of the complete directory tree metadata. In this way, by obtaining only the hash value of the root folder of the client and the hash value of the root folder of the server, the amount of data transmitted over the network is greatly reduced, and a basis is provided for quickly determining whether full synchronization is needed.

[0030] Step 103: Based on the first hash value and the second hash value, determine the directory changes that occurred on the client and server during the synchronization interval.

[0031] In this embodiment, directory change refers to changes such as the addition, modification, and deletion of files or folders that occur on the client or server within the synchronization interval. The hash value of any folder in the directory tree contains the hash value information of all its subfolders and files. Any change to its subfolders and files will cause a change in its hash value. By comparing hash values, it is possible to quickly determine whether the directory tree has changed and which parts have changed. Therefore, the client compares the hash value of its local root folder (first hash value) with the hash value of the server's root folder (second hash value). If they are the same (i.e., the first hash value equals the second hash value), it means that no directory change has occurred within the synchronization interval; if they are different (i.e., the first hash value does not equal the second hash value), it means that at least one end (i.e., the client or the server) has experienced a directory change, and further analysis of the specific changes is needed. In this way, a simple hash value comparison can quickly determine whether detailed synchronization is needed, thereby avoiding unnecessary full directory tree comparisons and significantly improving performance. It should be noted that, referring to... Figure 2 As shown, the moment when the change is completed can be denoted as T2.

[0032] Step 104: Generate synchronization operation information based on directory changes.

[0033] The synchronized operation information includes operations such as uploading, downloading, and deleting files or folders.

[0034] In this embodiment, the synchronization operation information is a set of specific operation instructions that need to be executed to eliminate directory differences between the client and the server; uploading refers to transferring files added or modified by the client to the server; downloading refers to retrieving files added or modified by the server to the client; deleting refers to removing files or folders that are no longer needed on the client or the server; when a directory change is detected, corresponding synchronization operation instructions can be generated based on the specific content of the change (such as adding, modifying, or deleting files, adding or deleting folders, etc.) and in conjunction with a predefined conflict handling strategy.

[0035] Step 105: Perform synchronization operation information to the client and server to complete directory synchronization.

[0036] In this embodiment, directory synchronization refers to the process of bringing the directory tree states of the client and server into consistency. According to the generated synchronization operation information, upload, download, and delete operations are executed in an orderly manner, following specific order rules, such as creating parent folders before creating child files, deleting files before deleting empty folders, etc. This achieves eventual consistency between the directory states of the client and server, and avoids operation conflicts and path errors through orderly execution. It should be noted that, referring to... Figure 2 As shown, the time when directory synchronization is completed can be denoted as T3.

[0037] The data synchronization method provided in this application can quickly determine whether a directory change has occurred within the synchronization interval by comparing only the hash values ​​of the root folders of the client and server directory trees, instead of requiring the overhead of fully fetching and comparing directory tree metadata as in related technologies. This not only reduces the amount of network data transmitted (only incremental metadata needs to be transmitted instead of the full data) and the consumption of client computing resources, but also improves the comparison efficiency and thus reduces time consumption. At the same time, the hash value-based change detection mechanism can sensitively capture any changes in the directory tree (because the hash value aggregates the information of all lower-level nodes), ensuring the timeliness and accuracy of synchronization.

[0038] Based on the foregoing embodiments, this application provides a data synchronization method, referring to... Figure 3 As shown, this is applied to the client side and can be implemented in the following way.

[0039] Step 201: Initiate a synchronization request after the synchronization interval.

[0040] The synchronization interval is the time interval between the completion time of the previous synchronization and the start time of the current synchronization.

[0041] Step 202: Respond to the synchronization request and obtain the first hash value of the root folder of the first directory tree on the client and the second hash value of the root folder of the second directory tree on the server.

[0042] The directory tree includes files and folders at multiple levels.

[0043] In this application embodiment, a directory tree is as follows: Figure 4 As shown, all folders and files in the directory tree have their own hash values, such as... Figure 4 The root folder of the directory tree shown is Dir1-1, and its hash value is Hash1-1 = hash(hashDir1-1 + Hash2-1 + Hash2-2).

[0044] Step 203: If the first hash value and the second hash value are different, obtain the incremental metadata that has changed since the last synchronization was completed from the server.

[0045] In this embodiment, incremental metadata refers to the set of metadata for files and folders that have changed on the server side between the last synchronization completion time (T0) and the current synchronization start time (T1). It only contains the changed parts, not the complete directory tree metadata. When the client detects that the hash values ​​(i.e., the first hash value and the second hash value) of the root folder on the local machine and the server are different, it indicates that at least one end has undergone directory changes during the synchronization interval. At this time, the client does not fetch the complete directory tree from the server, but instead requests the server to obtain the data that has changed during this period, i.e., the incremental metadata (refer to...). Figure 4 (as shown in the thick lines); In this way, not only can the amount of data transmitted over the network be greatly reduced and the synchronization efficiency improved, but unnecessary full data retrieval is also avoided, saving bandwidth resources.

[0046] Step 204: Based on the first metadata of the first directory tree of the client, the incremental metadata, and the second metadata of the second directory tree of the server, determine the directory changes that occurred on the client and server during the synchronization interval.

[0047] The second directory tree is the server-side directory tree that is stored locally on the client at the time of the last synchronization completion.

[0048] In this embodiment, the first metadata refers to a complete set of data describing the attributes of all nodes in the current directory tree (first directory tree) of the client, specifically including client file metadata (ClientFiles / CF) and client folder metadata (ClientDirs / CD). The client file metadata is a set of metadata for all files on the client, and the client folder metadata is a set of metadata for all folders on the client. The second metadata is the complete metadata of the server directory tree stored locally on the client at the last synchronization completion time (T0), specifically including server file metadata (ServerFiles / SF) and server folder metadata (ServerDirs / SD). The server file metadata is a set of metadata for all files on the server, and the server folder metadata is a set of metadata for all folders on the server. It should be noted that the metadata corresponding to the three time points T0, T1, and T3 can be named as shown in Table 2 below. Among them, CF0 and CD0 are stored on the client side, SF0 and SD0 are stored on the server side, CF1, CD1, CF3, and CD3 are obtained by traversing the client's synchronized folder, and SF1 and SD1 are incremental metadata returned by the server side, obtained by calling the server side's "Get Server Directory Tree" interface, and SF0 and SD0 are merged to obtain the metadata (i.e., the metadata of the target directory tree).

[0049] In this embodiment, the client combines three parts of data for analysis: the complete metadata of the current local directory tree (first metadata), the incremental change data returned by the server (incremental metadata), and the metadata of the server's directory tree at the time of the last synchronization stored locally (second metadata). By comparing these data, the specific changes that occurred on the client and server respectively within the synchronization interval can be accurately determined.

[0050] Table 2

[0051] It should be noted that step 204 can be achieved in the following way: Step 204A: Based on incremental metadata and second metadata, obtain the target metadata of the target directory tree of the server at the start of this synchronization.

[0052] In this embodiment, the target directory tree is the actual directory tree state of the server at the start time (T1) of this synchronization; the target metadata is the set of all metadata describing the complete state of the target directory tree. The client merges the incremental metadata returned by the server with the previously stored server directory tree metadata (second metadata) to reconstruct the complete directory tree state of the server at the current moment, thus obtaining the target metadata. It should be noted that by using incremental merging, a complete view of the latest directory tree of the server is efficiently constructed, and the large overhead operation of directly fetching the complete directory tree is avoided.

[0053] Step 204B: Based on the first metadata and the target metadata, determine the directory changes that occurred on the client and server during the synchronization interval.

[0054] In this embodiment of the application, after obtaining the metadata of the current directory tree of the client (first metadata) and the metadata of the latest directory tree of the server (target metadata), all directory changes that have occurred at both ends are accurately identified through comparative analysis.

[0055] It's important to note that to obtain changes on the server and client sides within the synchronization interval, metadata can be compared. Typically, this involves comparing SF0 and SF1 (server-side file metadata) and SD0 and SD1 (server-side folder metadata) to determine server-side file and folder changes during the synchronization interval. However, to optimize comparison speed, the incremental metadata obtained by calling the server's "Get Server Directory Tree" interface can also be compared with SF0. ​​Similarly, client-side file and folder changes during the synchronization interval can be obtained by comparing CF0 and CF1, and CD0 and CD1.

[0056] It should be noted that step 204B can be achieved in the following way: Step 204b1: Starting from the root folder of the target directory tree, compare the hash value of each node in the target directory tree with the hash value of the node under the same path in the first directory tree to obtain the second comparison result.

[0057] The metadata includes at least a hash value.

[0058] Table 1

[0059] In this embodiment, a node is the basic unit of a directory tree, which can be a file or a folder. Each node has corresponding metadata, including but not limited to type (file / folder), name, path, hash value, size, modification time, etc. (as shown in Table 1 above). A "same path" refers to a completely identical location identifier in two directory trees, such as " / documents / work / report.txt". The client uses long polling, starting from the root folder of the target directory tree on the server, and employs a depth-first or breadth-first traversal method to compare the hash values ​​of corresponding nodes along the same path in both directory trees. It determines whether the two hash values ​​are the same or different. If they are the same, the entire directory tree is consistent, and the comparison ends. If they are different, the client traverses the next level of child nodes, continuing to compare the hash values ​​of nodes along the same path, repeating this process until all the specific nodes that have changed are located.

[0060] It should be noted that after step 204b1, either step 204b2 or step 204b23 can be executed. Step 204b2: If the hash value of the first target folder in the target directory tree is different from the hash value of the first target folder under the same path in the first directory tree, it is determined that the subfolders or files of the first target folder have undergone directory changes.

[0061] The hash value of a folder is determined based on the hash value of the folder's name and the hash values ​​of all its subfolders and subfiles.

[0062] In this embodiment, the first target folder refers to the first folder node with an inconsistent hash value found during the directory tree hierarchy comparison. This folder exists in the same path in both the target directory tree (server) and the first directory tree (client), but with different hash values. Sub-folders or files refer to the sub-items directly contained in the first target folder, including its next-level subfolders and direct files. Specifically, the hash value of a folder is obtained by concatenating the hash value of the folder name with the hash values ​​of all sub-items (subfiles and subfolders) and then performing a hash operation. (Refer to...) Figure 4 As shown, folder Dir1-1 has a subfolder Dir2-1 and a subfile Dir2-2. Therefore, the hash value of folder Dir1-1 is Hash1-1 = hash(hashDir1-1 + Hash2-1 + Hash2-2).

[0063] In this embodiment, since the calculation rule for a folder's hash value is that the hash value of a folder is equal to the hash value of its name plus the hash values ​​of all its sub-items (subfiles and subfolders) concatenated before hashing, when the hash value of a folder changes, it must be due to one of the following reasons: the folder's name itself has changed; direct subfiles or subfolders under the folder have been added, deleted, or modified; or deeper-level descendant items have changed, affecting the current folder through the transitivity of hash values. Therefore, when the hash value of the first target folder is detected to be inconsistent, it can be determined that a directory change has occurred within the jurisdiction of that folder (i.e., its subordinates). In this way, the location of the change is quickly narrowed down from the entire directory tree to the subtree of a specific folder, and deep traversal of sibling folders with the same hash value is avoided, thereby reducing unnecessary comparisons.

[0064] Step 204b3: If the hash value of the first target file in the target directory tree is different from the hash value of the first target file under the same path in the first directory tree, it is determined that the first target file has undergone a directory change.

[0065] The hash value of a file is determined based on the hash value of its name and the hash value of its content.

[0066] In the application embodiment, the first target file refers to the file node with inconsistent hash values ​​found during the directory tree hierarchy comparison process. This file exists in the same path in both the target directory tree (server) and the first directory tree (client), but with different hash values. Specifically, the file's hash value is obtained by concatenating the hash value of the file name and the hash value of the file content, and then performing a hash operation. (Refer to...) Figure 4 As shown, the hash value of file File3-1 is Hash3-1 = hash(hashFile3-1 + hashFile3-1 file content).

[0067] In this embodiment, since the calculation rule for file hash value is that the hash value of a file is equal to the hash value of its name and the hash value of its content concatenated and then hashed, when the hash values ​​of the first target files are found to be inconsistent under the same path, one of the following situations can be accurately determined: the file content has changed (modification operation), the file name has changed but the path has not changed (renaming operation), or the file has been deleted and then a new file with the same name has been created in the same path. Since the file is a leaf node in the directory tree (not containing other child items), the change of its hash value directly reflects the change of the file itself, without the need for further tracing downwards. Therefore, in the case of different hash values, the change of the target file itself can be directly determined, so that the specific file that has been changed can be directly located.

[0068] Step 205: Based on the path of a folder or file, match the directory changes on the client side with the directory changes on the server side to obtain different combinations of operations.

[0069] Directory changes represent changes in the status of files and folders caused by corresponding operations within the synchronization interval. These operations include adding, modifying, and deleting files, as well as adding and deleting folders.

[0070] Table 2

[0071] In this embodiment, client-side directory changes refer to the set of file and folder changes occurring locally on the client within the synchronization interval; server-side directory changes refer to the set of file and folder changes occurring on the server within the synchronization interval; matching refers to the process of pairing and analyzing operations performed by the client and server on the same path; operation combination refers to the pairing relationship formed by the client's change status and the server's change status for the same path. Changes on the client and server can be matched one by one according to the path. For each path, the operation status (add, modify, delete, etc.) of both ends is analyzed to form a complete list of operation combinations. For example: path " / docs / a.txt": client status is "modified", server status is "deleted", i.e., operation combination is (client: modify, server: delete); path " / photos / ": client status is "add", server status is "no change", i.e., operation combination is (client: add, server: no change).

[0072] It should be noted that the specific operations (as shown in Table 2) can include changes to files and changes to folders. Changes to files can specifically include creating, modifying, renaming, copying, moving, and deleting, while changes to folders can specifically include creating, renaming, copying, moving, and deleting. Furthermore, merging Table 2 yields Table 3.

[0073] Table 3

[0074] It should be noted that the directory changes resulting from adding, modifying, and deleting files, as well as adding and deleting folders, are shown in Table 4 below, including uploads, downloads, deletions, and creations.

[0075] Table 4

[0076] Step 206: Determine synchronization operation information from the target conflict matrix based on different operation combinations.

[0077] The target conflict matrix defines the processing methods corresponding to different combinations of operations, and includes upload, download, delete, create, or conflict.

[0078] Table 5

[0079] In this embodiment, the target conflict matrix is ​​a predefined decision rule table (as shown in Table 5 above), which specifies the synchronization strategies to be adopted under various operation combinations. The processing method is the synchronization operation determined for a specific operation combination, including uploading, downloading, deleting, creating, or conflicting, etc., where conflict is a situation where the client and server operations on the same path cannot be automatically merged and require special handling; the synchronization operation information is the final generated set of executable synchronization instructions. The various operation combinations obtained in step 205 can be used as input to query the corresponding processing method in the target conflict matrix. This matrix has the client operation status as the row and the server operation status as the column, and each cell defines the processing strategy under a specific combination. By automatically determining the synchronization strategy under various scenarios through predefined rules, this ensures that 1) the same operation combination is processed consistently at different times and on different clients; 2) reduces the need for user decision-making during the synchronization process and reduces manual intervention. In one feasible implementation, when the client "modifies" a file and the server "deletes" it (i.e., the operation combination is (client: modify, server: delete)), the synchronization operation information determined from the target conflict matrix is ​​a conflict.

[0080] It should be noted that when a conflict is detected in the synchronization operation information, a preset conflict resolution strategy must be adopted. This includes overwriting the server-side file with the client's file, overwriting the client-side file with the server's file, or renaming the client-side file. Overwriting the server-side file with the client's file means that when a conflict is detected, the file version on the client will be unconditionally selected as the correct version, and the client's file will be uploaded to the server. The original file version on the server will be overwritten (replaced) by the client's version. This is suitable for individual users or scenarios where the current operating device is clearly prioritized (e.g., a user modifies a document on their personal computer and wants this version to be the final version, ignoring possible modifications on other devices). This ensures that modifications on the user's currently operating device (client) are not lost, and the user's immediate work results are given priority. Overwriting the client-side file with the server's file... The first method, renaming the client's file, means that when a conflict occurs, the server-stored file version is considered the authoritative version, and the latest version of the file is downloaded from the server. The client's local file version is overwritten (replaced) by the server's version. This ensures data consistency, with the server version being the most reliable. This is suitable for scenarios requiring centralized management or where the server-side file version is considered more reliable. The second method, renaming the client's file, means that when a conflict occurs, neither party's modifications are discarded. Instead, one version is renamed to preserve both versions. The server-side file remains unchanged in its original path, and the client's file is automatically renamed (e.g., adding "conflicted," "local," and a timestamp to the filename) and uploaded to the server. Ultimately, the server-side version exists in the original file path, and the renamed client version exists in the same directory. This avoids any data loss and preserves all conflicted versions.

[0081] Step 207: Perform synchronization operation information to the client and server to complete directory synchronization.

[0082] In this embodiment, when performing synchronization operations, specific file system operation rules must be followed to ensure the correctness and reliability of the process. Specifically, this includes: 1) Deleting the second target folder if its subfiles and subfolders have already been deleted; that is, for folder deletion operations, the server can usually only delete empty folders. Therefore, before deleting a second target folder (i.e., the target folder to be deleted), it must be ensured that all its subfiles and subfolders have been deleted first. 2) Creating the second target file if its parent folder has already been created. 3) Creating the third target folder if its parent folder has already been created; that is, for file or folder creation operations, the hierarchical creation principle must be followed. Before creating a second target file (i.e., the target file to be created) or a third target folder (i.e., the target folder to be created), it must be ensured that its parent folder has been successfully created. In one feasible implementation, if a file D: / docs / report.txt (i.e., the second target file) is to be created, it must be ensured that its parent folder D: / docs already exists before the target file can be created. It should be noted that by maintaining the hash value of the directory tree each time changes occur on both the client and server sides, the changes on both the local and server sides can be obtained, reducing the frequency of fetching directories and the comparison time.

[0083] It should be noted that, referring to Figure 5The diagram illustrates a multi-client-server architecture. This architecture clearly defines the roles, functions, and interactions of each component during data synchronization. Specifically, the core components are: client, server, and network. The client represents the terminal device directly used by the user, such as a personal computer, mobile phone, or tablet. The diagram typically shows multiple clients to illustrate the "multi-device synchronization" application scenario. Its core functional modules include: a local file system for storing the user's actual files and folders; a metadata cache for storing and maintaining the directory tree's metadata locally (this cache is the basis for fast hash comparison); and a synchronization engine, the client's core processing logic responsible for executing all method steps in the claims, including initiating synchronization requests, calculating hashes, comparing metadata, handling conflicts, and performing synchronization operations. The server acts as the central hub, responsible for coordinating the synchronization status of all clients and storing authoritative versions of files. Its core functional modules include a business layer and a data layer. The business layer provides basic file / folder operations, maintains the hash of the directory tree, and offers incremental / full directory tree retrieval and change notification capabilities. Specifically, it includes: 1) File / folder operation interfaces: providing five business interfaces for client calls: file upload, file download, file deletion, folder creation, and folder deletion; 2) Directory tree and hash maintenance: when performing file / folder operations, the hash values ​​of all its parent directories need to be updated synchronously; 3) Change notification and incremental retrieval: providing the ability to obtain incremental metadata, returning file and folder metadata that has changed since a specified point in time when a client queries. The data layer provides file storage and file metadata storage capabilities, specifically including: 1) Metadata refers to the ability to store file metadata, typically stored in a database. File metadata refers to attributes other than the binary content, such as filename, path, Message-Digest Algorithm 5 (MD5), and size. 2) File storage refers to the ability to store the binary content of a file, typically using mainstream distributed storage such as object storage and file storage. Without considering clustering and expansion, it can also be stored on the machine's disk. The network acts as a communication bridge between the client and server, enabling data interaction between them. The client sends requests to the server, such as obtaining the hash value of the root folder, obtaining incremental metadata, or uploading or downloading files. The server returns responses to the client, such as returning the requested hash value, a list of incremental metadata, and the file content.The overall interaction process between the client and the server is as follows: The client initiates a synchronization request to the server (business layer) via the network; the server processes the request and queries or updates data from the data layer; the server returns the processing results (such as the hash value of the root folder and incremental metadata) to the client; the client's synchronization engine compares and calculates the returned data with the local metadata (i.e., the first metadata) cache to generate synchronization operation information; based on the synchronization operation information, the client performs specific file upload, download, and other operations to the server via the network, ultimately completing directory synchronization.

[0084] It should be noted that the descriptions of the same steps and contents as in other embodiments in this embodiment can be found in the descriptions in other embodiments, and will not be repeated here.

[0085] The data synchronization method provided in this application can quickly determine whether a directory change has occurred within the synchronization interval by comparing only the hash values ​​of the root folders of the client and server directory trees, instead of requiring the overhead of fully fetching and comparing directory tree metadata as in related technologies. This not only reduces the amount of network data transmitted (only incremental metadata needs to be transmitted instead of the full data) and the consumption of client computing resources, but also improves the comparison efficiency and thus reduces time consumption. At the same time, the hash value-based change detection mechanism can sensitively capture any changes in the directory tree (because the hash value aggregates the information of all lower-level nodes), ensuring the timeliness and accuracy of synchronization.

[0086] Based on the foregoing embodiments, this application provides a data synchronization device that can be applied to... Figure 1 and Figure 3 In the data synchronization method provided in the corresponding embodiment, refer to Figure 6 As shown, the data synchronization device 3 may include: a synchronization request unit 31, an acquisition unit 32, a change determination unit 33, an operation generation unit 34, and a synchronization execution unit 35, wherein, Synchronization request unit 31 is used to initiate a synchronization request after a synchronization interval; wherein, the synchronization interval is the time interval between the completion time of the last synchronization and the start time of the current synchronization. The acquisition unit 32 is used to respond to the synchronization request and acquire the first hash value of the root folder of the first directory tree of the client and the second hash value of the root folder of the second directory tree of the server; wherein, the directory tree includes files and folders at multiple levels; The change determination unit 33 is used to determine the directory changes that occur on the client and server within the synchronization interval based on the first hash value and the second hash value. The operation generation unit 34 is used to generate synchronization operation information based on directory changes; wherein, the synchronization operation information includes operations such as uploading, downloading, and deleting files or folders; Synchronization execution unit 35 is used to execute synchronization operation information to the client and server to complete directory synchronization.

[0087] In other embodiments of this application, the change determination unit 33 is further configured to perform the following steps: If the first hash value and the second hash value are different, retrieve the incremental metadata that has changed since the last synchronization was completed from the server; Based on the first metadata of the first directory tree on the client, the incremental metadata, and the second metadata of the second directory tree on the server, the directory changes that occurred on the client and server during the synchronization interval are determined; wherein, the second directory tree is the server directory tree stored locally on the client at the time of the last synchronization completion.

[0088] In other embodiments of this application, the change determination unit 33 is further configured to perform the following steps: Based on incremental metadata and secondary metadata, the target metadata of the target directory tree of the server at the start of this synchronization is obtained. Based on the first metadata and the target metadata, determine the directory changes that occurred on the client and server during the synchronization interval.

[0089] In other embodiments of this application, the change determination unit 33 is further configured to perform the following steps: Starting from the root folder of the target directory tree, the hash value of each node in the target directory tree is compared with the hash value of the node under the same path in the first directory tree to obtain the second comparison result; the metadata includes at least the hash value; If the second comparison result indicates that the hash value of the first target folder in the target directory tree is different from the hash value of the first target folder under the same path in the first directory tree, it is determined that the subfolders or files of the first target folder have undergone directory changes; wherein, the hash value of the folder is obtained by concatenating the hash value of the folder name and the hash values ​​of all subfolders and subfiles included in the folder; If the second comparison result indicates that the hash value of the first target file in the target directory tree is different from the hash value of the first target file under the same path in the first directory tree, it is determined that the first target file has undergone a directory change; wherein, the hash value of the file is obtained by concatenating the hash value of the file name and the hash value of the file content.

[0090] In other embodiments of this application, the operation generation unit 34 is further configured to perform the following steps: Based on the path of a folder or file, the directory changes on the client side and the directory changes on the server side are matched to obtain different combinations of operations. Among them, the directory change represents the change in the state of files and folders caused by the corresponding operation within the synchronization interval. The operation includes adding, modifying and deleting files, as well as adding and deleting folders. Synchronization operation information is determined from the target conflict matrix based on different operation combinations; the target conflict matrix defines the processing method corresponding to different operation combinations, and includes upload, download, delete, create or conflict.

[0091] In other embodiments of this application, the operation generation unit 34 is further configured to perform the following steps: Overwrite the server's file with the client's file; Overwrite the client's file with the server's file; Rename the client's files.

[0092] In other embodiments of this application, the operation generation unit 34 is further configured to perform the following steps: Delete the second target folder if its subfiles and subfolders have already been deleted; If the parent folder of the second target file has already been created, then create the second target file. Create a new third target folder if the parent folder of the existing third target folder has already been created.

[0093] It should be noted that the specific implementation process of the steps performed by each module in the embodiments of this application can be referred to Figure 1 and Figure 3 The implementation process of the data synchronization method provided in the corresponding embodiment will not be described in detail here.

[0094] The data synchronization device provided in this application can quickly determine whether a directory change has occurred within the synchronization interval by comparing only the hash values ​​of the root folders of the client and server directory trees, instead of requiring the overhead of fully fetching and comparing the directory tree metadata as in related technologies. This not only reduces the amount of network data transmitted (only incremental metadata needs to be transmitted instead of the full data) and the consumption of client computing resources, but also improves the comparison efficiency and thus reduces the time consumption. At the same time, the hash value-based change detection mechanism can sensitively capture any changes in the directory tree (because the hash value aggregates the information of all lower-level nodes), ensuring the timeliness and accuracy of synchronization.

[0095] Based on the foregoing embodiments, embodiments of this application provide a client that can be applied to... Figure 1 and Figure 3 In the data synchronization method provided in the corresponding embodiment, refer to Figure 7 As shown, the client 4 may include: a processor 41, a memory 42, and a communication bus 43, wherein: Communication bus 43 is used to realize the communication connection between processor 41 and memory 42; The processor 41 is used to execute the data synchronization program in the memory 42 to perform the following steps: Initiate a synchronization request after the synchronization interval; where the synchronization interval is the time interval between the completion time of the last synchronization and the start time of the current synchronization. Respond to the synchronization request and obtain the first hash value of the root folder of the first directory tree on the client and the second hash value of the root folder of the second directory tree on the server; wherein, the directory tree includes files and folders at multiple levels; Based on the first hash value and the second hash value, determine whether directory changes have occurred on the client and server sides within the synchronization interval; If a directory change occurs, synchronization operation information is generated; this information includes operations such as uploading, downloading, and deleting files or folders. Perform synchronization operations to send information to both the client and server to complete directory synchronization.

[0096] In other embodiments of this application, the processor 41 is configured to execute a data synchronization program in the memory 42 to determine, based on a first hash value and a second hash value, whether a directory change has occurred on the client and server within the synchronization interval, in order to implement the following steps: If the first hash value and the second hash value are different, retrieve the incremental metadata that has changed since the last synchronization was completed from the server; Based on the first metadata of the first directory tree on the client, the incremental metadata, and the second metadata of the second directory tree on the server, the directory changes that occurred on the client and server during the synchronization interval are determined; wherein, the second directory tree is the server directory tree stored locally on the client at the time of the last synchronization completion.

[0097] In other embodiments of this application, processor 41 is used to execute the data synchronization program in memory 42 based on the first metadata of the first directory tree of the client, the incremental metadata, and the second metadata of the second directory tree of the server, to determine the directory changes that occur on the client and server during the synchronization interval, in order to implement the following steps: Based on incremental metadata and secondary metadata, the target metadata of the target directory tree of the server at the start of this synchronization is obtained. Based on the first metadata and the target metadata, determine the directory changes that occurred on the client and server during the synchronization interval.

[0098] In other embodiments of this application, processor 41 is configured to execute a data synchronization program in memory 42 based on first metadata and target metadata to determine directory changes that occur on the client and server during the synchronization interval, in order to implement the following steps: Starting from the root folder of the target directory tree, the hash value of each node in the target directory tree is compared with the hash value of the node under the same path in the first directory tree to obtain the second comparison result; the metadata includes at least the hash value; If the second comparison result indicates that the hash value of the first target folder in the target directory tree is different from the hash value of the first target folder under the same path in the first directory tree, it is determined that the subfolders or files of the first target folder have undergone directory changes; wherein, the hash value of the folder is obtained by concatenating the hash value of the folder name and the hash values ​​of all subfolders and subfiles included in the folder; If the second comparison result indicates that the hash value of the first target file in the target directory tree is different from the hash value of the first target file under the same path in the first directory tree, it is determined that the first target file has undergone a directory change; wherein, the hash value of the file is obtained by concatenating the hash value of the file name and the hash value of the file content.

[0099] In other embodiments of this application, the processor 41 is used to execute the data synchronization program in the memory 42 to generate synchronization operation information if a directory change occurs, in order to implement the following steps: Based on the path of a folder or file, the directory changes on the client side and the directory changes on the server side are matched to obtain different combinations of operations. Among them, the directory change represents the change in the state of files and folders caused by the corresponding operation within the synchronization interval. The operation includes adding, modifying and deleting files, as well as adding and deleting folders. Synchronization operation information is determined from the target conflict matrix based on different operation combinations; the target conflict matrix defines the processing method corresponding to different operation combinations, and includes upload, download, delete, create or conflict.

[0100] In other embodiments of this application, the processor 41's handling of conflicts in the data synchronization program in the memory 42 includes at least one of the following: Overwrite the server's file with the client's file; Overwrite the client's file with the server's file; Rename the client's files.

[0101] In other embodiments of this application, the processor 41 is used to execute the data synchronization program in the memory 42 to determine synchronization operation information based on target change information and target conflict matrix, in order to implement the following steps: Delete the second target folder if its subfiles and subfolders have already been deleted; If the parent folder of the second target file has already been created, then create the second target file. Create a new third target folder if the parent folder of the existing third target folder has already been created.

[0102] It should be noted that a detailed description of the steps performed by the processor can be found in [reference needed]. Figure 1 and Figure 3 The implementation process of the data synchronization method provided in the corresponding embodiment will not be described in detail here.

[0103] The client provided in this application embodiment can quickly determine whether a directory change has occurred within the synchronization interval by comparing only the hash values ​​of the root folders of the client and server directory trees, instead of requiring the overhead of fully fetching and comparing directory tree metadata as in related technologies. This not only reduces the amount of network data transmitted (only incremental metadata needs to be transmitted instead of the full data) and the consumption of client computing resources, but also improves the comparison efficiency and thus reduces the time consumption. At the same time, the hash value-based change detection mechanism can sensitively capture any changes in the directory tree (because the hash value aggregates the information of all lower-level nodes), ensuring the timeliness and accuracy of synchronization.

[0104] Based on the foregoing embodiments, this application provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to achieve... Figure 1 and Figure 3 The steps in the data synchronization method provided in the corresponding embodiment.

[0105] Based on the foregoing embodiments, this application provides a computer program product, including a computer program, which, when executed by a processor, implements... Figure 1 and Figure 3 The steps in the data synchronization method provided in the corresponding embodiment.

[0106] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A method of data synchronization, the method comprising: The method includes: A synchronization request is initiated after a synchronization interval; wherein, the synchronization interval is the time interval between the completion time of the previous synchronization and the start time of the current synchronization. In response to the synchronization request, the system obtains the first hash value of the root folder of the first directory tree on the client and the second hash value of the root folder of the second directory tree on the server; wherein the directory tree includes files and folders at multiple levels; Based on the first hash value and the second hash value, determine the directory changes that occurred on the client and server within the synchronization interval; If the directory change occurs, synchronization operation information is generated; wherein, the synchronization operation information includes operations such as uploading, downloading, and deleting files or folders; The synchronization operation information is sent to the client and server to complete the directory synchronization; The step of determining the directory changes that occurred on the client and server within the synchronization interval based on the first hash value and the second hash value includes: If the first hash value and the second hash value are different, obtain the incremental metadata that has changed since the last synchronization completion time from the server; Based on the first metadata of the first directory tree of the client, the incremental metadata, and the second metadata of the second directory tree of the server, the directory changes that occurred on the client and the server within the synchronization interval are determined; wherein, the second directory tree is the server directory tree stored locally on the client at the time of the last synchronization completion; If the directory change occurs, the generation of synchronization operation information includes: Based on the path of a folder or file, the directory changes on the client and the directory changes on the server are matched to obtain different combinations of operations; wherein, the directory change represents the change in the state of files and folders caused by the corresponding operation within the synchronization interval, and the operation includes adding, modifying and deleting files, as well as adding and deleting folders; The synchronization operation information is determined from the target conflict matrix based on the different operation combinations; wherein, the target conflict matrix defines the processing method corresponding to the different operation combinations, and the target conflict matrix includes upload, download, delete, create, or conflict.

2. The method of claim 1, wherein, The determination of directory changes occurring on the client and server within the synchronization interval, based on the first metadata of the first directory tree of the client, the incremental metadata, and the second metadata of the second directory tree of the server, includes: Based on the incremental metadata and the second metadata, the target metadata of the target directory tree of the server at the start time of this synchronization is obtained; Based on the first metadata and the target metadata, determine the directory changes that occurred on the client and the server within the synchronization interval.

3. The method of claim 2, wherein, The step of determining the directory changes that occurred on the client and the server within the synchronization interval based on the first metadata and the target metadata includes: Starting from the root folder of the target directory tree, the hash value of each node in the target directory tree is compared with the hash value of the node under the same path in the first directory tree to obtain a second comparison result; the metadata includes at least the hash value; If the second comparison result indicates that the hash value of the first target folder in the target directory tree is different from the hash value of the first target folder under the same path in the first directory tree, it is determined that the subfolders or files of the first target folder have undergone directory changes; wherein, the hash value of the folder is obtained by concatenating the hash value of the folder name and the hash values ​​of all subfolders and subfiles included in the folder; If the second comparison result indicates that the hash value of the first target file in the target directory tree is different from the hash value of the first target file under the same path in the first directory tree, it is determined that the first target file has undergone a directory change; wherein, the hash value of the file is obtained by concatenating the hash value of the file name and the hash value of the file content.

4. The method of claim 1, wherein, The methods for handling the aforementioned conflict include at least one of the following: The client's file overwrites the server's file; The server's file overwrites the client's file; Rename the files on the client.

5. The method of claim 1, wherein, The method further includes: If the subfiles and subfolders of the second target folder have already been deleted, then delete the second target folder. If the parent folder of the second target file has already been created, then create the second target file. If the parent folder of the newly created third target folder has already been created, then create the third target folder.

6. A client, characterized in that, The client includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the method according to any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the method as described in any one of claims 1-5.

8. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-5.

Images