Remote file rapid synchronization method and device

By determining the existence of the file's hash value and identifier in the database of the remote device, recording the file information and creating a hard link, the problems of slow file transfer and waste of storage space in the cloud machine environment are solved, and fast synchronization and storage savings are achieved.

CN120602476BActive Publication Date: 2025-10-03HAIMA CLOUD TIANJIN INFORMATION TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202511086867.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-08-05
Publication Date
2025-10-03
Estimated Expiration
2045-08-05

AI Technical Summary

Technical Problem

In a cloud machine environment, existing technologies cannot avoid duplicate storage of the same files on the target end during remote file transfer, resulting in slow transmission speeds and wasted storage space.

Method used

By determining whether the hash value, file access mode, owner identifier and group identifier of the file to be transferred exist in the database of the second device, the hash value, file access mode, owner identifier and group identifier are recorded in the first file or the second file, and a hard link is created for the file that does not exist in the second device to avoid repeated transmission.

Benefits of technology

The file transfer speed is improved, storage space is saved, and storage usage is reduced by sending files that do not exist in the second device in batches and building hard links in the second device.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120602476B_ABST
    Figure CN120602476B_ABST
Patent Text Reader

Abstract

The present application provides a method and device for rapid remote file synchronization, the method comprising: receiving information of files to be transmitted sent by a first device; for each information of the file to be transmitted, determining whether the hash value, file access mode, owner identifier and group identifier in the information of the file to be transmitted are respectively the same as the hash value, file access mode, owner identifier and group identifier of a piece of data in a database of a second device; if they are the same, recording part or all of the information of the file to be transmitted in a first file; otherwise, recording part or all of the information of the file to be transmitted in a second file; sending the first file and / or the second file to the first device; for each information of the file to be transmitted in the first file, if the paths of the third file and the fourth file are different, creating a hard link; and / or receiving and storing the files to be transmitted sent by the first device, which can improve the file transmission speed and save storage space.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to a method and device for rapid remote file synchronization. Background Art

[0002] In a cloud machine usage environment, it is often necessary to transfer files between cloud machines, such as game installation packages, game hot update packages, and container images. Currently, when transferring files remotely, it is necessary to determine whether the target file exists at the target transmission path in the target end. If it exists, by comparing the modification time and file size of the source file and the target file, or the file hash value, it is determined whether the target file is the same as the source file, and then it is decided whether to transfer the source file to overwrite the target file; if the target file does not exist, the source file needs to be transferred to the target transmission path. Since this solution compares the source file with the target file at the target transmission path, it cannot be compared with files at other paths. This will cause the same file to be stored repeatedly at the target end, resulting in slow file transfer speeds and waste of storage space. The applicant's description in the background technology section does not constitute a recognition of the prior art.

[0003] Therefore, how to provide a file transfer solution to improve file transfer speed and save storage space has become a technical problem that needs to be solved urgently. Summary of the Invention

[0004] In order to solve the technical problems existing in the prior art, the embodiments of the present application provide a method and device for rapid remote file synchronization.

[0005] In a first aspect, an embodiment of the present application provides a remote file fast synchronization method, applied to a second device, comprising:

[0006] Receiving file information to be transferred sent by the first device, wherein the file information to be transferred includes information of all files to be transferred, and the information of each file to be transferred includes a hash value, a file access mode, an owner identifier, and a group identifier of the file to be transferred;

[0007] For each piece of information of a file to be transferred in the file information to be transferred, determine whether a hash value, a file access mode, an owner identifier, and a group identifier in the information of the file to be transferred are respectively identical to a hash value, a file access mode, an owner identifier, and a group identifier of a piece of data in a database of the second device; if they are identical, record part or all of the information of the file to be transferred in the first file; otherwise, record part or all of the information of the file to be transferred in the second file, wherein the database stores information of multiple files in the second device, each piece of data in the database is information of one of the multiple files, and for each file in the multiple files, the file information includes the hash value, file access mode, owner identifier, and group identifier of the file;

[0008] sending the first file and / or the second file to the first device;

[0009] For the information of each file to be transferred in the first file, if the paths of the third file and the fourth file are different, a hard link is created for the third file and the fourth file so that the third file and the fourth file share the same physical data, wherein the path of the fourth file is the target transmission path of the file to be transferred corresponding to the information in the second device, and the hash value, file access mode, owner identifier and group identifier of the third file are respectively the same as the hash value, file access mode, owner identifier and group identifier of the file to be transferred corresponding to the information; and / or all files to be transferred corresponding to the second file sent by the first device are received and stored.

[0010] In a second aspect, an embodiment of the present application further provides a remote file fast synchronization device, applied to a second device, comprising:

[0011] a receiving unit, configured to receive information about files to be transferred sent by the first device, wherein the information about files to be transferred includes information about all files to be transferred, and the information about each file to be transferred includes a hash value, a file access mode, an owner identifier, and a group identifier of the file to be transferred;

[0012] a recording unit, configured to determine, for each piece of information about a file to be transferred in the file information to be transferred, whether a hash value, a file access mode, an owner identifier, and a group identifier in the information about the file to be transferred are respectively identical to a hash value, a file access mode, an owner identifier, and a group identifier of a piece of data in a database of the second device; if so, recording part or all of the information about the file to be transferred in the first file; otherwise, recording part or all of the information about the file to be transferred in the second file, wherein the database stores information about a plurality of files in the second device, each piece of data in the database is information about one of the plurality of files, and for each of the plurality of files, the file information includes the hash value, file access mode, owner identifier, and group identifier of the file;

[0013] a sending unit, configured to send the first file and / or the second file to the first device;

[0014] A processing unit is used to create a hard link for the third file and the fourth file for the information of each file to be transferred in the first file, if the paths of the third file and the fourth file are different, so that the third file and the fourth file share the same physical data, wherein the path of the fourth file is the target transmission path of the file to be transferred corresponding to the information in the second device, and the hash value, file access mode, owner identifier and group identifier of the third file are respectively the same as the hash value, file access mode, owner identifier and group identifier of the file to be transferred corresponding to the information; and / or receive and store all files to be transferred corresponding to the second file sent by the first device.

[0015] In summary, the remote file rapid synchronization method and device provided by the embodiments of the present application determine whether the hash value, file access mode, owner identifier and group identifier of the file to be transferred sent by the first device exist in the database of the second device. When the hash value, file access mode, owner identifier and group identifier of the file to be transferred exist in the database of the second device, it is determined that the corresponding file to be transferred exists in the second device, and the information of the file to be transferred existing in the second device is recorded in the first file, and the information of the file to be transferred not existing in the second device is recorded in the second file. The first file and / or the second file are sent to the first device so that the first device sends the files to be transferred that do not exist in the second device to the second device in batches. For the files to be transferred existing in the second device, the first device no longer sends these files to the second device, but the second device constructs corresponding hard links so that each of these files shares the same physical data with the corresponding files existing in the second device. In this way, the files to be transferred that do not exist in the second device can be quickly determined and transferred to the second device. The hard links occupy less storage space, thereby improving the file transfer speed and saving the storage space of the second device. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flowchart of an embodiment of a remote file rapid synchronization method provided by the present application;

[0017] Figure 2 A schematic structural diagram of an embodiment of a remote file rapid synchronization device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of illustration and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowcharts can be implemented out of sequence, and steps without logical context can be reversed or implemented simultaneously. In addition, those skilled in the art, under the guidance of the contents of this application, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0019] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0020] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0021] Reference Figure 1 As shown, an embodiment of the present application provides a flowchart of a method for rapid remote file synchronization, which is applied to a second device and includes:

[0022] S10. Receive information about files to be transferred sent by the first device, wherein the information about files to be transferred includes information about all files to be transferred, and the information about each file to be transferred includes a hash value, a file access mode, an owner identifier, and a group identifier of the file to be transferred;

[0023] In this embodiment, it should be noted that when synchronizing files to be transferred, the first device synchronizes them in folders. When synchronizing files with a second device, the first device may first send information about all files in at least one folder to the second device. The information about all files in the at least one folder is the information about the files to be transferred. When sending the information about the files to be transferred, the first device may notify the second device that file synchronization is about to begin and proactively send the information to the second device via a file transfer protocol (such as rsync). Alternatively, the first device may first notify the second device that file synchronization is about to begin, after which the second device downloads the information from the first device via a file transfer protocol (such as rsync). The information about each file to be transferred includes a hash value, a file access mode (FileAccessMode), an owner identifier (FileUid), and a group identifier (FileGid). The first and second devices may be terminal devices or cloud hosts.

[0024] S11. For each piece of information about a file to be transferred in the file information to be transferred, determine whether a hash value, a file access mode, an owner identifier, and a group identifier in the information about the file to be transferred are respectively identical to a hash value, a file access mode, an owner identifier, and a group identifier of a piece of data in a database of the second device; if so, record part or all of the information about the file to be transferred in the first file; otherwise, record part or all of the information about the file to be transferred in the second file, wherein the database stores information about a plurality of files in the second device, each piece of data in the database is information about one of the plurality of files, and for each of the plurality of files, the file information includes the hash value, file access mode, owner identifier, and group identifier of the file;

[0025] In this embodiment, it should be noted that, for each piece of information of the file to be transmitted in the file information to be transmitted, if the hash value, file access mode, owner identifier and group identifier in the information of the file to be transmitted are respectively the same as the hash value, file access mode, owner identifier and group identifier of a piece of data in the database of the second device, it means that the hash value, file access mode, owner identifier and group identifier in the information of the file to be transmitted exist in the database of the second device, and then the file to be transmitted exists in the second device. At this time, part or all of the information in the information of the file to be transmitted is recorded in the first file; and if there is no data in the database of the second device, and the hash value, file access mode, owner identifier and group identifier of the data are respectively the same as the hash value, file access mode, owner identifier and group identifier in the information of the file to be transmitted, it means that the hash value, file access mode, owner identifier and group identifier in the information of the file to be transmitted do not exist in the database of the second device, and then the file to be transmitted does not exist in the second device. At this time, part or all of the information in the information of the file to be transmitted is recorded in the second file.

[0026] S12. Send the first file and / or the second file to the first device;

[0027] In this embodiment, it should be noted that after sending the first file and / or the second file to the first device, the first device may, based on the first file or the second file, send to the second device any to-be-transmitted files that do not exist on the second device (if at least one to-be-transmitted file does not exist on the second device) or not send any to-be-transmitted files to the second device (if all to-be-transmitted files exist on the second device). For example, if the second device sends the first file to the first device, the first device may send to the second device all to-be-transmitted files except the file corresponding to the information in the first file (if all to-be-transmitted files except the file corresponding to the information in the first file exist) or not send any to-be-transmitted files to the second device (if all to-be-transmitted files except the file corresponding to the information in the first file do not exist). For another example, if the second device sends the first device the second file, the first device may directly send to the second device the file corresponding to the information in the second file (if the second file contains information about the file to be transferred) or not send any to-be-transmitted files to the second device (if the second file does not contain information about any file to be transferred). In addition, to increase file transfer speed, the first device may perform batch transfers when transmitting files to be transferred.

[0028] S13. For the information of each file to be transferred in the first file, if the paths of the third file and the fourth file are different, create a hard link for the third file and the fourth file so that the third file and the fourth file share the same physical data, wherein the path of the fourth file is the target transmission path of the file to be transferred corresponding to the information in the second device, and the hash value, file access mode, owner identifier and group identifier of the third file are respectively the same as the hash value, file access mode, owner identifier and group identifier of the file to be transferred corresponding to the information; and / or receive and store all files to be transferred corresponding to the second file sent by the first device.

[0029] In this embodiment, it should be noted that if the paths of the third file and the fourth file are the same, it can be decided whether to update the information of the third file based on the information of the file to be transferred corresponding to the information; if the paths of the third file and the fourth file are different, it is necessary to create a hard link for the third file and the fourth file so that the physical data of the fourth file on the disk is the physical data of the third file on the disk.

[0030] For example, suppose that the first device wants to synchronize all files under the folder AAA to the folder BBB in the second device. The first file contains information about file X1, file X2, and file X3. The second device contains files Y1, Y2, Y3, and Y4. File Y1 is located in the folder BBB. The relative path of file Y1 relative to the folder BBB is the same as the relative path of file X1 relative to the folder AAA (that is, the path of file Y1 is the target transmission path of file X1 in the second device). The hash value, file access mode, owner ID, and group ID of file Y1 are respectively the same as the hash value, file access mode, owner ID, and group ID of file X1. The file Y2 is not located in the folder BBB. The hash value, file access mode, owner ID, and group ID of file Y2 are respectively the same as the hash value, file access mode, owner ID, and group ID of file X2. There is no file in the folder BBB whose relative path relative to folder BBB is the same as the relative path of file X2 relative to folder AAA. File Y3 is located in the folder BBB. The relative path of file Y3 relative to folder BBB is the same as the relative path of file X3 relative to folder AAA (that is, the path of file Y3 is the target transfer path of file X3 in the second device). The hash value of file Y3 is the same as the hash value of file X3. Different, Y4 is not located in the folder BBB, the hash value, file access mode, owner identifier and group identifier of file Y4 are respectively the same as the hash value, file access mode, owner identifier and group identifier of file X3, then for file X1, it can be decided whether to update the information of file Y1 in the database based on the information of file X1. For example, if the information of file X1 includes the last modification time of the file, and the information of the file stored in the database of the second device includes the last modification time of the file, then when the last modification time of file X1 is later than the last modification time of file Y1 in the database, the last modification time of file Y1 in the database is updated to the last modification time of file X1, or If the last modification time of X1 is earlier than the last modification time of file Y1 in the database, the last modification time of file Y1 in the database is not updated. For file X2, file Y22 needs to be created, and a hard link is created between file Y2 (i.e., the third file) and file Y22 so that files Y2 and Y22 share the same physical data (i.e., the physical data of file Y22 on the disk is the physical data of file Y2 on the disk). File Y22 is located in folder BBB, and the relative path of file Y22 relative to folder BBB is the same as the relative path of file X2 relative to folder AAA (i.e., the path of file Y22 is the target transfer path of file X2 on the second device).For file X3, you need to delete file Y3, recreate file Y3 at the original path, and create a hard link between file Y3 and file Y4 (the third file) so that files Y3 and Y4 share the same physical data (that is, the physical data of file Y3 on the disk is the same as the physical data of file Y4 on the disk).

[0031] The remote file rapid synchronization method provided by the embodiment of the present application determines whether the hash value, file access mode, owner identifier and group identifier of the file to be transferred sent by the first device exist in the database of the second device. When the hash value, file access mode, owner identifier and group identifier of the file to be transferred exist in the database of the second device, it is determined that the corresponding file to be transferred exists in the second device, and the information of the file to be transferred existing in the second device is recorded in the first file, and the information of the file to be transferred not existing in the second device is recorded in the second file. The first file and / or the second file are sent to the first device so that the first device sends the files to be transferred that do not exist in the second device to the second device in batches. For the files to be transferred existing in the second device, the first device no longer sends these files to the second device, but the second device constructs corresponding hard links so that each of these files shares the same physical data with the corresponding files existing in the second device. In this way, the files to be transferred that do not exist in the second device can be quickly determined and transferred to the second device. The hard links occupy less storage space, thereby improving the file transfer speed and saving storage space of the second device.

[0032] Based on the foregoing method embodiment, before sending the first file and / or the second file to the first device, the method may further include:

[0033] Calculating the total size of all to-be-transferred files corresponding to the second file, and using the total size as the storage space size that needs to be reserved on the second device;

[0034] Obtaining the remaining storage space of the second device, and determining whether the storage space of the second device is sufficient based on the remaining storage space of the second device and the storage space required to be reserved by the second device;

[0035] The sending the first file and / or the second file to the first device may include:

[0036] When the storage space of the second device is sufficient, the first file and / or the second file is sent to the first device.

[0037] In this embodiment, it should be noted that before sending the first file and / or the second file to the first device, the second device may also evaluate whether the second device has sufficient storage space based on the sizes of all files to be transferred corresponding to the second file. The first file and / or the second file will only be sent to the first device if the second device has sufficient storage space. If the second device does not have sufficient storage space, the second device may send a notification to the first device, instructing the first device to terminate this file synchronization. The criteria for determining whether the second device has sufficient storage space can be set as needed. For example, if the amount of storage space required to be reserved on the second device is greater than the product of the remaining storage space on the second device and a preset value (the preset value is greater than 0 and less than 1 and can be set as needed, such as 0.7), the second device is considered to have insufficient storage space. Otherwise, the second device is considered to have insufficient storage space.

[0038] Based on the above method embodiment, the method may further include:

[0039] storing information of the fourth file in a database; and / or

[0040] The information of the file to be transmitted received and stored from the first device is stored in a database.

[0041] In this embodiment, it should be noted that, for the file to be transferred that exists in the second device, it is necessary to determine whether the relevant information of the file to be transferred needs to be stored in the database according to the data storage field in the database according to the situation. Specifically, if there is a file at the target transmission path of the file to be transferred in the second device, and the hash value, file access mode, owner identifier and group identifier of the file to be transferred are the same as those of the file to be transferred, then it is necessary to decide whether to update the information of the file in the database based on the information of the file to be transferred. The specific processing process can refer to the processing process involved in the file X1 in the aforementioned example; if there is no file at the target transmission path of the file to be transferred in the second device, or there is a file at the target transmission path of the file to be transferred in the second device, and the hash value, file access mode, owner identifier and group identifier of the file to be transferred are not exactly the same as those of the file to be transferred, then the information of the fourth file corresponding to the file to be transferred needs to be stored in the database. For example, for the file X2 in the aforementioned example, the information of the file Y22 needs to be stored in the database. For another example, for the file X3 in the aforementioned example, the information of the original file Y3 in the database needs to be deleted, and the information of the re-created file Y3 needs to be stored in the database. For files to be transferred that do not exist on the second device, the information about the files to be transferred needs to be stored in the database after the second device receives and stores the files to be transferred. Furthermore, it should be noted that if a file or files and / or folders on the second device need to be deleted, in addition to deleting the file or folders, the data corresponding to the file or folders in the database also needs to be deleted.

[0042] Based on the aforementioned method embodiment, the information of each file to be transferred may further include a relative path of the file to be transferred. For each of the multiple files, the information of the file may further include the folder location corresponding to the file and the relative path of the file.

[0043] In this embodiment, it should be noted that the information of each file to be transferred may also include a relative path of the file to be transferred. This relative path is the relative path of the file to be transferred relative to the folder to be transferred by the first device. For example, in the aforementioned example, the information of file X1 may also include the relative path of file X1 relative to folder AAA. For each of the multiple files, the file information may also include the file's corresponding folder location and the file's relative path. The file's corresponding folder location is the file's corresponding folder path, which can be specified as needed, and the file's relative path is the file's relative path relative to the file's corresponding folder. In the aforementioned example, for file Y22, the hash value, file access mode, owner identifier, group identifier, path of folder BBB (i.e., the location of folder BBB), and the relative path of file Y22 relative to folder BBB may be stored in the database. In addition, the information of each file to be transferred and the information of each file stored in the database may also include other information, such as the last modification time and size of the file to be transferred.

[0044] Reference Figure 2 FIG. 1 is a schematic diagram of a structure of a remote file rapid synchronization device provided by an embodiment of the present application. The device is applied to a second device and includes:

[0045] The receiving unit 20 is configured to receive the file information to be transferred sent by the first device, wherein the file information to be transferred includes information of all files to be transferred, and the information of each file to be transferred includes a hash value, a file access mode, an owner identifier, and a group identifier of the file to be transferred;

[0046] a recording unit 21 for determining, for each piece of information about a file to be transferred in the file information to be transferred, whether a hash value, a file access mode, an owner identifier, and a group identifier in the information about the file to be transferred are respectively identical to a hash value, a file access mode, an owner identifier, and a group identifier of a piece of data in a database of the second device; if so, recording part or all of the information about the file to be transferred in the first file; otherwise, recording part or all of the information about the file to be transferred in the second file, wherein the database stores information about a plurality of files in the second device, each piece of data in the database is information about one of the plurality of files, and for each of the plurality of files, the file information includes the hash value, file access mode, owner identifier, and group identifier of the file;

[0047] A sending unit 22, configured to send the first file and / or the second file to the first device;

[0048] The processing unit 23 is used to create a hard link for the third file and the fourth file for the information of each file to be transferred in the first file, if the paths of the third file and the fourth file are different, so that the third file and the fourth file share the same physical data, wherein the path of the fourth file is the target transmission path of the file to be transferred corresponding to the information in the second device, and the hash value, file access mode, owner identifier and group identifier of the third file are respectively the same as the hash value, file access mode, owner identifier and group identifier of the file to be transferred corresponding to the information; and / or receive and store all files to be transferred corresponding to the second file sent by the first device.

[0049] The remote file fast synchronization device provided in the embodiment of the present application determines whether the hash value, file access mode, owner identifier and group identifier of the file to be transferred sent by the first device exist in the database of the second device. When the hash value, file access mode, owner identifier and group identifier of the file to be transferred exist in the database of the second device, it determines that the corresponding file to be transferred exists in the second device, and records the information of the file to be transferred existing in the second device in the first file, and records the information of the file to be transferred not existing in the second device in the second file. The first file and / or the second file are sent to the first device so that the first device sends the files to be transferred that do not exist in the second device to the second device in batches. For the files to be transferred existing in the second device, the first device no longer sends these files to the second device, but the second device constructs corresponding hard links so that each of these files shares the same physical data with the corresponding files existing in the second device. In this way, the files to be transferred that do not exist in the second device can be quickly determined and transferred to the second device. The hard links occupy less storage space, thereby improving the file transfer speed and saving storage space of the second device.

[0050] Based on the aforementioned device embodiment, the sending unit may be specifically configured to:

[0051] Calculating the total size of all to-be-transferred files corresponding to the second file, and using the total size as the storage space size that needs to be reserved on the second device;

[0052] Obtaining the remaining storage space of the second device, and determining whether the storage space of the second device is sufficient based on the remaining storage space of the second device and the storage space required to be reserved by the second device;

[0053] When the storage space of the second device is sufficient, the first file and / or the second file is sent to the first device.

[0054] Based on the aforementioned device embodiment, the processing unit may also be used for:

[0055] storing information of the fourth file in a database; and / or

[0056] The information of the file to be transmitted received and stored from the first device is stored in a database.

[0057] Based on the aforementioned device embodiment, the information of each file to be transferred may further include a relative path of the file to be transferred. For each of the multiple files, the information of the file may further include the folder location corresponding to the file and the relative path of the file.

[0058] The remote file fast synchronization device provided in the embodiment of the present application has the same implementation process as the remote file fast synchronization method provided in the embodiment of the present application, and the effect that can be achieved is also the same as the remote file fast synchronization method provided in the embodiment of the present application, which will not be repeated here.

[0059] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A remote file rapid synchronization method, applied to a second device, characterized in that: include: Receiving file information to be transferred sent by the first device, wherein the file information to be transferred includes information of all files to be transferred, and the information of each file to be transferred includes a hash value, a file access mode, an owner identifier, and a group identifier of the file to be transferred; For each piece of information of a file to be transferred in the file information to be transferred, determine whether a hash value, a file access mode, an owner identifier, and a group identifier in the information of the file to be transferred are respectively identical to a hash value, a file access mode, an owner identifier, and a group identifier of a piece of data in a database of the second device; if they are identical, record part or all of the information of the file to be transferred in the first file; otherwise, record part or all of the information of the file to be transferred in the second file, wherein the database stores information of multiple files in the second device, each piece of data in the database is information of one of the multiple files, and for each file in the multiple files, the file information includes the hash value, file access mode, owner identifier, and group identifier of the file; sending the first file and / or the second file to the first device; For the information of each file to be transferred in the first file, if the paths of the third file and the fourth file are different, a hard link is created for the third file and the fourth file so that the third file and the fourth file share the same physical data, wherein the path of the fourth file is the target transmission path of the file to be transferred corresponding to the information in the second device, and the hash value, file access mode, owner identifier and group identifier of the third file are respectively the same as the hash value, file access mode, owner identifier and group identifier of the file to be transferred corresponding to the information; and / or all files to be transferred corresponding to the second file sent by the first device are received and stored.

2. The method according to claim 1, wherein Before sending the first file and / or the second file to the first device, the method further includes: Calculating the total size of all to-be-transferred files corresponding to the second file, and using the total size as the storage space size that needs to be reserved on the second device; Obtaining the remaining storage space of the second device, and determining whether the storage space of the second device is sufficient based on the remaining storage space of the second device and the storage space required to be reserved by the second device; The sending of the first file and / or the second file to the first device includes: When the storage space of the second device is sufficient, the first file and / or the second file is sent to the first device.

3. The method according to claim 1 or 2, wherein: Also includes: storing information of the fourth file in a database; and / or The information of the file to be transmitted received and stored from the first device is stored in a database.

4. The method according to claim 1, wherein The information of each file to be transferred further includes a relative path of the file to be transferred. For each of the multiple files, the information of the file further includes the folder location corresponding to the file and the relative path of the file.

5. A remote file rapid synchronization device, applied to a second device, characterized in that: include: a receiving unit, configured to receive information about files to be transferred sent by the first device, wherein the information about files to be transferred includes information about all files to be transferred, and the information about each file to be transferred includes a hash value, a file access mode, an owner identifier, and a group identifier of the file to be transferred; a recording unit, configured to determine, for each piece of information about a file to be transferred in the file information to be transferred, whether a hash value, a file access mode, an owner identifier, and a group identifier in the information about the file to be transferred are respectively identical to a hash value, a file access mode, an owner identifier, and a group identifier of a piece of data in a database of the second device; if so, recording part or all of the information about the file to be transferred in the first file; otherwise, recording part or all of the information about the file to be transferred in the second file, wherein the database stores information about a plurality of files in the second device, each piece of data in the database is information about one of the plurality of files, and for each of the plurality of files, the file information includes the hash value, file access mode, owner identifier, and group identifier of the file; a sending unit, configured to send the first file and / or the second file to the first device; A processing unit is used to create a hard link for the third file and the fourth file for the information of each file to be transferred in the first file, if the paths of the third file and the fourth file are different, so that the third file and the fourth file share the same physical data, wherein the path of the fourth file is the target transmission path of the file to be transferred corresponding to the information in the second device, and the hash value, file access mode, owner identifier and group identifier of the third file are respectively the same as the hash value, file access mode, owner identifier and group identifier of the file to be transferred corresponding to the information; and / or receive and store all files to be transferred corresponding to the second file sent by the first device.

6. The device according to claim 5, characterized in that The sending unit is specifically configured to: Calculating the total size of all to-be-transferred files corresponding to the second file, and using the total size as the storage space size that needs to be reserved on the second device; Obtaining the remaining storage space of the second device, and determining whether the storage space of the second device is sufficient based on the remaining storage space of the second device and the storage space required to be reserved by the second device; When the storage space of the second device is sufficient, the first file and / or the second file is sent to the first device.

7. The device according to claim 5 or 6, characterized in that The processing unit is further configured to: storing information of the fourth file in a database; and / or The information of the file to be transmitted received and stored from the first device is stored in a database.

8. The device according to claim 5, wherein The information of each file to be transferred further includes a relative path of the file to be transferred. For each of the multiple files, the information of the file further includes the folder location corresponding to the file and the relative path of the file.

Citation Information

Patent Citations

  • File storage method and device, electronic equipment and storage medium

    CN113259494A

  • Cloud storage data synchronization method and device and storage medium

    CN119046377A