Data synchronization method of a distributed file system and related components

By using an inode-based method to synchronize file content and metadata information in a distributed file system, the problem of manually configuring file attributes after asynchronous replication is solved, achieving automated file attribute setting and improving the reliability and efficiency of data replication.

CN116340279BActive Publication Date: 2026-01-30JINAN INSPUR DATA TECH CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202310314866.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-03-24
Publication Date
2026-01-30
Estimated Expiration
2043-03-24

AI Technical Summary

Technical Problem

In existing technologies, asynchronous replication in distributed file systems requires manual configuration of file attributes after file content is copied, which leads to low efficiency and may cause permission confusion, making it impossible to guarantee the reliability and efficiency of data replication.

Method used

The file content and metadata information are packaged using inodes as the unit, and the file content and metadata information are automatically synchronized to the slave end, including time attributes, mode attributes, user and group attributes, etc. The packaging form of the file copy package is optimized to include extended attributes, and write protection requests are sent to ensure that the master file is not changed.

Benefits of technology

It enables the replication of files without the need for manual configuration of file attributes, improving the reliability and efficiency of data replication and ensuring file permission consistency and synchronization accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116340279B_ABST
    Figure CN116340279B_ABST
Patent Text Reader

Abstract

This invention discloses a data synchronization method and related components for a distributed file system, relating to the field of data replication. The method includes acquiring a file replication packet sent by the master end, wherein the file replication packet includes the file content and metadata information of files under each inode of the distributed file system. Then, each file content is copied to a corresponding copy file on the slave end. Furthermore, the basic attributes of the copied files are set to the metadata information corresponding to the file content stored within the copied files. Therefore, this application, when synchronizing files from the master end to the slave end, not only copies the file content but also the file attributes, i.e., the metadata information. Thus, there is no need to manually set file attributes for the copied files on the slave end, improving the reliability and efficiency of data replication.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of data replication, and in particular to a data synchronization method and related components for a distributed file system. Background Technology

[0002] As digitalization increases, data is becoming the core of businesses, and companies are demanding greater stability from their data storage systems. Unpredictable natural disasters can cause permanent data loss, so existing technologies periodically replicate data from the master to the slave. The most common replication methods are synchronous and asynchronous replication. Asynchronous replication is preferred because it allows for greater distances between the master and slave and requires less bandwidth. Therefore, most companies use asynchronous replication to copy data from the master to the slave. Ensuring the reliability and efficiency of data replication during asynchronous replication is crucial. Summary of the Invention

[0003] The purpose of this invention is to provide a data synchronization method and related components for a distributed file system, which eliminates the need for manual setting of file attributes for copied files on the slave end, thereby improving the reliability and efficiency of data replication.

[0004] To address the aforementioned technical problems, this invention provides a data synchronization method for a distributed file system, comprising:

[0005] Obtain the file copy package sent by the master end, wherein the file copy package includes the file content of the files under each inode of the distributed file system and the metadata information of the files;

[0006] The contents of each of the aforementioned files are copied to the corresponding copy files in the slave device;

[0007] Set the basic attributes of the copied file to the metadata information corresponding to the file content stored within the copied file.

[0008] Preferably, the metadata information includes any one or more of the following: time attribute, mode attribute, user and group attributes, and user permissions.

[0009] Preferably, the file copy package also includes the extended attributes of each of the files;

[0010] After setting the basic attributes of the copied file to the metadata information corresponding to the file content stored within the copied file, the method further includes:

[0011] Set the extended attributes of the copied file to the extended attributes of the file whose content is stored within the copied file.

[0012] Preferably, the extended attributes include acl attributes, worm attributes, directory quotas, capacity quotas, xattr attributes, and any combination of one or more of the stored files.

[0013] Preferably, before copying the contents of each file to the corresponding copy file in the slave device, the method further includes:

[0014] A write protection request is sent to the master terminal so that the master terminal stops modifying, adding, and deleting the file content of the files under each of the inodes after receiving the write protection request.

[0015] Preferably, before copying the contents of each file to the corresponding copy file in the slave device, the method further includes:

[0016] The ctime timestamps marked on the master end of each file are compared with the ctime timestamps marked on the slave end to see if they are consistent.

[0017] If they match, then stop copying the file content corresponding to the file to the slave device;

[0018] If there is a discrepancy, proceed to the step of copying the file content to a corresponding copy file on the slave end.

[0019] This application also provides a data synchronization system for a distributed file system, including:

[0020] The file copy package acquisition unit is used to acquire the file copy package sent by the master end, wherein the file copy package includes the file content of the files under each index node of the distributed file system and the metadata information of the files;

[0021] The file content copying unit is used to copy the contents of each file to a corresponding copy file in the slave end;

[0022] The attribute setting unit is used to set the basic attributes of the copied file to the metadata information corresponding to the file content stored in the copied file.

[0023] This application also provides a data synchronization device for a distributed file system, comprising:

[0024] Memory, used to store computer programs;

[0025] A processor, used to implement the steps of the metadata synchronization method of the distributed file system as described above when executing the computer program.

[0026] This application also provides a distributed file system, including the data synchronization device for the distributed file system described above.

[0027] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the data synchronization method of the distributed file system described above.

[0028] In summary, this invention provides a data synchronization method and related components for a distributed file system. The method includes acquiring a file replication packet sent by the master end. The file replication packet includes the file content and metadata information of files under each inode of the distributed file system. Then, each file content is copied to a corresponding copy file on the slave end. Furthermore, the basic attributes of the copied file are set to the metadata information corresponding to the file content stored within the copied file. Therefore, this application, when synchronizing files from the master end to the slave end, not only copies the file content but also the file attributes, i.e., the metadata information. Thus, there is no need to manually set file attributes for the copied files on the slave end, improving the reliability and efficiency of data replication. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the prior art and embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 A flowchart illustrating a data synchronization method for a distributed file system provided by this invention;

[0031] Figure 2 This invention provides a schematic diagram of the structure of a data synchronization system for a distributed file system.

[0032] Figure 3 This is a schematic diagram of the structure of a data synchronization device for a distributed file system provided by the present invention;

[0033] Figure 4 This is a schematic diagram of the structure of a computer-readable storage medium provided by the present invention. Detailed Implementation

[0034] The core of this invention is to provide a data synchronization method and related components for a distributed file system, which eliminates the need for manual setting of file attributes for copied files on the slave end, thereby improving the reliability and efficiency of data copying.

[0035] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] As digitalization increases, data is becoming the core of enterprise business, and users are demanding greater stability from distributed file systems (FSS) that host this data. While many manufacturers can provide highly stable storage devices, they cannot prevent irreparable damage to FSS from natural disasters. There are various ways to protect data, the most common being to replicate data across multiple locations. This way, even if a storage device fails and data is lost, it can be recovered from other storage devices. To ensure the continuity, recoverability, and high availability of data storage, remote data replication technology has emerged. Remote replication can be synchronous or asynchronous. Because asynchronous replication allows for greater distances between the master and slave devices, it is more secure and requires less bandwidth, making it the most commonly used remote replication method today.

[0037] However, in existing technologies, whether asynchronous or synchronous copying is performed, only the file content of the file on the master end is usually copied to the slave end. Then, the user has to manually configure the relevant attribute information for each file on the slave end, which is inefficient and may also cause problems with file ownership and permissions after synchronization.

[0038] Please refer to Figure 1 , Figure 1 A flowchart of a data synchronization method for a distributed file system provided by the present invention, the method comprising:

[0039] S1: Obtain the file copy package sent by the master end, wherein the file copy package includes the file content and metadata information of the files under each inode of the distributed file system;

[0040] To address the aforementioned technical problems, this application provides a data synchronization method for a distributed file system. Using inodes as units, it synchronizes the file content and metadata (i.e., file attribute information) of all files under a given inode. The metadata information in this application includes file atime, ctime, mtime, user ID, group ID, and user group permissions, allowing the slave end to set corresponding attribute information for its files while copying the file content from the master end's inode, eliminating the need for manual attribute setting by the user. The master end packages the file content and metadata of the files under each inode into a file replication package and sends the package to the slave end. Furthermore, this application does not impose specific limitations on the specific packaging format of the file replication package; it can be configured according to actual needs.

[0041] S2: Copy the contents of each file to the corresponding copy file on the slave end;

[0042] S3: Set the basic attributes of the copied file to the metadata information of the file corresponding to the file content stored in the copied file.

[0043] After receiving the file copy packet from the master, the slave first copies the contents of each file under the inode to a pre-created copy file, and then automatically sets the attributes for the copy file. Specifically, the file contents and metadata information in the file copy packet are in one-to-one correspondence. Therefore, after copying the file contents from the file copy packet to the copy file, the basic attributes of the copy file need to be set to the metadata information corresponding to the file contents stored in the copy file, thereby achieving automatic configuration of file attributes and improving the efficiency of file copying, i.e., data synchronization.

[0044] In summary, this invention provides a data synchronization method for a distributed file system, comprising obtaining a file replication packet sent by the master end, wherein the file replication packet includes the file content and metadata information of files under each inode of the distributed file system, and then copying each file content to a corresponding copy file on the slave end, and further requiring setting the basic attributes of the copied file to the metadata information corresponding to the file content stored in the copied file. Therefore, this application, when synchronizing files from the master end to the slave end, not only copies the file content but also copies the file attributes, i.e., metadata information, thus eliminating the need for manual setting of file attributes for the copied files on the slave end, improving the reliability and efficiency of data replication.

[0045] Based on the above embodiments:

[0046] As a preferred embodiment, the metadata information includes any combination of one or more of the following: time attribute, mode attribute, user and group attributes, and user permissions.

[0047] In this embodiment, the metadata information specifically includes any combination of one or more of the following: time attribute, mode attribute, user and group attributes, and user permissions. Specifically, the time attribute includes atime, ctime, and mtime. atime refers to the time of the most recent file access, where access refers to the time the file was read or executed; ctime refers to the time of the most recent modification of the file's content or metadata, such as access permissions; and mtime refers to the time of the most recent modification, specifically referring to modifications to the file's content. The time attribute, mode attribute, and user permissions can be directly set by the slave end for each copied file, and will not be elaborated upon here. When configuring user and group attributes for file copying on the slave, it is necessary to first determine whether the user has configured not to convert user IDs and group IDs before file copying. If so, the user IDs and group IDs are directly set in the slave's metadata structure. If the user has configured a mapping relationship between user IDs and group IDs before file copying, the user IDs and group IDs are converted accordingly according to the mapping relationship, and the converted IDs are synchronized in the slave's metadata. If the user has not configured a mapping relationship between user IDs and group IDs before file copying, the user IDs and group IDs are converted to default IDs and then set in the metadata.

[0048] As a preferred embodiment, the file copy package also includes the extended attributes of each file;

[0049] After setting the basic attributes of the copied file to the metadata information corresponding to the file content stored within the copied file, it also includes:

[0050] Set the extended attributes of the copied file to the extended attributes of the file whose contents are stored within the copied file.

[0051] Considering that in addition to the basic attributes of a file, i.e., metadata information, files usually also include some advanced attributes, i.e., extended attributes in this embodiment, such as ACL attributes, worm attributes, directory quotas, capacity quotas, and xattr attributes, in order to further improve the accuracy and reliability of file copying, in this embodiment, when the master end packages the file copy package by inode, it also includes the extended attributes of each file under that inode in the file copy package. In this way, when the slave end receives the file copy package, in addition to copying the file content and setting the basic file attributes, it will also automatically set the extended attributes for the copied file, further improving the automation and reliability of file copying.

[0052] As a preferred embodiment, the extended attributes include acl attributes, worm attributes, directory quotas, capacity quotas, xattr attributes, and any combination of one or more of the stored files.

[0053] In this embodiment, the extended attributes specifically include ACL attributes, worm attributes, directory quotas, capacity quotas, xattr attributes, and any combination of one or more of the stored file attributes. ACL attributes refer to access list attributes. Before setting ACL attributes for copied files on the slave, it is first determined whether the user has configured not to convert user IDs and group IDs. If so, the ACL attributes are directly set in the slave's metadata structure. If the user previously configured a mapping relationship between user IDs and group IDs, the user IDs and group IDs in the ACL attributes are converted accordingly, and the converted IDs are synchronized with the slave's metadata. If the user did not configure a mapping relationship between user IDs and group IDs before copying the file, the user IDs and group IDs are converted to default IDs and then set in the metadata ACL attributes.

[0054] When synchronizing directory worm attributes on the slave end, the grace period and protection period are directly set into the corresponding directory metadata. When synchronizing file worm attributes on the slave end, the protection period expiration time and grace period expiration time of the master end metadata are directly synchronized without recalculating based on the parent directory worm.

[0055] Directory quotas and capacity quotas are both quota attributes. During quota synchronization, the quota settings on the slave device are read first. If quota settings are enabled on the slave device, these settings are synchronized for file copying. If quota settings are not enabled on the slave device, they are not synchronized for file copying. When quotas are synchronized to the slave device, if the directory is empty, the directory quota value is set directly, and the slave directory quota usage value is 0. If the directory is not empty, all files in the directory are traversed, and the number and size of files are calculated to set the number quota and capacity quota. The directory count and capacity count are calculated only during the first quota information synchronization to reduce performance overhead.

[0056] When synchronizing the master xattr (i.e., Linux system extended file attributes) to the slave xattr, you can configure filter attributes. If a certain attribute exists in the filter list, this xattr attribute will not be synchronized.

[0057] As a preferred embodiment, before copying the contents of each file to the corresponding copy file on the slave end, the method further includes:

[0058] Send a write protection request to the master so that the master will stop modifying, adding, and deleting the file content of the files under each inode after receiving the write protection request.

[0059] To ensure the accuracy and reliability of file copying, in this embodiment, when the slave performs file copying, the slave sends a write protection request to the master. When the master receives the write protection request, it stops adding, deleting, or modifying the file content under the inode, so that the file copied to the slave can be consistent with the file in the master, thus improving the reliability of file copying.

[0060] Metadata write operations include moving, deleting, renaming, creating symbolic links, creating new directories, modifying file information, modifying snapshots, and modifying file extended attributes. Metadata write operations are subject to write protection. During remote replication, the replication process sends a request to the MDS (Metadata Storage Service) to set protection attributes. After setting the protection attribute, the directory is in a resource-protected state. During write operations on the slave end, the write request is sent to the MDS. The MDS checks the xattr attributes of the inode and all its parent directory inodes. If any of the xattr attributes contains a write protection attribute, the inode is write-protected, and the metadata under protection cannot be modified.

[0061] As a preferred embodiment, before copying the contents of each file to the corresponding copy file on the slave end, the method further includes:

[0062] Compare the ctime timestamps of each file marked on the master end with the ctime timestamps of the files marked on the slave end to see if they are consistent.

[0063] If there is a discrepancy, stop copying the corresponding file content to the slave end;

[0064] If they match, proceed to the step of copying the file content to a corresponding copy file on the slave end.

[0065] In this embodiment, considering that the backup directory on the slave end is under resource protection, the ctime will only change due to synchronization with the master end. Before copying the file content to the copy file on the slave end, it is first determined whether the ctime timestamp marked on each file on the master end is consistent with the ctime timestamp marked on the slave end. If they are consistent, it proves that the master end has not modified the file during the file copying operation, so the file copying can continue, that is, the step of copying the file content to the copy file on the slave end that corresponds to each file content is entered. If they are inconsistent, it proves that the master end has modified the file during the file copying operation, so the slave end stops the operation of copying the file content corresponding to the file to the slave end, thereby ensuring the reliability of file copying.

[0066] Please refer to Figure 2 , Figure 2This is a schematic diagram of the structure of a data synchronization system for a distributed file system provided by the present invention. The data synchronization system for the distributed file system includes:

[0067] The file copy packet acquisition unit 11 is used to acquire the file copy packet sent by the master end. The file copy packet includes the file content and metadata information of the files under each inode of the distributed file system.

[0068] The file content copying unit 12 is used to copy the contents of each file to the corresponding copy file in the slave end;

[0069] The attribute setting unit 13 is used to set the basic attributes of the copied file to the metadata information corresponding to the file content stored in the copied file.

[0070] This invention provides a data synchronization system for a distributed file system, comprising acquiring a file replication packet sent by the master end, wherein the file replication packet includes the file content and metadata information of files under each inode of the distributed file system, and then copying each file content to a corresponding copy file on the slave end, further specifying that the basic attributes of the copied file are set to the metadata information corresponding to the file content stored in the copied file. Therefore, this application, when synchronizing files from the master end to the slave end, not only copies the file content but also the file attributes, i.e., the metadata information, thus eliminating the need for manual setting of file attributes for the copied files on the slave end, improving the reliability and efficiency of data replication.

[0071] For a detailed description of the data synchronization system of the distributed file system provided in this application, please refer to the above-described embodiments of the data synchronization method of the distributed file system. This application will not repeat the details here.

[0072] Based on the above embodiments:

[0073] As a preferred embodiment, the metadata information includes any combination of one or more of the following: time attribute, mode attribute, user and group attributes, and user permissions.

[0074] As a preferred embodiment, the file replication package also includes extended attributes of each file; the distributed file system data synchronization system also includes:

[0075] The extended attribute setting unit 13 is used to set the extended attributes of the copied file to the extended attributes of the file corresponding to the file content stored in the copied file after setting the basic attributes of the copied file to the metadata information of the file corresponding to the file content stored in the copied file.

[0076] As a preferred embodiment, the extended attributes include acl attributes, worm attributes, directory quotas, capacity quotas, xattr attributes, and any combination of one or more of the stored files.

[0077] As a preferred embodiment, it also includes:

[0078] The write protection unit is used to send a write protection request to the master before copying the contents of each file to the corresponding copy file on the slave end. This allows the master end to stop modifying, adding, or deleting the contents of the files under each inode after receiving the write protection request.

[0079] As a preferred embodiment, it also includes:

[0080] The comparison unit is used to compare the ctime timestamps marked on the master end and the ctime timestamps marked on the slave end of each file before copying the contents of each file to the corresponding copy file on the slave end. If they match, the copy stop unit is triggered; if they do not match, the file content copy unit 12 is triggered.

[0081] The stop copy unit is used to stop copying the file content corresponding to the file to the slave end.

[0082] Please refer to Figure 3 , Figure 3 This is a schematic diagram of a data synchronization device for a distributed file system provided by the present invention. The data synchronization device for the distributed file system includes:

[0083] Memory 21 is used to store computer programs;

[0084] Processor 22 is used to implement the metadata synchronization method of the distributed file system described above when executing computer programs.

[0085] This invention provides a data synchronization device for a distributed file system. The processor in this device, when executing a computer program, includes acquiring a file copy packet sent by the master end. The file copy packet includes the file content and metadata information of files under each inode of the distributed file system. Then, it copies each file content to a corresponding copy file on the slave end. Furthermore, it sets the basic attributes of the copied file to the metadata information corresponding to the file content stored within the copied file. Therefore, when synchronizing files from the master end to the slave end, this application copies not only the file content but also the file attributes, i.e., the metadata information. Thus, there is no need for manual setting of file attributes for the copied files on the slave end, improving the reliability and efficiency of data copying. For a detailed description of the distributed file system data synchronization device provided in this application, please refer to the embodiments of the distributed file system data synchronization method described above; further details are omitted here.

[0086] This application also provides a distributed file system, including the data synchronization device for the distributed file system described above.

[0087] For a detailed description of the distributed file system provided in this application, please refer to the above-described implementation of the data synchronization method for the distributed file system; further details will not be provided here.

[0088] Please refer to Figure 4 , Figure 4 This is a schematic diagram of a computer-readable storage medium provided by the present invention. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the data synchronization method of the distributed file system described above.

[0089] This invention provides a computer-readable storage medium. When a computer program in the computer-readable storage medium is executed, it includes acquiring a file copy package sent by a master end. The file copy package includes the file content and metadata information of files under each inode of a distributed file system. Then, it copies each file content to a corresponding copy file on the slave end. Furthermore, it sets the basic attributes of the copied file to the metadata information corresponding to the file content stored within the copied file. Therefore, when synchronizing files from the master end to the slave end, this application copies not only the file content but also the file attributes, i.e., the metadata information. Thus, there is no need for manual setting of file attributes for the copied files on the slave end, improving the reliability and efficiency of data copying. For a detailed description of the computer-readable storage medium provided in this application, please refer to the embodiments of the data synchronization method for the distributed file system described above; this application will not repeat them here.

[0090] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0091] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0092] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0093] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0094] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data synchronization method of a distributed file system, characterized by, The method comprises the following steps: acquiring a file copy package sent by a master end, wherein the file copy package comprises file contents of files under each index node of a distributed file system and metadata information of the files; copying each file content into a copy file corresponding to each file in a slave end respectively; setting basic attributes of the copy file as metadata information corresponding to the file contents stored in the file copy package; the metadata information comprises a combination of any one or more of time attributes, Mode attributes, user and group attributes, and user permissions; when the time attributes, the Mode attributes, and the user permissions are configured for the copy file, the slave end is directly set; when the user and group attributes are configured for the copy file, it is judged whether the user has configured non-conversion of user id and group id before file copying, if yes, the user id and the group id are directly set into a metadata structure body of the slave end; if the user has configured a mapping relationship of the user id and the group id before file copying, the user id and the group id are correspondingly converted according to the mapping relationship, and the slave end metadata synchronously converts the ids; if the user has not configured the mapping relationship of the user id and the group id before file copying, the user id and the group id are converted into default ids and then set into the metadata.

2. The metadata synchronization method of a distributed file system according to claim 1, wherein, the file copy package further comprises extension attributes of each file; after the basic attributes of the copy file are set as the metadata information corresponding to the file contents stored in the copy file, the method further comprises the following steps: the extension attributes of the copy file are set as extension attributes corresponding to the file contents stored in the copy file.

3. The data synchronization method of a distributed file system according to claim 2, wherein, the extension attributes comprise a combination of any one or more of acl attributes, worm attributes, directory quotas, volume quotas, xattr attributes, and storage files.

4. The data synchronization method of a distributed file system according to claim 1, wherein, before each file content is copied into a copy file corresponding to each file content in the slave end respectively, the method further comprises the following steps: sending a write protection request to the master end, so that the master end stops modifying, adding, and deleting the file contents of the files under each index node after receiving the write protection request.

5. The data synchronization method of the distributed file system according to any one of claims 1 to 4, wherein, before each file content is copied into a copy file corresponding to each file content in the slave end respectively, the method further comprises the following steps: sequentially comparing whether ctime time stamps of each file marked in the master end are consistent with ctime time stamps of the file marked in the slave end; if consistent, stopping copying the file contents corresponding to the file to the slave end; if inconsistent, entering the step of copying the file contents into a copy file corresponding to each file content in the slave end.

6. A data synchronization system of a distributed file system, characterized by, The method comprises the following steps: a file copy package acquisition unit is configured to acquire a file copy package sent by a master end, wherein the file copy package comprises file contents of files under each index node of a distributed file system and metadata information of the files; a file content copy unit is configured to copy each file content into a copy file corresponding to each file in a slave end respectively; The attribute setting unit is configured to set the basic attribute of the copied file as metadata information corresponding to the file content stored in the file copy package; The metadata information includes any one or more of a combination of time attribute, Mode attribute, user and group attribute, and user permission; when the time attribute, Mode attribute, and user permission are configured for the copied file, the direct setting is performed through the slave end; when the user and group attribute are configured for the copied file, it is judged whether the user has configured the non-conversion of user id and group id before the file copy; if yes, the user id and group id are directly set to the slave end metadata structure body; if the user has configured the mapping relationship of user id and group id before the file copy, the user id and group id are converted according to the mapping relationship, and the converted id is synchronized to the slave end metadata; if the user has not configured the mapping relationship of user id and group id before the file copy, the user id and group id are converted into default id and then set to the metadata.

7. A data synchronization apparatus of a distributed file system, characterized by comprising: Comprise: A memory for storing a computer program; A processor for executing the computer program to realize the steps of the metadata synchronization method of the distributed file system according to any one of claims 1 to 5.

8. A distributed file system, characterized by The data synchronization device of the distributed file system according to claim 7.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to realize the steps of the data synchronization method of the distributed file system according to any one of claims 1 to 5.

Citation Information

Patent Citations

  • Distributed metadata remote asynchronous replication method, device and equipment

    CN114968966A