A file system cloning method and device

Abstract

A method and device for cloning a file system, in which a target storage system first receives first information from a source storage system running a file system, the first information being used to indicate the data layout of the file system, and then the target storage system creates a clone file system of the file system according to the first information. After the target storage system creates a clone file system corresponding to the file system in the source storage system, applications running on the file system of the source storage system can also run on the clone file system. Since creating a clone file system only requires the data layout of the source file system, that is, there is no need to transmit the data of the files included in the file system, and the amount of data in the data layout is smaller than the amount of data included in the files, the delay of the application migration process can be reduced.

Description

Technical Field

[0001] The present application relates to the field of storage technology, and in particular to a method and device for cloning a file system. Background Art

[0002] Cloud bursting is an application deployment architecture widely used in storage systems. The main idea is that when an application is deployed in a local storage system and needs to use a large amount of resources (for example, computing resources or storage resources), the application can be migrated to the public cloud and processed using the resources in the public cloud to save resources in the local storage system.

[0003] Before using an application on a public cloud, all data related to the application must be migrated to the public cloud. For example, all data in the local storage system may need to be migrated to the public cloud. However, data migration takes a certain amount of time, especially when the amount of data to be migrated is large, which requires a longer wait for data migration, resulting in a longer processing delay and affecting user use. Summary of the invention

[0004] The present application provides a file system cloning method and device to solve the problem of long delay in application migration process.

[0005] In a first aspect, a method for cloning a file system is provided, in which a target storage system first receives first information from a source storage system running a file system, where the first information is used to indicate a data layout of the file system, and then the target storage system creates a clone file system of the file system based on the first information.

[0006] In the above technical solution, after the destination storage system creates a clone file system corresponding to the file system in the source storage system, the application running on the file system of the source storage system can also run on the clone file system. Since the creation of the clone file system only requires the data layout of the source file system, and the source storage system will not send the data included in any file to the destination storage system before receiving the data acquisition request sent by the destination storage system, that is, it is not necessary to transmit the data of the files included in the file system, and the data volume of the data layout is smaller than the data volume of the data included in the file, thereby reducing the delay of the application migration process.

[0007] In a possible design, the first information may include but is not limited to the following two contents:

[0008] First, the first information includes metadata of all files in the file system.

[0009] In the above technical solution, if the description method of the metadata in the source storage system and the target storage system is the same, the data layout of the file system can be directly indicated by the metadata, and the implementation method is simple.

[0010] Second, the first information includes semantic analysis information of metadata of all files in the file system, and the semantic analysis information of metadata of each file includes an identifier of the metadata of the file, an operation corresponding to the metadata of the file, and data required to create the metadata of the file.

[0011] In the above technical solution, if the source storage system and the destination storage system have different description methods for metadata, the data layout of the file system can be indicated by semantic analysis information of the metadata, which can verify the applicability of the solution. Of course, if the source storage system and the destination storage system have the same description method for metadata, this method can also be used to indicate the data layout of the file system, which is not limited here.

[0012] Furthermore, the data layout of the file system may be indicated by one of the above two contents, which may increase the flexibility of the solution.

[0013] In a possible design, the destination storage system may also receive second information sent by the source storage system, where the second information includes metadata of the files that have changed in the file system or semantic analysis information of the metadata of the files that have changed. Then, the destination storage system updates the clone file system based on the second information.

[0014] In the above technical solution, when a file in the source storage system is changed, the metadata of the changed file or semantic analysis information of the metadata can be sent to the destination storage system to update the clone file system and implement incremental update.

[0015] In one possible design, after the destination storage system receives a read data request including an identifier of a data block to be read, if the destination storage system determines that the data block to be read is not stored in the destination storage system, the destination storage system may send a first data acquisition request to the source storage system, wherein the first data acquisition request includes the identifier of the data block to be read, so as to obtain the data block to be read. After the destination storage system receives the data block to be read from the source storage system, the data block to be read may be saved.

[0016] In the above technical solution, the destination storage system can obtain data blocks from the source storage system on demand to meet the user's usage requirements. In this way, unnecessary data blocks do not need to be sent to the destination storage system, which can reduce the amount of data transmitted between the source storage system and the destination storage system.

[0017] In a possible design, after receiving the read data request, the destination storage system may also determine the pre-read data block according to the pre-fetch rule and the read data request, and send a second data acquisition request including the identifier of the pre-read data block to the source storage system to acquire the pre-read data block. After the destination storage system receives the pre-read data block from the source storage system, the pre-read data block is stored in the destination storage system.

[0018] In the above technical solution, the destination storage system can pre-fetch some data blocks and store them in the destination storage system, so that when a read data request for the pre-read data block is subsequently received, the data block can be directly read locally, which can reduce the response delay.

[0019] In a second aspect, a cloning device for a file system is provided. The cloning device for the file system may be a management node or a management server, or a device in the management node or the management server. The cloning device for the file system includes a processor for implementing the method described in the first aspect. The cloning device for the file system may also include a memory for storing program instructions and data. The memory is coupled to the processor, and the processor may call and execute program instructions stored in the memory to implement any one of the methods described in the first aspect.

[0020] In one possible design, the processor of the cloning device of the file system executes the program instructions in the memory to implement the following functions:

[0021] receiving first information sent by a source storage system, where a file system is running in the source storage system, and the first information is used to indicate a data layout of the file system;

[0022] A clone file system of the file system is created according to the first information.

[0023] In a possible design, the description of the first information is similar to the corresponding content in the first aspect and will not be repeated here.

[0024] In one possible design, the processor executes program instructions stored in the memory to implement the following functions:

[0025] receiving second information sent by the source storage system, where the second information includes metadata of the changed file in the file system or semantic analysis information of the metadata of the changed file;

[0026] The clone file system is updated according to the second information.

[0027] In one possible design, the processor executes program instructions stored in the memory to implement the following functions:

[0028] receiving a data read request, wherein the data read request includes an identifier of a data block to be read;

[0029] When it is determined that the data block to be read is not stored in the destination storage system, a first data acquisition request is sent to the source storage system, where the first data acquisition request includes an identifier of the data block to be read;

[0030] The to-be-read data block is received from the source storage system and saved.

[0031] In one possible design, the processor executes program instructions stored in the memory to implement the following functions:

[0032] Determine the pre-read data block according to the pre-fetch rule and the read data request;

[0033] Sending a second data acquisition request to the source storage system, wherein the second data acquisition request includes an identifier of the pre-read data block;

[0034] The pre-read data blocks are received from the source storage system, and the pre-read data blocks are stored in the destination storage system.

[0035] In a third aspect, a file system cloning device is provided, which may be a management node or a management server, or a device in a management node or a management server. The file system cloning device may include a transceiver unit and a creation unit, which may perform the corresponding functions performed in any one of the design examples of the first aspect, specifically:

[0036] The transceiver unit is used to receive first information sent by a source storage system, where a file system is running in the source storage system, and the first information is used to indicate a data layout of the file system;

[0037] The creating unit is used to create a clone file system of the file system according to the first information.

[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes any one of the methods described in the first aspect.

[0039] In a fifth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes any one of the methods described in the first aspect.

[0040] In a sixth aspect, the present application provides a chip system, which includes a processor and may also include a memory, for implementing the method described in the first aspect. The chip system may be composed of a chip, or may include a chip and other discrete devices.

[0041] In a seventh aspect, an embodiment of the present application provides a storage system, which includes a source storage system and a cloning device of the file system described in the second aspect and any one of the designs of the second aspect, or the storage system includes a source storage system and a cloning device of the file system described in the third aspect and any one of the designs of the third aspect.

[0042] The beneficial effects of the above-mentioned second to seventh aspects and their implementation methods can refer to the description of the beneficial effects of the method of the first aspect and its implementation method. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of an example of an application scenario of an embodiment of the present application;

[0044] Figure 2 A flowchart of a file system cloning method provided in an embodiment of the present application;

[0045] Figure 3 A schematic diagram of an example of the data layout of a file system provided in an embodiment of the present application;

[0046] Figure 4 A schematic diagram of an example of a cloned file system created in the storage system 110 in an embodiment of the present application;

[0047] Figure 5 A flowchart of processing a read data request for a cloned file system in an embodiment of the present application;

[0048] Figure 6 A flowchart of processing a write data request for a cloned file system in an embodiment of the present application;

[0049] Figure 7 A schematic diagram of an example structure of a file system cloning device provided in an embodiment of the present application;

[0050] Figure 8 A schematic diagram of the structure of another example of a file system cloning device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0052] In the embodiments of the present application, "multiple" refers to two or more than two. In view of this, in the embodiments of the present application, "multiple" can also be understood as "at least two". "At least one" can be understood as one or more, for example, one, two or more. For example, including at least one means including one, two or more, and there is no restriction on which ones are included. For example, including at least one of A, B and C, then A, B, C, A and B, A and C, B and C, or A and B and C may be included. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / ", unless otherwise specified, generally indicates that the objects associated before and after are in an "or" relationship. In the embodiments of the present application, "node" and "node" can be used interchangeably.

[0053] Unless otherwise specified, the ordinal numbers such as "first" and "second" mentioned in the embodiments of the present application are used to distinguish multiple objects and are not used to limit the order, timing, priority or importance of the multiple objects.

[0054] The file system cloning method provided in the embodiment of the present application can be applied to two storage systems. Figure 1 , is a schematic diagram of an example of an application scenario of an embodiment of the present application. Figure 1 In the embodiment, a storage system 100 and a storage system 110 are included. The storage system 100 includes a management module 101 and at least one storage node 102 (in Figure 1 In the example, three storage nodes 102 , namely storage node 1 to storage node 3 , are taken. The management module 101 is used to write data to each storage node 101 , and to read data from at least one storage node 102 .

[0055] Figure 1 The storage node 102 may be an independent server, or may be a storage array including at least one storage device, which may be a hard disk drive (HDD) disk device, a solid state drive (SSD) disk device, a serial advanced technology attachment (SATA) disk device, a small computer system interface (SCSI) disk device, a serial attached SCSI interface (SAS) disk device, or a fibre channel interface (FC) disk device, etc.

[0056] Figure 1 The management module 101 in the embodiment may be an independent server or controller, etc. The management module 101 and at least one storage node 102 may be independent devices, for example, the management module 101 is an independent server; or, the management module 101 may be a software module deployed on a storage node 102, for example, the management module 101 and a storage node 102 run on the same server, and the specific forms of the management module 101 and the storage node 102 are not limited here. Figure 1 In the description, the management module 101 and the storage node 102 are taken as an example as being independent devices.

[0057] The storage system 100 includes a file system 103. The management module 101 runs the file system 103 to control the way of writing and reading data into the storage node 102. If there is no file system, the information placed in the storage space of at least one storage node 102 will be a large data body, and it is impossible to distinguish the starting position and the ending position of an information. The data is separated and each data is named, so that it is easy to separate and identify the information. The naming is based on the naming method of the paper information system, and each group of data is called a "file". The structure and logical rules used to manage each group of information and its name are called a "file system". Accordingly, the file system 103 is responsible for organizing files and tracking when files are changed. Further, the metadata of all files is stored in the file system 103, and the metadata of each file includes information such as the file name, file ID (for example, index node number), the size of the file data block, the file location stored in the disk, the timestamp of creating the file, and multiple timestamps of writing each data block to the disk. The metadata of all files in the file system 103 is dispersed and stored in at least one storage node 102.

[0058] The hardware architecture of the storage system 110 is similar to the hardware architecture of the above-mentioned storage system 100. For example, the storage system 110 includes a management module 111 and at least one storage node 112 (respectively storage node 4 to storage node 6), the management module 111 is similar to the management module 101, and the at least one storage node 112 is similar to the at least one storage node 102, which will not be described in detail here.

[0059] Before creating a file system, a storage space for storing data required by the file system may be reserved in the storage system 110. The size of the storage space may be preset or determined by the management module 111. The storage space is distributed on all storage nodes 112 of the target storage system 110 or on some storage nodes 112. In addition, the description of metadata of files in different types of file systems is different. The administrator may configure the type and attributes of the file system through the management module 111. For example, the type of the file system may be a FAT 32 file system using a 32-bit file allocation table (FAT) or a new technology file system (NTFS) or a third extended file system (ETX3). The attributes of the file system may include whether the file system is a read-only file system or a readable and writable file system.

[0060] exist Figure 1 In the application scenario shown, storage system 100 is a distributed storage system, and storage system 110 is also a distributed storage system. In other application scenarios, storage system 100 or storage system 110 may also be a centralized storage system. In other application scenarios, storage system 110 may also be a cloud storage system such as a public cloud or a private cloud, or storage system 100 and storage system 110 may both be cloud storage systems, which is not limited here.

[0061] Below, Figure 1 Taking the application scenario shown in the figure as an example, the cloning method of the file system in the embodiment of the present application is described. Figure 2 , is a flow chart of the method, which is described as follows:

[0062] S201. The management module 101 obtains first information of the file system 103 at a first moment.

[0063] In the embodiment of the present application, the first information is used to indicate the data layout of the file system 103. The first moment can be used to refer to any moment. For example, the management module 101 can obtain the first information based on a clone request received from a client communicating with the storage system 100, and the first moment can be a moment after receiving the clone request. Alternatively, the management module 101 can also periodically and actively obtain the first information, and the first moment can be the start moment of any period, which is not limited here.

[0064] The management module 101 may obtain the first information in the following two ways, including but not limited to:

[0065] The first way to obtain:

[0066] Since the metadata of each file in the source storage system 100 records information such as the file name, file ID, file size, and location stored on the disk, if the metadata of all files in the file system 103 are obtained and identified, the data layout of the file system 103 can be obtained. For example, the file system 103 includes metadata corresponding to directory A, metadata corresponding to file a, metadata corresponding to directory B, and metadata corresponding to file b, wherein the metadata corresponding to directory A includes the name of directory A, the ID of the files included in directory A and the ID of the directory (target B), the size of the files included in directory A, or the size of the data blocks included in directory A, etc.; the metadata corresponding to file a includes the name of file a, the size of file a, the size of the data blocks included in file a, etc.; the metadata corresponding to directory B includes the name of directory B, the ID of the files included in directory B, the size of the data blocks included in directory B, etc.; the metadata corresponding to file b includes the name of file b, the size of file b, the size of the data blocks included in file b, etc. The above information can be referred to as follows. Figure 3 Therefore, in this manner, the first information is the metadata of all files in the file system 103 .

[0067] As an example, the management module 101 may obtain metadata of all files stored in the file system 103 at the first moment by taking a snapshot of the file system 103. For example, the first information of the file system 103 obtained by the management module 101 at the first moment is metadata corresponding to directory A, metadata corresponding to file a, metadata corresponding to directory B, and metadata corresponding to file b, wherein the content included in each metadata is similar to the aforementioned content and is not described in detail here.

[0068] As can be seen from the first acquisition method, the data layout in the file system 103 can be obtained through the metadata of all files in the file system 103. However, as can be seen from the above description of the file system, the metadata of files in different types of file systems are described in different ways. Therefore, the prerequisite for implementing the first method is that the management module receiving the first information (for example, the management module 111 of the storage system 110) describes the metadata of the files in the same way. If the management module receiving the first information describes the metadata of the files in a different way, the management module receiving the first information cannot obtain the correct metadata.

[0069] In view of this, a second acquisition method is provided, as follows:

[0070] After the management module 101 obtains the metadata of all files / directories of the file system 103 at the first moment, the metadata of each file can be semantically analyzed to generate semantic analysis information corresponding to the metadata of each file, each of which includes the identifier of the metadata, the operation corresponding to the metadata, and the data required to create the metadata. The identifier of the metadata and the data required to create the metadata can be obtained from the metadata of each file, which will not be described in detail here. The operations corresponding to the metadata in the semantic analysis information are explained below.

[0071] In an embodiment of the present application, in addition to the content included in the aforementioned metadata, the metadata of the file also records information indicating the type of metadata in the block header or data integrity field (DIF) of the metadata of each file. The type of metadata may include data type, directory type, access permission type, etc. The type of metadata may be divided by the management module 101 according to the file name of the file corresponding to the metadata or the content included in the file data block. For example, if the file name of a file includes a "file" field, the management module 101 determines that the type of metadata corresponding to the file is a data type; or, if the file data block of a file is user data, the management module 101 determines that the type of metadata corresponding to the file is a data type. Of course, other methods can also be used to determine the type of metadata for each file, which will not be repeated here.

[0072] Then, the management module 101 can determine the operation corresponding to the metadata of each file according to the type of the metadata of each file. In the file system 103, metadata of different types have different corresponding operations. For example, the file system 103 may include different metadata creation functions. When creating metadata of a data type, the corresponding metadata creation function may be function X, and the operation corresponding to the metadata of the data type is the operation of calling function X; when creating metadata of a directory type, the corresponding metadata creation function may be function Y, and the operation corresponding to the metadata of the directory type is the operation of calling function Y.

[0073] In this manner, the first information is semantic analysis information of metadata of all files in the file system 103 of the storage system 100 .

[0074] As an example, following the above example, the file system 103 includes metadata corresponding to directory A, metadata corresponding to file a, metadata corresponding to directory B, and metadata corresponding to file b. After obtaining the metadata corresponding to directory A, the management module 101 performs semantic analysis on the metadata, and obtains the semantic analysis information of the metadata corresponding to directory A: the metadata is identified as directory A, and the data required to create the metadata corresponding to directory A is: the IDs of the files included in directory A are file a and directory B, and the size of the data block is 2MB. Then, the management module 101 can determine that the type of the metadata is a directory type according to the metadata block header or the data integrity field DIF, and further determine that the operation on the metadata is an operation of calling function X, so as to obtain the semantic analysis information of the metadata corresponding to directory A: calling function X to create metadata identified as directory A, with a data block size of 2MB and including file a and directory B. The metadata of other files are semantically analyzed in the same way, which will not be repeated here.

[0075] In a possible implementation, when the metadata of files in the file system 103 are dispersedly stored in various storage nodes 102, the management module 101 can send a snapshot acquisition request to each storage node 102. After receiving the snapshot acquisition request, each storage node 102 takes a snapshot of the metadata of the files stored therein, and then sends the acquired snapshot of the metadata to the management module 101. The management module 101 then obtains the metadata stored in the file system 103 at the first moment from the snapshot corresponding to each storage node 102.

[0076] In another possible implementation, in the storage system 100, the storage space corresponding to at least one storage node 102 may be divided into multiple domains. For example, storage node 1 may include two domains, storage node 2 may include three domains, and the data in each domain may be managed independently. For example, the management module 101 may perform a hash operation based on the file name and ID of the metadata of each file to obtain a domain name corresponding to the metadata of each file, and then store the metadata in the corresponding domain. Since the data in each domain can be managed independently, each storage node 102 may take a snapshot of the metadata of the files stored in each domain, thereby obtaining a snapshot of the metadata corresponding to the multiple domains included in the storage node 102, and then use the first acquisition method or the second acquisition method to obtain the metadata of the files in each domain or the parsed information of the metadata.

[0077] S202 . The management module 101 of the storage system 100 sends the first information to the management module 111 of the storage system 110 .

[0078] The management module 101 can send the first information sequentially, or the management module 101 can divide the first information into multiple copies according to the source of each information in the first information (from different storage nodes or different domains), and send the multiple copies in parallel, thereby speeding up the data transmission speed, which is not limited here.

[0079] S203: The management module 111 of the storage system 110 creates a cloned file system according to the first information.

[0080] Depending on the content of the first information, the management module 111 of the storage system 110 creates a cloned file system according to the first information in different ways, which may include but are not limited to the following two ways.

[0081] The first creation method:

[0082] In the case where the first information is metadata of all files in the file system 103, the management module 111 obtains metadata corresponding to the file from the first information, and then the management module 111 can create cloned metadata on the storage space reserved in the storage system 110 for storing data required by the file system. When the cloned metadata of all files are created, the following is obtained: Figure 3 The cloned file system is shown.

[0083] The metadata of the multiple clones may be created on the same storage node, or may be created on different storage nodes respectively. In the embodiment of the present application, the storage node where the metadata of the clones is located is not limited.

[0084] The second creation method:

[0085] When the first information is semantic analysis information of metadata of all files in the file system 103 , the management module 111 creates cloned metadata on the corresponding storage node 112 according to the operation indicated in each semantic analysis information and the data required to create the metadata.

[0086] As an example, the management module 111 obtains 4 semantic analysis information from the first information, wherein the content of each semantic analysis information is the same as the aforementioned content and will not be repeated here, so that the management module 111 creates cloned metadata corresponding to the 4 semantic analysis information, and then establishes an association relationship between the metadata of each clone based on the file ID and / or directory ID included in the metadata of each directory type. For example, the file IDs included in the semantic analysis information corresponding to the metadata of directory A are file a and directory B, then the management module 111 establishes an association relationship between directory A, file a and directory B, and uses the same processing method to establish an association relationship between directory B and file b, thereby obtaining the following: Figure 3The cloned file system is shown.

[0087] In other embodiments, when cloning the metadata of each file, the management module 111 may update the file name of each file according to a preset rule. For example, if the file name of the metadata is directory A, the management module 111 may update the file name to directory A' when cloning the metadata of the file, thereby creating the metadata of directory A' and establishing a one-to-one mapping relationship between the source metadata and the cloned metadata (for example, the metadata of directory A and the metadata of directory A'). The metadata of other files is processed in the same way to obtain the following: Figure 4 (a) shows the cloned file system.

[0088] The above technical solution provides a solution for creating a cloned file system based on the metadata of the file system. Moreover, since only the metadata in the source storage system needs to be sent to the destination storage system without copying any user data, the cloned file system can be created in the destination storage system, thereby reducing the amount of data transmitted and improving the efficiency of cloning the file system.

[0089] Since the data in the file system changes dynamically, the file system cloned in the storage system 110 also supports incremental update operations. Figure 2 The flowchart shown also includes the following steps:

[0090] S204 : The management module 101 of the storage system 100 determines metadata updated between the first moment and the second moment.

[0091] The second moment may be any moment after the first moment, or the management module 101 periodically obtains updated metadata in the file system 103, and the second moment is the start moment of a cycle. The management module 101 may obtain a snapshot of the file system 103 at the second moment, and then compare the snapshot obtained at the first moment with the snapshot obtained at the second moment, and then obtain the metadata updated between the first moment and the second moment.

[0092] The updated metadata may include but is not limited to the following two situations:

[0093] In the first case, the data layout of the file system 103 remains unchanged, but the metadata changes due to changes in the data blocks. For example, at the first moment, the file system 103 includes metadata corresponding to directory A, metadata corresponding to file a, metadata corresponding to directory B, and metadata corresponding to file b, wherein the content included in the metadata of each file has been described in the previous text and will not be repeated here. After the first moment, the management module 101 updates the data blocks of file a according to the operation of the client, for example, updates file a from data block 1 to data block 2. Since the data blocks change, the metadata corresponding to file a will also change. In this case, the metadata corresponding to file a including data block 2 is the updated metadata.

[0094] In the second case, the data layout changes due to the addition of a new file. For example, after the first moment, the file system 103 adds a new file c to the directory B, so that metadata corresponding to the file c is added to the file system 103, where the metadata corresponding to the file c includes the file name of the file c and the size of the data block of 1MB. In this case, the metadata corresponding to the file c is the updated metadata.

[0095] Of course, the updated metadata may also include other situations, which are not listed here one by one.

[0096] S205 : The management module 101 sends the updated metadata to the management module 111 of the storage system 110 .

[0097] After acquiring the updated metadata, the management module 101 may send the updated metadata or the semantic analysis information of the updated metadata to the management module 111. Figure 2 For example, the metadata corresponding to the file a including the data block 2 and the metadata corresponding to the file c are sent to the management module 111.

[0098] S206 : The management module 111 of the storage system 110 updates the cloned file system according to the updated metadata.

[0099] After receiving the updated metadata, the management module 111 can first determine whether the updated metadata is metadata corresponding to a newly added file or metadata generated by updating an existing file, and then perform corresponding operations according to the determination result. If the updated metadata is metadata generated by updating an existing file, the management module 111 first deletes the old metadata corresponding to the updated metadata in the cloned file system, and then creates new cloned metadata of the file in the cloned file system based on the updated metadata. If the updated metadata is metadata corresponding to a newly added file, the management module 111 directly creates cloned metadata in the cloned file system based on the updated metadata.

[0100] For example, the management module 111 queries the file names of all metadata stored in the storage system 110 to determine whether there is metadata in the cloned file system with the same file name as the file name of the updated metadata. If so, it means that the updated metadata is metadata generated by updating an already existing file. For example, the file name included in the updated metadata is file a, and the management module 111 determines that the cloned metadata corresponding to file a already exists in the cloned file system, then the management module 111 deletes the metadata corresponding to file a in the cloned file system, and then recreates the cloned metadata corresponding to file a based on the updated metadata. For the convenience of explanation, the cloned metadata corresponding to file a created based on the updated metadata is marked as metadata corresponding to file a", and then the management module 111 establishes a mapping relationship between the metadata corresponding to file a and the metadata corresponding to file a", so that the following can be obtained: Figure 4 (b) shows the data layout.

[0101] The management module 111 queries the file names of all metadata stored in the storage system 110, and determines that there is no metadata with the same file name as the updated metadata in the cloned file system, which indicates that the updated metadata is metadata corresponding to the newly added file. For example, if the management module 111 determines that the cloned file system does not include the cloned metadata corresponding to file c, then the cloned metadata corresponding to file c is created in the cloned file system according to the updated metadata, thereby obtaining the following: Figure 4 (c) shows the data layout.

[0102] In the above technical solution, the source storage system may send updated metadata to the destination storage system to maintain the consistency of data in the source file system and the cloned file system.

[0103] The cloned file system created in the storage system 110 can provide data access to users. The following describes the process of processing a read data request and a write request of the cloned file system.

[0104] Please refer to Figure 5 , which is a flowchart of processing a read data request for a cloned file system. The description of the flowchart is as follows:

[0105] S501 . The management module 111 of the storage system 110 receives a data read request sent by a client. The data read request includes an identifier of a data block to be read.

[0106] In an embodiment of the present application, the identifier of the data block to be read includes the file name of the file to which the data block to be read belongs, the starting address and length of the data block to be read, the starting address of the data block to be read can be indicated by an offset from the starting address, and the length of the data block to be read can be the number of data blocks of a fixed size (for example, 8KB data blocks). As an example, the identifier of the data block to be read includes file a, the starting address is 0, and the length is 10, that is, the data block to be read is the first 80KB data block of file a.

[0107] S502 : The management module 111 determines whether the data block to be read is stored in the storage system 110 .

[0108] For metadata of data types, after the management module 111 creates cloned metadata in the storage system 110, it can also create a bitmap or bit-tree corresponding to the cloned metadata. The bitmap or bit-tree is used to indicate whether the storage space corresponding to the cloned metadata stores the data blocks corresponding to the metadata. Taking the bitmap as an example, the bitmap may include multiple bits, each bit is used to indicate whether a data block is stored in a physical address segment of a fixed size. For example, the fixed size is the size of a sector (for example, 4KB), and the size of the data block of file a is 160KB. Then the bitmap corresponding to file a includes 40 bits. The data block of file a is stored in the first disk of storage node 4, where the starting address is 0. Then the first bit is used to indicate whether the data block is stored in the first sector of the first disk, and the second bit is used to indicate whether the data block is stored in the second sector of the first disk, and so on. Since only the metadata of the file system 103 is cloned in the storage system 110, when the storage system 110 executes Figure 2 After the cloning method shown, the storage system 110 may not store data blocks of file a. In this case, each bit in the bitmap corresponding to file a indicates that no data blocks are stored in the corresponding physical address segment. For example, the values ​​of the 40 bits corresponding to file a are all 0.

[0109] As an example, when the management module 111 determines that the data block to be read is the first 80KB data block of file a, the management module 111 first determines whether the values ​​of the first 20 bits in the bitmap corresponding to file a are 0. If they are 0, the management module 111 determines that the data block to be read is not stored in the storage system 110.

[0110] As another example, when cloning the metadata of each file, the management module 111 may update the file name of each file according to a preset rule, thereby creating a file such as Figure 4 Thus, after receiving a read data request, the management module 111 first needs to determine that the read data request is to read a data block of file a' according to the one-to-one mapping relationship between the source metadata and the cloned metadata stored in the storage system 110, and then determine whether the storage system 110 stores the data block to be read according to the bitmap corresponding to the file a'.

[0111] S503 : When the data block to be read is not stored in the storage system 110 , the management module 111 sends a first data acquisition request to the source storage system 100 .

[0112] The first data acquisition request may include an identifier of the data block to be read, which is the same as that in step S501 and is not described again. As an example, the identifier of the data block to be read includes file a, a starting address of 0, and a length of 10, that is, the data block to be read is the first 80KB data block of file a.

[0113] It should be noted that the first data acquisition request may be the read data request obtained by the client in step S501, that is, the read data request is directly forwarded, or it may be the data acquisition request obtained after the management module 111 parses and processes the received read data request, which is not limited here.

[0114] S504 , the management module 101 obtains the data block to be read, and sends the data block to be read to the management module 111 .

[0115] After obtaining the data acquisition request, the management module 101 obtains the data block to be read according to the identifier of the data block to be read carried in the data acquisition request, for example, obtains the first 80KB data block of file a, and sends the data block to the management module 111.

[0116] S505: The management module 111 saves the data block to be read.

[0117] After the management module 111 obtains the data block to be read, it stores the data block to be read in the storage space corresponding to it. For example, the first 80KB data block of file a is stored in sectors 1 to 20 of storage node 4, then the management module 111 writes the data block to be read in sectors 1 to 20 of storage node 4. In this case, the following can be obtained: Figure 4 (d) shows the data layout, Figure 4 In (d), the corresponding file data blocks are displayed below file a, indicating that the data blocks included in the file have been stored in the destination storage system 110.

[0118] Furthermore, since the data blocks included in file a have already been saved in the storage system 110, the management module 111 changes the bitmap corresponding to file a, for example, by setting the values ​​of the first 20 bits in the bitmap corresponding to file a to 1, to indicate that the first 80KB of data blocks of file a have already been saved in the storage space corresponding to file a.

[0119] S506: The management module 111 sends the data block to be read to the client.

[0120] In the embodiment of the present application, the execution order of step S505 and step S506 is not limited.

[0121] In addition, if the data block to be read is stored in the target storage system 110, the management module 111 does not need to execute steps S503 to S505, but directly obtains the data block to be read from the corresponding storage space and sends the data block to the client. Therefore, steps S503 to S505 are optional steps. Figure 5 Marked by dotted line.

[0122] S507: The management module 111 sends a second data acquisition request to the storage system 100, where the second data acquisition request includes an identifier of the pre-read data block.

[0123] The management module 111 may also pre-fetch part of the data into the cache of the management module 111 according to the data read request. For example, if the data request is used to read the first 80KB data block of file a, the management module 111 may infer, based on a preset pre-fetch rule, that the user will subsequently read the last 80KB data block of file a. The management module 111 may send a second data acquisition request to the storage system 100, and the second data acquisition request may carry an identifier of the last 80KB data block of file a.

[0124] S508 , the management module 101 obtains the pre-read data block, and sends the pre-read data block to the management module 111 .

[0125] The management module 101 obtains the last 80KB data block of file a according to the second data acquisition request, and sends it to the management module 111 .

[0126] S509 , the management module 111 caches the pre-read data blocks.

[0127] After receiving the pre-read data block, the management module 111 stores the pre-read data block in the cache of the management module 111. In this way, when the management module 111 subsequently receives a read data request corresponding to the pre-read data block, the data block can be directly obtained from the cache and sent to the client, thereby reducing latency.

[0128] Steps S507 to S509 are optional steps. Figure 5 Indicated by dotted line.

[0129] When the cloned file system in the storage system 110 is a readable and writable file system, the management module 111 of the storage system 110 can also process a write request. In the embodiment of the present application, the write request may include a write data request, a create file request, or a delete file request, etc. In the following, the write request is taken as an example of a write data request.

[0130] Please refer to Figure 6 , is a flowchart of the storage system 110 processing a write data request, and the description of the flowchart is as follows:

[0131] S601: The management module 111 of the storage system 110 receives a write data request sent by a client. The write data request includes a data block to be written and an identifier of the data block to be written.

[0132] In an embodiment of the present application, the identifier of the data block to be written includes the file name of the file to which the data block to be written belongs and the size of the data block to be written. As an example, the identifier of the data block to be written includes file a, and the size of the data block to be written is 80KB.

[0133] S602 : The management module 111 writes the data block to be written into the corresponding storage space of the storage system 110 .

[0134] As an example, the management module 111 determines that the data block to be written is a data block corresponding to file a, then the management module 111 stores the data block to be written in the storage space corresponding to file a. For example, the storage space of file a is sector 1 to sector 40 of storage node 4, then the management module writes the data block to be written sequentially into sectors 1 to sector 40.

[0135] It should be noted that if the storage system 110 has already stored a data block corresponding to file a, the management module 111 may adopt an overwrite writing method to replace the previously stored data block corresponding to file a with the data block to be written.

[0136] In other embodiments, when cloning the metadata of each file, the management module 111 may update the file name of each file according to a preset rule, thereby creating a file such as Figure 4 (a) shows the cloned file system. Thus, after receiving the write data request, the management module 111 first needs to determine the metadata corresponding to the data block to be written according to the one-to-one mapping relationship between the source metadata and the cloned metadata stored in the storage system 110 and the identifier of the data block to be written included in the write data request. For example, if the file name carried in the write data request is file a, and the mapping relationship includes the correspondence between file a and file a', the management module 111 determines that the data block carried by the write data request is the data block of file a'. Then, the data block to be written is written into the storage space corresponding to the metadata.

[0137] When the write request is a request to create a file or a request to delete a file, the management module 111 may use Figure 2 or Figure 6 Similar steps are used to create new files or delete files on the cloned file system, which will not be repeated here.

[0138] In the above embodiments provided by the present application, in order to implement the functions of the method provided by the above embodiments of the present application, the storage system may include a hardware structure and / or a software module, and implement the above functions in the form of a hardware structure, a software module, or a hardware structure plus a software module. Whether one of the above functions is executed in the form of a hardware structure, a software module, or a hardware structure plus a software module depends on the specific application and design constraints of the technical solution.

[0139] Figure 7 FIG. 7 is a schematic diagram showing a structure of a file system cloning device 700. The file system cloning device 700 may be Figure 2 or Figure 5 or Figure 6 The device where the management module 111 in the illustrated embodiment is located, or in the device where the management module 111 is located, can be used to implement the functions of the management module 111. The file system cloning device 700 can be a hardware structure or a hardware structure plus a software module.

[0140] The file system cloning device 700 includes at least one memory for storing program instructions and / or data. The file system cloning device 700 also includes at least one processor, which is coupled to the memory and can execute program instructions stored in the memory.

[0141] The file system cloning device 700 may include a creating unit 701 and a transceiving unit 702 .

[0142] The creation unit 701 can call the processor to execute the program instructions stored in the memory to perform Figure 2 In the embodiment shown, step S203 and step S206 are performed, or Figure 5 In the embodiment shown, steps S502, S509 and S509, or executing Figure 6 Step S602 in the illustrated embodiment, and / or other processes for supporting the techniques described herein.

[0143] The transceiver unit 702 can call the processor to execute the program instructions stored in the memory to perform Figure 2 In the embodiment shown, step S202 and step S205 are performed, or Figure 5 Step S501, step S503, step S504, step S506 to step S508 in the embodiment shown, or executing Figure 6 Step S601 in the illustrated embodiment, and / or other processes for supporting the technology described herein. The transceiver unit 702 is used for the file system cloning device 700 to communicate with other modules, and can be a circuit, a device, an interface, a bus, a software module, a transceiver or any other device capable of achieving communication.

[0144] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and will not be repeated here.

[0145] Figure 7 The division of modules in the illustrated embodiments is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present application may be integrated into one processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0146] like Figure 8 The file system cloning device 800 provided in an embodiment of the present application is shown, wherein the file system cloning device 800 may be Figure 2 or Figure 5 or Figure 6 The device where the management module 111 in the illustrated embodiment is located, or in the device where the management module 111 is located, can be used to implement the functions of the management module 111.

[0147] The file system cloning device 800 includes at least one processor 820, which is used to implement or support the file system cloning device 800 to implement the functions of the management module 111 in the method provided in the embodiment of the present application. Exemplarily, the processor 820 can create a clone file system according to the first information, and the detailed description in the method example is referred to for details, which will not be repeated here.

[0148] The file system cloning device 800 may also include at least one memory 830 for storing program instructions and / or data. The memory 830 is coupled to the processor 820. The coupling in the embodiment of the present application is an indirect coupling or communication connection between devices, units or modules, which may be electrical, mechanical or other forms, and is used for information exchange between devices, units or modules. The processor 820 may operate in coordination with the memory 830. The processor 820 may execute program instructions stored in the memory 830. At least one of the at least one memory may be included in the processor.

[0149] The file system cloning device 800 may also include a communication interface 810 for communicating with other devices via a transmission medium, so that the file system cloning device 800 can communicate with other devices. Exemplarily, the other device may be a client or a storage device. The processor 820 may use the communication interface 810 to send and receive data.

[0150] The specific connection medium between the communication interface 810, the processor 820 and the memory 830 is not limited in the embodiment of the present application. Figure 8 In the embodiment, the memory 830, the processor 820 and the communication interface 810 are connected via a bus 850. Figure 8 The connections between the other components are shown in bold lines, which are only for illustration and are not intended to be limiting. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0151] In the embodiment of the present application, the processor 820 may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiment of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0152] In an embodiment of the present application, the memory 830 may be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), etc., or a volatile memory (volatile memory), such as a random access memory (RAM). The memory is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory in the embodiment of the present application may also be a circuit or any other device that can implement a storage function, for storing program instructions and / or data.

[0153] The present application also provides a computer-readable storage medium including instructions, which, when executed on a computer, causes the computer to execute Figure 2 or Figure 5 or Figure 6 The method executed by the management module 111 in the illustrated embodiment.

[0154] The present application also provides a computer program product, including instructions, which, when executed on a computer, causes the computer to execute Figure 2 or Figure 5 or Figure 6 The method executed by the management module 111 in the illustrated embodiment.

[0155] The present application provides a storage system, which includes a source storage system and a destination storage system. The destination storage system includes Figure 2 or Figure 5 or Figure 6 In the illustrated embodiment, a management module 111 is provided.

[0156] In the method provided in the embodiment of the present application, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., an SSD), etc.

Claims

1. A method for cloning a file system, characterized in that: include: The destination storage system receives first information sent by a source storage system, where a file system runs in the source storage system, and the first information includes metadata of files in the file system; The destination storage system creates a clone file system of the file system according to the first information; The destination storage system receives a first data read request, where the first data read request includes an identifier of a data block to be read; the data block to be read belongs to data of a file in the clone file system; In response to the first data read request, sending a first data acquisition request to the source storage system, wherein the first data acquisition request includes an identifier of the data block to be read; The destination storage system receives the to-be-read data block from the source storage system; wherein the data of the file in the clone file system in the destination storage system is acquired from the source storage system on demand based on a read data request; The destination storage system determines a pre-read data block; The destination storage system obtains the pre-read data block from the source storage system.

2. The method according to claim 1, characterized in that: The first information does not include data of files in the file system.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: The destination storage system receives second information sent by the source storage system, where the second information includes metadata of the file whose file system has changed; The destination storage system updates the clone file system according to the second information.

4. The method according to claim 1 or 2, characterized in that: The method further comprises: The destination storage system receives a request to write data; The destination storage system writes the data corresponding to the data write request into the storage space of the destination storage system.

5. The method according to claim 1 or 2, characterized in that: Before sending the first data acquisition request to the source storage system, the method further includes: The destination storage system determines that the to-be-read data block is not stored in the destination storage system.

6. The method according to claim 1 or 2, characterized in that: The identifier of the data block to be read includes the identifier, the start address and the length of the file in the clone file system.

7. A file system cloning device, characterized in that: The invention comprises a processor and a memory, wherein the memory stores computer executable instructions, and when the computer executable instructions are called by the processor, the processor is used to execute: Receiving first information sent by a source storage system, where a file system runs in the source storage system, and the first information includes metadata of files in the file system; creating a clone file system of the file system according to the first information; receiving a first data read request, wherein the first data read request includes an identifier of a data block to be read; the data block to be read belongs to data of a file in the clone file system; In response to the first data read request, sending a first data acquisition request to the source storage system, wherein the first data acquisition request includes an identifier of the data block to be read; Receiving the to-be-read data block from the source storage system; wherein the data of the file in the clone file system in the destination storage system is acquired from the source storage system on demand based on a read data request; Determine the pre-read data blocks; The destination storage system obtains the pre-read data block from the source storage system.

8. The cloning device according to claim 7, characterized in that: The first information does not include data of files in the file system.

9. The cloning device according to claim 7 or 8, characterized in that: When the computer executable instruction is called by the processor, it is also used to cause the processor to execute: receiving second information sent by the source storage system, where the second information includes metadata of the file whose file system has changed; The clone file system is updated according to the second information.

10. The cloning device according to claim 7 or 8, characterized in that: When the computer executable instruction is called by the processor, it is also used to cause the processor to execute: Receive a write data request; The data corresponding to the write data request is written into the storage space of the cloning device.

11. The cloning device according to claim 7 or 8, characterized in that: When the computer executable instruction is called by the processor, it is also used to cause the processor to execute: It is determined that the data block to be read is not stored in the clone device.

12. The cloning device according to claim 7 or 8, characterized in that: The identifier of the data block to be read includes the identifier, the start address and the length of the file in the clone file system.

13. A system for file system cloning, characterized in that: It includes a source storage system and a destination storage system; a file system is running in the source storage system; wherein, The source storage system is used to: send first information to the destination storage system, where the first information includes metadata of files in the file system; The destination storage system is used to: receive the first information sent by the source storage system, and create a clone file system of the file system according to the first information; receive a first data read request, the first data read request including an identifier of a data block to be read; in response to the first data read request, send a first data acquisition request to the source storage system, the first data acquisition request including an identifier of the data block to be read; wherein the data block to be read belongs to data of a file in the clone file system; The source storage system is further used to: receive the first data acquisition request, and send the to-be-read data block to the destination storage system; The destination storage system is further used to: receive the to-be-read data block from the source storage system; wherein the data of the file in the clone file system in the destination storage system is obtained from the source storage system on demand based on a read data request; The destination storage system is further used to: determine a pre-read data block, and receive the pre-read data block from the source storage system; The source storage system is further used to send the pre-read data block to the destination storage system.

14. The system according to claim 13, characterized in that The first information does not include data of files in the file system.

15. The system according to claim 13 or 14, characterized in that The source storage system is further used to: send second information, wherein the second information includes metadata of the file whose file system has changed; The destination storage system is further configured to: receive second information sent by the source storage system, and update the clone file system according to the second information.

16. The system according to claim 13 or 14, characterized in that The destination storage system is further configured to: receive a write data request, and write data corresponding to the write data request into a storage space of the destination storage system.

17. The system according to claim 13 or 14, characterized in that The destination storage system is further used to determine that the to-be-read data block is not stored in the destination storage system.

18. The system according to claim 13 or 14, characterized in that The identifier of the data block to be read includes the identifier, the start address and the length of the file in the clone file system.

19. A computer program product, characterized in that The computer program product stores instructions, and when the instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 6.

Images