Method, device, equipment, medium and product for data migration

Abstract

The present disclosure provides a method, apparatus, device, medium and product for data migration. The method includes: obtaining first identification information of a file object, second identification information of a starting snapshot in a snapshot corresponding to the file object, and third identification information of a target snapshot in the snapshot; determining metadata difference information based on the first identification information, the second identification information and the third identification information; and performing data migration based on the determined metadata difference information. The method provided by the present disclosure can accurately obtain which physical storage areas in the file system have been written to data through the snapshot information of the file system, so that data migration can be performed based on this later. Compared with the prior art, the method provided by the present disclosure has the advantages of small data transmission volume, high efficiency, and less system resource occupation.

Description

Technical Field

[0001] The present disclosure relates to the field of data migration, and more specifically, to a method, apparatus, device, medium and product for data migration. Background Art

[0002] With the advancement of digitalization in all walks of life, users have increasingly higher requirements for high availability and high performance of digital storage systems. Therefore, many storage vendors have launched advanced functions such as file system data backup and hot and cold data tiering, so that in the event of uncontrollable factors (such as natural disasters) where the primary cluster suffers irreversible damage, the secondary cluster can still be used to continue to provide high availability, high performance and other services.

[0003] However, when implementing the advanced functions of the above two file systems, it involves the acquisition and transmission of a large number of file difference data, so how to obtain the difference data becomes the core of the above two functions.

[0004] At present, in order to realize advanced functions such as data backup and hot and cold data tiering of file systems, a method for data migration based on the acquired incremental data differences is often required in distributed file systems. The traditional method is to obtain data of the entire file size and then migrate the data. This traditional method will bring about problems such as a lot of invalid data transmission, low efficiency, and high system resource usage.

[0005] Therefore, a new approach to data migration is needed. Summary of the invention

[0006] In view of the above problems, the present disclosure provides a method for data migration. The method provided by the present disclosure can accurately obtain which physical storage areas in the file system are written with data through the snapshot information of the file system, so that data migration can be performed based on this in the future. Compared with the prior art, the method provided by the present disclosure has the advantages of small data transmission volume, high efficiency, and less system resource occupation.

[0007] An embodiment of the present disclosure provides a method for data migration, including: obtaining first identification information of a file object, second identification information of a starting snapshot in a snapshot corresponding to the file object, and third identification information of a target snapshot in the snapshot; determining metadata difference information based on the first identification information, the second identification information, and the third identification information; and performing data migration based on the determined metadata difference information.

[0008] According to an embodiment of the present disclosure, determining metadata difference information based on first identification information, second identification information and third identification information includes: obtaining snapshot structure information based on the first identification information, wherein the snapshot structure information includes the snapshot corresponding to the file object and the relationship between the snapshots; determining the relationship between the starting snapshot and the target snapshot based on the snapshot structure information; determining the metadata difference information based on the determined relationship between the starting snapshot and the target snapshot.

[0009] According to an embodiment of the present disclosure, the snapshot structure information reflects the topological structure between snapshots, and the determining of the metadata difference information based on the determined relationship between the starting snapshot and the target snapshot includes: based on the parent-child relationship between the starting snapshot and the target snapshot, performing an operation on the snapshot between the starting snapshot and the target snapshot and the target snapshot to determine the metadata difference information; based on the fact that the starting snapshot and the target snapshot are not in a parent-child relationship, determining a common parent node snapshot of the starting snapshot and the target snapshot, performing an operation on the snapshot between the parent node snapshot and the starting snapshot and the starting snapshot to obtain a first operation result, performing an operation on the snapshot between the parent node snapshot and the target snapshot and the target snapshot to obtain a second operation result, and performing an operation on the first operation result and the second operation result to determine the metadata difference information.

[0010] According to an embodiment of the present disclosure, the step of performing an operation on the snapshot between the starting snapshot and the target snapshot and the target snapshot to determine the metadata difference information includes: performing a bitwise OR operation on first metadata and second metadata to determine the metadata difference information, wherein the first metadata corresponds to the snapshot between the starting snapshot and the target snapshot, and the second metadata corresponds to the target snapshot; the step of performing an operation on the snapshot between the parent node snapshot and the starting snapshot and the starting snapshot to obtain the first operation result includes: performing a bitwise OR operation on third metadata and fourth metadata to obtain the first operation result, wherein the third metadata corresponds to the snapshot between the parent node snapshot and the starting snapshot, and the fourth metadata corresponds to the starting snapshot. The method further comprises: performing a bitwise OR operation on fifth metadata and sixth metadata to obtain the second operation result, wherein the fifth metadata corresponds to the snapshot between the parent node snapshot and the target snapshot, and the sixth metadata corresponds to the target snapshot; performing an operation on the first operation result and the second operation result to determine the metadata difference information, comprising: performing a bitwise OR operation on the first operation result and the second operation result to determine the metadata difference information; wherein each of the first to sixth metadata indicates whether data is written in the physical storage area corresponding to the file object.

[0011] An embodiment of the present disclosure provides a device for data migration, including: a first acquisition unit, which is configured to acquire first identification information of a file object, second identification information of a starting snapshot in a snapshot corresponding to the file object, and third identification information of a target snapshot in the snapshot; a first determination unit, which is configured to determine metadata difference information based on the first identification information, the second identification information and the third identification information; and an execution unit, which is configured to execute data migration based on the determined metadata difference information.

[0012] According to an embodiment of the present disclosure, the determination unit includes: a second acquisition unit, which is configured to obtain snapshot structure information based on the first identification information, wherein the snapshot structure information includes the snapshot corresponding to the file object and the relationship between the snapshots; a second determination unit, which is configured to determine the relationship between the starting snapshot and the target snapshot based on the snapshot structure information; and a third determination unit, which is configured to determine the metadata difference information based on the determined relationship between the starting snapshot and the target snapshot.

[0013] According to an embodiment of the present disclosure, wherein the snapshot structure information reflects the topological structure between snapshots, the third determination unit is configured to: based on the parent-child relationship between the starting snapshot and the target snapshot, perform an operation on the snapshot between the starting snapshot and the target snapshot and the target snapshot to determine the metadata difference information; based on the fact that the starting snapshot and the target snapshot are not in a parent-child relationship, determine the parent node snapshot common to the starting snapshot and the target snapshot, perform an operation on the snapshot between the parent node snapshot and the starting snapshot and the starting snapshot to obtain a first operation result, perform an operation on the snapshot between the parent node snapshot and the target snapshot and the target snapshot to obtain a second operation result, perform an operation on the first operation result and the second operation result to determine the metadata difference information.

[0014] According to an embodiment of the present disclosure, the step of performing an operation on the snapshot between the starting snapshot and the target snapshot and the target snapshot to determine the metadata difference information includes: performing a bitwise OR operation on first metadata and second metadata to determine the metadata difference information, wherein the first metadata corresponds to the snapshot between the starting snapshot and the target snapshot, and the second metadata corresponds to the target snapshot; the step of performing an operation on the snapshot between the parent node snapshot and the starting snapshot and the starting snapshot to obtain the first operation result includes: performing a bitwise OR operation on third metadata and fourth metadata to obtain the first operation result, wherein the third metadata corresponds to the snapshot between the parent node snapshot and the starting snapshot, and the fourth metadata corresponds to the starting snapshot. The method further comprises: performing a bitwise OR operation on fifth metadata and sixth metadata to obtain the second operation result, wherein the fifth metadata corresponds to the snapshot between the parent node snapshot and the target snapshot, and the sixth metadata corresponds to the target snapshot; performing an operation on the first operation result and the second operation result to determine the metadata difference information, comprising: performing a bitwise OR operation on the first operation result and the second operation result to determine the metadata difference information; wherein each of the first to sixth metadata indicates whether data is written in the physical storage area corresponding to the file object.

[0015] An embodiment of the present disclosure provides a device for data migration, including: a component for obtaining first identification information of a file object, second identification information of a starting snapshot in a snapshot corresponding to the file object, and third identification information of a target snapshot in the snapshot; a component for determining metadata difference information based on the first identification information, the second identification information, and the third identification information; and a component for performing data migration based on the determined metadata difference information.

[0016] An embodiment of the present disclosure provides a device for data migration, including: a processor, and a memory, wherein the memory stores computer executable instructions, and when the computer executable instructions are executed by the processor, the processor is prompted to execute any of the above methods.

[0017] An embodiment of the present disclosure provides a computer-readable recording medium storing computer-executable instructions, wherein when the computer-executable instructions are executed by a processor, the processor is prompted to execute any of the above methods.

[0018] An embodiment of the present disclosure provides a computer program product storing computer executable instructions, wherein the computer executable instructions, when executed by a processor, prompt the processor to perform any of the above methods.

[0019] The present disclosure provides a method, device, equipment, medium and product for data migration. The method provided by the present disclosure can accurately obtain which physical storage areas in the file system are written with data through the snapshot information of the file system, so that data migration can be performed based on this in the future. Compared with the prior art, the method provided by the present disclosure has the advantages of small data transmission volume, high efficiency, and less system resource occupation. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some exemplary embodiments of the present disclosure, and a person of ordinary skill in the art can obtain other drawings based on these drawings without creative work.

[0021] Figure 1 A schematic diagram of a directory entry of a file system according to an embodiment of the present disclosure is shown;

[0022] Figure 2 A flow chart of a method 200 for data migration according to an embodiment of the present disclosure is shown;

[0023] Figure 3 A possible schematic diagram showing a snapshot tree structure of a file object according to an embodiment of the present disclosure is shown;

[0024] Figure 4 A schematic diagram of obtaining metadata differences for the first situation described above according to an embodiment of the present disclosure is shown;

[0025] Figure 5 A schematic diagram of obtaining metadata differences for the second situation described above according to an embodiment of the present disclosure is shown;

[0026] Figure 6 A block diagram of an apparatus 600 for data migration according to an embodiment of the present disclosure is shown;

[0027] Figure 7 is a block diagram showing a device 700 for data migration according to an embodiment of the present disclosure;

[0028] Figure 8 A schematic diagram showing a recording medium according to the present disclosure is shown. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0030] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, similar words such as "one", "one" or "the" do not indicate quantity restrictions, but indicate that there is at least one. Similar words such as "include" or "comprise" mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Similar words such as "connect" or "connected" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0031] In the prior art, in order to realize advanced functions such as data backup and hot and cold data tiering of the file system, a method for data migration based on the acquired incremental data differences is often required in the distributed file system. The traditional method is to obtain the data of the entire file size and then migrate the data. This traditional method will bring about problems such as a lot of invalid data transmission, low efficiency, and high system resource usage.

[0032] In order to solve the above problems, the present disclosure provides a method for data migration. The method provided by the present disclosure can accurately obtain which physical storage areas in the file system are written with data through the snapshot information of the file system, so that data migration can be performed based on this in the future. This allows the method provided by the present disclosure to minimize system overhead and reduce the impact on the normal business of users while meeting the functions. Compared with the existing technology, it has the advantages of small data transmission volume, high efficiency, and less system resource occupation.

[0033] The method provided by the present disclosure will be described in detail below with reference to the accompanying drawings.

[0034] Figure 1 A schematic diagram of a directory entry of a file system according to an embodiment of the present disclosure is shown.

[0035] According to an embodiment of the present disclosure, a file system generally includes data and metadata.

[0036] As an example, metadata may be data about data used to describe data. Metadata may be information that mainly describes data attributes to support functions such as indicating storage location, historical data, resource search, file records, etc.

[0037] As an example, the metadata of a file system may include a directory entry (dentry) and an index node (inode). A directory entry can be used to describe the organization of files or folders in a file system, and an index node is the attribute information of a file or folder. In a file system, each file or folder corresponds to an index node, which is a unique number for the file or folder. The file system accesses the information of the index node through the index node number (inodeid).

[0038] For example, the directory items in the metadata record information such as the file name and parent directory. By connecting all directory items in the form of parent directory / child directory, a tree structure that reflects the organization information of the file system can be formed. For example, in some examples, such as Figure 1As shown, using the directory entries of the file system, a tree structure reflecting the organization information of the file system can be obtained, and then the corresponding index node information can be obtained using the index node numbers of the files or folders dir1, dir2, dir3, file1, and file2, so that the complete information of the file system can be read. For example, in this example, dir1, dir2, and dir3 represent folders, and file1 and file2 represent files.

[0039] According to an embodiment of the present disclosure, the metadata of the file system provided by the present disclosure may include multiple directory entry entries and multiple index node entries. For example, the directory entry entry is the aforementioned directory entry (dentry), and the index node entry is the aforementioned index node (inode).

[0040] The plurality of directory entry entries may include at least one native directory entry entry and at least one snapshot directory entry entry, that is, the plurality of directory entry entries are divided into two categories: native directory entry entries and snapshot directory entry entries. The native directory entry entries may record the current organization information of the file system. The snapshot directory entry entries may record the organization information in the file system that may not currently exist but exists when the snapshot is generated.

[0041] The plurality of index node entries may include at least one native index node entry and at least one snapshot index node entry, that is, the plurality of index node entries are divided into two categories: native index node entries and snapshot index node entries. The native index node entry may record the current file attribute information of the file system. The snapshot index node entry may record the file attribute information that may not currently exist in the file system but exists when the snapshot is generated.

[0042] The method for data migration provided by the present disclosure is mainly based on snapshot information corresponding to file objects (such as the above-mentioned files or folders) in the file system. Therefore, the method provided by the present disclosure will be described in detail below using snapshot information corresponding to file objects as an example.

[0043] Figure 2 A flow chart of a method 200 for data migration according to an embodiment of the present disclosure is shown. As an example, the method 200 may be executed by a processor of a file system. As another example, the method 200 may be executed by a processor in a file system specifically used for data migration.

[0044] Reference Figure 2 In step S210, first identification information of the file object, second identification information of a starting snapshot in the snapshots corresponding to the file object, and third identification information of a target snapshot in the snapshots may be obtained.

[0045] As an example, the file object may be any file object in a file system, such as a word document, a video document, etc. The first identification information of the file object may be any suitable information for identifying the file object, such as the name of the file object, etc.

[0046] As an example, there may be multiple snapshots corresponding to the file object in the file system. For example, based on actual usage scenarios, the file object may have multiple snapshots corresponding to it, such as snapshots s1, s2, s3, s4, etc.

[0047] As an example, the starting snapshot in the snapshot corresponding to the file object may be a starting snapshot specified by the user based on data migration needs, and the second identification information of the starting snapshot may be any suitable information for identifying the snapshot, such as the name of the snapshot, such as s2 mentioned above.

[0048] As an example, the target snapshot in the snapshot corresponding to the file object may be a target snapshot specified by the user based on data migration needs, and the third identification information of the target snapshot may be any suitable information for identifying the snapshot, such as the name of the snapshot, such as s4 above.

[0049] Continue to refer to Figure 2 In step S220, metadata difference information may be determined based on the first identification information, the second identification information, and the third identification information.

[0050] According to an embodiment of the present disclosure, determining metadata difference information based on the first identification information, the second identification information, and the third identification information may include the following steps:

[0051] Step 1: Obtain snapshot structure information based on the first identification information.

[0052] As an example, the snapshot structure information may include the snapshots corresponding to the file object and the relationship between the snapshots. For example, the snapshots corresponding to the file object may be the above-mentioned snapshots s1, s2, s3, s4, etc. The relationship between the snapshots may be a parent-child relationship, a non-parent-child relationship (e.g., a relationship of having a common parent node), etc.

[0053] As an example, the snapshot structure information may be pre-constructed snapshot structure information, or may be created in real time based on needs.

[0054] Step 2: Based on the snapshot structure information, the relationship between the start snapshot and the target snapshot may be determined.

[0055] As an example, the starting snapshot may be the above-mentioned snapshot s2, and the target snapshot may be the above-mentioned snapshot s4. Assuming that snapshots s2 and s4 are in a parent-child relationship in the snapshot structure information, based on the above content, it can be determined that the relationship between the starting snapshot and the target snapshot is a parent-child relationship.

[0056] As another example, the starting snapshot may be the above-mentioned snapshot s3, and the target snapshot may be the above-mentioned snapshot s4. Assuming that snapshots s3 and s4 have a common parent node relationship in the snapshot structure information, based on the above content, it can be determined that the relationship between the starting snapshot and the target snapshot is a common parent node relationship.

[0057] Step 3: Based on the determined relationship between the starting snapshot and the target snapshot, the metadata difference information may be determined.

[0058] According to an embodiment of the present disclosure, the snapshot structure information can reflect the topological structure between snapshots, and determining the metadata difference information based on the determined relationship between the starting snapshot and the target snapshot can include: based on the parent-child relationship between the starting snapshot and the target snapshot, performing operations on the snapshots between the starting snapshot and the target snapshot and the target snapshot to determine the metadata difference information.

[0059] As an example, performing an operation on the snapshot between the starting snapshot and the target snapshot and the target snapshot to determine the metadata difference information may include: performing a bitwise OR operation on first metadata and second metadata to determine the metadata difference information, wherein the first metadata corresponds to the snapshot between the starting snapshot and the target snapshot, and the second metadata corresponds to the target snapshot. Both the first metadata and the second metadata may indicate whether data is written in a physical storage area corresponding to the file object.

[0060] According to an embodiment of the present disclosure, the snapshot structure information can reflect the topological structure between snapshots, and determining the metadata difference information based on the determined relationship between the starting snapshot and the target snapshot can include: based on the fact that the starting snapshot and the target snapshot are not in a parent-child relationship, determining the parent node snapshot common to the starting snapshot and the target snapshot, performing an operation on the snapshot between the parent node snapshot and the starting snapshot and the starting snapshot to obtain a first operation result, performing an operation on the snapshot between the parent node snapshot and the target snapshot and the target snapshot to obtain a second operation result, and performing an operation on the first operation result and the second operation result to determine the metadata difference information.

[0061] As an example, performing an operation on the snapshot between the parent node snapshot and the starting snapshot and the starting snapshot to obtain the first operation result may include: performing a bitwise OR operation on third metadata and fourth metadata to obtain the first operation result, wherein the third metadata corresponds to the snapshot between the parent node snapshot and the starting snapshot, and the fourth metadata corresponds to the starting snapshot. Both the third metadata and the fourth metadata may indicate whether data is written into the physical storage area corresponding to the file object.

[0062] As an example, performing an operation on the snapshot between the parent node snapshot and the target snapshot and the target snapshot to obtain the second operation result may include: performing a bitwise OR operation on the fifth metadata and the sixth metadata to obtain the second operation result, wherein the fifth metadata corresponds to the snapshot between the parent node snapshot and the target snapshot, and the sixth metadata corresponds to the target snapshot. Both the fifth metadata and the sixth metadata may indicate whether data is written into the physical storage area corresponding to the file object.

[0063] As an example, performing an operation on the first operation result and the second operation result to determine the metadata difference information may include: performing a bitwise OR operation on the first operation result and the second operation result to determine the metadata difference information.

[0064] Return to reference Figure 2 In step S230, data migration may be performed based on the determined metadata difference information.

[0065] As an example, the metadata difference information obtained through the above steps S210-S220 can indicate which areas of the file object in the corresponding physical storage area have data written between the starting snapshot and the target snapshot, so that data migration can be performed based on the areas where the data is written. For example, the data corresponding to the area where the data is written can be obtained based on an existing appropriate data acquisition method, and then data migration can be performed on the obtained data.

[0066] The above has been combined Figure 1-2 The method for data migration provided by the present disclosure is described in detail. The method provided by the present disclosure can accurately obtain which physical storage areas in the file system are written with data through the snapshot information of the file system, so that data migration can be performed based on this later. This allows the method provided by the present disclosure to minimize system overhead and reduce the impact on the normal business of users while meeting the functions. Compared with the prior art, it has the advantages of small data transmission volume, high efficiency, and less system resource occupation.

[0067] In order to facilitate a better understanding of the above method provided by the present disclosure, the above method provided by the present disclosure will be further explained in the form of examples.

[0068] In a file storage system, for a file of a certain size, the file is usually divided into several objects for data storage and management.

[0069] For example, for a file of size 1G, the system will generally cut the file into objects of size 4M for data storage management. In this case, 1024 / 4 pieces of data of size 4M will be generated. When storing the 4M data on a physical disk, the disk will generally use sectors (usually 8k in size) as the minimum data unit for storage management. In this way, if the value of a metadata (such as one of the first to sixth metadata mentioned above) is, for example, 1100, it can indicate that data has been written to the 1st and 2nd sectors (i.e., 8k in size), while data has not been written to the 3rd and 4th sectors.

[0070] In the file storage system provided by the present disclosure, a snapshot header identifier head can be pre-set, which can indicate whether data is written into the physical storage area corresponding to the file object, and its default value can be all 0 (indicating that no data is written). When data is written into the corresponding physical storage area, the corresponding bit is set to 1; when data is not written into the corresponding physical storage area, the corresponding bit is set to 0.

[0071] For example, the snapshot header identifier head is 00000000 at the beginning. After the file object is modified, head may become 10000000. At this time, if a snapshot s1 is taken for the file object, the value of s1 is the value of head at this time, that is, 10000000. Then head is reset to 00000000.

[0072] Then, the file object is modified again, and the head may become 00000001. At this time, a snapshot s2 is taken for the file object, and the value of s2 is the value of the head at this time, that is, 00000001. Then the head is reset to 00000000. At this time, the difference between snapshot s2 and snapshot s1 is that data is written to the physical storage area corresponding to the 8th sector. Then, through the appropriate data acquisition method, the data corresponding to the area can be obtained, and then the obtained data can be migrated.

[0073] To this end, the data organization method of the file storage system provided by the present disclosure can be organized and managed through two key-value pairs (i.e., key-value, abbreviated as kv) management, and the two key-value pairs are metadata kv and data block kv, wherein the metadata kv is mainly used to indicate where there is data, and the data block kv is mainly used to indicate where there is data on the disk. It should be noted that the above organization and management method is only an example, and any appropriate organization and management method can be used based on actual needs.

[0074] The organization of metadata kv is shown in Table 1 below:

[0075] Table 1 Organization of metadata kv

[0076]

[0077] In Table 1, the Key field means the following:

[0078] type indicates the type of the kv, such as the metadata kv type (meta key). version indicates the version information of the key; when the organizational form of the key is modified, subsequent version upgrades are supported. poolid indicates that there are multiple data pools in the storage system, which are distinguished by different poolids. inodeid indicates different objects. sliceid indicates the slice identifier (id), which indicates the number of data slices of a large file. For example, if a file is 8M in size, the sliceid of the first 4M is 0, and the sliceid of the second 4M is 1.

[0079] In Table 1, the Value field means the following:

[0080] version indicates the version information of the value in kv to support subsequent version upgrades. size indicates the size of the object. num_bitmap indicates how many bits the object bitmap has. bitmap: In the file storage system, data is organized in 8k units; when a data block is written, the bitmap bit is set to 1, and the data read or snapshot operation described before or after is based on the bit to determine the data validity. slice_size indicates the size of an object, and the number of bytes occupied by the bitmap can be determined based on this value. snapid_cnt indicates how many snapshots exist for the object. The snapid-tpid correspondence indicates the correspondence between the snapshot identifier (snapid) and the timepointid described in Table 2 below; when reading a snapshot based on the snapid sent by the user, its timepointid value can be found in the metadata kv, and then spelled into the data block kv based on Table 2 below to find the specific snapshot data storage address, and then obtain the data.

[0081] The organization of the data block kv is shown in Table 2 below:

[0082] Table 2 Organization of data block kv

[0083]

[0084] In Table 2, the meaning of the Key field is as follows:

[0085] type indicates the type of the kv, such as the data block kv type (block key). version indicates the version information of the key; when the key organization form is modified, subsequent version upgrades are supported. poolid indicates that there are multiple data pools in the storage system, and different poolids are used to distinguish them. inodeid indicates different objects. sliceid indicates the slice identifier (id), which indicates the number of data slices of a large file. For example, if a file is 8M in size, the sliceid of the first 4M is 0, and the sliceid of the second 4M is 1. blockid indicates the number of 8k. timepointid indicates the version of the snapshot (for example, snapshot version 1 (i.e. s1), version 2 (i.e. s2), version 3 (i.e. s3), etc. introduced above); when there is no snapshot, the default value of timepointid is 1, which can be saved in the head as described above; at this time, for the head, when the modification operation of the file object is sent, the timepointid is 1; when the snapshot is created, the value of the head is given to s1, for example, then the timepoint of the head is 2; the value of timepointid is an auto-increment value; the correspondence between the snapshot identifier (snapid) and the timepointid can be saved in the metadata kv.

[0086] In Table 2, the Value field means the following:

[0087] version indicates the version information of the value in kv to support subsequent code upgrades. disk_addr indicates on which disk the 8k data corresponding to the kv is stored. disk_offset: indicates the offset position of the 8k data corresponding to the kv is stored on a certain disk.

[0088] Figure 3 A possible schematic diagram of a snapshot tree structure of a file object according to an embodiment of the present disclosure is shown.

[0089] Reference Figure 3 s1 to s11 represent the names of snapshots created by the user and also represent the time sequence of snapshot creation. s1-s6 are created in a direct parent-child relationship. When creating s7, the file system is first rolled back to s1, and then snapshot s7 is created. When creating s8, the file system is first rolled back to s3, and then snapshot s8 is created. The creation process of s9-s11 is similar. The final result is as follows Figure 3 The snapshot tree structure shown (as shown above combined with Figure 2 The snapshot structure information described by

[0090] Therefore, when obtaining metadata differences, there may be the following two situations: the first is to obtain the difference data between [s1-s6], where there is a direct parent-child relationship between s1-s6; the second is to obtain the difference data between [s8-s9], where s8-s9 have a common ancestor (i.e., a common parent node) but no direct parent-child relationship. Therefore, when obtaining metadata difference data, it is necessary to distinguish between the above two situations.

[0091] Figure 4 A schematic diagram of obtaining metadata differences for the first situation described above according to an embodiment of the present disclosure is shown. Figure 5 A schematic diagram of obtaining metadata differences for the second situation described above according to an embodiment of the present disclosure is shown.

[0092] Reference Figure 4 , when the user directly calls the application programming interface (API) through the command line or internal scheduled tasks (for example, Figure 4 The interface corresponding to the remote replication (REP) process or the client (CLIENT) process shown in S410 in FIG. Figure 3 When the metadata difference information between the s1 and s4 snapshot versions is generated, the start snapshot identifier (start_snapid, such as s1, such as the above combination Figure 2 The second identification information described), the identification of the target snapshot (end_snapid, such as s4, such as the above combination Figure 2 The third identification information described) and the file object's identifier oid (for example, the above combination Figure 2 The first identification information described) is sent to the server corresponding to the object data service process of the file storage system (such as Figure 4 S420 in the TDS server, and then the parameters are parsed through the TDS process on the TDS server (such as Figure 4 S430 and S440 in the .

[0093] Next, in step S450, the TDS process first obtains the snapshot tree structure corresponding to the file object according to oid (as described above in conjunction with Figure 2 The snapshot tree structure of the snapshot stores the parent-child relationship of the snapshots. Combining the snapshot tree structure, start_snapid and end_snapid, it can be determined that, for example, snapshots s1 and s4 are in a parent-child relationship, that is, the parent node of s4 is s3, the parent node of s3 is s2, and the parent node of s2 is s1.

[0094] In step S460, snapshot information start_snapinfo corresponding to the start snapshot start_snapid is obtained.

[0095] The TDS service provided by the present disclosure improves the method of obtaining the difference data of a file object between snapshots to further reduce the occupation of system resources. Specifically, if the user creates a snapshot (whether one or more) for the file object, and the file object has not undergone any data modification, the created snapshot information will not be transmitted to the TDS service. Only after the file object has undergone data modification, the snapshot information corresponding to the file object will be transmitted to the TDS service. It should be noted that the above-mentioned improvement may not be performed (i.e., it is not necessary to determine whether start_snapinfo and end_snapinfo exist), and the metadata difference information may be determined directly based on the above-mentioned first identification information, second identification information and third identification information.

[0096] In step S461, it is determined whether the snapshot information start_snapinfo exists. If not, it means that there is no data difference between start_snapid and end_snapid. At this time, in step S462, it is returned to the caller that there is no data difference. If it exists, it proceeds to step S470.

[0097] In step S470, snapshot information end_snapinfo corresponding to the target snapshot end_snapid is obtained.

[0098] In step S471, it is determined whether the snapshot information end_snapinfo exists. If it exists, the process proceeds to step S480 to obtain the snapshot information end_snapinfo. If it does not exist, the snapshot header identifier head information is used as the snapshot information end_snapinfo in step S472.

[0099] In step S490, the acquired snapshot information end_snapinfo is used as the current snapshot information current_snapinfo to start acquiring metadata difference information.

[0100] In step S491, metadata information corresponding to the current_snapinfo is obtained, such as the second metadata mentioned above, for example, its value may be 010100.

[0101] In step S492, the parent node snapshot information parent_snapinfo corresponding to the current_snapinfo is obtained (eg Figure 3 Then, in step S493, it is determined whether the parent node snapshot information is equal to the snapshot information start_snapinfo of the starting snapshot.

[0102] If they are equal, it means that all metadata difference information has been found. For example, if the metadata difference information between the s3 and s4 snapshot versions is found at the beginning. In this case, the value of the second metadata 010100 above indicates the metadata difference information between the s3 and s4 snapshot versions. In this way, only the physical storage area interval indicated by the value (i.e. the 2nd and 4th 8k, or storage intervals such as [8k-16k] and [24k-32k]) needs to be merged and returned to the caller.

[0103] If they are not equal, then in step S494, the parent node snapshot information is used as the current snapshot information current_snapinfo at this time. At this time, the metadata information corresponding to the parent node snapshot information parent_snapinfo is, for example, Figure 3 The information in s3 (as mentioned above combined Figure 2 The method then returns to step S492 until all metadata information between (start_snapinfo - end_snapinfo] is obtained, and then the intersection is taken, and finally the continuous interval is converted into the continuous interval and returned to the client, so that data migration can be performed based on the obtained interval later.

[0104] Reference Figure 5 , when the user directly calls the application programming interface (API) through the command line or internal scheduled tasks (for example, Figure 5 The interface corresponding to the remote replication (REP) process or the client (CLIENT) process shown in S510 in FIG. Figure 3 When the metadata difference information between the s8 and s9 snapshot versions is generated, the start snapshot identifier (start_snapid, such as s8, for example, the above combination Figure 2 The second identification information described), the identification of the target snapshot (end_snapid, such as s9, such as the above combination Figure 2 The third identification information described) and the file object's identifier oid (for example, the above combination Figure 2 The first identification information described) is sent to the server corresponding to the object data service process of the file storage system (such as Figure 5S520 in the TDS server, and then the parameters are parsed through the TDS process on the TDS server (such as Figure 5 S530 and S540 in the .

[0105] Next, in step S550, the TDS process first obtains the snapshot tree structure corresponding to the file object according to oid (as described above in conjunction with Figure 2 The snapshot tree structure of the snapshot stores not only the parent-child relationship of the snapshot but also the non-parent-child relationship of the snapshot (such as the relationship of having a common parent node). Combining the above snapshot tree structure, as well as start_snapid and end_snapid, it can be determined that, for example, snapshots s8 and s9 are not in a parent-child relationship, that is, s8 and s9 have a common parent node s1.

[0106] In step S560, snapshot information start_snapinfo corresponding to the start snapshot start_snapid is obtained.

[0107] The TDS service provided by the present disclosure improves the method of obtaining the difference data of a file object between snapshots to further reduce the occupation of system resources. Specifically, if the user creates a snapshot (whether one or more) for the file object, and the file object has not undergone any data modification, the created snapshot information will not be transmitted to the TDS service. Only after the file object has undergone data modification, the snapshot information corresponding to the file object will be transmitted to the TDS service. It should be noted that the above-mentioned improvement may not be performed (i.e., it is not necessary to determine whether start_snapinfo and end_snapinfo exist), and the metadata difference information may be determined directly based on the above-mentioned first identification information, second identification information and third identification information.

[0108] In step S561, it is determined whether the snapshot information start_snapinfo exists. If not, it means that there is no data difference between start_snapid and end_snapid. At this time, in step S562, it is returned to the caller that there is no data difference. If it exists, it proceeds to step S570.

[0109] In step S570, snapshot information end_snapinfo corresponding to the target snapshot end_snapid is obtained.

[0110] In step S571, it is determined whether the snapshot information end_snapinfo exists. If it exists, the process proceeds to step S580 to obtain the snapshot information end_snapinfo. If it does not exist, the snapshot header identifier head information is used as the snapshot information end_snapinfo in step S572.

[0111] In step S590, the common parent node snapshot information common_snapinfo corresponding to start_snapinfo and end_snapinfo is obtained, for example, the common parent node s1 of s8 and s9.

[0112] In step S591, all metadata information between (common_snapinfo - start_snapinfo] is obtained, and then the intersection (ie, bitwise OR operation) is taken to obtain, for example, b2. For the specific process, see the above description of Figure 4 Describe the process.

[0113] In step S592, all metadata information between (common_snapinfo - end_snapinfo] is obtained, and then the intersection (ie, bitwise OR operation) is taken to obtain, for example, b3. For the specific process, see the above description of Figure 4 Describe the process.

[0114] In step S593, the intersection of b2 and b3 is obtained to obtain b1, which is the metadata difference information between the final s8 and s9 snapshot versions. In this way, it is only necessary to merge the physical storage area intervals indicated by the values ​​corresponding to the metadata difference information and return them to the caller (i.e., step S594). In this way, data migration can be performed based on the obtained intervals later.

[0115] In addition to providing the above-mentioned method for data migration, the present disclosure also provides an apparatus for data migration. The above-mentioned description of the method is also applicable to the apparatus to be described below, unless otherwise explicitly stated.

[0116] Figure 6 A block diagram of an apparatus 600 for data migration according to an embodiment of the present disclosure is shown.

[0117] Reference Figure 6 , the device 600 may include a first acquiring unit 610 , a first determining unit 620 and an executing unit 630 .

[0118] According to an embodiment of the present disclosure, the first acquisition unit 610 may be configured to acquire first identification information of a file object, second identification information of a starting snapshot in snapshots corresponding to the file object, and third identification information of a target snapshot in the snapshots.

[0119] As an example, the file object may be any file object in a file system, such as a word document, a video document, etc. The first identification information of the file object may be any suitable information for identifying the file object, such as the name of the file object, etc.

[0120] As an example, there may be multiple snapshots corresponding to the file object in the file system. For example, based on actual usage scenarios, the file object may have multiple snapshots corresponding to it, such as snapshots s1, s2, s3, s4, etc.

[0121] As an example, the starting snapshot in the snapshot corresponding to the file object may be a starting snapshot specified by the user based on data migration needs, and the second identification information of the starting snapshot may be any suitable information for identifying the snapshot, such as the name of the snapshot, such as s2 mentioned above.

[0122] As an example, the target snapshot in the snapshot corresponding to the file object may be a target snapshot specified by the user based on data migration needs, and the third identification information of the target snapshot may be any suitable information for identifying the snapshot, such as the name of the snapshot, such as s4 above.

[0123] Continue to refer to Figure 6 The first determining unit 620 may be configured to determine metadata difference information based on the first identification information, the second identification information, and the third identification information.

[0124] According to an embodiment of the present disclosure, the first determination unit 620 may include a second acquisition unit, a second determination unit, and a third determination unit. It should be noted that the first determination unit, the second determination unit, and the third determination unit may be the same unit or different units. The first acquisition unit and the second acquisition unit may be the same unit or different units.

[0125] The second acquisition unit may be configured to obtain snapshot structure information based on the first identification information.

[0126] As an example, the snapshot structure information may include the snapshots corresponding to the file object and the relationship between the snapshots. For example, the snapshots corresponding to the file object may be the above-mentioned snapshots s1, s2, s3, s4, etc. The relationship between the snapshots may be a parent-child relationship, a non-parent-child relationship (e.g., a relationship of having a common parent node), etc.

[0127] As an example, the snapshot structure information may be pre-constructed snapshot structure information, or may be created in real time based on needs.

[0128] The second determining unit may be configured to determine a relationship between the start snapshot and the target snapshot based on the snapshot structure information.

[0129] As an example, the starting snapshot may be the above-mentioned snapshot s2, and the target snapshot may be the above-mentioned snapshot s4. Assuming that snapshots s2 and s4 are in a parent-child relationship in the snapshot structure information, based on the above content, it can be determined that the relationship between the starting snapshot and the target snapshot is a parent-child relationship.

[0130] As another example, the starting snapshot may be the above-mentioned snapshot s3, and the target snapshot may be the above-mentioned snapshot s4. Assuming that snapshots s3 and s4 have a common parent node relationship in the snapshot structure information, based on the above content, it can be determined that the relationship between the starting snapshot and the target snapshot is a common parent node relationship.

[0131] The third determining unit may be configured to determine the metadata difference information based on the determined relationship between the start snapshot and the target snapshot.

[0132] According to an embodiment of the present disclosure, the snapshot structure information can reflect the topological structure between snapshots, and the third determination unit can be configured to: based on the parent-child relationship between the starting snapshot and the target snapshot, perform operations on the snapshots between the starting snapshot and the target snapshot and the target snapshot to determine the metadata difference information.

[0133] As an example, performing an operation on the snapshot between the starting snapshot and the target snapshot and the target snapshot to determine the metadata difference information may include: performing a bitwise OR operation on first metadata and second metadata to determine the metadata difference information, wherein the first metadata corresponds to the snapshot between the starting snapshot and the target snapshot, and the second metadata corresponds to the target snapshot. Both the first metadata and the second metadata may indicate whether data is written in a physical storage area corresponding to the file object.

[0134] According to an embodiment of the present disclosure, the snapshot structure information can reflect the topological structure between snapshots, and the third determination unit can be configured to: based on the fact that the starting snapshot and the target snapshot are not in a parent-child relationship, determine the parent node snapshot common to the starting snapshot and the target snapshot, perform an operation on the snapshot between the parent node snapshot and the starting snapshot and the starting snapshot to obtain a first operation result, perform an operation on the snapshot between the parent node snapshot and the target snapshot and the target snapshot to obtain a second operation result, perform an operation on the first operation result and the second operation result to determine the metadata difference information.

[0135] As an example, performing an operation on the snapshot between the parent node snapshot and the starting snapshot and the starting snapshot to obtain the first operation result may include: performing a bitwise OR operation on third metadata and fourth metadata to obtain the first operation result, wherein the third metadata corresponds to the snapshot between the parent node snapshot and the starting snapshot, and the fourth metadata corresponds to the starting snapshot. Both the third metadata and the fourth metadata may indicate whether data is written into the physical storage area corresponding to the file object.

[0136] As an example, performing an operation on the snapshot between the parent node snapshot and the target snapshot and the target snapshot to obtain the second operation result may include: performing a bitwise OR operation on the fifth metadata and the sixth metadata to obtain the second operation result, wherein the fifth metadata corresponds to the snapshot between the parent node snapshot and the target snapshot, and the sixth metadata corresponds to the target snapshot. Both the fifth metadata and the sixth metadata may indicate whether data is written into the physical storage area corresponding to the file object.

[0137] As an example, performing an operation on the first operation result and the second operation result to determine the metadata difference information may include: performing a bitwise OR operation on the first operation result and the second operation result to determine the metadata difference information.

[0138] Return to reference Figure 6 The execution unit 630 may be configured to perform data migration based on the determined metadata difference information.

[0139] As an example, the metadata difference information obtained by the first acquisition unit 610 and the first determination unit 620 can indicate which areas of the file object in the corresponding physical storage area between the starting snapshot and the target snapshot have data written therein, so that data migration can be performed based on the areas where the data is written therein. For example, data corresponding to the area where the data is written can be obtained based on an existing appropriate data acquisition method, and then data migration can be performed on the obtained data.

[0140] The above has been combined Figure 1-6 The device for data migration provided by the present disclosure is described in detail. The device provided by the present disclosure can accurately obtain which physical storage areas in the file system are written with data through the snapshot information of the file system, so that data migration can be performed based on this later. This allows the device provided by the present disclosure to minimize system overhead and reduce the impact on the normal business of users while meeting the functions. Compared with the prior art, it has the advantages of small data transmission volume, high efficiency, and less system resource occupation.

[0141] As can be seen from the above description, the method and device for data migration provided by the present disclosure is a new solution for obtaining data differences. Through the snapshot information of the file system, the modifications that occurred in the file system within the snapshot interval can be accurately obtained. In this way, in data backup, when the master and slave nodes need to synchronize data, it is only necessary to call the method provided by the solution to obtain the metadata difference information of the file system, and then obtain the data of the corresponding physical storage area, and apply it to the slave end in the future to complete the synchronization of differential data. At the same time, in data migration, when migrating multiple versions of snapshot data, it is only necessary to call the method provided by the solution to complete the migration of differential data between snapshot versions.

[0142] The advantage of the technical solution provided by the present invention is that only the modified data is synchronized, the amount of synchronized data is small, the snapshot information records the modified data, the difference information is obtained quickly without affecting the native file system, and the difference information is also recorded in the database, so the incremental information is efficient when applied.

[0143] In addition to providing the above-mentioned method and apparatus for data migration, the present disclosure also provides a device and medium for data migration. The following will be described in conjunction with the accompanying drawings. The above description of the method and apparatus also applies to the device and medium to be described below, unless otherwise explicitly stated.

[0144] Figure 7 is a block diagram illustrating a device 700 for data migration according to an embodiment of the present disclosure.

[0145] See also Figure 7 , the device 700 may include a processor 701 and a memory 702. The processor 701 and the memory 702 may be connected via a bus 703.

[0146] The processor 701 can perform various actions and processes according to the program stored in the memory 702. Specifically, the processor 701 can be an integrated circuit chip with signal processing capabilities. The above processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The various methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., which can be an X86 architecture or an ARM architecture.

[0147] The memory 702 stores computer instructions, and the above-mentioned method for data migration is implemented when the computer instructions are executed by the processor 701. The memory 702 can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of exemplary but not limiting explanation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous connection dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM). It should be noted that the memory of the method described herein is intended to include but is not limited to these and any other suitable types of memory.

[0148] Figure 8 A schematic diagram showing a recording medium according to the present disclosure is shown.

[0149] like Figure 8 As shown, a computer executable instruction 810 is stored on the recording medium 800. When the computer executable instruction 810 is executed by a processor, the method according to the embodiment of the present disclosure described with reference to the above figures can be executed. The recording medium in the embodiment of the present disclosure can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous connection dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM). It should be noted that memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0150] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0151] In general, various example embodiments of the present disclosure may be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. When various aspects of the embodiments of the present disclosure are illustrated or described as block diagrams, flow charts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general purpose hardware or controllers or other computing devices, or some combination thereof as non-limiting examples.

[0152] The exemplary embodiments of the present invention described in detail above are merely illustrative, rather than restrictive. It should be understood by those skilled in the art that various modifications and combinations may be made to these embodiments or their features without departing from the principles and spirit of the present invention, and such modifications should fall within the scope of the present invention.

[0153] In the above specific embodiments, it can be seen that different features are combined together in the examples. This disclosure should not be understood as an intention that the example clauses have more features than the features explicitly mentioned in each clause. On the contrary, various aspects of the present disclosure may include fewer features than all the features of the disclosed single example clauses. Therefore, the following clauses should be considered to be included in the specification, where each clause itself can be used as a separate example. Although each subordinate clause can refer to a specific combination with one of the other clauses in these clauses, the aspects of the subordinate clauses are not limited to specific combinations. It should be understood that other example clauses may also include a combination of the subject matter of the subordinate clauses with any other subordinate clauses or independent clauses, or any combination of features with other subordinate clauses and independent clauses. Various aspects disclosed herein explicitly include these combinations, unless it is explicitly expressed or it can be easily inferred that a specific combination is not intended (for example, contradictory aspects, such as defining an element as both an insulator and a conductor). In addition, it is also intended that various aspects of the clauses may also be included in any other independent clauses, even if the clause is not directly subordinate to the independent clause.

[0154] The following numbered clauses describe implementation examples:

[0155] Clause 1. A method for data migration, comprising: obtaining first identification information of a file object, second identification information of a starting snapshot in a snapshot corresponding to the file object, and third identification information of a target snapshot in the snapshot; determining metadata difference information based on the first identification information, the second identification information, and the third identification information; and performing data migration based on the determined metadata difference information.

[0156] Clause 2. The method according to Clause 1, wherein determining metadata difference information based on first identification information, second identification information and third identification information includes: obtaining snapshot structure information based on the first identification information, wherein the snapshot structure information includes the snapshot corresponding to the file object and the relationship between the snapshots; determining the relationship between the starting snapshot and the target snapshot based on the snapshot structure information; and determining the metadata difference information based on the determined relationship between the starting snapshot and the target snapshot.

[0157] Clause 3. A method according to any of the above clauses, wherein the snapshot structure information reflects the topological structure between snapshots, and the determining of the metadata difference information based on the determined relationship between the starting snapshot and the target snapshot includes: based on the parent-child relationship between the starting snapshot and the target snapshot, performing an operation on the snapshot between the starting snapshot and the target snapshot and the target snapshot to determine the metadata difference information; based on the fact that the starting snapshot and the target snapshot are not in a parent-child relationship, determining a common parent node snapshot of the starting snapshot and the target snapshot, performing an operation on the snapshot between the parent node snapshot and the starting snapshot and the starting snapshot to obtain a first operation result, performing an operation on the snapshot between the parent node snapshot and the target snapshot and the target snapshot to obtain a second operation result, and performing an operation on the first operation result and the second operation result to determine the metadata difference information.

[0158] Clause 4. A method according to any of the above clauses, wherein the step of performing an operation on the snapshot between the starting snapshot and the target snapshot and the target snapshot to determine the metadata difference information comprises: performing a bitwise OR operation on first metadata and second metadata to determine the metadata difference information, wherein the first metadata corresponds to the snapshot between the starting snapshot and the target snapshot, and the second metadata corresponds to the target snapshot; the step of performing an operation on the snapshot between the parent node snapshot and the starting snapshot and the starting snapshot to obtain the first operation result comprises: performing a bitwise OR operation on third metadata and fourth metadata to obtain the first operation result, wherein the third metadata corresponds to the snapshot between the parent node snapshot and the starting snapshot, and the fourth metadata corresponds to the target snapshot. The starting snapshot corresponds; the snapshot between the parent node snapshot and the target snapshot is operated with the target snapshot to obtain the second operation result, including: performing a bitwise OR operation on the fifth metadata and the sixth metadata to obtain the second operation result, wherein the fifth metadata corresponds to the snapshot between the parent node snapshot and the target snapshot, and the sixth metadata corresponds to the target snapshot; the first operation result and the second operation result are operated to determine the metadata difference information, including: performing a bitwise OR operation on the first operation result and the second operation result to determine the metadata difference information; wherein each of the first to sixth metadata indicates whether data is written in the physical storage area corresponding to the file object.

[0159] Clause 5. A device for data migration, comprising: a first acquisition unit, which is configured to acquire first identification information of a file object, second identification information of a starting snapshot in a snapshot corresponding to the file object, and third identification information of a target snapshot in the snapshot; a first determination unit, which is configured to determine metadata difference information based on the first identification information, the second identification information and the third identification information; and an execution unit, which is configured to execute data migration based on the determined metadata difference information.

[0160] Clause 6. The device according to Clause 5, wherein the first determination unit comprises: a second acquisition unit, which is configured to obtain snapshot structure information based on the first identification information, wherein the snapshot structure information includes the snapshot corresponding to the file object and the relationship between the snapshots; a second determination unit, which is configured to determine the relationship between the starting snapshot and the target snapshot based on the snapshot structure information; and a third determination unit, which is configured to determine the metadata difference information based on the determined relationship between the starting snapshot and the target snapshot.

[0161] Clause 7. An apparatus according to any of the above clauses, wherein the snapshot structure information reflects the topological structure between snapshots, and the third determination unit is configured to: based on the parent-child relationship between the starting snapshot and the target snapshot, perform an operation on the snapshot between the starting snapshot and the target snapshot and the target snapshot to determine the metadata difference information; based on the fact that the starting snapshot and the target snapshot are not in a parent-child relationship, determine the parent node snapshot common to the starting snapshot and the target snapshot, perform an operation on the snapshot between the parent node snapshot and the starting snapshot and the starting snapshot to obtain a first operation result, perform an operation on the snapshot between the parent node snapshot and the target snapshot and the target snapshot to obtain a second operation result, and perform an operation on the first operation result and the second operation result to determine the metadata difference information.

[0162] Clause 8. An apparatus according to any of the above clauses, wherein the step of performing an operation on the snapshot between the starting snapshot and the target snapshot and the target snapshot to determine the metadata difference information comprises: performing a bitwise OR operation on the first metadata and the second metadata to determine the metadata difference information, wherein the first metadata corresponds to the snapshot between the starting snapshot and the target snapshot, and the second metadata corresponds to the target snapshot; the step of performing an operation on the snapshot between the parent node snapshot and the starting snapshot and the starting snapshot to obtain the first operation result comprises: performing a bitwise OR operation on the third metadata and the fourth metadata to obtain the first operation result, wherein the third metadata corresponds to the snapshot between the parent node snapshot and the starting snapshot, and the fourth metadata corresponds to the target snapshot. The starting snapshot corresponds; the snapshot between the parent node snapshot and the target snapshot is operated with the target snapshot to obtain the second operation result, including: performing a bitwise OR operation on the fifth metadata and the sixth metadata to obtain the second operation result, wherein the fifth metadata corresponds to the snapshot between the parent node snapshot and the target snapshot, and the sixth metadata corresponds to the target snapshot; the first operation result and the second operation result are operated to determine the metadata difference information, including: performing a bitwise OR operation on the first operation result and the second operation result to determine the metadata difference information; wherein each of the first to sixth metadata indicates whether data is written in the physical storage area corresponding to the file object.

[0163] Clause 9. A device for data migration, comprising: a component for obtaining first identification information of a file object, second identification information of a starting snapshot in a snapshot corresponding to the file object, and third identification information of a target snapshot in the snapshot; a component for determining metadata difference information based on the first identification information, the second identification information and the third identification information; and a component for performing data migration based on the determined metadata difference information.

[0164] Clause 10. A device for data migration, comprising: a processor, and a memory, wherein the memory stores computer executable instructions, and when the computer executable instructions are executed by the processor, the processor is prompted to perform the method as described in any one of the preceding clauses.

[0165] Clause 11. A computer-readable recording medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, cause the processor to perform the method as described in any of the preceding clauses.

[0166] Clause 12. A computer program product storing computer executable instructions, wherein the computer executable instructions, when executed by a processor, cause the processor to perform the method as described in any of the preceding clauses.

Claims

1. A method for data migration, comprising: Acquire first identification information of a file object, second identification information of a starting snapshot in the snapshots corresponding to the file object, and third identification information of a target snapshot in the snapshots; Determine metadata difference information based on the first identification information, the second identification information, and the third identification information, wherein the metadata is data used to describe data, and the metadata indicates whether data is written into a physical storage area corresponding to the file object; Based on the determined metadata difference information, perform data migration, The step of determining metadata difference information based on the first identification information, the second identification information, and the third identification information includes: Based on the first identification information, obtaining snapshot structure information, wherein the snapshot structure information includes snapshots corresponding to the file object and relationships between the snapshots; Determine a relationship between the start snapshot and the target snapshot based on the snapshot structure information; determining the metadata difference information based on the determined relationship between the starting snapshot and the target snapshot, The snapshot structure information reflects the topological structure between snapshots, and the determining of the metadata difference information based on the determined relationship between the starting snapshot and the target snapshot includes: Based on the parent-child relationship between the starting snapshot and the target snapshot, performing an operation on the snapshot between the starting snapshot and the target snapshot and the target snapshot to determine the metadata difference information; Based on the fact that the starting snapshot and the target snapshot are not in a parent-child relationship, determining a common parent node snapshot of the starting snapshot and the target snapshot, performing an operation on the snapshot between the parent node snapshot and the starting snapshot and the starting snapshot to obtain a first operation result, performing an operation on the snapshot between the parent node snapshot and the target snapshot and the target snapshot to obtain a second operation result, performing an operation on the first operation result and the second operation result to determine the metadata difference information, in, The step of performing an operation on the snapshot between the starting snapshot and the target snapshot and the target snapshot to determine the metadata difference information comprises: performing a bitwise OR operation on the first metadata and the second metadata to determine the metadata difference information, wherein the first metadata corresponds to the snapshot between the starting snapshot and the target snapshot, and the second metadata corresponds to the target snapshot; The performing an operation on the snapshot between the parent node snapshot and the starting snapshot and the starting snapshot to obtain the first operation result includes: performing a bitwise OR operation on third metadata and fourth metadata to obtain the first operation result, wherein the third metadata corresponds to the snapshot between the parent node snapshot and the starting snapshot, and the fourth metadata corresponds to the starting snapshot; The step of performing an operation on the snapshot between the parent node snapshot and the target snapshot and the target snapshot to obtain the second operation result includes: performing a bitwise OR operation on the fifth metadata and the sixth metadata to obtain the second operation result, wherein the fifth metadata corresponds to the snapshot between the parent node snapshot and the target snapshot, and the sixth metadata corresponds to the target snapshot; The performing an operation on the first operation result and the second operation result to determine the metadata difference information includes: performing a bitwise OR operation on the first operation result and the second operation result to determine the metadata difference information; Each of the first to sixth metadata indicates whether data is written into the physical storage area corresponding to the file object.

2. A device for data migration, comprising: A first acquisition unit is configured to acquire first identification information of a file object, second identification information of a starting snapshot in the snapshots corresponding to the file object, and third identification information of a target snapshot in the snapshots; A first determining unit is configured to determine metadata difference information based on the first identification information, the second identification information, and the third identification information, wherein the metadata is data used to describe data, and the metadata indicates whether the data is written in the physical storage area corresponding to the file object; an execution unit, configured to execute data migration based on the determined metadata difference information, Wherein, the first determining unit includes: A second obtaining unit is configured to obtain snapshot structure information based on the first identification information, wherein the snapshot structure information includes snapshots corresponding to the file object and relations between the snapshots; a second determining unit configured to determine a relationship between the start snapshot and the target snapshot based on the snapshot structure information; a third determining unit configured to determine the metadata difference information based on the determined relationship between the starting snapshot and the target snapshot, The snapshot structure information reflects the topological structure between snapshots, and the third determining unit is configured to: Based on the parent-child relationship between the starting snapshot and the target snapshot, performing an operation on the snapshot between the starting snapshot and the target snapshot and the target snapshot to determine the metadata difference information; Based on the fact that the starting snapshot and the target snapshot are not in a parent-child relationship, determining a common parent node snapshot of the starting snapshot and the target snapshot, performing an operation on the snapshot between the parent node snapshot and the starting snapshot and the starting snapshot to obtain a first operation result, performing an operation on the snapshot between the parent node snapshot and the target snapshot and the target snapshot to obtain a second operation result, performing an operation on the first operation result and the second operation result to determine the metadata difference information, in, The step of performing an operation on the snapshot between the starting snapshot and the target snapshot and the target snapshot to determine the metadata difference information comprises: performing a bitwise OR operation on the first metadata and the second metadata to determine the metadata difference information, wherein the first metadata corresponds to the snapshot between the starting snapshot and the target snapshot, and the second metadata corresponds to the target snapshot; The performing an operation on the snapshot between the parent node snapshot and the starting snapshot and the starting snapshot to obtain the first operation result includes: performing a bitwise OR operation on third metadata and fourth metadata to obtain the first operation result, wherein the third metadata corresponds to the snapshot between the parent node snapshot and the starting snapshot, and the fourth metadata corresponds to the starting snapshot; The step of performing an operation on the snapshot between the parent node snapshot and the target snapshot and the target snapshot to obtain the second operation result includes: performing a bitwise OR operation on the fifth metadata and the sixth metadata to obtain the second operation result, wherein the fifth metadata corresponds to the snapshot between the parent node snapshot and the target snapshot, and the sixth metadata corresponds to the target snapshot; The performing an operation on the first operation result and the second operation result to determine the metadata difference information includes: performing a bitwise OR operation on the first operation result and the second operation result to determine the metadata difference information; Each of the first to sixth metadata indicates whether data is written into the physical storage area corresponding to the file object.

3. A device for data migration, comprising: A component for obtaining first identification information of a file object, second identification information of a starting snapshot in the snapshots corresponding to the file object, and third identification information of a target snapshot in the snapshots; A component for determining metadata difference information based on the first identification information, the second identification information and the third identification information, wherein the metadata is data used to describe data, and the metadata indicates whether the data is written in the physical storage area corresponding to the file object; means for performing data migration based on the determined metadata difference information, The step of determining metadata difference information based on the first identification information, the second identification information, and the third identification information includes: Based on the first identification information, obtaining snapshot structure information, wherein the snapshot structure information includes snapshots corresponding to the file object and relationships between the snapshots; Determine a relationship between the start snapshot and the target snapshot based on the snapshot structure information; determining the metadata difference information based on the determined relationship between the starting snapshot and the target snapshot, The snapshot structure information reflects the topological structure between snapshots, and the determining of the metadata difference information based on the determined relationship between the starting snapshot and the target snapshot includes: Based on the parent-child relationship between the starting snapshot and the target snapshot, performing an operation on the snapshot between the starting snapshot and the target snapshot and the target snapshot to determine the metadata difference information; Based on the fact that the starting snapshot and the target snapshot are not in a parent-child relationship, determining a common parent node snapshot of the starting snapshot and the target snapshot, performing an operation on the snapshot between the parent node snapshot and the starting snapshot and the starting snapshot to obtain a first operation result, performing an operation on the snapshot between the parent node snapshot and the target snapshot and the target snapshot to obtain a second operation result, performing an operation on the first operation result and the second operation result to determine the metadata difference information, in, The step of performing an operation on the snapshot between the starting snapshot and the target snapshot and the target snapshot to determine the metadata difference information comprises: performing a bitwise OR operation on the first metadata and the second metadata to determine the metadata difference information, wherein the first metadata corresponds to the snapshot between the starting snapshot and the target snapshot, and the second metadata corresponds to the target snapshot; The performing an operation on the snapshot between the parent node snapshot and the starting snapshot and the starting snapshot to obtain the first operation result includes: performing a bitwise OR operation on third metadata and fourth metadata to obtain the first operation result, wherein the third metadata corresponds to the snapshot between the parent node snapshot and the starting snapshot, and the fourth metadata corresponds to the starting snapshot; The step of performing an operation on the snapshot between the parent node snapshot and the target snapshot and the target snapshot to obtain the second operation result includes: performing a bitwise OR operation on the fifth metadata and the sixth metadata to obtain the second operation result, wherein the fifth metadata corresponds to the snapshot between the parent node snapshot and the target snapshot, and the sixth metadata corresponds to the target snapshot; The performing an operation on the first operation result and the second operation result to determine the metadata difference information includes: performing a bitwise OR operation on the first operation result and the second operation result to determine the metadata difference information; Each of the first to sixth metadata indicates whether data is written into the physical storage area corresponding to the file object.

4. A device for data migration, comprising: processor, and A memory storing computer executable instructions, which when executed by a processor cause the processor to perform the method of claim 1.

5. A computer-readable recording medium storing computer-executable instructions, wherein: The computer executable instructions, when executed by a processor, cause the processor to perform the method of claim 1 .

6. A computer program product storing computer executable instructions, wherein: The computer executable instructions, when executed by a processor, cause the processor to perform the method of claim 1 .

Images