Data migration method, first storage device, storage medium and program product

Abstract

The invention provides a data migration method which comprises the steps that target data are determined in response to a data migration instruction, the data migration instruction indicates that the target data are migrated to a target storage position from a source storage position, the source storage position is located in one of local storage equipment and remote storage equipment, and the target storage position is located in the other of local storage equipment and remote storage equipment; the destination storage position is located at the other one of the local storage device and the remote storage device, and the remote storage device is mounted on the local storage device; based on the metadata information of the target data, the target application program is utilized to migrate the target data from the source storage position to the target storage position, the target application program is a user state application program deployed in a local storage device, and the metadata information is used for positioning the target data. The invention further provides a first storage device, a storage medium and a program product.

Description

Technical Field

[0001] This application relates to the technical fields of data processing and storage, and more particularly, to a data migration method, a first storage device, a storage medium, and a program product. Background Art

[0002] With the rapid development of information technology, the amount of data has grown explosively, and the demand of enterprises and individuals for data storage and management has also increased day by day. Exemplarily, storage devices can be used to centrally store and manage data.

[0003] However, with the change of data storage requirements and the replacement of technologies, users may need to migrate data between different storage devices. In the process of implementing the inventive concept of this application, the inventors found that related data migration methods have problems such as high data migration cost and low data migration efficiency. Summary of the Invention

[0004] In view of the above problems, this application provides a data migration method, a first storage device, a storage medium, and a program product.

[0005] According to one aspect of this application, a data migration method is provided, including: in response to a data migration instruction, determining target data, where the data migration instruction indicates migrating the target data from a source storage location to a destination storage location, the source storage location is in one of a local storage device and a remote storage device, the destination storage location is in the other of the local storage device and the remote storage device, and the remote storage device is mounted on the local storage device; based on the metadata information of the target data, using a target application program to migrate the target data from the source storage location to the destination storage location, where the target application program is a user-mode application program deployed on the local storage device, and the metadata information is used to locate the target data.

[0006] Another aspect of this application provides a first storage device, including: a storage module for storing and managing data; a communication module for implementing a communication connection between the first storage device and a second storage device, the second storage device being mounted on the first storage device; a data migration module for determining target data in response to a data migration instruction and migrating the target data from a source storage location to a destination storage location, where the source storage location is in one of the first storage device and the second storage device, and the destination storage location is in the other of the first storage device and the second storage device.

[0007] Another aspect of this application provides an electronic device, including: one or more processors; a memory for storing one or more computer programs, where the above one or more processors execute the above one or more computer programs to implement the steps of the above method.

[0008] Another aspect of the present application also provides a computer-readable storage medium, on which a computer program or instruction is stored, and when the computer program or instruction is executed by a processor, the steps of the above method are implemented.

[0009] Another aspect of the present application also provides a computer program product, including a computer program or instruction, and when the computer program or instruction is executed by a processor, the steps of the above method are implemented.

[0010] According to the embodiments of the present application, since the target application program is deployed on a local storage device, there is no need to deploy additional independent servers / virtual machines / containers, etc. during data migration, nor is it necessary to purchase third-party data migration software additionally, which can effectively save hardware resources and migration costs. Moreover, when the target application program accesses the local storage device, it can bypass the Network File System (NFS) standard protocol and directly call the data interface of the local storage device to implement operations such as reading and writing local files, thereby improving data migration efficiency and saving central processing unit resources. In addition, since the target application program runs entirely in the user space and avoids modifying the storage kernel code, the failure of the target application program only affects its own process, thereby reducing the risk of system crashes caused by migration operations and simplifying the deployment and maintenance processes. Thus, the data migration method provided by the embodiments of the present application can at least partially overcome problems such as high data migration costs and low data migration efficiency, and is well applicable to data migration scenarios with strict requirements for efficiency and stability, such as enterprise-level storage and cloud computing. Description of the Drawings

[0011] Through the following description of the embodiments of the present application with reference to the drawings, the above content and other objects, features and advantages of the present application will become clearer.

[0012] Figure 1 The application scenario diagram of the data migration method, the first storage device, the storage medium and the program product according to the embodiments of the present application is shown.

[0013] Figure 2 The flowchart of the data migration method according to the embodiments of the present application is shown.

[0014] Figure 3 The schematic diagram of the principle of the data migration method according to the embodiments of the present application is shown.

[0015] Figure 4 The schematic diagram of the data migration method according to an embodiment of the present application is shown.

[0016] Figure 5 The schematic diagram of the data migration method according to another embodiment of the present application is shown.

[0017] Figure 6 Shows a schematic diagram of multi-threaded parallel migration according to an embodiment of the present application.

[0018] Figure 7 Shows a structural block diagram of a first storage device according to an embodiment of the present application.

[0019] Figure 8 Shows a structural block diagram of an electronic device to which a data migration method can be applied according to an embodiment of the present application. Detailed implementation manners

[0020] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present application. In the following detailed description, for the sake of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present application. However, obviously, one or more embodiments can also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present application.

[0021] The terms used herein are merely for describing specific embodiments and are not intended to limit the present application. The terms "including", "comprising" and the like used herein indicate the presence of the described features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

[0022] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0023] In the case of using expressions such as "at least one of A, B, and C", generally, it should be interpreted according to the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but not be limited to a system having only A, having only B, having only C, having A and B, having A and C, having B and C, and / or having A, B, and C).

[0024] With the rapid development of information technology, the amount of data has grown explosively, and the demand of enterprises and individuals for data storage and management has also increased day by day. Exemplarily, a storage device can be used to centrally store and manage data.

[0025] As an example, a Network Attached Storage (NAS) device can be used to centrally store and manage data. A Network Attached Storage device is a dedicated data storage server, typically including storage devices and embedded system software. The Network Attached Storage device can provide centralized and easily managed storage services over a network, supporting functions such as file sharing and backup for multiple devices.

[0026] However, with the change of data storage requirements and the update of technology, users may need to migrate data between different storage devices. In implementing the inventive concept of this application, the inventors found that there are problems such as high data migration cost and low data migration efficiency in related data migration methods.

[0027] In one embodiment, data migration can be performed between a local Network Attached Storage device and a remote Network Attached Storage device. As an example, data migration can be achieved through a third-party dedicated data migration software. However, dedicated data migration software usually requires additional payment and also requires an additional server to deploy the data migration software, resulting in a relatively high cost. As another example, a virtual machine or a container can also be deployed on one of the storage devices participating in the data migration (for example, it can be a local Network Attached Storage device), and then the data migration software can be deployed in the virtual machine or the container. However, virtual machines will occupy a large amount of CPU and memory resources, which are likely to affect the I / O (Input / Output) performance of the storage device, while the management of containers is relatively complex, and data migration software failures may affect the kernel, resulting in a relatively low migration efficiency. In addition, the above two data migration methods need to be implemented through a file sharing protocol when accessing the local storage device and the remote storage device.

[0028] An embodiment of this application provides a data migration method, a first storage device, a storage medium, and a program product. The data migration method includes: in response to a data migration instruction, determining target data, where the data migration instruction indicates migrating the target data from a source storage location to a destination storage location, the source storage location is located in one of a local storage device and a remote storage device, the destination storage location is located in the other of the local storage device and the remote storage device, and the remote storage device is mounted on the local storage device; based on the metadata information of the target data, using a target application program to migrate the target data from the source storage location to the destination storage location, where the target application program is a user-mode application program deployed on the local storage device, and the metadata information is used to locate the target data.

[0029] Figure 1 The application scenario diagram of the data migration method, the first storage device, the storage medium, and the program product according to the embodiment of this application is shown.

[0030] As shown Figure 1 in FIG. 1, the application scenario 100 according to an embodiment of the present application may include a local storage device 101, a remote storage device 102, and a network 103. Both the local storage device 101 and the remote storage device 102 may store data, and the data may be stored in the local storage device 101 and / or the remote storage device 102 in the form of files. For example, the data stored in the local storage device 101 may be referred to as local files, and the data stored in the remote storage device 102 may be referred to as remote files.

[0031] It should be noted that the local storage device 101 may be understood as a storage device that can be directly connected to a user computer system. For example, the local storage device 101 may be directly connected to the user computer system through a physical connection, and the user computer system can directly access and manage the local storage device 101 without using network protocols or remote access technologies. The remote storage device 102 may be understood as a storage device that is not directly connected to the user computer system but can be connected to the user computer system through a network (such as the Internet, a wide area network, a VPN, etc.). The user computer system can remotely access and manage the remote storage device 102 through network protocols.

[0032] Exemplarily, the local storage device 101 and the remote storage device 102 may be heterogeneous devices to each other. In other words, the local storage device 101 and the remote storage device 102 may be storage devices of different structures and different standards from different manufacturers.

[0033] In one embodiment, the local storage device 101 and the remote storage device 102 may be selected as Network Attached Storage (NAS) devices. A Network Attached Storage device may be understood as a special dedicated data storage server, which may include storage devices (such as disk arrays, CD / DVD drives, tape drives, or removable storage media) and embedded system software, and can provide cross-platform file sharing functions. For example, the system software of the local storage device 101 may be referred to as the local file system, and the system software of the remote storage device 102 may be referred to as the remote file system.

[0034] The Network Attached Storage device can be directly connected to a disk storage array through a network to allow users to read data on the network. Exemplarily, a user can store files on the hard disk of the Network Attached Storage device and perform operations such as creating folders, setting permissions, and data backup through the management interface (such as a graphical user interface or a command prompt input interface) provided by the Network Attached Storage device.

[0035] As shown Figure 1As shown, the local storage device 101 and the remote storage device 102 can be connected via the network 103. For example, the local storage device 101 and the remote storage device 102 can communicate via a standard protocol, such as communicating using the Network File System (NFS) standard protocol or the Common Internet File System (CIFS) standard protocol, etc. Exemplarily, the network file system standard protocol, i.e., the NFS protocol, can allow users to access files on the file systems of other storage devices on the network as if they were accessing local files.

[0036] As Figure 1 shown, data can be migrated between the local storage device 101 and the remote storage device 102 in the form of files. For example, the data stored in the local storage device 101 can be migrated to the remote storage device 102. Or, for example, the data stored in the remote storage device 102 can be migrated to the local storage device 101.

[0037] In one embodiment, the scenario of data migration can be that the user has purchased a new storage device and needs to transfer the data stored on the old storage device to the new storage device. In another embodiment, the scenario of data migration can also be that the user, due to actual needs (such as business requirements), migrates the data stored on one storage device to another storage device.

[0038] Figure 2 The flowchart of the data migration method according to an embodiment of the present application is shown.

[0039] As Figure 2 shown, the method 200 includes operation S210 to operation S220.

[0040] In operation S210, in response to a data migration instruction, the target data is determined, where the data migration instruction indicates migrating the target data from a source storage location to a destination storage location, the source storage location is in one of the local storage device and the remote storage device, the destination storage location is in the other of the local storage device and the remote storage device, and the remote storage device is mounted on the local storage device.

[0041] In operation S220, based on the metadata information of the target data, the target data is migrated from the source storage location to the destination storage location using the target application, where the target application is a user-mode application deployed on the local storage device, and the metadata information is used to locate the target data.

[0042] The data migration method provided by the embodiments of this application can be applied to data migration between a local storage device and a remote storage device. Among them, the remote storage device can be mounted on the local storage device. The mounting operation can attach the file system of the remote storage device to a certain directory in the file system of the local storage device.

[0043] In one embodiment, the user can use a mounting command (such as the mount command in Linux) to mount the remote storage device under a certain directory of the local storage device. This directory can be called the "mount point" (Mount Point). After mounting, the user can remotely access the data in the remote storage device through the mount point in the local storage device, just like accessing a local hard disk.

[0044] For example, the IP address of the remote storage device can be 100.60.40.23, and there is a shared folder / shared on the remote storage device. The user can mount the folder / shared on the / mnt / nas directory of the local storage device. After mounting, the user can remotely access the files in the / shared folder on the remote storage device through the / mnt / nas directory of the local storage device.

[0045] Only as an example, the mounting operation can be implemented based on the NFS protocol. For example, a libNFS client can be deployed on the local storage device. This software runs in the user space and can mount the file system of the remote storage device to the local storage device, and then access the remote storage through the NFS protocol. libNFS is a client library for accessing NFS file systems, aiming to access NFS shares across the network. It provides a set of API functions that allow direct interaction with the NFS server in the application without going through the operating system's mounting mechanism. This enables flexible access and operation of NFS shared resources without changing the operating system's mounting configuration. Among them, a library is a set of pre-compiled reusable code modules that provide specific functions (such as mathematical calculations, file operations, network communications, etc.) for programs to call to simplify development. Exemplarily, the process of the mounting operation can be as follows:

[0046] 1) According to the obtained shared IP and path, such as 100.60.40.23: / share1, the nfs_parse_url_full function can be called through the interface of libnfs to parse the URL (Uniform Resource Locator) of the remote nfs. This function will parse the URL and fill the struct nfs_mount_data structure, which contains all the information required for mounting;

[0047] 2) Mount the remote NFS to the local. For example, you can call the nfs_mount function through the linbnfs interface to mount the remote NFS to the local. This function requires the NFS context, the parsed URL, and the mount data as parameters.

[0048] In response to a data migration instruction, the target data can be determined. Herein, the data migration instruction indicates migrating the target data from a source storage location to a destination storage location. In one embodiment, the data migration instruction can be triggered based on a user's data migration operation. For example, the user can trigger the data migration instruction based on a relevant human-machine interface. The user can specify the source storage location and the destination storage location as needed.

[0049] As an example, when the source storage location is a local storage device and the destination storage location is a remote storage device, the data migration instruction indicates migrating the target data from the local storage device to the remote storage device. As another example, when the source storage location is a remote storage device and the destination storage location is a local storage device, the data migration instruction indicates migrating the target data from the remote storage device to the local storage device.

[0050] Metadata can be used to describe information about the attributes of data to support functions such as indicating the storage location, historical data, resource lookup, file recording, etc. For example, the metadata information of a file can include information such as the file name, size, permissions, modification time, etc.

[0051] The metadata information of the target data can be used to locate the target data. For example, it can include the directory, file name, file size, modification time, etc. of the target data. Based on the metadata information of the target data, the target application can be used to migrate the target data from the source storage location to the destination storage location. Herein, the target application is a user-mode application deployed in the local storage device.

[0052] User Mode can be understood as a restricted operating environment designed by the operating system for applications, forming a privilege isolation with Kernel Mode. In User Mode, the program can only access user space memory and cannot directly operate the hardware or execute privileged instructions (such as modifying system configurations, accessing the memory of other processes). For example, the target application runs in User Mode and needs to request kernel services through system calls to complete operations such as file reading and writing. Kernel Mode can be understood as a privileged mode in which the operating system kernel runs, having full control over all hardware resources and system memory. In Kernel Mode, the kernel can execute privileged instructions (such as setting interrupts, managing memory paging), directly schedule devices such as the CPU and disk. For example, functions such as process scheduling, interrupt handling, and device drivers of the operating system all run in Kernel Mode.

[0053] Figure 3 The schematic diagram of the data migration method according to an embodiment of the present application is shown.

[0054] As Figure 3 shown, the target application 301 is a user-mode application deployed on the local storage device 101.

[0055] As Figure 3 shown, the target application 301 can directly access and manage the local file system of the local storage device 101 by bypassing the NFS protocol 302, and remotely access and manage the remote file system of the remote storage device 102 through the NFS protocol 302. Thus, the target application 301 can migrate local files to the remote storage device 102 and / or migrate remote files to the local storage device 101.

[0056] It can be understood that since the target application is deployed on the local storage device, no additional independent server / virtual machine / container, etc. needs to be deployed during data migration, nor does it need to purchase third-party data migration software additionally, which can effectively save hardware resources and migration costs. Moreover, when the target application accesses the local storage device, it can bypass the NFS protocol and directly call the data interface of the local storage device to implement operations such as reading and writing local files, thereby improving data migration efficiency and saving central processing unit resources. In addition, since the target application runs entirely in the user mode, modifying the storage kernel code is avoided, and the failure of the target application only affects its own process, thereby reducing the risk of system crash caused by migration operations and simplifying the deployment and maintenance processes. Thus, the data migration method provided by the embodiment of the present application can at least partially overcome problems such as high data migration cost and low data migration efficiency, and can be well applied to data migration scenarios with strict requirements for efficiency and stability, such as enterprise-level storage and cloud computing.

[0057] According to an embodiment of the present application, the data migration instruction includes source access information and destination access information. The source access information represents the path of the source storage location, and the destination access information represents the path of the destination storage location; in response to the data migration instruction, determining the target data includes: respectively determining first metadata information corresponding to the source storage location and second metadata information corresponding to the destination storage location according to the source access information and the destination access information; in the case where the second metadata information meets the preset conditions, determining the first metadata information as the metadata information, and determining the target data based on the metadata information; in the case where the second metadata information does not meet the preset conditions, determining the metadata information based on the difference between the first metadata information and the second metadata information, and determining the target data based on the metadata information.

[0058] In one embodiment, a human-computer interaction interface for user operation can be provided, enabling the user to input the path of the source storage location and the path of the destination storage location in the human-computer interaction interface. The human-computer interaction interface can be, for example, a graphical user interface for a target application or an input interface of a command prompt. The target application can determine first metadata information corresponding to the source storage location according to the source access information. The target application can determine second metadata information corresponding to the destination storage location according to the destination access information. Among them, the first metadata information and the second metadata information can be stored in a preset database for subsequent data migration operations.

[0059] Taking the case where the source storage location is a local storage device and the destination storage location is a remote storage device as an example, the target application can traverse the metadata of all directories and files at the source path in the local storage device according to the path of the source storage location specified by the user (hereinafter simply referred to as the source path) to obtain the first metadata information. The target application can traverse the metadata of all directories and files at the destination path in the remote storage device according to the path of the destination storage location specified by the user (hereinafter simply referred to as the destination path) to obtain the second metadata information.

[0060] Taking the case where the source storage location is a remote storage device and the destination storage location is a local storage device as an example, the target application can traverse the metadata of all directories and files at the source path in the remote storage device according to the path of the source storage location specified by the user (hereinafter simply referred to as the source path) to obtain the first metadata information. The target application can traverse the metadata of all directories and files at the destination path in the local storage device according to the path of the destination storage location specified by the user (hereinafter simply referred to as the destination path) to obtain the second metadata information.

[0061] As an example, the process of traversing the metadata information of all directories and files at a certain path (such as / mnt / nas / photos) in the local storage device can be as follows:

[0062] 1. The target application opens the local root directory (i.e., / mnt / nas / photos) by calling the local lib library interface. Among them, the local lib library runs in the user mode of the local storage device and can provide the basic ability to read and write the file system of the local storage device to ensure that the target application can efficiently process the data in the local storage. The local lib library allows the target application to directly interact with the local storage device (such as hard disk, SSD, etc.) and perform operations such as file creation, reading, writing, and deletion;

[0063] 2. Traverse all subdirectories under the above local root directory recursively and store them in a preset database;

[0064] 3. The traversal of subdirectories ends;

[0065] 4. Read all files in each directory in sequence, and store metadata information such as the directory name, file name, file size, permissions, and modification time in a preset database.

[0066] It can be understood that based on the local lib library running in the user mode of the local storage device, when the target application accesses the local storage device, it can bypass the NFS protocol and directly call the local lib library interface to implement operations such as reading and writing local files, thereby improving the data migration efficiency and saving CPU resources.

[0067] As an example, for instance, a shared folder / shared on a remote storage device is mounted on the / mnt / nas directory of the local storage device. The method for traversing the metadata information of all directories and files at a certain path (such as / shared / share1) in the remote storage device can be as follows:

[0068] 1. The target application opens the NFS shared directory (i.e., / shared / share1) by calling the libNFS library interface. Among them, the libNFS library runs in the user mode of the local storage device, enabling the target application to mount the file system of the remote storage device to the local storage device and then access the remote storage through the NFS protocol;

[0069] 2. Traverse all subdirectories under the above NFS shared directory recursively and save them in a preset database;

[0070] 3. The traversal of subdirectories ends;

[0071] 4. Read all files in each directory in sequence, and record metadata information such as the directory name, file name, and size in a preset database.

[0072] According to an embodiment of the present application, the preset condition includes that the content at the path of the destination storage location represented by the second metadata information is empty.

[0073] In one embodiment, the content at the path of the destination storage location is empty, which means that there is no existing data at the destination storage location that will conflict with the data to be migrated. In the case where the content at the path of the destination storage location represented by the second metadata information is not empty, the first metadata information can be determined as the metadata information, and the target data can be determined based on the metadata information.

[0074] In another embodiment, the content at the path of the destination storage location is not empty, which means that there are already some files and directories at the destination storage location, and these files and directories may have the same name or different content from the files and directories under the source path. This may be because the user specifies a non-empty destination storage location, or it may be due to a previous interrupted data migration. When the second metadata information characterizes that the content at the path of the destination storage location is not empty, the metadata information can be determined based on the difference between the first metadata information and the second metadata information. Thereby, the accuracy and integrity of data migration can be ensured, and data duplication or conflict can be avoided.

[0075] According to an embodiment of the present application, when the second metadata information does not meet the preset conditions, determining the metadata information based on the difference between the first metadata information and the second metadata information includes: comparing the first metadata information with the second metadata information to determine the difference data between the first metadata information and the second metadata information, and the difference data includes at least one of the following: newly added data at the source storage location, inconsistent data at the source storage location; determining the metadata information of the difference data as the metadata information.

[0076] In one embodiment, when the second metadata information does not meet the preset conditions, that is, when the second metadata information characterizes that the content at the path of the destination storage location is not empty, the directories, file names, and file sizes under the source storage location and the destination storage location can be compared one by one to determine the difference data. The difference data may include newly added data at the source storage location compared to the destination storage location, such as data that exists at the source storage location but not at the destination storage location. The difference data may also include inconsistent data at the source storage location compared to the destination storage location, such as data that exists at both the source storage location and the destination storage location, but has the same file name but different file sizes. The metadata information of the above difference data can be determined as the metadata information, and the target data can be determined based on the metadata information.

[0077] According to an embodiment of the present application, the target data includes at least one target sub-data, and the metadata information includes at least one metadata sub-information corresponding to the at least one target sub-data; based on the metadata information of the target data, migrating the target data from the source storage location to the destination storage location by using the target application includes: writing at least one metadata sub-information into a preset database, where the preset database is deployed in a local storage device; determining a data queue to be migrated based on preset configuration information, the data queue to be migrated includes a preset number of metadata sub-information, where the metadata sub-information is used to locate the target sub-data, and the preset number of metadata sub-information is read from the preset database in batches based on the preset configuration information; based on a preset thread pool and the data queue to be migrated, executing at least one migration task, where the migration task is used to migrate the target sub-data to be migrated from the source storage location to the destination storage location by using the target application based on the target metadata sub-information and the second metadata information, and the target metadata sub-information is used to locate the target sub-data to be migrated.

[0078] In a local storage device and / or a remote storage device, data can be stored in the form of files. In one embodiment, the target data to be migrated may include at least one file, then the target data may include at least one target sub-data, and correspondingly, the metadata information may include at least one metadata sub-information corresponding to the at least one target sub-data. Among them, the metadata sub-information of the target sub-data may include, for example, the directory and file name of the target sub-data.

[0079] At least one metadata sub-information can be written into a preset database deployed in a local storage device. Storing at least one metadata in the preset database can centrally manage and maintain the information of all target sub-data to be migrated, which makes the data migration process more orderly and facilitates tracking and management. In addition, the preset database can provide persistent storage, and even if an unexpected interruption occurs during the migration process, the unfinished tasks can be restored from the preset database.

[0080] A preset number of metadata sub-information can be read from the preset database in batches according to the preset configuration information, and a data queue to be migrated is determined based on the preset number of metadata sub-information. Among them, the preset configuration information can be used to indicate control parameters for migrating at least one target sub-data based on multi-threaded concurrency, and the preset configuration information can be, for example, set by the user.

[0081] The data queue to be migrated may include a preset number of metadata sub-information, for example. The data queue to be migrated can be understood as a queue structure for storing target metadata sub-information. Among them, the target metadata sub-information is used to locate the target sub-data to be migrated. In other words, the metadata sub-information of the data queue to be migrated can be used as the target metadata sub-information, and the aforementioned target metadata sub-information is the metadata sub-information of the target sub-data to be migrated.

[0082] At least one migration task can be executed based on a preset thread pool and the data queue to be migrated. Among them, the migration task is used to migrate the target sub-data to be migrated from the source storage location to the destination storage location based on the target metadata sub-information and the second metadata information by using the target application. The preset thread pool can be understood as a mechanism for managing threads, which can efficiently process concurrent tasks. As an example, the data queue to be migrated can be, for example, a first-in-first-out (FIFO) data structure for managing at least one migration task. The data queue to be migrated can be used as a buffer for the data migration task to ensure that the threads in the preset thread pool always have migration tasks to process. The preset thread pool can dynamically allocate migration tasks to the threads according to the number of target metadata sub-information in the data queue to be migrated to ensure load balancing of the threads in the preset thread pool.

[0083] According to an embodiment of the present application, by writing at least one metadata sub-information into a preset database and batch-reading a preset number of metadata sub-information from the preset database, centralized management and efficient scheduling of the target sub-data can be achieved. The data queue to be migrated can be used to store and allocate migration tasks, and the thread pool is used to manage and schedule threads. Based on this, multi-threaded parallel migration of at least one target sub-data can be achieved through the preset database, the data queue to be migrated, and the preset thread pool, thereby significantly improving the migration speed, resource utilization rate, reliability, and flexibility.

[0084] According to an embodiment of the present application, in the case where the source storage location is a local storage device, migrating the target sub-data to be migrated from the source storage location to the destination storage location based on the target metadata sub-information and the second metadata information includes: using the target application to call a first data interface to read the target sub-data to be migrated from the source storage location based on the target metadata sub-information, where the first data interface is deployed on the local storage device; using the target application to remotely call a second data interface to write the target sub-data to be migrated into the destination storage location based on the second metadata information, where the second data interface is deployed on a remote storage device.

[0085] In one embodiment, the source storage location may be located in a local storage device. A first data interface is deployed in the local storage device. The first data interface can be used to define the data format, transmission method, operation specifications, etc. in the local storage device, so that operations such as reading, writing, updating, and deleting the data stored in the local storage device can be performed according to established rules. Exemplarily, the implementation manner of the first data interface may include, but is not limited to, a library (lib), an API (Application Programming Interface), a framework, etc.

[0086] The target application can be utilized to call the first data interface to read the target sub-data to be migrated from the source storage location based on the target metadata sub-information. As an example, the first data interface can be a local lib library deployed in the local storage device. For example, the target application can read the target sub-data to be migrated from the source storage location by calling the local lib library function based on the target metadata sub-information.

[0087] A second data interface is deployed in the remote storage device. The second data interface can be used to define the data format, transmission method, operation specifications, etc. in the remote storage device, so that operations such as reading, writing, updating, and deleting the data stored in the remote storage device can be performed according to established rules. Exemplarily, the implementation manner of the second data interface may include, but is not limited to, a library (lib), an API (Application Programming Interface), a framework, etc.

[0088] The target application can be remotely utilized to call the second data interface to write the target sub-data to be migrated to the destination storage location based on the second metadata information.

[0089] As an example, the second data interface can be a remote lib library deployed in the remote storage device. For example, the target application can write the target sub-data to be migrated to the destination storage location by remotely calling the remote lib library function based on the second metadata information.

[0090] In one embodiment, the target application can remotely call the second data interface through a third data interface. Among them, the third data interface is deployed in the local storage device. The third data interface can access the shared remote data through a network protocol (such as the NFS protocol), enabling the target application to access and operate the remote file system on the remote storage device through the third data interface.

[0091] As an example, the third data interface can be the libNFS library deployed on the local storage device. For example, the target application can remotely call the remote lib library function by invoking the libNFS library, and then write the target sub-data to be migrated to the destination storage location based on the second metadata information. It should be noted that the libNFS library encapsulates all the details of communicating with the remote storage device, and the aforementioned writing operation can be implemented by the target application invoking the libNFS library. In other words, the target application can write data to the remote storage device by invoking the libnfs library without directly calling the remote lib library on the remote storage device.

[0092] Table 1 exemplarily shows some of the API functions of the local lib library and the libNFS library respectively.

[0093] Table 1

[0094]

[0095] Figure 4 Shows a schematic diagram of a data migration method according to an embodiment of the present application.

[0096] As Figure 4 shown, the local storage device 101 can be deployed with the target application 301, the preset database 401, the local lib library, and the libNFS library. Among them, the target application 301, the local lib library, and the libNFS library all run in the user mode.

[0097] For example, the target application 301 can write at least one metadata sub-information to the preset database 401. For example, the target application 301 can allow the user to set the preset configuration information, so that the target metadata sub-information corresponding to the target sub-data to be migrated can be read from the preset database 401 based on the preset configuration information.

[0098] For example, the target application 301 can read the target sub-data to be migrated from the source storage location based on the target metadata sub-information by invoking the local lib library function.

[0099] For example, the target application 301 can remotely call the remote lib library function by invoking the libNFS library, and then write the target sub-data to be migrated to the destination storage location based on the second metadata information.

[0100] According to an embodiment of the present application, remotely invoking a second data interface by a target application to write target sub-data to be migrated to a destination storage location based on second metadata information includes: sending a data write request to a remote storage device by the target application through a mount point, where the data write request includes the second metadata information and the target sub-data to be migrated, so that the remote storage device writes the target sub-data to be migrated to the destination storage location by invoking the second data interface and based on the second metadata information; receiving a data write response from the remote storage device, where the data write response includes the write result of the target sub-data to be migrated.

[0101] The second metadata information may include the path of the destination storage location. In one embodiment, a shared folder / shared on a remote storage device is mounted on the / mnt / nas directory of the local storage device. The file name of the target sub-data to be migrated is, for example, A, its corresponding source path (i.e., the path of the source storage location) is, for example, / local / path / fileA, and the destination path (i.e., the path of the destination storage location) is, for example, / mnt / nas / destination / fileA.

[0102] For example, the target application may send a data write request to the remote storage device through the mount point, and the data write request may include the destination path and file A. After receiving the data write request, the remote storage device may invoke the remote lib library to write file A to the destination storage location / shared / destination / fileA. Among them, the foregoing data write operation may be implemented by the target application by invoking the libNFS library.

[0103] The target application may receive a data write response returned by the remote storage device, and the data write response may include the write result of file A, such as write success or write failure.

[0104] According to an embodiment of the present application, in the case where the source storage location is located on a remote storage device, migrating the target sub-data to be migrated from the source storage location to the destination storage location based on the target metadata sub-information and the second metadata information includes: remotely invoking a second data interface by the target application to read the target sub-data to be migrated from the source storage location based on the target metadata sub-information, where the second data interface is deployed on the remote storage device; invoking a first data interface by the target application to write the target sub-data to be migrated to the destination storage location based on the second metadata information, where the first data interface is deployed on the local storage device.

[0105] In one embodiment, the source storage location may be located in a remote storage device. The second data interface is deployed in the remote storage device, and the second data interface can be used to define the data format, transmission method, operation specifications, etc. in the remote storage device, so that operations such as reading, writing, updating, and deleting the data stored in the remote storage device can be performed according to established rules. Exemplarily, the implementation manner of the second data interface may include, but is not limited to, a library (lib), an API (Application Programming Interface), a framework, etc.

[0106] The target application can remotely call the second data interface to read the target sub-data to be migrated from the source storage location based on the target metadata sub-information. As an example, the second data interface can be a remote lib library deployed in the remote storage device. For example, the target application can remotely call the remote lib library function to read the target sub-data to be migrated from the source storage location based on the target metadata sub-information.

[0107] In one embodiment, the target application can remotely call the second data interface through a third data interface. Among them, the third data interface is deployed in the local storage device. The third data interface can access the shared remote data through a network protocol (such as the NFS protocol), so that the target application can access and operate the remote file system on the remote storage device through the third data interface.

[0108] As an example, the third data interface can be a libNFS library deployed in the local storage device. For example, the target application can call the libNFS library to remotely call the remote lib library function, and further implement reading the target sub-data to be migrated from the source storage location based on the target metadata sub-information. It should be noted that the libNFS library encapsulates all the details of communicating with the remote storage device, and the foregoing reading operation can be implemented by the target application calling the libNFS library. In other words, the target application can read the data on the remote storage device by calling the libnfs library without directly calling the remote lib library on the remote storage device.

[0109] The first data interface is deployed in the local storage device, and the first data interface can be used to define the data format, transmission method, operation specifications, etc. in the local storage device, so that operations such as reading, writing, updating, and deleting the data stored in the local storage device can be performed according to established rules. Exemplarily, the implementation manner of the first data interface may include, but is not limited to, a library (lib), an API (Application Programming Interface), a framework, etc.

[0110] The target application can be utilized to call the first data interface to write the target sub-data to be migrated to the destination storage location based on the second metadata information. As an example, the first data interface can be a local lib library deployed on a local storage device. For example, the target application can write the target sub-data to be migrated to the destination storage location by calling the local lib library function based on the second metadata information.

[0111] Figure 5 FIG. shows a schematic diagram of a data migration method according to another embodiment of the present application.

[0112] As Figure 5 shown, the local storage device 101 can be deployed with a target application 301, a preset database 401, a local lib library, and a libNFS library. Among them, the target application 301, the local lib library, and the libNFS library all run in the user state.

[0113] For example, the target application 301 can write at least one metadata sub-information into the preset database 401. For example, the target application 301 can allow the user to set preset configuration information, so that the target metadata sub-information corresponding to the target sub-data to be migrated can be read from the preset database 401 based on the preset configuration information.

[0114] For example, the target application 301 can remotely call the remote lib library function by calling the libNFS library, and then read the target sub-data to be migrated from the source storage location based on the target metadata sub-information.

[0115] For example, the target application 301 can write the target sub-data to be migrated to the destination storage location by calling the local lib library function based on the second metadata information.

[0116] According to an embodiment of the present application, using the target application to remotely call the second data interface to read the target sub-data to be migrated from the source storage location based on the target metadata sub-information includes: using the target application to send a data read request to the remote storage device through the mount point, the data read request includes the target metadata sub-information, so that the remote storage device reads the target sub-data to be migrated from the source storage location by calling the second data interface and based on the target metadata sub-information, wherein the second data interface is deployed on the remote storage device; receiving a data read response from the remote storage device, the data read response includes the target sub-data to be migrated.

[0117] The target metadata sub - information may include the path and file name of the target sub - file to be migrated. In one embodiment, a shared folder / shared on a remote storage device is mounted on the / mnt / nas directory of the local storage device. The file name of the target sub - data to be migrated is, for example, A, and its corresponding source path (i.e., the path of the target sub - file to be migrated) is, for example, / mnt / nas / destination / fileA, and the destination path (i.e., the path of the destination storage location) is, for example, / local / path / fileA.

[0118] For example, the target application may send a data read request to the remote storage device through the mount point. The data read request may include the source path (i.e., / mnt / nas / destination / fileA) and the file name (i.e., A). After receiving the data read request, the remote storage device may read the file A at / shared / destination / fileA by calling the remote lib library. Among them, the foregoing data read operation may be implemented by the target application by calling the libNFS library.

[0119] The target application may receive a data read response returned by the remote storage device. The data read response may include the file A, so that the target application can write the file A to / local / path / fileA by calling the local lib library.

[0120] According to an embodiment of the present application, determining the data queue to be migrated based on preset configuration information includes: reading a preset number of metadata sub - information in batches from a preset database based on the preset configuration information, and storing the preset number of metadata sub - information in the data queue to be migrated.

[0121] Exemplarily, the preset configuration information may include the read batch size of the metadata sub - information. The read batch size indicates the number of metadata sub - information read in each batch, that is, the above - mentioned preset number. For example, based on the preset configuration information, a preset number of metadata sub - information may be read in batches from the preset database, and the preset number of metadata sub - information may be stored in the data queue to be migrated.

[0122] According to an embodiment of the present application, the preset thread pool includes multiple threads. Executing at least one migration task based on the preset thread pool and the data queue to be migrated includes: allocating a preset number of metadata sub - information to at least one thread, determining the allocation result, where the allocation result is used to determine at least one migration task corresponding to each of the at least one thread; calling at least one thread to execute at least one migration task.

[0123] As an example, the preset thread pool may include 10 threads. The data queue to be migrated may include 100 metadata sub-information, and the 100 metadata sub-information may respectively correspond to 100 migration tasks, and each migration task is used to migrate a target sub-data. Among them, the aforementioned metadata sub-information may be, for example, the directory and file name of the file to be migrated.

[0124] For example, 100 metadata sub-information may be assigned to 10 threads to determine the assignment result, and the assignment result may characterize at least one metadata sub-information corresponding to each thread. More preferably, the metadata sub-information may be dynamically assigned according to the load conditions of the 10 threads respectively, so as to ensure the load balance of the 10 threads in the preset thread pool and avoid some threads being overloaded while other threads are idle.

[0125] Based on the assignment result, at least one migration task corresponding to each of the 10 threads may be determined, and then the 10 threads may be called to execute the above 100 migration tasks in parallel.

[0126] According to the embodiments of the present application, multiple threads may simultaneously perform file reading and writing operations. By parallel processing at least one migration task by multiple threads, the CPU and I / O resources may be fully utilized, thereby improving the overall utilization rate of the system, as well as the speed and efficiency of data migration.

[0127] According to the embodiments of the present application, the preset configuration information includes the number of threads in the preset thread pool, the preset number, and the reading condition of the metadata sub-information.

[0128] Among them, the reading condition of the metadata sub-information may be used to control when and how to read the metadata sub-information from the preset database to ensure the efficiency and correctness of the data migration process. For example, the reading condition of the metadata sub-information may be set as: when the number of metadata sub-information in the data queue to be migrated is lower than the preset threshold, the thread may read the metadata sub-information from the preset database and supplement it to the data queue to be migrated until all the metadata sub-information stored in the preset database is processed.

[0129] In one embodiment, a human-computer interaction interface for user operation may be provided, so that the user may input the above-mentioned number of threads, preset number, and preset threshold in the human-computer interaction interface. The human-computer interaction interface may be, for example, a graphical user interface for the target application program, or an input interface of a command prompt. Those skilled in the art may reasonably set the number of threads, preset number, preset threshold, etc. according to actual needs or application scenarios, etc., and no specific limitation is made here.

[0130] Figure 6 Shows a schematic diagram of multi-thread parallel migration according to an embodiment of the present application.

[0131] As shown Figure 6 in the figure, the preset database can store all metadata sub-information. Based on the database thread, the metadata sub-information can be read in batches from the preset database according to the preset number and preset threshold set by the user, and the read metadata sub-information is placed in the data queue to be migrated. The metadata sub-information in the data queue to be migrated can be referred to as target metadata sub-information. For example, 10,000 directories and files can be read each time. When the number of target metadata sub-information in the data queue to be migrated is less than 200, the database processing thread will read the next batch of directories and files and supplement them to the data queue to be migrated. Until all the metadata sub-information in the preset database has been processed.

[0132] As shown Figure 6 in the figure, the preset thread pool can be configured according to the number of threads set by the user. For example, the preset thread pool can include 10 threads.

[0133] As shown Figure 6 in the figure, 10,000 directories and files in the data queue to be migrated can be allocated to 10 threads, with each file corresponding to a migration task. The 10 threads can execute multiple migration tasks in parallel. After one thread finishes execution, it will read the next file to be migrated from the data queue to be migrated and then continue to execute the migration task until all migration tasks are completed.

[0134] Figure 7 The block diagram of the first storage device according to an embodiment of the present application is shown.

[0135] As shown Figure 7 in the figure, the first storage device 700 includes a storage module 710, a communication module 720, and a data migration module 730.

[0136] The storage module 710 is used to store and manage data.

[0137] The communication module 720 is used to implement the communication connection between the first storage device and the second storage device, and the second storage device is mounted on the first storage device.

[0138] The data migration module 730 is used to determine the target data in response to the data migration instruction and migrate the target data from the source storage location to the destination storage location, where the source storage location is located in one of the first storage device and the second storage device, and the destination storage location is located in the other of the first storage device and the second storage device.

[0139] Figure 8 The block diagram of the electronic device to which the data migration method according to an embodiment of the present application can be applied is shown.

[0140] As shown Figure 8 in FIG. Figure 8 , the electronic device 800 according to an embodiment of the present application includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage section 808 into a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., CPU), an instruction set processor, and / or a related chipset, and / or a dedicated microprocessor (e.g., an application specific integrated circuit (ASIC)), etc. The processor 801 may also include on-board memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present application.

[0141] In the RAM 803, various programs and data required for the operation of the electronic device 800 are stored. The processor 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. The processor 801 performs various operations of the method flow according to an embodiment of the present application by executing the program in the ROM 802 and / or the RAM 803. It should be noted that the program may also be stored in one or more memories other than the ROM 802 and the RAM 803. The processor 801 may also perform various operations of the method flow according to an embodiment of the present application by executing the program stored in the one or more memories.

[0142] According to an embodiment of the present application, the electronic device 800 may further include an input / output (I / O) interface 805, and the input / output (I / O) interface 805 is also connected to the bus 804. The electronic device 800 may further include one or more of the following components connected to the input / output (I / O) interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed so that a computer program read from it can be installed into the storage section 808 as needed.

[0143] In one embodiment, the electronic device 800 may be a user computer system. The electronic device 800 may be directly connected to a local storage device and may be connected to a remote storage device via a network.

[0144] The present application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist separately without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the one or more programs are executed, the method according to the embodiments of the present application is implemented.

[0145] According to an embodiment of the present application, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present application, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present application, the computer-readable storage medium may include the above-described ROM 802 and / or RAM 803 and / or one or more memories other than ROM 802 and RAM 803.

[0146] An embodiment of the present application also includes a computer program product, which includes a computer program that contains program code for executing the method shown in the flowchart. Among them, the computer program product can be understood as the target application program in the user state deployed on the local storage device described above. When the computer program product runs on the local storage device, the program code is used to enable the local storage device to implement the data migration method provided by the embodiments of the present application.

[0147] When the computer program is executed by the processor 801, the above functions defined in the system / apparatus of the embodiments of the present application are executed. According to an embodiment of the present application, the above-described systems, apparatuses, modules, units, etc. may be implemented by computer program modules.

[0148] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices and magnetic storage devices. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and be downloaded and installed through the communication part 809, and / or be installed from the removable medium 811. The program code included in the computer program can be transmitted by any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0149] In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the processor 801, the above functions defined in the system of the embodiments of the present application are performed. According to the embodiments of the present application, the above-described systems, devices, apparatuses, modules, units, etc. can be implemented by computer program modules.

[0150] According to the embodiments of the present application, the program code for executing the computer program provided by the embodiments of the present application can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level procedures and / or object-oriented programming languages, and / or assembly / machine languages. The programming languages include, but are not limited to, such as Java, C++, Python, the "C" language, or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, by connecting through an Internet service provider via the Internet).

[0151] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0152] Those skilled in the art can understand that the features described in the various embodiments of the present application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present application. In particular, without departing from the spirit and teachings of the present application, the features described in the various embodiments of the present application can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present application.

[0153] The embodiments of the present application have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present application. Although the embodiments have been described separately above, this does not mean that the measures in each embodiment cannot be used advantageously in combination. Without departing from the scope of the present application, those skilled in the art can make various substitutions and modifications, and all such substitutions and modifications should fall within the scope of the present application.

Claims

1. A data migration method, characterized in that, The method includes: In response to a data migration instruction, determining target data, where the data migration instruction instructs to migrate the target data from a source storage location to a destination storage location, the source storage location is in one of a local storage device and a remote storage device, the destination storage location is in the other of the local storage device and the remote storage device, and the remote storage device is mounted on the local storage device; Based on the metadata information of the target data, using a target application to migrate the target data from the source storage location to the destination storage location, where the target application is a user-mode application deployed on the local storage device, and the metadata information is used to locate the target data.

2. The method according to claim 1, wherein The data migration instruction includes source access information and destination access information, the source access information represents the path of the source storage location, and the destination access information represents the path of the destination storage location; The determining the target data in response to the data migration instruction includes: According to the source access information and the destination access information, respectively determining first metadata information corresponding to the source storage location and second metadata information corresponding to the destination storage location; When the second metadata information meets a preset condition, determining the first metadata information as the metadata information, and determining the target data based on the metadata information; When the second metadata information does not meet the preset condition, determining the metadata information based on the difference between the first metadata information and the second metadata information, and determining the target data based on the metadata information.

3. The method according to claim 2, wherein The target data includes at least one target sub-data, and the metadata information includes at least one metadata sub-information corresponding to the at least one target sub-data; The using the target application to migrate the target data from the source storage location to the destination storage location based on the metadata information of the target data includes: Writing the at least one metadata sub-information into a preset database, where the preset database is deployed on the local storage device; Determining a data queue to be migrated based on preset configuration information, the data queue to be migrated includes a preset number of metadata sub-informations, where the metadata sub-information is used to locate the target sub-data, and the preset number of metadata sub-informations is read in batches from the preset database based on the preset configuration information; Based on a preset thread pool and the data queue to be migrated, executing at least one migration task, where the migration task is used to migrate the target sub-data to be migrated from the source storage location to the destination storage location using the target application based on target metadata sub-information and the second metadata information, and the target metadata sub-information is used to locate the target sub-data to be migrated.

4. The method according to claim 3, wherein When the source storage location is in the local storage device, the migrating the target sub-data to be migrated from the source storage location to the destination storage location using the target application based on the target metadata sub-information and the second metadata information includes: Use the target application to call a first data interface to read the target sub-data to be migrated from the source storage location based on the target metadata sub-information, where the first data interface is deployed on the local storage device; Use the target application to remotely call a second data interface to write the target sub-data to be migrated to the destination storage location based on the second metadata information, where the second data interface is deployed on the remote storage device.

5. The method according to claim 4, wherein The step of using the target application to remotely call a second data interface to write the target sub-data to be migrated to the destination storage location based on the second metadata information includes: Use the target application to send a data write request to the remote storage device through a mount point, where the data write request includes the second metadata information and the target sub-data to be migrated, so that the remote storage device calls the second data interface and writes the target sub-data to be migrated to the destination storage location based on the second metadata information; Receive a data write response from the remote storage device, where the data write response includes the write result of the target sub-data to be migrated.

6. The method according to claim 3, wherein When the source storage location is the remote storage device, the step of using the target application to migrate the target sub-data to be migrated from the source storage location to the destination storage location based on the target metadata sub-information and the second metadata information includes: Use the target application to remotely call a second data interface to read the target sub-data to be migrated from the source storage location based on the target metadata sub-information, where the second data interface is deployed on the remote storage device; Use the target application to call a first data interface to write the target sub-data to be migrated to the destination storage location based on the second metadata information, where the first data interface is deployed on the local storage device.

7. The method according to claim 6, characterized in that, The step of using the target application to remotely call a second data interface to read the target sub-data to be migrated from the source storage location based on the target metadata sub-information includes: Use the target application to send a data read request to the remote storage device through a mount point, where the data read request includes the target metadata sub-information, so that the remote storage device calls the second data interface and reads the target sub-data to be migrated from the source storage location based on the target metadata sub-information, where the second data interface is deployed on the remote storage device; Receive a data read response from the remote storage device, where the data read response includes the target sub-data to be migrated.

8. The method according to claim 2, wherein The preset condition includes that the second metadata information indicates that the content at the path of the destination storage location is empty.

9. The method according to claim 2 or 5, characterized in that, When the second metadata information does not meet the preset condition, the step of determining the metadata information based on the difference between the first metadata information and the second metadata information includes: Compare the first metadata information with the second metadata information to determine the difference data of the first metadata information relative to the second metadata information, where the difference data includes at least one of the following: newly added data at the source storage location, inconsistent data at the source storage location; Determine the metadata information of the difference data as the metadata information.

10. The method according to claim 3, wherein The determining the data queue to be migrated based on preset configuration information includes: Based on the preset configuration information, read the preset number of metadata sub-information from the preset database in batches, and store the preset number of metadata sub-information in the data queue to be migrated.

11. The method according to claim 10, wherein The preset thread pool includes multiple threads. The executing at least one migration task based on the preset thread pool and the data queue to be migrated includes: Allocate the preset number of metadata sub-information to at least one thread, and determine the allocation result, where the allocation result is used to determine at least one migration task corresponding to each of the at least one thread; Invoke the at least one thread to execute the at least one migration task.

12. The method according to claim 11, wherein, The preset configuration information includes the number of threads in the preset thread pool, the preset number, and the reading conditions of the metadata sub-information.

13. A first storage device, comprising: A storage module for storing and managing data; A communication module for establishing a communication connection between the first storage device and a second storage device, where the second storage device is mounted on the first storage device; A data migration module for determining target data in response to a data migration instruction, and migrating the target data from a source storage location to a destination storage location, where the source storage location is in one of the first storage device and the second storage device, and the destination storage location is in the other of the first storage device and the second storage device.

14. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, The computer program or instruction, when executed by a processor, implements the steps of the method according to any one of claims 1 to 12.

15. A computer program product, comprising a computer program or instructions, characterized in that, The computer program or instruction, when executed by a processor, implements the steps of the method according to any one of claims 1 to 12.

Images