Cloud storage data migration method and related device

Abstract

Disclosed in embodiments of the present application is a cloud storage data migration method. In the method, when second data stored in a first cloud storage space needs to be migrated, instead of relying on a third-party device, the second data can be directly migrated out of the first cloud storage space and then stored in a storage destination, by means of a local storage device of a user. Hence, in the present solution, the second data in the first cloud storage space can be migrated via a storage device that is already present in an existing system architecture and used for users to access data, without requiring the deployment of an additional third-party server, thereby fully utilizing existing resources and avoiding extra costs. In addition, metadata such as second storage position information of the second data is stored locally in the storage device, and is updated and maintained by the storage device rather than by a cloud storage space, thereby avoiding storage semantic loss during data migration performed by the third-party device.

Description

A cloud storage data migration method and related equipment

[0001] This application claims priority to Chinese patent application No. CN202411750479.0, filed with the State Intellectual Property Office of China on November 29, 2024, entitled “A Cloud Storage Data Migration Method and Related Equipment”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of data migration technology, specifically to a cloud storage data migration method and related equipment. Background Technology

[0003] When users utilize cloud storage services through local storage devices, some scenarios may involve migrating stored data between cloud storage spaces provided by different cloud providers. For example, if a user uses cloud storage provided by multiple cloud providers and one provider ceases to provide cloud storage services, the user will need to migrate the stored data from that provider's cloud storage space to cloud storage spaces provided by other cloud providers.

[0004] When data migration is required, the traditional method is to deploy migration software on servers provided by cloud vendors or third parties. This software is used to read stored data from cloud storage space provided by one cloud vendor and write it to cloud storage space provided by another cloud vendor.

[0005] However, this traditional data migration method requires additional resource overhead, such as additional server costs, resulting in significant resource consumption. Summary of the Invention

[0006] This application provides a cloud storage data migration method, which can conveniently and efficiently migrate user data stored in cloud storage spaces, avoiding additional server and other resource overhead. This application also provides corresponding apparatus, devices, computer-readable storage media, and computer program products.

[0007] The first aspect of this application provides a cloud storage data migration method, which is applied to a storage device. The method includes: the storage device receiving first data from a host and storing the first data locally; the storage device receiving second data from the host, storing the second data in a first cloud storage space, and recording second storage location information of the second data; the storage device obtaining a data migration request, the data migration request being used to request the second data to be moved out of the first cloud storage space; the storage device responding to the data migration request, according to the second storage location information, obtaining the second data from the first cloud storage space and storing the second data to a storage destination, the storage destination being either the storage device or the second cloud storage space; and the storage device updating the second storage location information.

[0008] In the first aspect, the storage device provides data access for users. It has local storage space and manages remote cloud storage space, thus enabling the storage of external data locally or in remote cloud storage. For example, hot data can be stored locally, while cold data can be stored in cloud storage. When data migration of second data stored in the first cloud storage space is required, it is no longer necessary to use third-party devices. Instead, the second data can be directly migrated from the first cloud storage space to the storage destination via the user's local storage device. Therefore, in this solution, the second data located in the first cloud storage space can be migrated using the existing storage device within the existing system architecture, without the need for additional third-party servers. This fully utilizes existing resources and avoids additional costs.

[0009] Furthermore, in this new data migration method, metadata such as the second storage location information of the second data is stored locally on the storage device and updated and maintained by the storage device. Since the migration is performed by the storage device that provides data access to the outside world, this metadata can facilitate user access (e.g., data access during migration). This avoids the loss of storage semantics caused during the migration process when data migration is performed by a third-party device.

[0010] In one possible implementation of the first aspect, the method further includes: during the migration of second data in response to a data migration request, the storage device updates second storage location information in real time, the updated second storage location information indicating the location of the first sub-data in the second data in a first cloud storage space or a second cloud storage space, and indicating the location of the second sub-data in the second data on the storage device; during the migration of second data in response to a data migration request, the storage device receives a data access request for the second data; in response to the data access request, the storage device retrieves the first sub-data from the first cloud storage space or the second cloud storage space and retrieves the second sub-data from the local storage according to the current second storage location information; the storage device returns the complete second data.

[0011] In this possible implementation, during the data migration process, the different parts of the second data reside in different storage spaces, and the specific data distribution of these different parts is recorded in real-time updated second storage location information. Since the second storage location information is located locally on the storage device and maintained by the storage device, the specific data distribution of the second data is always controllable and visible to the storage device. Therefore, when a front-end service needs to access the second data, even if the second data is in a data migration state, the storage device can obtain the complete second data based on its second storage location information and return it, without waiting for the data migration to complete. This ensures that the front-end service is not affected by the data migration operation.

[0012] In one possible implementation of the first aspect, the storage device, in response to a data migration request, retrieves second data from a first cloud storage space and stores the second data to a storage destination based on second storage location information. This includes: if the amount of the second data is not greater than a first specified threshold and / or the storage space occupied in the storage device is not greater than a second specified threshold, the storage device, in response to the data migration request, retrieves complete second data from the first cloud storage space based on the second storage location information; the storage device temporarily stores the second data completely before storing it to the storage destination; and the storage device determines the storage destination based on target status information to store the second data thereto. The target status information includes one or more of the following: the resource status of the storage device, the resource status of the second cloud storage space, and the data status of the second data, wherein the data status of the second data indicates whether the second data is cold data or hot data.

[0013] In traditional cloud data migration scenarios using third-party servers, the lack of information interaction between the third-party server and the front-end business makes it difficult to flexibly adjust the migration destination and timing based on actual needs such as device requirements and front-end business requirements. However, in this possible implementation, the storage device retrieves complete second data from the first cloud storage space, temporarily stores it, and then flexibly determines the storage destination based on different application scenarios. Furthermore, the specific timing of storing the second data at the destination can also be adjusted according to the actual application scenario, offering greater flexibility and making full use of device resources.

[0014] In one possible implementation of the first aspect, the target state information includes the data state of the second data, and the storage device determines the storage destination based on the target state information to store the second data at the storage destination, including: if the second data is cold data, the storage device sends the second data to a second cloud storage space; or, if the second data is hot data, the storage device stores the second data locally.

[0015] In this possible implementation, after determining whether the second data is hot or cold data, the storage device can flexibly determine the storage destination of the second data based on its hot or cold status. Specifically, if the second data is cold data, it can be sent to a second cloud storage space to store infrequently used data using the lower-cost cloud storage. Conversely, if the second data is hot data, it can be stored locally so that front-end services can efficiently access the second data from the local storage device.

[0016] In one possible implementation of the first aspect, the target status information includes the resource status of the storage device. The storage device determines the storage destination based on the target status information to store the second data to the storage destination. This includes: after obtaining the complete second data, if the resource status of the storage device indicates that the occupied storage space in the storage device is greater than a second specified threshold, the storage device sends the second data to the second cloud storage space.

[0017] In this possible implementation, when there is little unused storage space on the local storage device, the second data can be sent to the second cloud storage space to ensure that the storage device has a certain amount of unused space, thereby ensuring the normal use of the storage device.

[0018] In one possible implementation of the first aspect, the target state information includes the resource state of the storage device. The storage device determines the storage destination based on the target state information to store the second data to the storage destination. This includes: after the storage device obtains the complete second data, during a target time period, and when the resource state of the storage device indicates that the idle level of the processing resources of the storage device meets specified conditions, the storage device sends the second data to the second cloud storage space.

[0019] In this possible implementation, second data can be transmitted to the second cloud storage space when the storage device is idle, thereby ensuring the execution of normal tasks of the storage device (such as ensuring the execution of tasks related to front-end business) and making full use of the storage device's resources.

[0020] In one possible implementation of the first aspect, the second data includes third sub-data and fourth sub-data; the storage device, in response to a data migration request, retrieves the second data from the first cloud storage space and stores the second data to the storage destination based on the second storage location information, including: the storage device receiving third sub-data from the first cloud storage space when the second data is greater than a first specified threshold and / or the storage space occupied in the storage device is greater than the second specified threshold; the storage device sending the third sub-data to the second cloud storage space in response to receiving the third sub-data from the first cloud storage space; the storage device receiving fourth sub-data from the first cloud storage space; and the storage device sending the fourth sub-data to the second cloud storage space in response to receiving the fourth sub-data from the first cloud storage space.

[0021] In this possible implementation, when the amount of the second data is large and / or the unused storage space in the storage device is relatively limited, a fast flow strategy can be triggered. The storage device is used as a pipeline to transfer the second data in segments to the second cloud storage space, instead of persisting the second data to local storage, so as to ensure the storage performance of the local storage device.

[0022] In one possible implementation of the first aspect, the method further includes: the storage device determining the storage location of the second data according to a hierarchical storage strategy; the storage device obtaining a data migration request, including: the storage device generating a data migration request when it determines, according to the hierarchical storage strategy, that the second data needs to be migrated out of the first cloud storage space.

[0023] In this possible implementation, the storage device can dynamically determine the storage location of the second data in the multi-cloud according to the hierarchical storage strategy during the multi-cloud management process, and can trigger a data migration request when it is determined that the second data needs to be moved out of the first cloud storage space, thereby realizing the flow of data between clouds, realizing data storage management of cloud storage space and storage device, and ensuring storage performance.

[0024] A second aspect of this application provides a cloud storage data migration apparatus, which has the function of implementing the method described in the first aspect or any possible implementation of the first aspect. This function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above function, such as an interface module, a storage module, and a processing module.

[0025] A third aspect of this application provides a storage device including at least one processor, a memory, and computer-executable instructions stored in the memory and executable on the processor. When the computer-executable instructions are executed by the processor, the processor performs a method as described in the first aspect or any possible implementation thereof.

[0026] The fourth aspect of this application provides a computer-readable storage medium storing one or more computer-executable instructions, wherein when the computer-executable instructions are executed by a processor, the processor performs a method as described in the first aspect or any possible implementation thereof.

[0027] The fifth aspect of this application provides a computer program product that stores one or more computer-executable instructions, wherein when the computer-executable instructions are executed by a processor, the processor executes a method as described in the first aspect or any possible implementation thereof.

[0028] A sixth aspect of this application provides a chip system including a processor for supporting a storage device in implementing the functions described in the first aspect or any possible implementation thereof. In one possible design, the chip system may further include a memory for storing necessary program instructions and data. This chip system may be composed of chips or may include chips and other discrete devices.

[0029] The technical effects of the second to sixth aspects or any of the possible implementations thereof can be found in the first aspect or the technical effects of the relevant possible implementations thereof, and will not be repeated here. Attached Figure Description

[0030] Figure 1 is an exemplary schematic diagram of conventional cloud data transmission provided in an embodiment of this application;

[0031] Figure 2 is a schematic diagram of an exemplary system architecture provided in an embodiment of this application;

[0032] Figure 3 is a schematic diagram of an embodiment of the cloud storage data migration method provided in this application;

[0033] Figure 4 is an exemplary schematic diagram of a data migration method provided in an embodiment of this application;

[0034] Figure 5 is another exemplary schematic diagram of the data migration method provided in the embodiments of this application;

[0035] Figure 6 is another exemplary schematic diagram of the data migration method provided in the embodiments of this application;

[0036] Figure 7 is a schematic diagram of an embodiment of the cloud storage data migration device provided in this application;

[0037] Figure 8 is a structural schematic diagram of a storage device provided in an embodiment of this application. Detailed Implementation

[0038] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0039] As will be known to those skilled in the art, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0040] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, such that a process, method, system, product, or apparatus that comprises a series of units is not necessarily limited to those units, but may include other units not expressly listed or inherent to those processes, methods, products, or apparatus.

[0041] First, some concepts involved in this application will be introduced by way of example.

[0042] 1. Cloud storage

[0043] Cloud storage is an online storage model where users store their data on multiple virtual servers, typically provided by cloud providers or other third parties, rather than on their own dedicated servers. Cloud providers operate data centers, and users requiring data storage hosting meet their needs by purchasing or leasing storage space from cloud providers. Based on customer needs, cloud providers prepare virtualized storage resources in the backend and provide these resources to users as storage pools. Users can then use these storage pools to store files, objects, and other data as needed. In practice, these resources may be distributed across numerous server hosts.

[0044] In practical applications, users can access cloud storage services through web application programming interfaces (APIs) or user interfaces.

[0045] Cloud storage can provide low-cost, virtually unlimited scale and elastic capacity for storing data.

[0046] 2. Tiered storage

[0047] Tiered storage refers to a storage system storing data on different storage devices based on one or more of the following indicators: importance of the data, access frequency, retention time, and capacity.

[0048] Users can implement tiered data storage based on cloud storage. Specifically, users can purchase or rent cloud storage services, associate local storage with cloud storage, and identify cold and hot data in their user data. Then, according to a tiered strategy, the user data stored locally can be tiered and stored in the cloud storage space.

[0049] 3. Cold data and hot data

[0050] Based on metrics such as access frequency and access time over a period of time, data can be categorized into hot data and cold data. Hot data refers to data that needs to be frequently accessed by computing nodes and can be considered online data; for example, data from the past six months that users frequently query. Cold data refers to data that is not frequently accessed and can be considered offline data; for example, backups used for disaster recovery or data used more than six months ago.

[0051] Hot data and cold data are stored differently in storage systems. Current storage systems widely employ tiered storage technology to store data in different levels. This not only reduces the space occupied by non-critical data on the primary local disk but also improves the overall storage performance of the system. Furthermore, storage systems further subdivide the data, storing frequently accessed data on high-performance storage devices and less frequently accessed data on relatively lower-performance storage devices, thereby reducing storage costs.

[0052] With the explosive growth of data in the information age, users' local storage devices are gradually becoming insufficient to meet the demands, necessitating a reduction in local storage costs. Therefore, in some scenarios, infrequently used, cold data can be migrated to lower-cost cloud storage, effectively expanding storage space and reducing costs.

[0053] In some scenarios, users may need to migrate stored data between cloud storage spaces provided by different cloud providers when using cloud storage services. For example, if a user uses cloud storage spaces provided by multiple cloud providers, and one cloud provider stops providing cloud storage services, the user will need to migrate the stored data from that provider's cloud storage space to cloud storage spaces provided by other cloud providers.

[0054] When data migration is required, the current practice is to deploy migration software on servers provided by cloud vendors or third parties. This software is used to read stored data from cloud storage space provided by one cloud vendor and write it to cloud storage space provided by another cloud vendor.

[0055] For example, referring to the example shown in Figure 1, in a traditional solution, when a user needs to transfer storage data from cloud storage space A provided by cloud vendor A to cloud storage space B provided by cloud vendor B, it is usually necessary to connect cloud storage space A of cloud vendor A and cloud storage space B of cloud vendor B through migration software on a third-party server in order to achieve data migration from cloud storage space A of cloud vendor A to cloud storage space B of cloud vendor B.

[0056] Since the storage space of a user's local storage device is usually limited, it is difficult to directly download the storage data in cloud storage space A of cloud provider A to the local storage device and then upload it to cloud storage space B of cloud provider B. In other words, in traditional solutions, due to the limited storage space of the user's local storage device, it is difficult to achieve data migration between cloud providers through the user's local storage device.

[0057] However, the traditional method of data migration using migration software on third-party servers has at least the following problems:

[0058] 1. This method requires additional resource overhead, such as additional server costs;

[0059] 2. Different cloud vendors often have different definitions of storage objects and supported functions, which leads to differences in the metadata descriptions of stored data in the cloud storage space of different cloud vendors. Incompatibility may also exist, resulting in the loss of metadata content after data migration, causing some attributes of the stored data to be lost.

[0060] It is evident that the storage semantics, such as metadata, of stored data may differ among different cloud vendors. Therefore, data migration may result in a loss of storage semantics.

[0061] 3. During the data migration process between cloud vendors, because the user's local storage device cannot know the specific address of the data being migrated, the user's local storage device cannot obtain the stored data being migrated, resulting in the inability to access the user's front-end business related to the stored data, causing the front-end business to be interrupted.

[0062] It is evident that during the data migration process between cloud vendors, users' local storage devices may experience disruptions in their front-end business access.

[0063] Based on this, this application provides a cloud storage data migration method that can migrate data stored in cloud storage space through the user's local storage device, avoiding additional server and other resource overhead. Furthermore, since the metadata such as the storage location information of the data stored in cloud storage space is stored locally on the storage device, storage semantic loss during the data migration process is avoided. In addition, the foreground services supported by the user's local storage device do not need to be interrupted during the data migration process.

[0064] Referring to the example shown in Figure 2, the system architecture involved in the cloud storage data migration method in this application embodiment may include a storage device and at least two cloud storage spaces connected to the storage device (e.g., the first cloud storage space and the second cloud storage space shown in Figure 2). In addition, it may also include hosts (e.g., host 1, host 2 and host 3 shown in Figure 2).

[0065] The following provides an exemplary description of the different parts of this system architecture and the information interaction between them.

[0066] 1. The host sends business data to the storage device for storage.

[0067] In this system architecture, the host sends business data to the storage device for storage. This business data can be stored on the storage device and / or in the cloud storage space connected to the storage device.

[0068] In real-world business scenarios, the host can be a device different from the storage device; the specific type of host is not limited here. For example, the host can be a terminal device, a server, a server cluster, or even a virtual machine or container.

[0069] There can be one or more hosts. For example, in the example shown in Figure 2, it can include host 1, host 2, and host 3. Each host can deploy its own application software, and each application software can generate business data on its respective host and distribute it to the storage device. The business executed on this host can be user-facing; therefore, the business on this host can be considered the front-end business. In this embodiment, the type of business executed in the front-end can be determined according to the actual application scenario requirements and is not limited here.

[0070] In other examples, the storage device can also function as a storage unit within the host, receiving business data from the host's processor or similar components for storage. In other words, in this example, the storage device is a component within the host, rather than a device independent of the host.

[0071] There are various ways for a host to store business data on a storage device, and it can use existing or future data storage methods. For example, the host can write business data to the storage device through protocols such as Network File System (NFS), Common Internet File System (CIFS), or Simple Storage Service (S3). The storage method between storage devices can be network attached storage (NAS) file storage or object storage, etc.

[0072] 2. Data storage management between storage devices and cloud storage space.

[0073] After receiving business data from the host, the storage device can perform storage management on the received business data. Specifically, it can store the received business data locally or in one or more cloud storage spaces connected to the storage device, according to methods such as tiered storage strategies.

[0074] In this embodiment of the application, there may be various types of cloud storage spaces connected to the storage device and cloud vendors involved, which are not limited here.

[0075] In the example shown in Figure 2, the cloud storage space connected to the storage device includes a first cloud storage space and a second cloud storage space. These first and second cloud storage spaces can be cloud storage spaces provided by different cloud vendors.

[0076] The storage device may include a storage system, which can be considered as a software system that manages the storage space of the storage device.

[0077] In practical use, a correspondence can be established between the storage space of the storage device and the cloud storage space, so that data can be stored from the storage device to the cloud storage space based on this correspondence.

[0078] It is understood that the data storage methods of storage devices and cloud storage spaces are not limited in this application embodiment. Regardless of whether the data storage method used by the storage device and cloud storage space is a file system or object storage, and regardless of whether the data storage methods of the storage device and cloud storage space are consistent, a correspondence between the storage space of the storage device and the cloud storage space can be established.

[0079] For example, in the example shown in Figure 2, the storage device uses NAS storage, and the NAS storage provides file systems (such as filesystem1 and filesystem2 shown in Figure 2) as shared file directories, which contain different directories and files. The first cloud storage space uses Amazon Web Services (AWS) object storage, which also manages objects through buckets, such as bucket1 and bucket2 shown in Figure 2. Different vendors may use different specific names for the concepts when using object storage, but similar concepts may exist, such as volume or container. For example, in the example shown in Figure 2, the second cloud storage space may include container1 and container2.

[0080] The following is an example of how to establish a correspondence between the storage space of a storage device and the cloud storage space.

[0081] First, the storage system of the storage device can interface with cloud storage space. For example, the cloud management platform managing the cloud storage space provides the storage device's storage system with information such as the bucket and account of the AWS Object Storage cloud storage space, including a key signing key (KSK), to enable the interface between the storage device and the cloud storage space. This allows the storage device's storage system to be authenticated and authorized by the cloud management platform. In this step, the storage device can act as a client. For example, depending on the data storage method, the storage device can be considered an NFS client, a CIFS client, or an S3 client, etc.

[0082] Then, a one-to-one or one-to-many correspondence can be established between the file system or bucket of the storage device's storage system and the remote cloud storage space. For example, it can be determined which file systems in the storage device's file system need to be tiered, thereby establishing a correspondence between one file system in the storage device that needs tiered storage and one or more buckets in the cloud storage space. This allows for the tiered storage of files in the one file system to be tiered and stored in the corresponding one or more buckets in the cloud storage space according to the tiered storage strategy. This enables finer-grained control over data tiering or data backup during data storage, achieving tiered storage of files in the file system.

[0083] After establishing the correspondence between the storage space of the storage device and the cloud storage space, a reasonable hierarchical storage strategy can be configured to store data in hierarchical order or according to file type, hot and cold data information, etc. In addition, data backup can also be achieved through cloud storage space.

[0084] The specific tiered storage strategy can adopt existing or future development strategies, and there are no restrictions here.

[0085] For example, when the storage space of the local file system is insufficient, or when a file with a certain file extension is configured as an important disclosure file, or when access time is used to determine which storage data in the storage device is stored in the cloud storage space.

[0086] For example, if certain data stored in a storage device has not been accessed for a long time (i.e., the interval between the most recent access time and the current time exceeds a specified threshold), this data can be considered cold data and stored in a specific cloud storage space (e.g., the first cloud storage space). If, at some later time, this data is frequently accessed, it can be considered hot data. In this case, the data needs to be retrieved from the cloud storage space and stored on the local storage device, thus providing it as hot data for front-end business access.

[0087] As can be seen, based on the above system architecture, the host's business data can be stored in storage devices or cloud storage spaces (such as the first cloud storage space and / or the second cloud storage space).

[0088] For example, in the example shown in Figure 2, business data may include first data and second data.

[0089] The host sends the first and second data to the storage device.

[0090] The storage system of the storage device determines the storage locations of the first and second data according to a tiered storage strategy. For example, if the first data is "hot" data, it will be stored locally; in the example shown in Figure 2, the first data can be stored in a local database. Conversely, if the second data is "cold" data, it can be stored in the first cloud storage space.

[0091] Based on the above system architecture, this application provides a cloud storage data migration method that can conveniently and efficiently migrate user storage data between cloud storage spaces of multiple cloud vendors, avoiding additional server and other resource overhead.

[0092] In this embodiment of the application, the cloud storage data migration method can be implemented by means of a storage engine, which can be the storage system shown in the example in Figure 2, or a module, plug-in or other form of software tool in the storage system. Alternatively, the storage engine can be a separate software to implement the cloud storage data migration method of this embodiment of the application.

[0093] In practical use, this data engine can be provided to users as a standalone toolkit, or its functionality can be provided through licenses or other means.

[0094] Specifically, as shown in Figure 3, the cloud storage data migration method may include steps 301-305.

[0095] Step 301: The storage device receives the first data from the host and stores the first data locally.

[0096] In this embodiment of the application, the storage device can receive service data from the host. This service data includes first data.

[0097] After receiving the first data, the storage device can determine whether to store it locally based on a tiered storage strategy. For example, if the storage device determines that the first data is hot data or data in a specified format, it can store the first data locally so that the host can quickly access the first data from the storage device when performing foreground operations.

[0098] Step 302: The storage device receives the second data from the host, stores the second data in the first cloud storage space, and records the second storage location information of the second data.

[0099] In this embodiment of the application, the storage device can receive service data from the host, which includes second data.

[0100] It is understood that the first data and the second data can be contained in the same data packet received by the storage device from the host, or they can be contained in different data packets received by the storage device from the host. The first data and the second data can be received by the storage device together, or they can be received separately at different times based on different information exchange operations. Furthermore, there can be one or more hosts, and the first data and the second data can come from the same host or from different hosts.

[0101] After receiving the second data, the storage device can determine whether to store the second data in the first cloud storage space based on tiered storage strategies. For example, if the storage device determines that the first data is cold data or a large file, it can store the second data in the first cloud storage space to reduce the storage burden on the local storage device.

[0102] The storage device can store the second data in the first cloud storage space. Specifically, it can send the second data to the first cloud storage space so that the first cloud storage space can store the received second data.

[0103] Furthermore, in this embodiment, the storage device may store second storage location information of the second data, which is used to record the storage location of the second data. For example, when the second data is stored in the first cloud storage space, the second storage location information indicates that the second data is stored in the first cloud storage space.

[0104] The second storage location information can be considered as metadata of the second data in a broad sense, used to describe the second data.

[0105] In storage devices, there are multiple ways to record metadata such as the second storage location information of the second data.

[0106] For example, the second storage location information of the second data can be recorded by specifying a list to facilitate updates, queries, etc. This specified list may include, but is not limited to, the second storage location information of the second data. For example, it may include the storage location information of all business data received by the storage device from the host, or it may only include the storage location information of some data received by the storage device from the host (e.g., data stored in the first cloud storage space and / or the second cloud storage space).

[0107] Other metadata of the second data, such as filename, size, file type, etc., can be stored in a different way than the second storage location information. For example, other metadata can be stored in a list different from the second storage location information; or, it can be stored in a specified database. Both other metadata of the second data and metadata such as the second storage location information can be stored in the storage device. That is to say, the storage device can store all the metadata of the second data, but different content within all metadata can be stored separately using different methods.

[0108] Alternatively, in some examples, the second storage location information of the second data in the storage device can be stored in the same way as other metadata such as the size of the second data, file type, and file name of the second data in the storage device, for example, all stored in the same database of the storage device.

[0109] In this way, even if the second data is stored in the first cloud storage space, since the second storage location information is stored locally on the storage device, the location of the second data is always controllable and visible to the storage device, which facilitates efficient management and access of the second data by the storage device.

[0110] Furthermore, it should be noted that the embodiments of this application do not limit the metadata of the second data recorded in the first cloud storage space. For example, when the second data is stored in the first cloud storage space, the first cloud storage space may also store information about the specific storage location of the second data in the first cloud storage space, for information backup or for retrieving and reading the second data in the first cloud storage space. In addition, the first cloud storage space may also store metadata information such as the size of the second data, file type, and filename of the second data in the first cloud storage space. This metadata information can also be used for information backup or for retrieving and reading the second data in the first cloud storage space.

[0111] Step 303: The storage device obtains a data migration request.

[0112] A data migration request is used to request that second data be moved out of the first cloud storage space.

[0113] In this application embodiment, the specific content of the data migration request can be varied.

[0114] For example, the data migration request could simply request the removal of the second data from the first cloud storage space without specifying its storage destination. Alternatively, the data migration request could specify both the removal of the second data from the first cloud storage space and its storage destination. Furthermore, the data migration request could also specify time constraints for removing the second data from the first cloud storage space; for example, it could request that the second data be removed from the first cloud storage space before a certain timeframe.

[0115] There are several ways to generate a data migration request.

[0116] In one example, a user could submit the data migration request to the storage device via a host or similar device.

[0117] In another example, the data migration request could be generated by an application on the storage device that is distinct from the storage engine and then passed to the storage engine.

[0118] In another example, the storage engine can be used for multi-cloud management. Specifically, it can determine the data stored on each storage device and in the multiple cloud storage spaces connected to it, based on a tiered storage strategy. In this example, the storage engine can automatically trigger data migration when data needs to be migrated between different clouds or storage devices, based on the tiered storage strategy.

[0119] Specifically, in some embodiments, the method further includes:

[0120] The storage device determines the storage location of the second data according to a tiered storage strategy;

[0121] Step 303 includes:

[0122] If the storage device determines that the second data needs to be migrated out of the first cloud storage space based on the tiered storage strategy, it generates a data migration request.

[0123] In this embodiment of the application, there can be various specific methods for the hierarchical storage strategy, which can be flexibly configured by the user and are not limited here.

[0124] For example, determining that second data needs to be migrated out of the first cloud storage space according to the tiered storage strategy may specifically include:

[0125] Based on one or more of the following: the data volume and / or importance information of the second data, the space information of the first cloud storage space, the space information of the storage device, the space information of the second cloud storage space, or the specified network status, it is determined that the second data needs to be migrated out of the first cloud storage space.

[0126] The spatial information may include the size and proportion of unoccupied storage space, and may also include space configuration information, such as configuration information indicating that data migration will be initiated when the size of the unoccupied storage space is less than a specified size. The specified network status may include network status information between the first cloud storage space and the storage device, and / or network status information between the second cloud storage space and the storage device. The importance information of the second data may be determined based on a user-preconfigured importance identifier, or it may be determined based on information such as the data type of the second data.

[0127] As can be seen in this example, during the multi-cloud management process, the storage device can dynamically determine the storage location of the second data in the multi-cloud according to the hierarchical storage strategy, and can trigger a data migration request when it is determined that the second data needs to be moved out of the first cloud storage space, thereby realizing the flow of data between clouds, realizing data storage management of cloud storage space and storage device, and ensuring storage performance.

[0128] Step 304: In response to the data migration request, the storage device retrieves the second data from the first cloud storage space and stores the second data to the storage destination based on the second storage location information.

[0129] The storage destination is a storage device or a second cloud storage space.

[0130] In this embodiment, the storage device can determine the current storage location of the second data as the first cloud storage space based on the second storage location information, thereby obtaining the second data from the first cloud storage space. The storage device can send a request to the first cloud storage space instructing the second data to be moved out of the first cloud storage space, and the first cloud storage space responds to the request by sending the second data back to the storage device.

[0131] In this embodiment, the second data can be migrated from the first cloud storage space and then stored in the storage destination via a storage device. Therefore, in this embodiment, the second data located in the first cloud storage space is migrated using existing storage devices in the current system architecture, without requiring additional third-party servers as in traditional technologies. This fully utilizes existing resources and avoids additional costs.

[0132] In different application scenarios, there are various ways for the storage device to obtain the second data from the first cloud storage space and store the second data to the storage destination. Examples are given below.

[0133] Data migration solution 1: Download the second data completely to the storage device and then store it to the storage destination.

[0134] Specifically, in some embodiments, step 304 includes:

[0135] If the amount of the second data is not greater than the first specified threshold and / or the storage space occupied in the storage device is not greater than the second specified threshold, in response to the data migration request, the storage device obtains the complete second data from the first cloud storage space according to the second storage location information;

[0136] The storage device temporarily stores the second data completely before storing it at the storage destination.

[0137] The storage device determines the storage destination based on the target status information in order to store the second data at the storage destination. The target status information includes one or more of the following: the resource status of the storage device, the resource status of the second cloud storage space, and the data status of the second data. The data status of the second data is used to indicate whether the second data is cold data or hot data.

[0138] In this embodiment, the first and second specified thresholds can be pre-configured by the user or automatically configured based on parameters such as the storage space size of the storage device. For example, the first specified threshold can be 10% of the storage space size of the storage device, and the second specified threshold can be 70% of the storage space size of the storage device, etc. A data volume of the second data not exceeding the first specified threshold can be considered as a small data volume, while a storage space occupied in the storage device not exceeding the second specified threshold can be considered as a large amount of unoccupied storage space remaining in the storage device.

[0139] Based on, but not limited to, conditions such as the amount of the second data not exceeding a first specified threshold and / or the storage space occupied in the storage device not exceeding a second specified threshold, the storage device can be considered to be able to obtain and temporarily store the complete second data. Then, according to the actual scenario, the second data can be flexibly stored to the storage destination.

[0140] Temporarily storing the second data refers to the period after the complete second data is acquired but before it is stored at the storage destination, during which the second data is stored on the storage device. However, the method of temporarily storing the second data during this period is not limited. For example, the second data can be temporarily stored in volatile storage media such as the memory or cache of the storage device, or it can be temporarily stored in non-volatile storage media such as the hard disk of the storage device.

[0141] The storage device can determine the storage location that is suitable for storing the second data based on the target status information, and use it as the storage destination for the second data.

[0142] For example, in the example shown in Figure 4, the storage device can obtain complete second data from a first cloud storage space, temporarily store the complete second data, and then store the temporarily stored second data to the storage destination after determining the storage destination. Specifically, the storage destination can be the local storage device or the second cloud storage space.

[0143] The following describes an exemplary method for determining the storage destination.

[0144] Method 1: Determine the storage destination based on the hot / cold status of the data.

[0145] Specifically, in some embodiments, the target state information includes the data state of the second data, and the storage device determines the storage destination based on the target state information to store the second data at the storage destination, including:

[0146] If the second data is cold data, the storage device sends the second data to the second cloud storage space;

[0147] or,

[0148] If the second data is hot data, the storage device stores the second data locally.

[0149] In this embodiment, whether the second data is hot data can be determined based on the timestamp of the access operation of the second data by the foreground service in the host. For example, if the timestamp indicates that the second data was accessed recently (e.g., within a specified time period), then the second data is determined to be hot data. Alternatively, other conditions such as the frequency of recent access can also be used to determine if the second data is hot data. Otherwise, the second data is cold data. The method for determining whether the second data is hot or cold data can refer to existing or future related technologies. This embodiment does not limit the method for determining whether the second data is hot or cold data.

[0150] After determining whether the second data is hot or cold data, the storage device can flexibly determine the storage destination of the second data based on its hot or cold status. Specifically, if the second data is cold data, it can be sent to a second cloud storage space to store infrequently used data using the lower-cost cloud storage. Conversely, if the second data is hot data, it can be stored locally to allow front-end services to efficiently access the second data from the local storage device.

[0151] Method 2: Determine whether the second data needs to be uploaded to the second cloud storage space based on the local storage situation.

[0152] Specifically, in some embodiments, the target state information includes the resource state of the storage device. The storage device determines the storage destination based on the target state information to store the second data at the storage destination, including:

[0153] After obtaining complete second data, if the storage device's resource status indicates that the occupied storage space in the storage device is greater than a second specified threshold, the storage device sends the second data to the second cloud storage space.

[0154] In this embodiment, when there is little unoccupied storage space on the local storage device, second data can be sent to the second cloud storage space to ensure that the storage device has a certain amount of unoccupied space, thereby ensuring the normal use of the storage device.

[0155] Method 3: Upload the second data to the second cloud storage space when the processing resources of the storage device are idle.

[0156] Specifically, in some embodiments, the target state information includes the resource state of the storage device. The storage device determines the storage destination based on the target state information to store the second data at the storage destination, including:

[0157] After obtaining complete second data, the storage device sends the second data to the second cloud storage space within the target time period, and when the resource status of the storage device indicates that the idle level of the storage device's processing resources meets the specified conditions.

[0158] In many scenarios, the migration of secondary data is not limited to immediate completion, but rather can be completed within a target time period. This target time period can be included in the data migration request or configured by the user through other means.

[0159] Processing resources may include one or more of the following: communication resources such as bandwidth, processor resources, etc.

[0160] The idle level of processing resources meeting the specified conditions can be, for example, the proportion of processing resources in an idle state being greater than a specified proportion. Furthermore, the conditions for sending second data to the second cloud storage space may include, but are not limited to, the idle level of the storage device's processing resources meeting the specified conditions; for example, it may also include other conditions such as being at night.

[0161] In this embodiment, second data can be transmitted to the second cloud storage space when the storage device is idle, thereby ensuring the execution of normal tasks of the storage device (e.g., ensuring the execution of tasks related to front-end business) and making full use of the storage device's resources.

[0162] It is understood that in real-world scenarios, any one of the above exemplary methods can be used to determine the storage destination, or a combination of the above exemplary methods can be used to determine the storage destination. In addition, other methods can also be used to determine the storage destination. For example, the second data can be sent to the second cloud storage space only if the size of the unoccupied storage space in the second cloud storage space is greater than a third specified threshold.

[0163] In traditional cloud data migration scenarios using third-party servers, the lack of information interaction between the third-party server and the front-end business makes it difficult to flexibly adjust the migration destination and timing based on actual needs such as device requirements and front-end business requirements. However, in this embodiment, the storage destination can vary depending on the actual application scenario, and the specific timing of storing the second data at the destination can also be adjusted according to the actual application scenario, offering greater flexibility and making full use of device resources.

[0164] Data migration solution two: a strategy that triggers rapid flow, where the storage device acts as a conduit to transfer the second data in segments to the second cloud storage space, instead of persisting the second data to local storage.

[0165] Specifically, in some embodiments, the second data includes a third sub-data and a fourth sub-data; in response to a data migration request, the storage device retrieves the second data from the first cloud storage space and stores the second data to the storage destination based on the second storage location information, including:

[0166] If the storage device receives third sub-data from the first cloud storage space when the second data exceeds the first specified threshold and / or the storage space occupied in the storage device exceeds the second specified threshold;

[0167] In response to receiving the third sub-data from the first cloud storage space, the storage device sends the third sub-data to the second cloud storage space;

[0168] The storage device receives the fourth sub-data from the first cloud storage space;

[0169] In response to receiving the fourth sub-data from the first cloud storage space, the storage device sends the fourth sub-data to the second cloud storage space.

[0170] In this embodiment of the application, when the amount of the second data is large and / or the unoccupied storage space in the storage device is relatively limited, a fast flow strategy can be triggered to use the storage device as a pipeline to transfer the second data in segments to the second cloud storage space instead of persisting the second data to local storage, so as to ensure the storage performance of the local storage device.

[0171] Specifically, this application embodiment takes the third and fourth sub-data in the second data as examples to introduce a scheme of using the storage device as a pipeline to transfer the second data in segments to the second cloud storage space.

[0172] The third and fourth sub-data can be considered as different segments of the second data. They can be obtained by dividing the second data according to the segmented transmission method of the second data. At this time, based on the transmission protocol between the first cloud storage space and the storage device, the third and fourth sub-data can be transmitted separately in different segments.

[0173] As for storage devices, they do not need to receive complete second data from the first cloud storage space before forwarding it to the second cloud storage space. Instead, they act as a conduit, triggering the forwarding of any received fragment to the second cloud storage space.

[0174] Taking the example shown in Figure 5 as an example, an exemplary data transmission process of an embodiment of this application is described.

[0175] In the example shown in Figure 5, a segmented transmission method is used to migrate the second data from the first cloud storage space.

[0176] When the third sub-data in the second data is transmitted to the storage device, the storage device is triggered to transmit the received third sub-data to the second cloud storage space. At this time, the fourth sub-data has not yet been transmitted to the storage device.

[0177] Then, the first cloud storage space sends the fourth sub-data to the storage device, and the storage device sends the third sub-data to the second cloud storage space, so that the storage device can receive the fourth sub-data and the second cloud storage space can receive the third sub-data.

[0178] As can be seen, in one exemplary intermediate state in Figure 5, the third sub-data in the second data can exist in a storage device, while the fourth sub-data in the second data can exist in a first cloud storage space. In another exemplary intermediate state in Figure 5, the fourth sub-data in the second data can exist in a storage device, while the third sub-data in the second data can exist in a second cloud storage space.

[0179] Next, when the fourth sub-data is transmitted to the storage device, the storage device is triggered to transmit the received fourth sub-data to the second cloud storage space.

[0180] Referring to the transmission process shown in Figure 5, the storage device can be used as a pipeline to transmit the second data from the first cloud storage space to the second cloud storage space in segments.

[0181] It should be noted that Figure 5 is merely an exemplary illustration of an embodiment of this application and is not intended to limit the scope. For example, in some examples, the fourth sub-data may be sent to the storage device before the third sub-data; or, the transmission process of the fourth sub-data may overlap with the transmission process of the third sub-data in time, rather than strictly guaranteeing that the third sub-data and the fourth sub-data are transmitted sequentially.

[0182] Step 305: The storage device updates the second storage location information.

[0183] During and after data migration, the storage device can manage and update the secondary storage location information of the secondary data in real time. For example, during data migration, the address and other status information of each segment of the secondary data can be updated in real time, thereby managing the specific data distribution of the data being migrated and ensuring that the specific storage location of each sub-data segment in the migrating secondary data remains controllable and visible to the storage device. After the data migration is complete, the secondary storage location information of the secondary data can be updated according to the storage destination of the secondary data.

[0184] In this embodiment, the storage device provides data access for users. It has local storage space and manages remote cloud storage space, thus enabling the storage of external data locally or in remote cloud storage. For example, hot data can be stored locally, and cold data can be stored in cloud storage. When migrating second data stored in the first cloud storage space, third-party devices are no longer needed. Instead, the second data can be directly migrated from the first cloud storage space to the destination storage device via the user's local storage device. Therefore, in this solution, the second data located in the first cloud storage space can be migrated using the existing storage device within the existing system architecture, eliminating the need for additional third-party servers. This fully utilizes existing resources and avoids additional costs.

[0185] Furthermore, in this new data migration method, metadata such as the second storage location information of the second data is stored locally on the storage device and updated and maintained by the storage device. Since the migration is performed by the storage device that provides data access to the outside world, this metadata can facilitate user access (e.g., data access during migration). This avoids the loss of storage semantics caused during the migration process when data migration is performed by a third-party device.

[0186] Furthermore, in this embodiment, since the second storage location information of the second data is located locally on the storage device and maintained by the storage device, the location of the second data is always controllable and visible to the storage device. Therefore, when the front-end service needs to access the second data, even if the second data is in a data migration state, the storage device can obtain the complete second data based on the second storage location information of the second data and output it, without having to wait for the data migration to be completed before obtaining the second data, thus ensuring that the front-end service is not affected by the data migration operation.

[0187] Specifically, in some embodiments, the method further includes:

[0188] During the process of migrating the second data in response to a data migration request, the storage device updates the second storage location information in real time. The updated second storage location information is used to indicate the location of the first sub-data in the second data in the first cloud storage space or the second cloud storage space, and to indicate the location of the second sub-data in the second data on the storage device.

[0189] During the process of migrating second data in response to a data migration request, the storage device receives a data access request for the second data;

[0190] In response to a data access request, the storage device retrieves the first sub-data from the first cloud storage space or the second cloud storage space, and retrieves the second sub-data from the local storage, based on the current second storage location information.

[0191] The storage device returns complete second data.

[0192] This data access request can be considered as an access request from the front-end business, for example, it could be an access request from any of the hosts shown in Figure 2; or it could be an access request from other devices.

[0193] A data access request is used to request access to second data, which can specifically be an operation such as reading, writing, updating, or deleting the second data.

[0194] In this embodiment, the specific division method of the first sub-data and the second sub-data may be the same as or different from the specific division method of the third sub-data and the fourth sub-data in any of the above embodiments, and no restriction is imposed here.

[0195] During the process of migrating second data in response to a data migration request, the storage device can update the second storage location information in real time.

[0196] In this embodiment of the application, taking an updated second storage location information as an example, it is explained that when the front-end service accesses the second data, the storage device performs data acquisition operations on the second data.

[0197] Specifically, referring to the example shown in Figure 6, the updated second storage location information is used to indicate the location of the first sub-data in the second data in the first cloud storage space or the second cloud storage space (for example, in the example shown in Figure 6, the first sub-data in the data migration is located in the first cloud storage space), and to indicate the location of the second sub-data in the second data in the storage device.

[0198] As can be seen, during the data migration process, the different parts of the second data are located in different storage spaces, and the specific data distribution of the different parts of the second data is recorded in the real-time updated second storage location information. Therefore, the specific data distribution is controllable and visible to the stored data.

[0199] In this specific data distribution scenario, referring to the example shown in Figure 6, when the storage device receives a data access request, it can retrieve the first sub-data from the first cloud storage space or the second cloud storage space based on the current second storage location information (for example, in the example shown in Figure 6, the storage device retrieves the first sub-data from the first cloud storage space), retrieve the second sub-data from the local storage, obtain the complete second data, return it to the designated recipient, and update the second storage location information. The designated recipient can be the sender of the data access request (i.e., the host) or a data recipient specified by the sender of the data access request.

[0200] Furthermore, in some examples, when the storage device receives a data access request, it can identify the second data as hot data. In this case, the storage destination of the second data is updated to the storage device. That is, if the initial storage destination indicated by the initial data migration request is the second cloud storage space, then after receiving the data access request, the initial storage destination can be updated to the storage device. Thus, when the storage device responds to the data access request, it retrieves the first sub-data from the first or second cloud storage space and the second sub-data from its local storage location based on the current second storage location information. After obtaining the complete second data and returning it, the storage device can store the complete second data as hot data in the storage device. Without a new data migration instruction, there is no need to send the complete second data to the second cloud storage space again.

[0201] The cloud storage data migration method provided by the embodiments of this application has been described above from multiple aspects. The cloud storage data migration device provided by the embodiments of this application will be described below with reference to the accompanying drawings.

[0202] As shown in Figure 7, this application embodiment provides a cloud storage data migration device 70, which includes:

[0203] Interface module 701 is used for:

[0204] Receive the first data from the host and store it locally;

[0205] Receive the second data from the host, store the second data in the first cloud storage space, and record the second storage location information of the second data;

[0206] Obtain a data migration request, which is used to request that the second data be moved out of the first cloud storage space;

[0207] In response to a data migration request, the system retrieves the second data from the first cloud storage space based on the second storage location information and stores the second data to a storage destination, which is either a storage device or the second cloud storage space.

[0208] Processing module 702 is used to update the second storage location information.

[0209] Optionally, the processing module 702 is used to: during the process of migrating the second data in response to a data migration request, update the second storage location information in real time, wherein the updated second storage location information is used to indicate the location of the first sub-data in the second data in the first cloud storage space or the second cloud storage space, and to indicate the location of the second sub-data in the second data in the storage device;

[0210] Interface module 701 is used for:

[0211] During the migration of second data in response to a data migration request, a data access request for the second data is received;

[0212] In response to a data access request, the storage device retrieves the first sub-data from the first cloud storage space or the second cloud storage space, and retrieves the second sub-data from the local storage, based on the current second storage location information.

[0213] Return the complete second data.

[0214] Optionally, the cloud storage data migration device 70 also includes a storage module 703;

[0215] Interface module 701 is used to: in response to a data migration request, obtain complete second data from the first cloud storage space based on the second storage location information, provided that the amount of second data is not greater than the first specified threshold and / or the storage space occupied in the storage device is not greater than the second specified threshold.

[0216] Storage module 703 is used to: temporarily store the second data completely before storing the second data to the storage destination;

[0217] The interface module 701 is used to: determine the storage destination based on the target status information, so as to store the second data to the storage destination. The target status information includes one or more of the following: the resource status of the storage device, the resource status of the second cloud storage space, and the data status of the second data. The data status of the second data is used to indicate whether the second data is cold data or hot data.

[0218] Optionally, interface module 701 is used for:

[0219] If the second data is cold data, send the second data to the second cloud storage space;

[0220] or,

[0221] If the second data is hot data, store the second data locally.

[0222] Optionally, the interface module 701 is used to: after obtaining complete second data, if the resource status of the storage device indicates that the occupied storage space in the storage device is greater than a second specified threshold, send the second data to the second cloud storage space.

[0223] Optionally, the interface module 701 is used to: after obtaining complete second data, during a target time period, and when the resource status of the storage device indicates that the idle level of the processing resources of the storage device meets specified conditions, send the second data to the second cloud storage space.

[0224] Optionally, the second data includes the third sub-data and the fourth sub-data;

[0225] Interface module 701 is used for:

[0226] If the second data is greater than the first specified threshold and / or the storage space occupied in the storage device is greater than the second specified threshold, the third sub-data from the first cloud storage space is received;

[0227] In response to receiving third sub-data from the first cloud storage space, the third sub-data is sent to the second cloud storage space;

[0228] Receive the fourth sub-data from the first cloud storage space;

[0229] In response to receiving the fourth sub-data from the first cloud storage space, the fourth sub-data is sent to the second cloud storage space.

[0230] Optionally, the processing module 702 is used for:

[0231] The storage location of the second data is determined according to the hierarchical storage strategy;

[0232] If, based on the tiered storage strategy, it is determined that the second piece of data needs to be migrated from the first cloud storage space, a data migration request is generated.

[0233] Figure 8 shows a possible logical structure diagram of a storage device 80 provided in an embodiment of this application. The storage device 80 is used to implement the functions of the storage device involved in any of the above embodiments. The storage device 80 includes: a memory 801, a processor 802, a communication interface 803, and a bus 804. The memory 801, processor 802, and communication interface 803 are interconnected via the bus 804.

[0234] The memory 801 can be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 801 can store programs, and when the program stored in the memory 801 is executed by the processor 802, the processor 802 and the communication interface 803 are used to perform one or more steps in the above-described cloud storage data migration method embodiments.

[0235] The processor 802 can be a central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), graphics processing unit (GPU), digital signal processing (DSP), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or any combination thereof, to execute relevant programs to achieve the functions required by the interface module, storage module, and processing module in the cloud storage data migration device of the above embodiments, or to execute one or more steps in the method embodiments of this application. The steps of the method disclosed in the embodiments of this application can be executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 801. The processor 802 reads the information in memory 801 and, in conjunction with its hardware, executes one or more steps in the cloud storage data migration method embodiments described above.

[0236] The communication interface 803 uses transceiver devices, such as, but not limited to, transceivers, to enable communication between the storage device 80 and other devices or communication networks.

[0237] Bus 804 enables the transmission of information between various components of storage device 80 (e.g., memory 801, processor 802, and communication interface 803). Bus 804 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one thick line is used in Figure 8, but this does not indicate that there is only one bus or one type of bus.

[0238] In another embodiment of this application, a computer-readable storage medium is also provided, which stores computer-executable instructions. When the processor of the device executes the computer-executable instructions, the device performs the steps performed by the processor in FIG8 above.

[0239] In another embodiment of this application, a computer program product is also provided, which includes computer-executable instructions stored in a computer-readable storage medium; when the processor of the device executes the computer-executable instructions, the device performs the steps performed by the processor in FIG8 above.

[0240] In another embodiment of this application, a chip system is also provided, which includes a processor for implementing the steps performed by the processor in FIG8. In one possible design, the chip system may further include a memory for storing program instructions and data necessary for data writing. This chip system may be composed of chips or may include chips and other discrete devices.

[0241] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0242] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0243] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0244] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0245] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A cloud storage data migration method, characterized in that, Applied to a storage device, the method includes: The storage device receives first data from the host and stores the first data locally; The storage device receives the second data from the host, stores the second data in the first cloud storage space, and records the second storage location information of the second data; The storage device obtains a data migration request, which is used to request that the second data be moved out of the first cloud storage space; In response to the data migration request, the storage device retrieves the second data from the first cloud storage space and stores the second data to a storage destination, which is either the storage device or the second cloud storage space, based on the second storage location information. The storage device updates the second storage location information.

2. The method according to claim 1, characterized in that, The method further includes: During the process of migrating the second data in response to the data migration request, the storage device updates the second storage location information in real time. The updated second storage location information is used to indicate the location of the first sub-data in the second data in the first cloud storage space or the second cloud storage space, and to indicate the location of the second sub-data in the second data in the storage device. During the process of migrating the second data in response to the data migration request, the storage device receives a data access request for the second data; In response to the data access request, the storage device retrieves the first sub-data from the first cloud storage space or the second cloud storage space, and retrieves the second sub-data from the local storage space, based on the current second storage location information. The storage device returns the complete second data.

3. The method according to claim 1 or 2, characterized in that, In response to the data migration request, the storage device retrieves the second data from the first cloud storage space and stores the second data to the storage destination based on the second storage location information, including: If the amount of the second data is not greater than a first specified threshold and / or the storage space occupied in the storage device is not greater than a second specified threshold, the storage device, in response to the data migration request, retrieves the complete second data from the first cloud storage space according to the second storage location information. The storage device temporarily stores the second data completely before storing it to the storage destination; The storage device determines the storage destination based on the target status information in order to store the second data in the storage destination. The target status information includes one or more of the following: the resource status of the storage device, the resource status of the second cloud storage space, and the data status of the second data. The data status of the second data is used to indicate whether the second data is cold data or hot data.

4. The method according to claim 3, characterized in that, The target status information includes the data status of the second data. The storage device determines the storage destination based on the target status information to store the second data at the storage destination, including: When the second data is cold data, the storage device sends the second data to the second cloud storage space; or, The storage device stores the second data locally when the second data is hot data.

5. The method according to claim 3, characterized in that, The target status information includes the resource status of the storage device. The storage device determines the storage destination based on the target status information to store the second data at the storage destination, including: After obtaining the complete second data, if the resource status of the storage device indicates that the occupied storage space in the storage device is greater than a second specified threshold, the storage device sends the second data to the second cloud storage space.

6. The method according to claim 3, characterized in that, The target status information includes the resource status of the storage device. The storage device determines the storage destination based on the target status information to store the second data at the storage destination, including: After acquiring the complete second data, the storage device sends the second data to the second cloud storage space during the target time period, and when the resource status of the storage device indicates that the idle level of the processing resources of the storage device meets the specified conditions.

7. The method according to claim 1 or 2, characterized in that, The second data includes the third sub-data and the fourth sub-data; In response to the data migration request, the storage device retrieves the second data from the first cloud storage space and stores the second data to the storage destination based on the second storage location information, including: The storage device receives the third sub-data from the first cloud storage space when the second data is greater than a first specified threshold and / or the storage space occupied in the storage device is greater than a second specified threshold. In response to receiving the third sub-data from the first cloud storage space, the storage device sends the third sub-data to the second cloud storage space; The storage device receives the fourth sub-data from the first cloud storage space; In response to receiving the fourth sub-data from the first cloud storage space, the storage device sends the fourth sub-data to the second cloud storage space.

8. The method according to any one of claims 1-7, characterized in that, The method further includes: The storage device determines the storage location of the second data according to a hierarchical storage strategy; The storage device acquires a data migration request, including: When the storage device determines, according to the tiered storage strategy, that the second data needs to be migrated from the first cloud storage space, it generates the data migration request.

9. A cloud storage data migration device, characterized in that, include: The interface module is used for: Receive the first data from the host and store the first data locally; Receive the second data from the host, store the second data in the first cloud storage space, and record the second storage location information of the second data; Obtain a data migration request, the data migration request being used to request that the second data be moved out of the first cloud storage space; In response to the data migration request, the second data is obtained from the first cloud storage space according to the second storage location information and the second data is stored to the storage destination, which is the storage device or the second cloud storage space; The processing module is used to update the second storage location information.

10. The apparatus according to claim 9, characterized in that, The processing module is used to: during the process of migrating the second data in response to the data migration request, update the second storage location information in real time, and the updated second storage location information is used to indicate the location of the first sub-data in the second data in the first cloud storage space or the second cloud storage space, and to indicate the location of the second sub-data in the second data in the storage device; The interface module is used for: During the process of migrating the second data in response to the data migration request, a data access request for the second data is received; In response to the data access request, the storage device retrieves the first sub-data from the first cloud storage space or the second cloud storage space, and retrieves the second sub-data from the local storage space, based on the current second storage location information. Return the complete second data.

11. The apparatus according to claim 9 or 10, characterized in that, It also includes a storage module; The interface module is used to: in response to the data migration request, obtain the complete second data from the first cloud storage space according to the second storage location information, provided that the amount of the second data is not greater than a first specified threshold and / or the storage space occupied in the storage device is not greater than a second specified threshold. The storage module is used to: temporarily store the second data completely before storing the second data to the storage destination; The interface module is used to: determine the storage destination based on the target status information, so as to store the second data in the storage destination. The target status information includes one or more of the following: the resource status of the storage device, the resource status of the second cloud storage space, and the data status of the second data. The data status of the second data is used to indicate whether the second data is cold data or hot data.

12. The apparatus according to claim 11, characterized in that, The interface module is used for: If the second data is cold data, the second data is sent to the second cloud storage space; or, If the second data is hot data, the second data is stored locally.

13. The apparatus according to claim 11, characterized in that, The interface module is used to: after obtaining the complete second data, if the resource status of the storage device indicates that the occupied storage space in the storage device is greater than a second specified threshold, send the second data to the second cloud storage space.

14. The apparatus according to claim 11, characterized in that, The interface module is used to: after obtaining the complete second data, during the target time period, and when the resource status of the storage device indicates that the idle level of the processing resources of the storage device meets the specified conditions, send the second data to the second cloud storage space.

15. The apparatus according to claim 9 or 10, characterized in that, The second data includes the third sub-data and the fourth sub-data; The interface module is used for: If the second data is greater than a first specified threshold and / or the storage space occupied in the storage device is greater than a second specified threshold, the third sub-data from the first cloud storage space is received; In response to receiving the third sub-data from the first cloud storage space, the third sub-data is sent to the second cloud storage space; Receive the fourth sub-data from the first cloud storage space; In response to receiving the fourth sub-data from the first cloud storage space, the fourth sub-data is sent to the second cloud storage space.

16. The apparatus according to any one of claims 9-15, characterized in that, The processing module is used for: The storage location of the second data is determined according to the hierarchical storage strategy; If it is determined, based on the tiered storage strategy, that the second data needs to be migrated from the first cloud storage space, the data migration request is generated.

17. A storage device, characterized in that, The storage device includes at least one processor, a memory, and instructions stored in the memory and executable by the at least one processor, wherein the at least one processor executes the instructions to implement the steps of the method according to any one of claims 1-8.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-8.

19. A computer program product containing instructions, characterized in that, When the instructions are executed by the processor, the method described in any one of claims 1-8 is implemented.

Images