Data migration method, system, device, electronic device and storage medium

Abstract

The present disclosure provides a data migration method, system, device, electronic equipment and storage medium, and relates to the technical field of cloud storage. The method comprises the following steps: determining target data on an object storage server to be migrated to a data migration client; creating a cloud server instance according to the target data; and pulling the target data from the object storage server through the cloud server instance and delivering the target data to the data migration client. The present disclosure can reduce the data migration cost of migrating data from the object storage server.

Description

Technical Field

[0001] This disclosure relates to the field of cloud storage technology, and in particular to a data migration method, system, apparatus, electronic device, and storage medium. Background Technology

[0002] Object Storage Service (OBS) is a massive object-based storage service provided by major cloud platforms. It is suitable for storing files of any type and is suitable for ordinary users, websites, enterprises, and developers.

[0003] Currently, in order to bind users to their cloud services, major cloud platforms charge extra for public network outbound traffic on their object storage servers. Furthermore, the larger the amount of data a user migrates, the higher the data migration cost for the user, resulting in a poor cloud service experience for the user.

[0004] Therefore, how to provide a low-cost data migration solution to migrate data from object storage servers to users' clients is a technical problem that urgently needs to be solved.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] This disclosure provides a data migration method, system, apparatus, electronic device, and storage medium, which at least to some extent overcomes the technical problem of high cost associated with migrating data directly from an object storage server in related technologies.

[0007] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0008] According to one aspect of this disclosure, a data migration method is provided, the method comprising: determining target data to be migrated from an object storage server to a data migration client; creating a cloud server instance based on the target data; pulling the target data from the object storage server through the cloud server instance and sending it to the data migration client.

[0009] In some embodiments, after transferring the target data on the cloud server instance to the data migration client, the method further includes: destroying the cloud server instance.

[0010] In some embodiments, before transmitting the target data on the cloud server instance to the data migration client, the method further includes: constructing an early warning model; and monitoring the data migration process from the cloud server instance to the data migration client according to the early warning model.

[0011] In some embodiments, before transferring the target data from the object storage server to the cloud server instance, the method further includes: obtaining the total data volume of the target data; determining, based on the total data volume of the target data, the traffic price and maximum allowed duration required to directly download the target data from the object storage server; determining, based on the total data volume of the target data and the maximum allowed duration, the public network bandwidth required to download the target data; selecting a target cloud server from a plurality of cloud servers that meets preset conditions based on the traffic price required to directly download the target data from the object storage server and the public network bandwidth required to download the target data, wherein the traffic price of the target cloud server is less than the traffic price required to directly download the target data from the object storage server, and the public network bandwidth of the target cloud server is greater than the public network bandwidth required to download the target data; and creating the cloud server instance based on the hardware configuration information and public network bandwidth of the target cloud server.

[0012] In some embodiments, before selecting a target cloud server that meets preset conditions from a plurality of cloud servers based on the traffic price required to download the target data directly from the object storage server and the public network bandwidth required to download the target data, the method further includes: obtaining hardware configuration information and public network bandwidth of a plurality of cloud servers; constructing a price matrix of the plurality of cloud servers based on the hardware configuration information and public network bandwidth of the plurality of cloud servers, wherein each element in the price matrix represents the traffic price of the cloud server under different hardware configuration information and different public network bandwidth.

[0013] In some embodiments, before selecting a target cloud server that meets preset conditions from a plurality of cloud servers based on the traffic price required to download the target data directly from the object storage server and the public network bandwidth required to download the target data, the method further includes: virtualizing the plurality of cloud servers to construct a virtualized cloud platform.

[0014] According to another aspect of this disclosure, a data migration system is also provided, comprising: a data migration client, a cloud platform, and an object storage server; wherein, the data migration client is used to initiate a data migration request to the object storage server; the object storage server is used to determine the target data to be migrated to the data migration client based on the data migration request initiated by the data migration client; the cloud platform is used to create a cloud server instance based on the target data, and pull the target data from the object storage server through the cloud server instance, and distribute it to the data migration client.

[0015] According to another aspect of this disclosure, a data migration apparatus is also provided, comprising: a data to be migrated determination module, configured to determine target data on an object storage server to be migrated to a data migration client; a cloud server instance creation module, configured to create a cloud server instance based on the target data; and a data migration module, configured to pull the target data from the object storage server through the cloud server instance and send it to the data migration client.

[0016] According to another aspect of this disclosure, an electronic device is also provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the data migration method described in any of the preceding claims by executing the executable instructions.

[0017] According to another aspect of this disclosure, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the data migration method described in any of the preceding claims.

[0018] According to another aspect of this disclosure, a computer program product is also provided, including a computer program that, when executed by a processor, implements the data migration method described above.

[0019] The data migration methods, systems, apparatuses, electronic devices, and storage media provided in the embodiments of this disclosure, when migrating data from an object storage server to a data migration client, first migrate the data to a cloud server, and then the cloud server distributes it to the data migration client. Through the embodiments of this disclosure, by utilizing the fact that data transmission between the cloud server and the object storage server is free, while the cloud server's downlink traffic is charged based on bandwidth, the data migration cost of migrating data from the object storage server can be reduced.

[0020] Furthermore, monitoring the data migration process through early warning models can improve the reliability of data migration.

[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0023] Figure 1 This diagram illustrates an application system architecture according to an embodiment of the present disclosure.

[0024] Figure 2 This diagram illustrates a data migration method according to an embodiment of the present disclosure.

[0025] Figure 3 This diagram illustrates a flowchart of an optional data migration method according to an embodiment of the present disclosure.

[0026] Figure 4 This diagram illustrates a flowchart of creating a cloud server instance according to an embodiment of the present disclosure;

[0027] Figure 5 This diagram illustrates a method for implementing cloud platform virtualization and early warning based on Fusioncomputer, as described in an embodiment of this disclosure.

[0028] Figure 6 This diagram illustrates a detailed implementation flowchart of the data migration method in an embodiment of this disclosure.

[0029] Figure 7 This diagram illustrates a data migration system according to an embodiment of the present disclosure.

[0030] Figure 8 This diagram illustrates a data migration apparatus according to an embodiment of the present disclosure.

[0031] Figure 9 This diagram illustrates a structural block diagram of an electronic device according to an embodiment of the present disclosure;

[0032] Figure 10 A schematic diagram of a computer-readable storage medium is shown in an embodiment of the present disclosure. Detailed Implementation

[0033] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0034] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0035] To facilitate understanding, before introducing the embodiments of this disclosure, the following explanations are provided for several terms involved in the embodiments of this disclosure:

[0036] The specific implementation methods of the embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0037] Currently, public cloud platforms have been widely deployed across various industries. From an industry perspective, each sector has built its own industry-specific solutions based on cloud computing technology. On the other hand, in line with technological development trends, the current trend in big data platform infrastructure is modularization and public cloud platformization. This public cloud platform not only provides the scheduling of physical underlying resources but also needs to and can provide various big data-related middleware PaaS (Platform as a Service) services. These services can be easily and flexibly combined to meet the customized needs of various application scenarios. Hybrid cloud platforms offer flexible resource definitions, covering various resource types in the big data processing process and providing support for different, targeted computing, storage, and network resources.

[0038] Large enterprise customers often use cloud services from multiple public cloud platforms simultaneously to minimize total cost while achieving load balancing and disaster recovery. Meanwhile, cloud disaster recovery vendors also perform off-site backups of customer data stored in cloud storage.

[0039] Figure 1 A schematic diagram of an exemplary application system architecture to which the data migration method of the embodiments of this disclosure can be applied is shown. For example... Figure 1 As shown, the system architecture may include: a data migration client 10, a cloud platform 20, and an object storage server 30.

[0040] Among them, the object storage server 30 is used to provide object-based data storage services. The basic components of the object storage service are buckets and objects. A bucket is a container for storing objects in the object storage service. Each bucket has its own storage category, access permissions, and region attributes. Users locate buckets on the Internet through the bucket's access domain name. An object is the basic unit of data storage in the object storage service. An object is actually a collection of data of a file and its related attribute information, including key-value pairs, metadata, and data.

[0041] The communication between the data migration client 10 and the cloud platform 20, and between the data migration client 10 and the object storage server 30, can be via wired network or wireless network.

[0042] Optionally, the aforementioned wireless or wired networks use standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to Local Area Networks (LANs), Metropolitan Area Networks (MANs), Wide Area Networks (WANs), mobile, wired or wireless networks, private networks, or any combination of virtual private networks. In some embodiments, technologies and / or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Networks (VPNs), and Internet Protocol Security (IPsec) can be used to encrypt all or some links. In other embodiments, custom and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0043] The data migration client 10 can be various electronic devices, including but not limited to smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, wearable devices, augmented reality devices, virtual reality devices, etc.

[0044] Optionally, the application clients installed in different data migration clients 10 may be the same, or clients of the same type of application based on different operating systems. Depending on the terminal platform, the specific form of the application client may also differ; for example, the application client may be a mobile client, a PC client, etc.

[0045] The cloud platform 20 can be a server that provides various cloud services. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers.

[0046] Those skilled in the art will know that Figure 1 The number of data migration clients, cloud platforms, and object storage servers shown is merely illustrative; any number of terminal devices, networks, and servers can be included depending on actual needs. This disclosure does not limit the scope of the embodiments.

[0047] Under the above system architecture, this disclosure provides a data migration method that can be executed by any electronic device with computing capabilities.

[0048] In some embodiments, the data migration method provided in this disclosure can be executed by the cloud platform of the above-described system architecture; in other embodiments, the data migration method provided in this disclosure can be implemented by the cloud platform in the above-described system architecture interacting with the object storage server and the data migration client; in other embodiments, the data migration method provided in this disclosure can also be executed by a third-party device that communicates with the data migration client, the cloud platform and the object storage server in the above-described system architecture.

[0049] Figure 2 This diagram illustrates a data migration method according to an embodiment of the present disclosure, such as... Figure 1 As shown, the data migration method provided in this embodiment includes the following steps:

[0050] S202, Determine the target data on the object storage server to be migrated to the data migration client.

[0051] It should be noted that the object storage server in this disclosure refers to a server that provides data storage services based on objects; in some embodiments, it may be a cloud storage server provided by various cloud platforms. The data migration client in this disclosure refers to various clients that want to download data from the object storage server; in some embodiments, the target data to be migrated from the object storage server to the data migration client can be determined based on the data migration request initiated by the data migration client to the object storage server. The target data in this disclosure may be various types of data stored on the object storage server, including but not limited to: data in file, image, and other formats. In some embodiments, when the target data is stored as objects on the object storage server, the above-described S202 can determine a list of objects containing the target data to be migrated.

[0052] S204. Create a cloud server instance based on the target data.

[0053] It should be noted that the cloud server instance in this embodiment can be a temporarily created virtual cloud server containing computing components such as CPU, memory, operating system, network, and disk. After determining the target data to be migrated, a corresponding cloud server instance can be created based on the target data. This server instance can pull the target data from the object storage server and distribute it to the data migration client.

[0054] S206 retrieves target data from the object storage server via a cloud server instance and distributes it to the data migration client.

[0055] Since clients are charged based on bandwidth when downloading data directly from object storage servers, the larger the amount of data downloaded, the higher the charge; while cloud server instances are free to pull data from object storage servers, and cloud servers are charged based on fixed public network bandwidth when sending data to clients, the target data to be migrated on the object storage server can be transferred to the cloud server instance, and then the cloud server instance can send it to the corresponding data migration client.

[0056] In some embodiments, after pulling the target data from the object storage server via a cloud server instance and sending it to the data migration client, such as Figure 3 As shown, the data migration method provided in this embodiment may further include the following steps:

[0057] S208, Destroy the cloud server instance.

[0058] To prevent data transmission failures caused by data discrepancies and potential faults during the data migration process from the cloud server to the data migration client, this disclosure employs an early warning model to monitor and warn of the data transmission process. In some embodiments, before pulling the target data from the object storage server via the cloud server instance and sending it to the data migration client, such as... Figure 3 As shown, the data migration method provided in this embodiment may further include the following steps:

[0059] S210, Construct an early warning model;

[0060] S212 monitors the data migration process from cloud server instances to data migration clients based on the early warning model.

[0061] In one embodiment, the following early warning model can be constructed:

[0062] X(k+1)=X(k)×P (1)

[0063] Where X(k+1) represents the state vector of the object to be predicted at time t = k+1; X(k) represents the state vector of the object to be predicted at time t = k; and P represents the one-step transition probability matrix.

[0064] After constructing the early warning model, a two-step transition matrix is ​​used to generate a dataset and monitor data transmission traffic to provide early warning for the migration process of target data.

[0065] For example, in a one-step transfer process, the flow value in time period 1 is 10M, of which the in direction flow is 3M and the out direction flow is 7M. 60% of the in direction flow may continue in the in direction and 40% may be converted to the out direction. The flow is [0.6, 0.4]in + out = total flow.

[0066] During the two-step transfer process, 70% of the traffic in time period 2 may still be out-direction traffic, and 30% may be transferred to the in-direction [0.3, 0.7]. in + out = total traffic.

[0067] (3M, 7M)

[0068] Time period 1 [0.6, 0.4]

[0069] Time period 2 [0.3, 0.7]

[0070] The in direction of time period 2 = 0.3 x 0.6 + 0.3 x 0.7 = 3.9M

[0071] Out direction = 0.3 x 0.4 + 0.7 x 0.7 = 6.1M

[0072] Based on the inflow and outflow of data transmission, predict the probability of failures that may occur during the target data synchronization process.

[0073] In some embodiments, such as Figure 4 As shown, the data migration method provided in this embodiment can also create a cloud server example through the following steps:

[0074] S402, Obtain the total amount of target data.

[0075] S404, based on the total amount of target data, determine the traffic price and maximum allowed duration required to download the target data directly from the object storage server.

[0076] It should be noted that the maximum allowable duration in this embodiment can be determined based on a variety of different factors, such as the urgency of the migration, the total amount of data to be migrated, and the expected cost of the migration.

[0077] S406: Determine the public network bandwidth required to download the target data based on the total data volume and the maximum allowed duration.

[0078] In one embodiment, the public network bandwidth required to download the target data can be calculated using the following formula:

[0079] B = DataVolume / T max (2)

[0080] Where B represents the public network bandwidth required to download the target data; DataVolume represents the total amount of target data; T max This indicates the maximum allowed duration.

[0081] S408 selects a target cloud server from multiple cloud servers that meets preset conditions based on the traffic cost required to download the target data directly from the object storage server and the public network bandwidth required to download the target data.

[0082] It should be noted that the above preset conditions can be: the traffic price of the target cloud server is less than the traffic price required to download the target data directly from the object storage server, and the public network bandwidth of the target cloud server is greater than the public network bandwidth required to download the target data. When multiple target cloud servers meet the preset conditions, the target cloud server with the lowest price can be selected.

[0083] S410 creates a cloud server instance based on the target cloud server's hardware configuration information and public network bandwidth.

[0084] In some embodiments, before selecting a target cloud server that meets preset conditions from multiple cloud servers based on the traffic cost required to download the target data directly from the object storage server and the public network bandwidth required to download the target data, such as Figure 4 As shown, the data migration method provided in this embodiment may further include the following steps:

[0085] S412, obtains hardware configuration information and public network bandwidth of multiple cloud servers;

[0086] S414 constructs a price matrix for multiple cloud servers based on their hardware configuration information and public network bandwidth. Each element in the price matrix represents the traffic price of the cloud server under different hardware configuration information and different public network bandwidth.

[0087] It should be noted that the price matrix in this embodiment may include hardware configuration information of multiple cloud servers, including but not limited to: the number of CPU cores and the memory size.

[0088] In one embodiment, the constructed price matrix can be represented as:

[0089] Price = [P] ij (2)

[0090] Where i represents the hardware configuration information of the cloud server, j represents the public network bandwidth of the cloud server; P ij This indicates the price of traffic for cloud servers.

[0091] The above preset conditions can then be expressed as:

[0092] Min(P ij )=Min{P ij |BW(j)>B,P ij <P out} (3)

[0093] Where BW(j) represents the public network bandwidth of the cloud server; B represents the public network bandwidth required to download the target data directly from the object storage server; P out This represents the cost of downloading target data directly from the object storage server; Min() represents the minimum value function.

[0094] It should be noted that public network bandwidth can fluctuate due to changes in network conditions. To ensure the data migration task runs smoothly on T... max Completed internally, a threshold condition α can be added, the value of which can be manually adjusted, with a default of 20%. The above preset condition can be optimized as follows:

[0095] Min(P ij )=Min{Pij |BW(j)>B×(1+α),P ij <P out} (4)

[0096] In some embodiments, before selecting a target cloud server that meets preset conditions from multiple cloud servers based on the traffic price required to download the target data directly from the object storage server and the public network bandwidth required to download the target data, the data migration method provided in this disclosure may further include the following steps: virtualizing multiple cloud servers to build a virtualized cloud platform.

[0097] Figure 5 A schematic diagram illustrating cloud platform virtualization and early warning based on Fusioncomputer is shown, such as... Figure 5 As shown, a virtualized cloud platform can be built by virtualizing multiple cloud servers using Fusioncomputer. Through Fusioncomputer's open API interface, data can be pulled from object storage servers and synchronized with data on the virtualized multi-cloud platform, thereby predicting the probability of failures that may occur during the synchronization process.

[0098] Figure 6 This document shows a flowchart illustrating the specific implementation of the data migration method in an embodiment of this disclosure. Figure 6 As shown, the specific steps include the following:

[0099] S602, determine the target data to be migrated, calculate the total data volume of the target data, and obtain the traffic price for transmitting the target data to the object storage server;

[0100] S604, Determine the maximum allowable duration for migrating the target data;

[0101] S606, determine the public network bandwidth for transmitting target data;

[0102] S608 constructs a cloud server price matrix, where the columns of the cloud server price matrix are the hardware configuration information of the cloud server, the columns of the cloud server price matrix are the public network bandwidth of the cloud server, and each element in the cloud server price matrix represents the traffic price of the cloud server under different hardware configuration information and different public network bandwidth.

[0103] S610: Select target cloud servers that meet preset conditions from the cloud server price matrix;

[0104] S612, determine if there is a target cloud server that meets the preset conditions; if yes, execute S616; if no, execute S614.

[0105] S614, downloads the target data directly from the object storage server;

[0106] S616 creates a corresponding cloud server instance based on the target cloud server's hardware configuration information and public network bandwidth;

[0107] S618 installs a data download agent module on a cloud server instance to pull target data from the object storage server and send it to the data migration client according to the data migration request from the data migration client.

[0108] S620: When all target data is sent to the data migration client, the cloud server instance is destroyed, and the data migration process ends.

[0109] As can be seen from the above, the data migration method provided in this embodiment of the present disclosure calculates the public network bandwidth required to directly download the target data from the object storage server by evaluating the total amount of target data to be migrated from the cloud storage server and the maximum allowable time required for migration. Then, based on the traffic price and public network bandwidth for directly downloading the target data from the object storage server, it selects target cloud servers that meet preset conditions from the constructed cloud server price matrix. Based on the hardware configuration information and public network bandwidth of the target cloud server, it creates a cloud server instance and installs a data download proxy module in the cloud server instance. The data download proxy module is then used to download the target data to the data migration client, thereby avoiding the additional charges required to directly download the target data from the object storage server to the data migration client, and greatly reducing the data migration cost.

[0110] Furthermore, by building an early warning model, traffic monitoring can be performed on the data migration process, thereby improving the reliability of data migration.

[0111] Based on the same inventive concept, this disclosure also provides a data migration system, as described in the following embodiments. Since the principle by which this system embodiment solves the problem is similar to that of the method embodiment described above, the implementation of this system embodiment can refer to the implementation of the method embodiment described above, and repeated details will not be repeated.

[0112] Figure 7 This diagram illustrates a data migration system according to an embodiment of the present disclosure, such as... Figure 7 As shown, the system includes: a data migration client 10, a cloud platform 20, and an object storage server 30;

[0113] The data migration client 10 is used to initiate a data migration request to the object storage server 30; the object storage server 30 is used to determine the target data to be migrated to the data migration client based on the data migration request initiated by the data migration client; the cloud platform 20 is used to create a cloud server instance based on the target data, and pull the target data from the object storage server through the cloud server instance and distribute it to the data migration client 30.

[0114] In practice, a cloud server instance can be created on the cloud platform 20, and a data download agent module can be installed in the cloud server instance. The data download agent module can be used to pull data from the object storage server and distribute it to the data migration client.

[0115] Based on the same inventive concept, this disclosure also provides a data migration apparatus, as described in the following embodiments. Since the principle by which this apparatus solves the problem is similar to that of the method embodiments described above, the implementation of this apparatus embodiment can refer to the implementation of the method embodiments described above, and repeated details will not be repeated.

[0116] Figure 8 This diagram illustrates a data migration apparatus according to an embodiment of the present disclosure, such as... Figure 8 As shown, the device includes: a data to be migrated determination module 81, a cloud server instance creation module 82, and a data migration module 83.

[0117] The module 81, which determines the target data to be migrated from the object storage server to the data migration client, is used to determine the target data to be migrated from the object storage server to the data migration client. The cloud server instance creation module 82 is used to create a cloud server instance based on the target data. The data migration module 83 is used to pull the target data from the object storage server through the cloud server instance and send it to the data migration client.

[0118] In some embodiments, such as Figure 8 As shown, the data migration device provided in this embodiment may further include: a cloud server instance destruction module 84, used to destroy cloud server instances.

[0119] In some embodiments, such as Figure 8 As shown, the data migration device provided in this embodiment may further include: a data migration process monitoring module 85, used to build an early warning model; and to monitor the data migration process from the cloud server instance to the data migration client according to the early warning model.

[0120] In some embodiments, the cloud server instance creation module 82 is further configured to: obtain the total data volume of the target data; determine the traffic price and maximum allowed duration required to download the target data directly from the object storage server based on the total data volume of the target data; determine the public network bandwidth required to download the target data based on the total data volume of the target data and the maximum allowed duration; select a target cloud server that meets preset conditions from multiple cloud servers based on the traffic price and public network bandwidth required to download the target data directly from the object storage server, wherein the traffic price of the target cloud server is less than the traffic price required to download the target data directly from the object storage server, and the public network bandwidth of the target cloud server is greater than the public network bandwidth required to download the target data; and create a cloud server instance based on the hardware configuration information and public network bandwidth of the target cloud server.

[0121] In some embodiments, such as Figure 8 As shown, the data migration device provided in this embodiment may further include: a cloud server price matrix construction module 86, used to obtain hardware configuration information and public network bandwidth of multiple cloud servers; and to construct a price matrix of multiple cloud servers based on the hardware configuration information and public network bandwidth of multiple cloud servers, wherein each element in the price matrix represents the traffic price of the cloud server under different hardware configuration information and different public network bandwidth.

[0122] In some embodiments, such as Figure 8 As shown, the data migration apparatus provided in this embodiment may further include: a cloud service virtualization module 87, used to virtualize multiple cloud servers and build a virtualized cloud platform.

[0123] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above method embodiments. It should also be noted that the above modules, as part of an apparatus, can be executed in a computer system such as a set of computer-executable instructions.

[0124] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0125] The following reference Figure 9 To describe an electronic device 900 according to such an embodiment of the present disclosure. Figure 9 The electronic device 900 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0126] like Figure 9 As shown, the electronic device 900 is manifested in the form of a general-purpose computing device. The components of the electronic device 900 may include, but are not limited to: at least one processing unit 910, at least one storage unit 920, and a bus 930 connecting different system components (including the storage unit 920 and the processing unit 910).

[0127] The storage unit stores program code that can be executed by the processing unit 910, causing the processing unit 910 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processing unit 910 can perform the following steps of the above method embodiments: determining the target data to be migrated from the object storage server to the data migration client; creating a cloud server instance based on the target data; pulling the target data from the object storage server through the cloud server instance and sending it to the data migration client.

[0128] Storage unit 920 may include readable media in the form of volatile storage units, such as random access memory (RAM) 9201 and / or cache memory 9202, and may further include read-only memory (ROM) 9203.

[0129] Storage unit 920 may also include a program / utility 9204 having a set (at least one) program module 9205, such program module 9205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0130] Bus 930 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0131] Electronic device 900 can also communicate with one or more external devices 940 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 900, and / or with any device that enables electronic device 900 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 950. Furthermore, electronic device 900 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 960. As shown, network adapter 960 communicates with other modules of electronic device 900 via bus 930. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0132] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0133] In particular, according to embodiments of this disclosure, the process described above with reference to the flowchart can be implemented as a computer program product, which includes a computer program that, when executed by a processor, implements the above-described data migration method.

[0134] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. Figure 10 This illustration shows a schematic diagram of a computer-readable storage medium according to an embodiment of the present disclosure, such as... Figure 10 As shown, the computer-readable storage medium 1000 stores a program product capable of implementing the methods described above in this disclosure. In some possible embodiments, various aspects of this disclosure may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0135] More specific examples of computer-readable storage media in this disclosure may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0136] In this disclosure, a computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device.

[0137] Optionally, the program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0138] In practical implementation, program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0139] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0140] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0141] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0142] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A data migration method, characterized in that, include: Determine the target data on the object storage server to be migrated to the data migration client; Create a cloud server instance based on the target data; The target data is pulled from the object storage server through the cloud server instance and sent to the data migration client; The process of creating a cloud server instance based on the target data includes: obtaining the total data volume of the target data; determining the traffic price and maximum allowed duration required to download the target data directly from the object storage server based on the total data volume of the target data; determining the public network bandwidth required to download the target data based on the total data volume of the target data and the maximum allowed duration; selecting a target cloud server from multiple cloud servers that meets preset conditions based on the traffic price and public network bandwidth required to download the target data directly from the object storage server, wherein the traffic price of the target cloud server is less than the traffic price required to download the target data directly from the object storage server, and the public network bandwidth of the target cloud server is greater than the public network bandwidth required to download the target data; and creating the cloud server instance based on the hardware configuration information and public network bandwidth of the target cloud server.

2. The data migration method according to claim 1, characterized in that, After pulling the target data from the object storage server via the cloud server instance and sending it to the data migration client, the method further includes: Destroy the cloud server instance.

3. The data migration method according to claim 1, characterized in that, Before pulling the target data from the object storage server via the cloud server instance and sending it to the data migration client, the method further includes: Build an early warning model; The data migration process from the cloud server instance to the data migration client is monitored based on the aforementioned early warning model.

4. The data migration method according to claim 1, characterized in that, Before selecting a target cloud server that meets preset conditions from multiple cloud servers based on the traffic cost required to download the target data directly from the object storage server and the public network bandwidth required to download the target data, the method further includes: Obtain hardware configuration information and public network bandwidth of multiple cloud servers; Based on the hardware configuration information and public network bandwidth of the multiple cloud servers, a price matrix for the multiple cloud servers is constructed, wherein each element in the price matrix represents the traffic price of the cloud server under different hardware configuration information and different public network bandwidth.

5. The data migration method according to claim 1, characterized in that, Before selecting a target cloud server that meets preset conditions from multiple cloud servers based on the traffic cost required to download the target data directly from the object storage server and the public network bandwidth required to download the target data, the method further includes: The multiple cloud servers are virtualized to build a virtualized cloud platform.

6. A data migration system, characterized in that, include: Data migration clients, cloud platforms, and object storage servers; The data migration client is used to initiate a data migration request to the object storage server. The object storage server is used to determine the target data to be migrated to the data migration client based on the data migration request initiated by the data migration client; The cloud platform is used to create a cloud server instance based on the target data, and pull the target data from the object storage server through the cloud server instance, and send it to the data migration client. The cloud platform is further configured to: acquire the total data volume of the target data; determine the traffic price and maximum allowed duration required to download the target data directly from the object storage server based on the total data volume of the target data; determine the public network bandwidth required to download the target data based on the total data volume of the target data and the maximum allowed duration; select a target cloud server from multiple cloud servers that meets preset conditions based on the traffic price and public network bandwidth required to download the target data directly from the object storage server, wherein the traffic price of the target cloud server is less than the traffic price required to download the target data directly from the object storage server, and the public network bandwidth of the target cloud server is greater than the public network bandwidth required to download the target data; and create a cloud server instance based on the hardware configuration information and public network bandwidth of the target cloud server.

7. A data migration device, characterized in that, include: The data to be migrated module is used to determine the target data on the object storage server to be migrated to the data migration client; The cloud server instance creation module is used to create a cloud server instance based on the target data. The data migration module is used to pull the target data from the object storage server through the cloud server instance and send it to the data migration client; The cloud server instance creation module is further configured to: obtain the total data volume of the target data; determine the traffic price and maximum allowed duration required to download the target data directly from the object storage server based on the total data volume of the target data; determine the public network bandwidth required to download the target data based on the total data volume of the target data and the maximum allowed duration; select a target cloud server from multiple cloud servers that meets preset conditions based on the traffic price and public network bandwidth required to download the target data directly from the object storage server, wherein the traffic price of the target cloud server is less than the traffic price required to download the target data directly from the object storage server, and the public network bandwidth of the target cloud server is greater than the public network bandwidth required to download the target data; and create the cloud server instance based on the hardware configuration information and public network bandwidth of the target cloud server.

8. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the data migration method of any one of claims 1 to 5 by executing the executable instructions.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the data migration method according to any one of claims 1 to 5.

Images